Evaluating bioassessment designs and decision thresholds using simulation techniques

Craig D. Snyder, Nathaniel P Hitt, David R. Smith, Jonathan P. Daily

Research output: Chapter in Book/Report/Conference proceedingChapter

1 Citation (Scopus)

Abstract

Natural resource managers face numerous choices when developing bioassessment programs but seldom have the opportunity to compare the performance of alternative designs. As a result, managers often lack a basis for establishing decision thresholds based on their objectives for evaluating resource condition, accounting for uncertainty, and controlling costs. In this chapter, we illustrate how simulation techniques may be used to optimize bioassessment decision thresholds and sampling designs with a case study of benthic macroinvertebrate communities in Shenandoah National Park, USA. We evaluated the effects of sampling effort (6 levels) and taxonomic resolution (family vs. genus) on the sensitivity of a commonly used index of stream condition (Macroinvertebrate Biotic Integrity Index, MBII) to classify resource condition as affected by ecological change. We computed expected utility values to compare decision thresholds, which integrated statistical power and differential risk tolerance for misclassification (i.e., type I and II error rates). Our analysis revealed important differences among bioassessment designs. MBII sensitivity increased with sampling effort, but improvements were modest across the highest sampling levels. Genus-level assessments were generally most sensitive to ecological change, even though precision increased at the family level due to decreased variation in reference communities. However, the sensitivity-cost relationship revealed no single, optimal combination of taxonomic resolution and sampling effort. Rather, we found that for a given cost, equivalent sensitivities could be obtained from larger samples at the family-level or smaller samples at the genus level. An analysis of expected utility demonstrated that the optimal decision threshold depends on prior probability of resource condition, i.e., reference, early warning, or impaired. We conclude that simulation methods provide a flexible approach to evaluate and optimize bioassessment designs and decision thresholds based on objective-specific utility values.

Original languageEnglish (US)
Title of host publicationApplication of Threshold Concepts in Natural Resource Decision Making
PublisherSpringer New York
Pages157-197
Number of pages41
Volume9781489980410
ISBN (Electronic)9781489980410
ISBN (Print)1489980407, 9781489980403
DOIs
StatePublished - Oct 1 2014

Fingerprint

Sampling
macroinvertebrate
sampling
Index of Biotic Integrity
simulation
Managers
resource
cost
Costs
Natural resources
national park
natural resource
tolerance
decision
family
index
analysis

All Science Journal Classification (ASJC) codes

  • Engineering(all)
  • Environmental Science(all)

Cite this

Snyder, C. D., Hitt, N. P., Smith, D. R., & Daily, J. P. (2014). Evaluating bioassessment designs and decision thresholds using simulation techniques. In Application of Threshold Concepts in Natural Resource Decision Making (Vol. 9781489980410, pp. 157-197). Springer New York. https://doi.org/10.1007/978-1-4899-8041-0_9
Snyder, Craig D. ; Hitt, Nathaniel P ; Smith, David R. ; Daily, Jonathan P. / Evaluating bioassessment designs and decision thresholds using simulation techniques. Application of Threshold Concepts in Natural Resource Decision Making. Vol. 9781489980410 Springer New York, 2014. pp. 157-197
@inbook{33ece2e1097644c5835fa50212ba3539,
title = "Evaluating bioassessment designs and decision thresholds using simulation techniques",
abstract = "Natural resource managers face numerous choices when developing bioassessment programs but seldom have the opportunity to compare the performance of alternative designs. As a result, managers often lack a basis for establishing decision thresholds based on their objectives for evaluating resource condition, accounting for uncertainty, and controlling costs. In this chapter, we illustrate how simulation techniques may be used to optimize bioassessment decision thresholds and sampling designs with a case study of benthic macroinvertebrate communities in Shenandoah National Park, USA. We evaluated the effects of sampling effort (6 levels) and taxonomic resolution (family vs. genus) on the sensitivity of a commonly used index of stream condition (Macroinvertebrate Biotic Integrity Index, MBII) to classify resource condition as affected by ecological change. We computed expected utility values to compare decision thresholds, which integrated statistical power and differential risk tolerance for misclassification (i.e., type I and II error rates). Our analysis revealed important differences among bioassessment designs. MBII sensitivity increased with sampling effort, but improvements were modest across the highest sampling levels. Genus-level assessments were generally most sensitive to ecological change, even though precision increased at the family level due to decreased variation in reference communities. However, the sensitivity-cost relationship revealed no single, optimal combination of taxonomic resolution and sampling effort. Rather, we found that for a given cost, equivalent sensitivities could be obtained from larger samples at the family-level or smaller samples at the genus level. An analysis of expected utility demonstrated that the optimal decision threshold depends on prior probability of resource condition, i.e., reference, early warning, or impaired. We conclude that simulation methods provide a flexible approach to evaluate and optimize bioassessment designs and decision thresholds based on objective-specific utility values.",
author = "Snyder, {Craig D.} and Hitt, {Nathaniel P} and Smith, {David R.} and Daily, {Jonathan P.}",
year = "2014",
month = "10",
day = "1",
doi = "10.1007/978-1-4899-8041-0_9",
language = "English (US)",
isbn = "1489980407",
volume = "9781489980410",
pages = "157--197",
booktitle = "Application of Threshold Concepts in Natural Resource Decision Making",
publisher = "Springer New York",
address = "United States",

}

Snyder, CD, Hitt, NP, Smith, DR & Daily, JP 2014, Evaluating bioassessment designs and decision thresholds using simulation techniques. in Application of Threshold Concepts in Natural Resource Decision Making. vol. 9781489980410, Springer New York, pp. 157-197. https://doi.org/10.1007/978-1-4899-8041-0_9

Evaluating bioassessment designs and decision thresholds using simulation techniques. / Snyder, Craig D.; Hitt, Nathaniel P; Smith, David R.; Daily, Jonathan P.

Application of Threshold Concepts in Natural Resource Decision Making. Vol. 9781489980410 Springer New York, 2014. p. 157-197.

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - Evaluating bioassessment designs and decision thresholds using simulation techniques

AU - Snyder, Craig D.

AU - Hitt, Nathaniel P

AU - Smith, David R.

AU - Daily, Jonathan P.

PY - 2014/10/1

Y1 - 2014/10/1

N2 - Natural resource managers face numerous choices when developing bioassessment programs but seldom have the opportunity to compare the performance of alternative designs. As a result, managers often lack a basis for establishing decision thresholds based on their objectives for evaluating resource condition, accounting for uncertainty, and controlling costs. In this chapter, we illustrate how simulation techniques may be used to optimize bioassessment decision thresholds and sampling designs with a case study of benthic macroinvertebrate communities in Shenandoah National Park, USA. We evaluated the effects of sampling effort (6 levels) and taxonomic resolution (family vs. genus) on the sensitivity of a commonly used index of stream condition (Macroinvertebrate Biotic Integrity Index, MBII) to classify resource condition as affected by ecological change. We computed expected utility values to compare decision thresholds, which integrated statistical power and differential risk tolerance for misclassification (i.e., type I and II error rates). Our analysis revealed important differences among bioassessment designs. MBII sensitivity increased with sampling effort, but improvements were modest across the highest sampling levels. Genus-level assessments were generally most sensitive to ecological change, even though precision increased at the family level due to decreased variation in reference communities. However, the sensitivity-cost relationship revealed no single, optimal combination of taxonomic resolution and sampling effort. Rather, we found that for a given cost, equivalent sensitivities could be obtained from larger samples at the family-level or smaller samples at the genus level. An analysis of expected utility demonstrated that the optimal decision threshold depends on prior probability of resource condition, i.e., reference, early warning, or impaired. We conclude that simulation methods provide a flexible approach to evaluate and optimize bioassessment designs and decision thresholds based on objective-specific utility values.

AB - Natural resource managers face numerous choices when developing bioassessment programs but seldom have the opportunity to compare the performance of alternative designs. As a result, managers often lack a basis for establishing decision thresholds based on their objectives for evaluating resource condition, accounting for uncertainty, and controlling costs. In this chapter, we illustrate how simulation techniques may be used to optimize bioassessment decision thresholds and sampling designs with a case study of benthic macroinvertebrate communities in Shenandoah National Park, USA. We evaluated the effects of sampling effort (6 levels) and taxonomic resolution (family vs. genus) on the sensitivity of a commonly used index of stream condition (Macroinvertebrate Biotic Integrity Index, MBII) to classify resource condition as affected by ecological change. We computed expected utility values to compare decision thresholds, which integrated statistical power and differential risk tolerance for misclassification (i.e., type I and II error rates). Our analysis revealed important differences among bioassessment designs. MBII sensitivity increased with sampling effort, but improvements were modest across the highest sampling levels. Genus-level assessments were generally most sensitive to ecological change, even though precision increased at the family level due to decreased variation in reference communities. However, the sensitivity-cost relationship revealed no single, optimal combination of taxonomic resolution and sampling effort. Rather, we found that for a given cost, equivalent sensitivities could be obtained from larger samples at the family-level or smaller samples at the genus level. An analysis of expected utility demonstrated that the optimal decision threshold depends on prior probability of resource condition, i.e., reference, early warning, or impaired. We conclude that simulation methods provide a flexible approach to evaluate and optimize bioassessment designs and decision thresholds based on objective-specific utility values.

UR - http://www.scopus.com/inward/record.url?scp=84929899525&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84929899525&partnerID=8YFLogxK

U2 - 10.1007/978-1-4899-8041-0_9

DO - 10.1007/978-1-4899-8041-0_9

M3 - Chapter

AN - SCOPUS:84929899525

SN - 1489980407

SN - 9781489980403

VL - 9781489980410

SP - 157

EP - 197

BT - Application of Threshold Concepts in Natural Resource Decision Making

PB - Springer New York

ER -

Snyder CD, Hitt NP, Smith DR, Daily JP. Evaluating bioassessment designs and decision thresholds using simulation techniques. In Application of Threshold Concepts in Natural Resource Decision Making. Vol. 9781489980410. Springer New York. 2014. p. 157-197 https://doi.org/10.1007/978-1-4899-8041-0_9