The landing page at https://verra.org states that:
“Verra catalyzes tangible climate action and sustainable development outcomes. Verra’s standards drive large-scale investment towards high-impact activities that tackle some of the most pressing environmental and social issues of our day.”
Verra’s six listed standards and programs includes one entitled, “Sustainable Development Verified Impact Standards.” Two documents providing details on this kind of standard are available for download. One concerns “Methodology for Coastal Resilience Benefits from Restoration and Protection of Tidal Wetlands.” The methodology lays out a descriptive group-level statistical model of an ordinal unit, and not a prescriptive individual-level measurement model of an interval unit. Even though the stem ‘measur-‘ at the root of ‘measured,’ ‘measurement,’ etc. appears 119 times in the standard’s 51 pages, there is no definition of a composite Coastal Resilience Benefits interval unit quantity and associated uncertainty, nor is there any mention of experimental tests of the hypothesis that such a unit quantity can be identified and estimated.
The standard takes it for granted that physical measurements of distance, mass, volume, time, temperature, etc. are sufficient to the task of measuring coastal resilience benefits. But this is what is termed in logic as a category mistake, an ecological fallacy, or what Alfred North Whitehead (1925, pp. 52-58) called the “fallacy of misplaced concreteness.” Gregory Bateson (1972, pp. 73, 180-185, 491-495) similarly made much of the epistemological errors committed when the map is confused for the territory, the forest for the trees. In short, measuring coastal resilience benefits demands that this construct (also known in metrological terms as a measurand) itself be modeled, estimated, and calibrated in an identified and defined interval unit quantity.
Extensive and longstanding authoritative resources on measurement models supporting metrologically quality-assured instrument calibration traceability of this kind are available (Luce & Tukey, 1964; Rasch, 1960, 1961; Wright, 1977, 1997; Bond & Fox, 2015; Fisher & Wilson, 2015; Fisher & Wright, 1994; Mari & Wilson, 2014; Mari, et al., 2021; Pendrill, 2019; Pendrill & Fisher, 2015; Wilson, 2005, 2013a/b; Wilson & Fisher, 2016, 2019; etc.), with a similarly voluminous array of sustainable development applications (Cano, et al., 2019; Fisher, 2020a/b, 2021a/b; Fisher, et al., 2019, 2021; Fisher & Wilson, 2019; Madhala & Fisher, 2022; Moral, et al., 2006, 2014, 2016; Kaiser & Wilson, 2000, 2004; etc.). Writing in 1986, Narens and Luce (1986, pp. 167-169) pointed out that additive conjoint log-interval models developed in the 1960s (Luce & Tukey, 1964; Rasch, 1960, 1961; Wright, 1997) were “widely accepted” as providing access to fundamental measurement. Unfortunately, we have yet to even begin capitalizing on the opportunities for scientific, economic, social, and environmental progress offered by these models (Fisher, 2011, 2012a/b, 2020a).
Where statistical models are concerned with group-level processes occurring in the relations between variables, measurement models focus on substantive processes as they impact individuals. Actionable, meaningful management gets a grip on things in the world only in terms of measurements that give insight into what can be done in specific instances, and that can then be communicated in a common language across those instances. Statistical models have a number of debilitating shortcomings that make them highly unsatisfactory as a basis for quantification (Fisher, 2022). In addition to not positing and testing for interval quantities, these models do not:
- articulate the individual-level response process;
- map the development continuum;
- provide individual level quantity, uncertainty, or consistency estimates;
- meaningfully reduce data volume by an order of magnitude;
- support the development of a metrologically quality assured instrument calibration network;
- report out individual and group measurements in a way showing what has been accomplished relative to overall goals, what comes next, and special strengths and weaknesses;
- enable the cost accounting, arbitrage, and pricing of unit outcomes;
- nonreductively quantify living processes in ways that make them objectively reproducible over time and space;
- represent individual- and group-level properties in comparable terms that support legal title to personal stocks of human, social, and natural capital, and profitable investments in and returns from those stocks.
For instance, Sections 9.1 and 9.2 in the coastal wetlands standard lists all the parameters to be monitored. Two comments pertain. First, these “parameters” are actually indicators that ought to be combined into an overarching composite model testing the statistical sufficiency of the observations–i.e., their capacity to serve as a basis for estimating interval unit quantities and uncertainties. In measurement theory and practice, the model parameters are the mathematical terms in the equation specifying the stimulus and response variables being quantified. Estimates signify the value obtained as a result of the combined inputs of all the indicators, no matter which particular subset of them is administered, and no matter which particular sample is measured.
The second point concerns the content of the indicators, which have been chosen because they are fairly easily measured in the physical values of distance, mass, volume, time, temperature, etc. A composite model and metrological unit system should also include a more actionable and meaningful definition of the measurand, one articulated as a developmental progression defined along a continuum ranging from most easily implemented to least easily implemented. This kind of integrated psychophysics is eminently suited to taking advantage of advanced measurement modeling (Camargo & Henson, 2013, 2015; Fisher, Melin, & Möller, 2021; Massof & McDonnell, 2012; Pendrill & Fisher, 2015; Powers & Fisher, 2018, 2022).
The practical application of the metrics and their comparability depends on obtaining usefully precise (a) theoretical predictions of indicator and project locations on the instrument (a construct map); (b) repeated demonstrations of reproducible and empirically stable data-based indicator and project location estimates (Wright maps); and (c) end user reports displaying indicator response values ordered along the mapped variable showing what has been accomplished, where the project stands in relation to its goals, what comes next in advancing its program, and where its actionable strengths and weaknesses lie.
Clear and significant progress in addressing the urgent needs for solutions to today’s pressing challenges cannot reasonably be expected until advanced measurement modeling in support of quality-assured metrological traceability is explicitly included in the design and implementation of Verra’s and others’ sustainable development standards. Hopefully the day will soon arrive when that will be the case.
References
Bateson, G. (1972). Steps to an ecology of mind: Collected essays in anthropology, psychiatry, evoltion, and epistemology. Chicago: University of Chicago Press.
Bond, T., & Fox, C. (2015). Applying the Rasch model: Fundamental measurement in the human sciences, 3d edition. New York: Routledge.
Camargo, F. R., & Henson, B. (2013). Aligning physical elements with persons’ attitude: An approach using Rasch measurement theory. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012009/pdf/1742-6596_459_1_012009.pdf.
Camargo, F. R., & Henson, B. (2015). Conceptualising computerized adaptive testing for measurement of latent variables associated with physical objects. Journal of Physics Conference Series, 588(1), 012012. doi:10.1088/1742-6596/588/1/012012
Cano, S., Pendrill, L., Melin, J., & Fisher, W. P., Jr. (2019). Towards consensus measurement standards for patient-centered outcomes. Measurement, 141, 62-69. Retrieved from https://doi.org/10.1016/j.measurement.2019.03.056
Fisher, W. P., Jr. (2011). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.
Fisher, W. P., Jr. (2012a). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.
Fisher, W. P., Jr. (2012b, June 1). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].
Fisher, W. P., Jr. (2020a). Contextualizing sustainable development metric standards: Imagining new entrepreneurial possibilities. Sustainability, 12(9661), 1-22. Retrieved from https://doi.org/10.3390/su12229661
Fisher, W. P., Jr. (2020b). Measuring genuine progress: An example from the UN Millennium Development Goals project. Journal of Applied Measurement, 21(1), 110-133
Fisher, W. P., Jr. (2021a). Bateson and Wright on number and quantity: How to not separate thinking from its relational context. Symmetry, 13(1415). Retrieved from https://doi.org/10.3390/sym13081415
Fisher, W. P., Jr. (2021b). Separation theorems in econometrics and psychometrics: Rasch, Frisch, two Fishers, and implications for measurement. Journal of Interdisciplinary Economics, OnlineFirst, 1-32. Retrieved from https://journals.sagepub.com/doi/10.1177/02601079211033475
Fisher, W. P., Jr. (2022). Contrasting roles of measurement knowledge systems in confounding or creating sustainable change. Acta IMEKO, in press.
Fisher, W. P., Jr., Melin, J., & Möller, C. (2021). Metrology for climate-neutral cities (RISE Research Institutes of Sweden AB No. RISE Report 2021:84). Gothenburg, Sweden:. RISE. Retrieved from http://ri.diva-portal.org/smash/record.jsf?pid=diva2%3A1616048&dswid=-7140
Fisher, W. P., Jr., Pendrill, L., Lips da Cruz, A., & Felin, A. (2019). Why metrology? Fair dealing and efficient markets for the United Nations’ Sustainable Development Goals. Journal of Physics: Conference Series, 1379(012023 [http://iopscience.iop.org/article/10.1088/1742-6596/1379/1/012023]). doi:10.1088/1742-6596/1379/1/012023
Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo: Revista de Investigacion Educacional Latinoamericana, 52(2), 55-78. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2688260
Fisher, W. P., Jr., & Wilson, M. (2019). The BEAR Assessment System Software as a platform for developing and applying UN SDG metrics. Journal of Physics Conference Series, 1379(012041). Retrieved from https://doi.org/10.1088/1742-6596/1379/1/012041
Fisher, W. P., Jr., & Wright, B. D. (Eds.). (1994). Applications of probabilistic conjoint measurement. International Journal of Educational Research, 21(6), 557-664.
Kaiser, F. G., & Wilson, M. (2000). Assessing people’s general ecological behavior: A cross-cultural measure. Journal of Applied Social Psychology, 30(5), 952-978.
Kaiser, F. G., & Wilson, M. (2004). Goal-directed conservation behavior: The specific composition of a general performance. Personality and Individual Differences, 36(7), 1531-1544. Retrieved from https://doi.org/10.1016/j.paid.2003.06.003
Luce, R. D., & Tukey, J. W. (1964). Simultaneous conjoint measurement: A new kind of fundamental measurement. Journal of Mathematical Psychology, 1(1), 1-27.
Madhala, T., & Fisher, W. P., Jr. (2022). Clothing, textile, and fashion industry sustainable impact measurement and management. Acta IMEKO, in press.
Mari, L., & Wilson, M. (2014, May). An introduction to the Rasch measurement approach for metrologists. Measurement, 51, 315-327. Retrieved from http://www.sciencedirect.com/science/article/pii/S0263224114000645
Mari, L., Wilson, M., & Maul, A. (2021). Measurement across the sciences (R. Morawski, G. Rossi, & others, Eds.). Springer Series in Measurement Science and Technology. Cham: Springer.
Massof, R. W., & McDonnell, P. J. (2012, April). Latent dry eye disease state variable. Investigative Ophthalmology & Visual Science, 53(4), 1905-1916. Retrieved from https://iovs.arvojournals.org/article.aspx?articleid=2188166
Moral, F. J., Álvarez, P., & Canito, J. L. (2006). Mapping and hazard assessment of atmospheric pollution in a medium sized urban area using the Rasch model and geostatistics techniques. Atmospheric Environment, 40(8), 1408-1418.
Moral, F. J., Rebollo, F. J., & Méndez, F. (2014). Using an objective model to estimate overall ozone levels at different urban locations. Stochastic Environmental Research and Risk Assessment, 28(3), 455-465.
Moral, F. J., Rebollo, F. J., Paniagua, L. L., García, A., & de Salazar, E. M. (2016). Application of climatic indices to analyse viticultural suitability in Extremadura, south-western Spain. Theoretical and Applied Climatology, 123(1-2), 277-289.
Narens, L., & Luce, R. D. (1986). Measurement: The theory of numerical assignments. Psychological Bulletin, 99(2), 166-180.
Pendrill, L. R. (2019). Quality assured measurement: Unification across social and physical sciences. Cham: Springer.
Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46-55. doi: http://dx.doi.org/10.1016/j.measurement.2015.04.010
Powers, M., & Fisher, W. P., Jr. (2018). Toward a standard for measuring functional binocular vision: Modeling visual symptoms and visual skills. Journal of Physics Conference Series, 1065(132009). doi:10.1088/1742-6596/1065/13/132009
Powers, M., & Fisher, W. P., Jr. (2021). Physical and psychological measures quantifying functional binocular vision. Measurement: Sensors, 18, 100320. Retrieved from https://doi.org/10.1016/j.measen.2021.100320
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.
Rasch, G. (1961). On general laws and the meaning of measurement in psychology. In J. Neyman (Ed.), Proceedings of the fourth Berkeley symposium on mathematical statistics and probability: Volume IV: Contributions to biology and problems of medicine (pp. 321-333 [http://www.rasch.org/memo1960.pdf]). Berkeley, California: University of California Press.
Whitehead, A. N. (1925). Science and the modern world. New York: Macmillan.
Wilson, M. R. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.
Wilson, M. R. (2013a, April). Seeking a balance between the statistical and scientific elements in psychometrics. Psychometrika, 78(2), 211-236.
Wilson, M. R. (2013b). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774. Retrieved from http://www.sciencedirect.com/science/article/pii/S0263224113001061
Wilson, M., & Fisher, W. P., Jr. (2016). Preface: 2016 IMEKO TC1-TC7-TC13 Joint Symposium: Metrology across the Sciences: Wishful Thinking? Journal of Physics Conference Series, 772(1), 011001. Retrieved from http://iopscience.iop.org/article/10.1088/1742-6596/772/1/011001/pdf
Wilson, M., & Fisher, W. P., Jr. (2019). Preface of special issue, Psychometric Metrology. Measurement, 145, 190. Retrieved from https://www.sciencedirect.com/journal/measurement/special-issue/10C49L3R8GT
Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14(2), 97-116 [http://www.rasch.org/memo42.htm].
Wright, B. D. (1997). A history of social science measurement. Educational Measurement: Issues and Practice, 16(4), 33-45, 52 [http://www.rasch.org/memo62.htm]. Retrieved from https://doi.org/10.1111/j.1745-3992.1997.tb00606.x
Livingcapitalmetrics's Blog 