Before the metrology of intangible assets can get off the ground, before we will be able to buy calibrated instruments for measuring abilities, attitudes, performances, and outcomes off the shelf like we do clocks, thermometers, and bathroom scales, we will need to develop trust in the dependability of our measures by building out persistent and reliable social network connections across different sectors (Alder, 2002; Andrade-Garcia, et al., 2020; Ashworth, 2004; Latour, 1987, 2005; O’Connell, 1993; Porter, 1995). Demand for the tools and unit standards will emerge and grow as awareness of the value obtained spreads, and as habitual usage leads to confidence in the processes and outcomes. As demand, awareness, and habits like these arise, researchers and practitioners will focus less on isolated and centralized data analyses, and more on collaborating in the creation and maintenance of shared quantity values (Fisher, et al., 1995; Fisher, 1997, 1998, 1999; Fisher & Stenner, 2016; Smith & Taylor, 2004; Stenner & Burdick, 1997; Stenner, et al., 2013, 2016; Stone, 2002; Stone & Stenner, 2018). Far from being a merely technical task, a number of social, emotional, aesthetic, ethical, political, economic, and other factors complicate matters considerably.
But to start from a plausible beginning, a better term than “networks” is what Latour (2005, p. 143) calls “worknets.” In the Rasch world, we can see these as ecosystems functioning at individual (kidmap), instrumentation and unit standard (Wright map), and predictive theory (construct map) levels of semiotic levels of concrete, abstract, and formal complexity (Fisher, 2018b, 2020; Fisher & Stenner, 2018a; Fisher & Wilson, 2015).
Different communities tend to focus their interests on just one of the different levels. though they implicitly make use of the others. In the natural sciences, the experimentalist, instrument-maker, and theoretician communities are more disconnected and disunified than one might expect, but in a complex way that makes them more effective and coherent than they could be if they were homogenously interconnected (Galison, 1997, pp. 46, 785-799, 843-844; 1999; Galison & Stump, 1996; Woolly & Fuchs, 2011).
Though these communities see and value the objects of their interests differently, they find enough common ground to coordinate and align their activities. The position of the objects of interest on the borders between the various scientific communities and others (accountants, lawyers, bankers, shareholders, customers, managers, etc.), as well, leads to them being referred to as boundary objects (Star & Griesemer, 1989, p. 392; also see Star, 2010; Star & Ruhleder, 1996; Bowker, 2015; Bowker, et al., 2014, 2015):
“Boundary objects are objects which are both plastic enough to adapt to local needs and the constraints of several parties employing them, yet robust enough to maintain a common identity across sites. They are weakly structured in common use, and become strongly structured in individual site use. These objects may be abstract or concrete. They have different meanings in different social worlds but their structure is common enough to more than one world to make them recognizable, a means of translation. The creation and management of boundary objects is a key process in developing and maintaining coherence across intersecting social worlds.”
We can expect the same kind of disunity across communities in education, health care, management, etc. Indeed, we are already quite familiar with it, given the theme of this thread concerned with whether it is realistic to try to train teachers as functional Rasch measurement experts. But in addition to this experience, we also have the array of tools we need to meet each community where they are:
- Practitioners like teachers, clinicians, and managers want kidmaps (Wright, et al., 1980; Wilson, 2004; Chien, et al., 2018) or their group-level equivalents, which provide the contextualized concrete data and formative guidance they need to know what to do next, to review the developmental, healing, or improvement trajectory, and to keep the goal in mind.
- Researchers and analysts focus on abstract standards of representation, and so want Wright maps (Wright & Masters, 1982; Wilson, 2011) and the technical demands of instrument and experimental designs, statistical comparisons, and invariance assessments.
- Theoreticians focus on formal conceptual structures, and so want explanatory models and construct maps (Stone, et al., 1999; Wilson, 2005) capable of demonstrating that the variable measured is the intended one, demonstrated by being able to write items to specifications and obtain the expected calibrations.
Putting all of this together effectively extends the thing-word-concept semiotic triangle of natural language into the data-instrument-theory semiotic triangle of science (Fisher, 2018b). This extension into contexts explicitly conceived and designed as multilevel ecosystems is well underway (Morrison & Fisher, 2018, 2019, 2020). This extension sets the stage for creating a new demand for communicable results reporting, where research papers and practical applications within a domain are meaningful, interpretable, and transferable for individuals over time, across individuals and instruments (collections of items), ways they currently are not.
This all undoubtedly sounds unrealistic to the point of impossible. The same was true of every advance in science. If matters were merely technical, analytic, and logical, the unrealistic demands to be made would truly put goals like these out of reach. In extending the semiotic triangle in new ways, however, this proposal has solid grounding in past successes, a grounding that includes thorough integration with the affective and bodily experiences of communication (Abram, 1996; Boedker & Chua, 2013), metaphoric process (Fisher, 1994, 2011, 2012), aesthetics (Fisher, 2018a, 2020; Fisher & Stenner, 2018b), ethics (Fisher, 2016), interpretation theory (Fisher, 2003a/b, 2004), econometrics (Fisher, 2010, 2019b), and the politics balancing powers across executive, legislative, and judicial branches of government (Fisher, 2009, 2019a; Nielsen, et al., 2007). We will also need to expand the number of levels of complexity we include in our models and results (Morrison & Fisher, 2020, slides 67-75; Scribner, et al., 2000), but that’s another story for another time.
References
Abram, D. (1996). The spell of the sensuous: Perception and language in a more-than-human world. New York: Vintage Books.
Alder, K. (2002). The measure of all things: The seven-year odyssey and hidden error that transformed the world. New York: The Free Press.
Andrade-Garda, J., Anguera, Á., Ares-Casal, J., García-Vázquez, R., Lara, J. A., Lizcano, D., .Rodriguez-Yánez, S., & Suárez-Garaboa, S. (2020). A metrology-based approach for measuring the social dimension of cognitive trust in collaborative networks. Cognition, Technology & Work, 22(2), 235-248.
Ashworth, W. J. (2004, November 19). Metrology and the state: Science, revenue, and commerce. Science, 306(5700), 1314-1317.
Boedker, C., & Chua, W. F. (2013). Accounting as an affective technology: A study of circulation, agency and entrancement. Accounting, Organizations and Society, 38, 245-267.
Bowker, G. C. (2015). Susan Leigh Star Special Issue. Mind, Culture, and Activity, 22(2), 89-91.
Bowker, G., Star, S. L., Gasser, L., & Turner, W. (Eds.). (2014). Social science, technical systems, and cooperative work: Beyond the great divide. Psychology Press.
Bowker, G., Timmermans, S., Clarke, A. E., & Balka, E. (Eds). (2015). Boundary objects and beyond: Working with Leigh Star. Cambridge, MA: MIT Press.
Fisher, W. P., Jr. (1994). Counting as metaphor. Rasch Measurement Transactions, 8(2), 358 [http://www.rasch.org/rmt/rmt82.htm].
Fisher, W. P., Jr. (1997). Physical disability construct convergence across instruments: Towards a universal metric. Journal of Outcome Measurement, 1(2), 87-113. Retrieved from http://jampress.org/JOM_V1N2.pdf
Fisher, W. P., Jr. (1998). A research program for accountable and patient-centered health status measures. Journal of Outcome Measurement, 2(3), 222-239. Retrieved from http://jampress.org/JOM_V2N3.pdf
Fisher, W. P., Jr. (1999). Foundations for health status metrology: The stability of MOS SF-36 PF-10 calibrations across samples. Journal of the Louisiana State Medical Society, 151(11), 566-578.
Fisher, W. P., Jr. (2003a). The mathematical metaphysics of measurement and metrology: Towards meaningful quantification in the human sciences. In A. Morales (Ed.), Renascent pragmatism: Studies in law and social science (pp. 118-153). Brookfield, VT: Ashgate Publishing Co.
Fisher, W. P., Jr. (2003b). Mathematics, measurement, metaphor, metaphysics: Parts I & II. Theory & Psychology, 13(6), 753-828.
Fisher, W. P., Jr. (2004). Meaning and method in the social sciences. Human Studies: A Journal for Philosophy and the Social Sciences, 27(4), 429-454.
Fisher, W. P., Jr. (2009, November 19). Draft legislation on development and adoption of an intangible assets metric system. Retrieved 12 September 2020, from Living Capital Metrics blog: https://livingcapitalmetrics.wordpress.com/2009/11/19/draft-legislation/
Fisher, W. P., Jr. (2011). Metaphor as measurement, and vice versa: Convergence and separation of figure and meaning in a Mawri proverb [Modified version of a paper presented to the African Studies Association, 1996]. Social Science Research Network. Retrieved 7 April 2014, from http://ssrn.com/abstract=1747967
Fisher, W. P., Jr. (2012, July 12). Metaphor as measurement and vice versa: Love is a rose. In A. Maul (Chair), Metaphors and measurement: An invited symposium on validity. International Meeting of the Psychometric Society, Lincoln, Nebraska. Retrieved from http://www.slideshare.net/wpfisherjr/fisher-imps2012c-metaphor
Fisher, W. P., Jr. (2016, December 29). Living Capital Metrics Blog Post: Measuring values to apply the Golden Rule. Retrieved from https://livingcapitalmetrics.wordpress.com/2016/12/29/measuring-values-to-apply-the-golden-rule/
Fisher, W. P., Jr. (2018a, October 30). Beauty, meaning, and measurement. BEAR Seminar. BEAR Center, Graduate School of Education, University of California, Berkeley. http://bearcenter.berkeley.edu/seminar/beauty-meaning-and-measurement
Fisher, W. P., Jr. (2018b, January 30). Discontinuous levels of complexity in coherent educational measurement: The roles of kidmaps, Wright maps, and construct maps. BEAR Seminar. BEAR Center, Graduate School of Education, University of California, Berkeley. http://bearcenter.berkeley.edu/seminar/discontinuous-levels-complexity-coherent-educational-measurement-roles-kidmaps-wright-maps
Fisher, W. P., Jr. (2019a, July 12). New work in progress: Commentary on Commons and Goodheart (2008). Retrieved 12 September 2020, from Living Capital Metrics blog: https://livingcapitalmetrics.wordpress.com/2019/07/12/new-work-in-progress/.
Fisher, W. P., Jr. (2019b). Separation theorems in econometrics and psychometrics: Rasch, Frisch, two Fishers, and implications for measurement (in revision).
Fisher, W. P., Jr. (2020, August 12). New goals for managing what we measure [Slides as presented: https://www.dropbox.com/s/h6a7lv6gvpmz0n4/Fisher2020NewGoals3.pdf?dl=0%5D (I. Rothmann, Ed.). Optentia Research Focus Area, North-West University, Vanderbijlpark, South Africa. https://vimeo.com/447502427.
Fisher, W. P., Jr., Harvey, R. F., Taylor, P., Kilgore, K. M., & Kelly, C. K. (1995). Rehabits: A common language of functional assessment. Archives of Physical Medicine and Rehabilitation, 76(2), 113-122.
Fisher, W. P., Jr., & Stenner, A. J. (2016). Theory-based metrological traceability in education: A reading measurement network. Measurement, 92, 489-496. Retrieved from http://www.sciencedirect.com/science/article/pii/S0263224116303281
Fisher, W. P., Jr., & Stenner, A. J. (2018a). Ecologizing vs modernizing in measurement and metrology. Journal of Physics Conference Series, 1044(012025), [http://iopscience.iop.org/article/10.1088/1742-6596/1044/1/012025].
Fisher, W. P., Jr., & Stenner, A. J. (2018b). On the complex geometry of individuality and growth: Cook’s 1914 ‘Curves of Life’ and reading measurement. Journal of Physics Conference Series, 1065, 072040 [https://iopscience.iop.org/article/10.1088/1742-6596/1065/7/072040/pdf].
Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.
Galison, P. (1999). Trading zone: Coordinating action and belief. In M. Biagioli (Ed.), The science studies reader (pp. 137-160). New York: Routledge.
Galison, P., & Stump, D. J. (1996). The disunity of science: Boundaries, contexts, and power. Palo Alto, California: Stanford University Press.
Latour, B. (1987). Science in action: How to follow scientists and engineers through society. New York: Harvard University Press.
Latour, B. (2005). Reassembling the social: An introduction to Actor-Network-Theory. (Clarendon Lectures in Management Studies). Oxford, England: Oxford University Press.
Morrison, J., & Fisher, W. P., Jr. (2018). Connecting learning opportunities in STEM education: Ecosystem collaborations across schools, museums, libraries, employers, and communities. Journal of Physics: Conference Series, 1065(022009). doi:10.1088/1742-6596/1065/2/022009
Morrison, J., & Fisher, W. P., Jr. (2019). Measuring for management in Science, Technology, Engineering, and Mathematics learning ecosystems. Journal of Physics: Conference Series, 1379(012042). doi:10.1088/1742-6596/1379/1/012042
Morrison, J., & Fisher, W. P., Jr. (2020, September 1). The Measure STEM Caliper Development Initiative [Online]. BEAR Seminar Series. BEAR Center, Graduate School of Education: University of California, Berkeley. http://bearcenter.berkeley.edu/seminar/measure-stem-caliper-development-initiative-online
Nielsen, T., Kreiner, S., & Styles, I. (2007). Mental self-government: Development of the additional democratic learning style scale using Rasch measurement models. Journal of Applied Measurement, 8(2), 124-148.
O’Connell, J. (1993). Metrology: The creation of universality by the circulation of particulars. Social Studies of Science, 23, 129-173.
Scribner, R. A., Cohen, D. A., & Fisher, W. P. (2000). Evidence of a structural effect for alcohol outlet density: A multilevel analysis. Alcoholism: Clinical & Experimental Research, 24(2), 188-195.
Smith, R. M., & Taylor, P. (2004). Equating rehabilitation outcome scales: Developing common metrics. Journal of Applied Measurement, 5(3), 229-242.
Star, S. L. (2010). This is not a boundary object: Reflections on the origin of a concept. Science, Technology, & Human Values, 35(5), 601-617.
Star, S. L., & Griesemer, J. R. (1989, August). Institutional ecology, ‘translations,’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907-39. Social Studies of Science, 19(3), 387-420.
Star, S. L., & Ruhleder, K. (1996, March). Steps toward an ecology of infrastructure: Design and access for large information spaces. Information Systems Research, 7(1), 111-134.
Stenner, A. J. (2001). The Lexile Framework: A common metric for matching readers and texts. California School Library Journal, 25(1), 41-2.
Stenner, A. J., & Burdick, D. S. (1997, January 3). The objective measurement of reading comprehension. http://files.eric.ed.gov/fulltext/ED435978.pdf (visited 18 April 2014) (Ed.), [Response to technical questions raised by the California Department of Education Technical Study Group], Durham, North Carolina: MetaMetrics, Inc.
Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2013, August). Causal Rasch models. Frontiers in Psychology: Quantitative Psychology and Measurement, 4(536), 1-14 [doi: 10.3389/fpsyg.2013.00536].
Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2016). Causal Rasch models in language testing: An application rich primer. In Q. Zhang (Ed.), Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings (pp. 1-14). Singapore: Springer.
Stenner, A. J., & Stone, M. (2010). Generally objective measurement of human temperature and reading ability: Some corollaries. Journal of Applied Measurement, 11(3), 244-252.
Stenner, A. J., Stone, M. H., & Fisher, W. P., Jr. (2018). The unreasonable effectiveness of theory based instrument calibration in the natural sciences: What can the social sciences learn? Journal of Physics Conference Series, 1044(012070), http://iopscience.iop.org/article/10.1088/1742-6596/1044/1/012070.
Stone, M. H. (2002). Knox’s cube test – revised. Wood Dale: Stoelting.
Stone, M. H., & Stenner, J. (2018). Substantive theory and constructive measures: A collection of chapters and measurement commentary on causal science. Bloomington, IN: IUniverse, Inc.
Stone, M. H., Wright, B., & Stenner, A. J. (1999). Mapping variables. Journal of Outcome Measurement, 3(4), 308-322. Retrieved from http://jampress.org/JOM_V3N4.pdf
Woolley, A. W., & Fuchs, E. (2011, October 1). Collective intelligence in the organization of science. Organization Science, 22(5), 1359-1367.
Livingcapitalmetrics's Blog 