Posts Tagged ‘metrology’

Feminist Diffractions, Stochastic Resonance, and Education, Revisited

May 25, 2015

Lehrer (2015) offers an insightful commentary on Saxe et al’s (2015) recent article in Human Development that prompts some observations.

Two areas for questions and comments come to mind. The first has to do with construing the development and revision of new ways of understanding as contested, which implicitly aligns with Latour’s (1987, pp. 89, 93) sense of the way new constructs are subjected to tests of strength. Haraway (1996) makes an important point in her critique of what she sees as the overly masculinist metaphors of heroic competition and (perhaps not so) sublimated violence in these contests. Her sense of “feminist diffractions” stops short of what I have in mind, but opens the door to an alternative approach to what Lehrer calls the “close coupling of definitions with the development and revision of new concepts and ways of understanding.”

Galison (1997, pp. 843-844), for instance, seeks a metaphor capable of expressing what happens in the conceptual, practical, and argumentative contests between different communities of scientists (instrumentalist technicians, theoreticians, and experimentalists). He wants a metaphor that does justice to the disunified chaos and disorder one finds in the relationships between these different groups, which paradoxically results in such productive and coherent innovations. He recalls Peirce’s and Wittgenstein’s metaphors of cables and threads that take their strength from being intertwined from smaller wires and bits of fiber but finds these images too mechanical for his purposes. He wants something more akin to amorphous semiconductors or laminated materials that can fail microscopically but hold macroscopically better than more structurally homogenous materials.

Berg and Timmermans (2000, pp. 55-56) make a similar observation in their study of the constitution of universalities in medical fields:

“In order for a statistical logistics to enhance precise decision making, it has to incorporate imprecision; in order to be universal, it has to carefully select its locales. … Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.”

The general problem is taken up by Ricoeur (1992, p. 289), who raises the notion of “universals in context or of potential or inchoate universals” that embody the paradox in which

“on the one hand, one must maintain the universal claim attached to a few values where the universal and the historical intersect, and on the other hand, one must submit this claim to discussion, not on a formal level, but on the level of the convictions incorporated in concrete forms of life.”

To repeat another theme that comes up again and again in this blog, this kind of noise-induced order sounds like the phenomenon of stochastic resonance (Fisher, 1992, 2011). The importance of stochastic resonance is that it opens up a way to connect the phenomena of emergent understanding with measurement, both at the local individual and general systemic levels.

This is the crux of some very important issues in the philosophy of science and in philosophy generally. Haraway (1996, pp. 439-440), for instance, points out that “embedded relationality is the prophylaxis for both relativism and transcendence.” And Golinski (2012, p. 35) similarly says, “Practices of translation, replication, and metrology have taken the place of the universality that used to be assumed as an attribute of singular science.”

A start in the direction of embedded relationality, translation, replication, and metrology in education is apparent, for instance, in work that enables teachers to usefully relate individual student performances to general learning progressions, connecting instructional applications with accountability (Fisher & Wilson, 2015; Lehrer, 2013; Lehrer & Jones, 2014; Wilson, 2004). As Lehrer (2015, p. 49) says about the Saxe et al. work, “Recurrent forms of mathematical practice enabled the authors to create compelling trajectories of collective activity and learning over time while preserving the contributions of individual development.”

The second of the two topics I’d like to address comes up here in the closing paragraph of his short commentary, where Lehrer says a “hoped-for future innovation would make it possible to visualize individual and collective trajectories simultaneously.” Though future improvements can certainlty be expected, visualizations of individual and collective trajectories for growth in reading are already being recognized in both educational and metrological contexts (Stenner, Swartz, Hanlon, & Emerson, 2012; Stenner & Fisher, 2013, p. 4) for their potential to serve as the media of an embedded relationality capable of undercutting both the relativism of uncontrolled local variation and the universalist pretensions often built into accountability programs.

With emerging recognition of the potential Rasch’s stochastic approaches to construct mapping (Bond & Fox, 2007; Wilson, 2005) offer in the way of metrological translation networks (Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2015; Fisher & Wilson, 2015; Stenner & Fisher, 2013; Wilson, 2013; Wilson, Mari, Maul, & Torres Irribarra 2015), there are good reasons to expect significant new kinds of progress in fields that rely on assessments and surveys for outcome measurement and management.

References

Berg, M.,& Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Fisher, W. P., Jr. (1992). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [http://www.rasch.org/rmt/rmt54k.htm].

Fisher, W. P., Jr. (2011). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Fisher, W. P., Jr., & Stenner, A. J. (2015). The role of metrology in mobilizing and mediating the language and culture of scientific facts. Journal of Physics Conference Series, 588(012043).

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo, in review.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Haraway, D. J. (1996). Modest witness: Feminist diffractions in science studies. In P. Galison & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts, and power (pp. 428-441). Stanford, California: Stanford University Press.

Latour, B. (1987). Science in action: How to follow scientists and engineers through society. New York: Harvard University Press.

Lehrer, R. (2013, April 29). (Chair). In A learning progression emerges in a trading zone of professional community and identity. American Educational Research Association, Division C on Learning and Instruction, Section 2b on Learning and Motivation in Social and Cultural Contexts, San Francisco, CA.

Lehrer, R., & Jones, S. (2014, 2 April). Construct maps as boundary objects in the trading zone. In W. P. Fisher Jr. (Chair), Session 3-A: Rating Scales and Partial Credit, Theory and Applied. International Objective Measurement Workshop, Philadelphia, PA.

Lehrer, R. (2015). Designing for development: Commentary on Saxe, de Kirby, Kang, Le and Schneider. Human Development, 58(1), 45-49.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012002/pdf/1742-6596_459_1_012002.pdf.

Pendrill, L. (2014). Man as a measurement instrument [Special Feature]. NCSLi Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46-55.

Ricoeur, P. (1992). Oneself as another. Chicago, Illinois: University of Chicago Press.

Saxe, G. B., de Kirby, K., Kang, B., Le, M., & Schneider, A. (2015). Studying cognition through time in a classroom community: The interplay between “everyday” and “scientific” concepts. Human Development, 58(1), 5-44.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459(012025), http://iopscience.iop.org/1742-6596/459/1/012025.

Stenner, A. J., Swartz, C., Hanlon, S., & Emerson, C. (2012, February). Personalized learning platforms. Presented at the Pearson Global Research Conference, Fremantle, Western Australia.

Wilson, M. (Ed.). (2004). National Society for the Study of Education Yearbooks. Vol. 103, Part II: Towards coherence between classroom assessment and accountability. Chicago, Illinois: University of Chicago Press.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Wilson, M. R. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.

 

Moore’s Law at 50

May 13, 2015

Thomas Friedman interviewed Gordon Moore on the occasion of the 50th anniversary of Moore’s 1965 article predicting that computing power would exponentially increase at little additional cost. Moore’s ten-year prediction for the doubling rate of the numbers of transistors on microchips held up, and has now, with small adjustments, guided investments and expectations in electronics for five decades.

Friedman makes an especially important point, saying:

But let’s remember that it [Moore’s Law] was enabled by a group of remarkable scientists and engineers, in an America that did not just brag about being exceptional, but invested in the infrastructure and basic scientific research, and set the audacious goals, to make it so. If we want to create more Moore’s Law-like technologies, we need to invest in the building blocks that produced that America.”

These kinds of calls for investments in infrastructure and basic research, for new audacious goals, and for more Moore’s Law-like technologies are, of course, some of the primary and recurring themes of this blog (here, here, here, and here) and presentations and publications of the last several years. For instance, Miller and O’Leary’s (2007) close study of how Moore’s Law has aligned and coordinated investments in the electronics industry has been extrapolated into the education context (Fisher, 2012; Fisher & Stenner, 2011).

Education already has had over 60 years experience with a close parallel to Moore’s Law in reading measurement. Stenner’s Law retrospectively predicts exactly the same doubling period for the increasing numbers from 1960 to 2010 of children’s reading abilities measured in a common (or equatable) unit with known uncertainty and personalized consistency indicators. Knowledge of this kind has enabled manufacturers, suppliers, marketers, customers, and other stakeholders in the electronics industry to plan five and ten years into the future, preparing products and markets to take advantage of increased power and speed at the same or lower cost. Similarly, that same kind of knowledge could be used in education, health care, social services, and natural resource management to define the rules, roles, and responsibilities of actors and institutions involved in literacy, health, community, and natural capital markets.

Reading instruction, for example, requires text complexities to be matched to reader abilities at a comprehension rate that challenges but does not discourage the reader. Uniform grade-level textbooks are often too easy for a third of a given classroom, and too hard for another third. Individualized instruction by teachers in classrooms of 25 and more students is too cumbersome to implement. Connecting classroom reading assessments with known text complexity measures informed by judicious teacher input sets the stage for the realization of new potentials in educational outcomes. Electronic resources tapping existing text complexity measures for millions of articles and books connect individual students’ high stakes and classroom assessments in a common instructional framework (for instance, see here for an offering from Pearson). As the numbers of student reading measures made in a common unit continues to grow exponentially, capacities for connecting readers to texts, and for communicating about what works and what doesn’t in education, will grow as well.

This model is exactly the kind of infrastructure, basic scientific research, and audacious goal setting that’s needed if we are to succeed in creating more Moore’s Law-like technologies. If we as a society made the decision to invest deliberately, intentionally, and massively in infrastructure of this kind across education, health care, social services, and natural resource management, who knows what kinds of powerful results might be attained?

References

Fisher, W. P., Jr. (2012). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr., & Stenner, A. J. (2011, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium, http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf, Jena, Germany.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Living Capital Metrics for Financial and Sustainability Accounting Standards

May 1, 2015

I was very happy a few days ago to come across Jane Gleeson-White’s new book, Six Capitals, or Can Accountants Save the Planet? Rethinking Capitalism for the 21st Century. The special value for me in this book comes in the form of an accessible update on what’s been going on in the world of financial accounting standards. Happily, there’s been a lot of activity (check out, for instance, Amato & White, 2013; Rogers & White, 2015). Less fortunately, the activity seems to be continuing to occur in the same measurement vacuum it always has, despite my efforts in this blog to broaden the conversation to include rigorous measurement theory and practice.

But to back up a bit, recent events around sustainability metric standards don’t seem to be connected to previous controversies around financial standards and economic modeling, which were more academically oriented to problems of defining and expressing value. Gleeson-White doesn’t cite any of the extensive literature in those areas (for instance, Anielski, 2007; Baxter, 1979; Economist, 2010; Ekins, 1992, 1999; Ekins, Dresner, & Dahlstrom, 2008; Ekins, Hillman, & Hutchins, 1992; Ekins & Voituriez, 2009; Fisher, 2009b, 2009c, 2011; Young & Williams, 2010). Valuation is still a problem, of course, as is the analogy between accounting standards and scientific standards (Baxter, 1979). But much of the sensitivity of the older academic debate over accounting standards seems to have been lost in the mad, though well-intentioned, rush to devise metrics for the traditionally externalized nontraditional forms of capital.

Before addressing the thousands of metrics in circulation and the science that needs to be brought to bear on them (the ongoing theme of posts in this blog), some attention to terminology is important. Gleeson-White refers to six capitals (manufactured, liquid, intellectual, human, social, and natural), in contrast with Ekins (1992; Ekins, et al., 2008), who describes four (manufactured, human, social, and natural). Gleeson-White’s liquid capital is cash money, which can be invested in capital (a means of producing value via ongoing services) and which can be extracted as a return on capital, but is not itself capital, as is shown by the repeated historical experience in many countries of printing money without stimulating economic growth and producing value. Of her remaining five forms of capital, intellectual capital is a form of social capital that can satisfactorily be categorized alongside the other forms of organization-level properties and systems involving credibility and trust.

On pages 209-227, Gleeson-White takes up questions relevant to the measurement and information quality topics of this blog. The context here is informed by the International Integrated Reporting Council’s (IIRC) December 2013 framework for accounting reports integrating all forms of capital (Amato & White, 2013), and by related efforts of the Sustainability Accounting Standards Board (SASB) (Rogers & White, 2015). Following the IIRC, Gleeson-White asserts that

“Not all the new capitals can be quantified, yet or perhaps ever–for example, intellectual, human and social capital, much of natural capital–and so integrated reports are not expected to provide quantitative measures of each of the capitals.”

Of course, this opinion flies in the face of established evidence and theory accepted by both metrologists (weights and measures standards engineers and physicists) and psychometricians as to the viability of rigorous measurement standards for the outcomes of education, health care, social services, natural resource management, etc. (Fisher, 2009b, 2011, 2012a, 2012b; Fisher & Stenner, 2011a, 2013, 2015; Fisher & Wilson, 2015; Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2013, 2015; Wilson, 2013; Wilson, Mari, Maul, & Torres Irribarra, 2015). Pendrill (2014, p. 26), an engineer, physicist, and past president of the European Association of National Metrology Institutes, for instance, states that “The Rasch approach…is not simply a mathematical or statistical approach, but instead [is] a specifically metrological approach to human-based measurement.” As is repeatedly shown in this blog, access to scientific measures sets the stage for a dramatic transformation of the potential for succeeding in the goal of rethinking capitalism.

Next, Gleeson-White’s references to several of the six capitals as the “living” capitals (p. 193) is a literal reference to the fact that human, social, and natural capital are all carried by people, organizations/communities, and ecosystems. The distinction between dead and living capital elaborated by De Soto (2000) and Fisher (2002, 2007, 2010b, 2011), which involves making any form of capital fungible by representing it in abstract forms negotiable in banks and courts of law, is not taken into account, though this would seem to be a basic requirement that must be fulfilled before the rethinking of capitalism could said to have been accomplished.

Gleeson-White raises the pointed question as to exactly how integrated reporting is supposed to provoke positive growth in the nontraditional forms of capital. The concept of an economic framework integrating all forms of capital relative to the profit motive, as described in Ekins’ work, for instance, and as is elaborated elsewhere in this blog, seems just over the horizon, though repeated mention is made of natural capitalism (Hawken, Lovins, & Lovins, 1999). The posing of the questions provided by Gleeson-White (pp. 216-217) is priceless, however:

“…given integrated reporting’s purported promise to contribute to sustainable development by encouraging more efficient resource allocation, how might it actually achieve this for natural and social capitals on their own terms? It seems integrated reporting does nothing to address a larger question of resource allocation….”

“To me the fact that integrated reporting cannot address such questions suggests that as with the example of human capital, its promise to foster efficient resource allocation pertains only to financial capital and not to the other capitals. If we accept that the only way to save our societies and planet is to reconceive them in terms of capital, surely the efficient valuing and allocation of all six capitals must lie at the heart of any economics and accounting for the planet’s scarce resources in the twenty-first century.
“There is a logical inconsistency here: integrated reporting might be the beginning of a new accounting paradigm, but for the moment it is being practiced by an old-paradigm corporation: essentially, one obliged to make a return on financial capital at the cost of the other capitals.”

The goal requires all forms of capital to be integrated into the financial bottom line. Where accounting for manufactured capital alone burns living capital resources for profit, a comprehensive capital accounting framework defines profit in terms of reduced waste. This is a powerful basis for economics, as waste is the common root cause of human suffering, social discontent and environmental degradation (Hawken, Lovins, & Lovins, 1999).

Multiple bottom lines are counter-productive, as they allow managers the option of choosing which stakeholder group to satisfy, often at the expense of the financial viability of the firm (Jensen, 2001; Fisher, 2010a). Economic sustainability requires that profits be legally, morally, and scientifically contingent on a balance of powers distributed across all forms of capital. Though the devil will no doubt lurk in the details, there is increasing evidence that such a balance of powers can be negotiated.

A key point here not brought up by Gleeson-White concerns the fact that markets are not created by exchange activity, but rather by institutionalized rules, roles, and responsibilities (Miller & O’Leary, 2007) codified in laws, mores, technologies, and expectations. Translating historical market-making activities as they have played out relative to manufactured capital in the new domains of human, social, and natural capital faces a number of significant challenges, adapting to a new way of thinking about tests, assessments, and surveys foremost among them (Fisher & Stenner, 2011b).

One of the most important contributions advanced measurement theory and practice (Rasch, 1960; Wright, 1977; Andrich, 1988, 2004; Fisher & Wright, 1994; Wright & Stone, 1999; Bond & Fox, 2007; Wilson, 2005; Engelhard, 2012; Stenner, Fisher, Stone, & Burdick, 2013) can make to the process of rethinking capitalism involves the sorting out of the myriad metrics that have erupted in the last several years. Gleeson-White (p. 223) reports, for instance, that the Bloomberg financial information network now has over 750 ESG (Environmental, Social, Governance) data fields, which were extracted from reports provided by over 5,000 companies in 52 countries.  Similarly, Rogers and White (2015) say that

“…today there are more than 100 organizations offering more than 400 corporate sustainability ratings products that assess some 50,000 companies on more than 8,000 metrics of environmental, social and governance (ESG) performance.”

As is also the case with the UN Millennium Development Goals (Fisher, 2011b), the typical use of these metrics as single-item “quantities” is based in counts of relevant events. This procedure misses the basic point that counts of concrete things in the world are not measures. Is it not obvious that I can have ten rocks to your two, and you can still have more rock than I do? The same thing applies to any kind of performance ratings, survey responses, or test scores. We assign the same numeric increase to every addition of one more count, but hardly anyone experimentally tests the hypothesis that the counts all work together to measure the same thing. Those who think there’s no need for precision science in this context are ignoring the decades of successful and widespread technical work in this area, at their own risk.

The repetition of history here is fascinating. As Ashworth (2004, p. 1,314) put it, historically, “The requirements of increased trade and the fiscal demands of the state fuelled the march toward a regular form of metrology.” For instance, in 1875 it was noted that “the existence of quantitative correlations between the various forms of energy, imposes upon men of science the duty of bringing all kinds of physical quantity to one common scale of comparison” (Everett, 1875, p. 9). The moral and economic  value of common scales was recognized during the French revolution, when, Alder (2002, p. 32) documents, it was asked:

“Ought not a single nation have a uniform set of measures, just as a soldier fought for a single patrie? Had not the Revolution promised equality and fraternity, not just for France, but for all the people of the world? By the same token, should not all of the world’s people use a single set of weights and measures to encourage peaceable commerce, mutual understanding, and the exchange of knowledge? That was the purpose of measuring the world.”

The value of rigorously measuring human, social and natural capital includes meaningfully integrating qualitative substance with quantitative convenience, reduced data volume, augmenting measures with uncertainty and consistency indexes, and the capacity to take missing data into account (making possible instrument equating, item banking, etc.)  In contrast with the usual methods, rigorous science demands that experiments determine which indicators cohere to measure the same thing by repeatedly giving the same values across samples, over time and space, and across subsets of indicators. Beyond such data-based results, advanced theory makes it possible to arrive at explanatory, predictive methods that add a whole new layer of efficiency to the generation of indicators (de Boeck & Wilson, 2004; Stenner, et al., 2013).

Finally, Gleeson-White (pp. 220-221) reports that “In July 2011, the SASB [Sustainability Accounting Standards Board] was launched in the United States to create standardized measures for the new capitals.” “Founded by environmental engineer and sustainability expert Jean Rogers in San Francisco, SASB is creating a full set of industry-specific standards for sustainability accounting, with the aim of making this information more consistent and comparable.” As of May 2014, the SASB vice chair is Mary Schapiro, former SEC chair, and the chairman of SASB is Michael Bloomfield, former mayor of NYC and founder of the financial information empire. The “SASB is developing nonfinancial standards for eighty-nine industries grouped in ten different sectors and aims to have completed this grueling task by February 2015. It is releasing each set of metrics as they are completed.”

Like the SASB and other groups, Gleeson-White (p. 222) reports, Bloomberg

“aims to use its metrics to start ‘standardizing the discourse around sustainability, so we’re all talking about the same things in the same way,’ as Bloomberg’s senior sustainability strategist Andrew Park put it. What companies ‘desperately want,’ he says, is ‘a legitimate voice’ to tell them: ‘This is what you need to do. You exist in this particular sector. Here are the metrics that you need to be reporting out on. So SASB will provide that. And we think that’s important, because that will help clean up the metrics that ultimately the finance community will start using.’
“Bloomberg wants to price environmental, social and governance externalities to legitimize them in the eyes of financial capital.”

Gleeson-White (p. 225) continues, saying

“Bloomberg wants to do more generally what Trucost did for Puma’s natural capital inputs: create standardized measures for the new capitals–such as ecosystem services and social impacts–so that this information can be aggregated and used by investors. Park and Ravenel call the failure to value clean air, water, stable coastlines and other environmental goods ‘as much a failure to measure as it is a market failure per se–one that could be addressed in part by providing these ‘unpriced’ resources with quantitative parameters that would enable their incorporation into market mechanisms. Such mechanisms could then appropriately ‘regulate’ the consumption of those resources.'”

Integrating well-measured living capitals into the context of appropriately configured institutional rules, roles, and responsibilities for efficient markets (Fisher, 2010b) should indeed involve a capacity to price these resources quantitatively, though this capacity alone would likely prove insufficient to the task of creating the markets (Miller & O’Leary, 2007; Williamson, 1981, 1991, 2005). Rasch’s (1960, pp. 110-115) deliberate patterning of his measurement models on the form of Maxwell’s equations for Newton’s Second Law provides a mathematical basis for connecting psychometrics with both geometry and natural laws, as well as with the law of supply and demand (Fisher, 2010c, 2015; Fisher & Stenner, 2013a).

This perspective on measurement is informed by an unmodern or amodern, post-positivist philosophy (Dewey, 2012; Latour, 1990, 1993), as opposed to a modern and positivist, or postmodern and anti-positivist, philosophy (Galison, 1997). The essential difference is that neither a universalist nor a relativist perspective is necessary to the adoption of practices of traceability to metrological standards. Rather, focusing on local, situated, human relationships, as described by Wilson (2004) in education, for instance, offers a way of resolving the false dilemma of that dichotomous contrast. As Golinski (2012, p. 35) puts it, “Practices of translation, replication, and metrology have taken the place of the universality that used to be assumed as an attribute of singular science.” Haraway (1996, pp. 439-440) harmonizes, saying “…embedded relationality is the prophylaxis for both relativism and transcendance.” Latour (2005, pp. 228-229) elaborates, saying:

“Standards and metrology solve practically the question of relativity that seems to intimidate so many people: Can we obtain some sort of universal agreement? Of course we can! Provided you find a way to hook up your local instrument to one of the many metrological chains whose material network can be fully described, and whose cost can be fully determined. Provided there is also no interruption, no break, no gap, and no uncertainty along any point of the transmission. Indeed, traceability is precisely what the whole of metrology is about! No discontinuity allowed, which is just what ANT [Actor Network Theory] needs for tracing social topography. Ours is the social theory that has taken metrology as the paramount example of what it is to expand locally everywhere, all while bypassing the local as well as the universal. The practical conditions for the expansion of universality have been opened to empirical inquiries. It’s not by accident that so much work has been done by historians of science into the situated and material extension of universals. Given how much modernizers have invested into universality, this is no small feat.
“As soon as you take the example of scientific metrology and standardization as your benchmark to follow the circulation of universals, you can do the same operation for other less traceable, less materialized circulations: most coordination among agents is achieved through the dissemination of quasi-standards.”

As Rasch (1980: xx) understood, “this is a huge challenge, but once the problem has been formulated it does seem possible to meet it.” Though some metrologically informed traceability networks have begun to emerge in education and health care (for instance, Fisher & Stenner, 2013, 2015; Stenner & Fisher, 2013), virtually everything remains to be done to make the coordination across stakeholders as fully elaborated as the standards in the natural sciences.

References

Alder, K. (2002). The measure of all things: The seven-year odyssey and hidden error that transformed the world. New York: The Free Press.

Amato, N., & White, S. (2013, December 7). IIRC releases International Integrated Reporting Framework. Journal of Accountancy. Retrieved from http://www.journalofaccountancy.com/news/2013/dec/20139207.html

Andrich, D. (1988). Sage University Paper Series on Quantitative Applications in the Social Sciences. Vol. series no. 07-068: Rasch models for measurement. Beverly Hills, California: Sage Publications.

Andrich, D. (2004, January). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1), I-7–I-16.

Andrich, D. (2010). Sufficiency and conditional estimation of person parameters in the polytomous Rasch model. Psychometrika, 75(2), 292-308.

Anielski, M. (2007). The economics of happiness: Building genuine wealth. Gabriola, British Columbia: New Society Publishers.

Ashworth, W. J. (2004, 19 November). Metrology and the state: Science, revenue, and commerce. Science, 306(5700), 1314-1317.

Baxter, W. T. (1979). Accounting standards: Boon or curse? In The Emmanuel Saxe distinguished lectures in accounting. http://newman.baruch.cuny.edu/digital/saxe/saxe_1978/baxter_79.htm.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. Statistics for Social and Behavioral Sciences). New York: Springer-Verlag.

De Soto, H. (2000). The mystery of capital: Why capitalism triumphs in the West and fails everywhere else. New York: Basic Books.

Dewey, J. (2012). Unmodern philosophy and modern philosophy (P. Deen, Ed.). Carbondale, Illinois: Southern Illinois University Press.

Editorial. (2010, 10 June). Accounting standards: To FASB or not to FASB? The Economist, http://www.economist.com/node/16319655.

Ekins, P. (1992). A four-capital model of wealth creation. In P. Ekins & M. Max-Neef (Eds.), Real-life economics: Understanding wealth creation (pp. 147-155). London: Routledge.

Ekins, P. (1999). Economic growth and environmental sustainability: The prospects for green growth. New York: Routledge.

Ekins, P., Dresner, S., & Dahlstrom, K. (2008, March/April). The four-capital method of sustainable development evaluation. European Environment, 18(2), 63-80.

Ekins, P., Hillman, M., & Hutchison, R. (1992). The Gaia atlas of green economics (Foreword by Robert Heilbroner). New York: Anchor Books.

Ekins, P., & Voituriez, T. (2009). Trade, globalization and sustainability impact assessment: A critical look at methods and outcomes. London, England: Earthscan Publications Ltd.

Engelhard, G., Jr. (2012). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. New York: Routledge Academic.

Everett, J. D. (1875). Illustrations of the C. G. S. system of units. London, England: Taylor & Francis.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [http://www.rasch.org/rmt/rmt154j.htm].

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009a, November 19). Draft legislation on development and adoption of an intangible assets metric system. Retrieved 6 January 2011, from https://livingcapitalmetrics.wordpress.com/2009/11/19/draft-legislation/

Fisher, W. P., Jr. (2009b, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009c). NIST Critical national need idea White Paper: metrological infrastructure for human, social, and natural capital (Tech. Rep. No. http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC:. National Institute for Standards and Technology.

Fisher, W. P., Jr. (2010a, 22 November). Meaningfulness, measurement, value seeking, and the corporate objective function: An introduction to new possibilities., LivingCapitalMetrics.com, Sausalito, California. Retrieved from http://ssrn.com/abstract=1713467

Fisher, W. P., Jr. (2010b). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital, Bridge to Business Postdoctoral Certification, Freeman School of Business, Tulane University (http://ssrn.com/abstract=2340674).

Fisher, W. P., Jr. (2010c). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics: Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2011b). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com. Retrieved 18 January 2011, from Social Science Research Network: http://ssrn.com/abstract=1739386.

Fisher, W. P., Jr. (2012a). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr. (2012b, May/June). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].

Fisher, W. P., Jr. (2015). A Rasch perspective on the law of supply and demand. Rasch Measurement Transactions, in press.

Fisher, W. P., Jr., Harvey, R. F., & Kilgore, K. M. (1995). New developments in functional assessment: Probabilistic models for gold standards. NeuroRehabilitation, 5(1), 3-25.

Fisher, W. P., Jr., Harvey, R. F., Taylor, P., Kilgore, K. M., & Kelly, C. K. (1995, February). Rehabits: A common language of functional assessment. Archives of Physical Medicine and Rehabilitation, 76(2), 113-122.

Fisher, W. P., Jr., & Stenner, A. J. (2011a, January). Metrology for the social, behavioral, and economic sciences (Social, Behavioral, and Economic Sciences White Paper Series). Retrieved 12 January 2014, from National Science Foundation: http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36.

Fisher, W. P., Jr., & Stenner, A. J. (2011b, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium, http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf, Jena, Germany.

Fisher, W. P., Jr., & Stenner, A. J. (2013a). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Fisher, W. P., Jr., & Stenner, A. J. (2013b). Overcoming the invisibility of metrology: A reading measurement network for education and the social sciences. Journal of Physics: Conference Series, 459(012024), http://iopscience.iop.org/1742-6596/459/1/012024.

Fisher, W. P., Jr., & Stenner, A. J. (2015). The role of metrology in mobilizing and mediating the language and culture of scientific facts. Journal of Physics Conference Series, 588(012043).

Fisher, W. P., Jr., & Stenner, A. J. (2015). Theory-based metrological traceability in education: A reading measurement network. Measurement, in review.

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo, in review.

Fisher, W. P., Jr., & Wright, B. D. (1994). Introduction to probabilistic conjoint measurement theory and applications (W. P. Fisher, Jr., & B. D. Wright, Eds.) [Special issue]. International Journal of Educational Research, 21(6), 559-568.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Gleeson-White, J. (2015). Six capitals, or can accountants save the planet? Rethinking capitalism for the 21st century. New York: Norton.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Haraway, D. J. (1996). Modest witness: Feminist diffractions in science studies. In P. Galison & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts, and power (pp. 428-441). Stanford, California: Stanford University Press.

Hawken, P., Lovins, A., & Lovins, H. L. (1999). Natural capitalism: Creating the next industrial revolution. New York: Little, Brown, and Co.

Jensen, M. C. (2001, Fall). Value maximization, stakeholder theory, and the corporate objective function. Journal of Applied Corporate Finance, 14(3), 8-21.

Latour, B. (1990). Postmodern? No, simply amodern: Steps towards an anthropology of science. Studies in History and Philosophy of Science, 21(1), 145-71.

Latour, B. (1993). We have never been modern. Cambridge, Massachusetts: Harvard University Press.

Latour, B. (2005). Reassembling the social: An introduction to Actor-Network-Theory. Clarendon Lectures in Management Studies). Oxford, England: Oxford University Press.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012002/pdf/1742-6596_459_1_012002.pdf.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Pendrill, L. (2014, December). Man as a measurement instrument [Special Feature]. NCSLI Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2013). Quantifying human response: Linking metrological and psychometric characterisations of man as a measurement instrument. Journal of Physics: Conference Series, 459, http://iopscience.iop.org/1742-6596/459/1/012057.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, p. in press. doi: http://dx.doi.org/10.1016/j.measurement.2015.04.010.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.

Rogers, J., & White, A. (2015, April 28). Focusing corporate sustainability ratings on what matters. Huffington Post. Retrieved from http://www.huffingtonpost.com/jean-rogers/focusing-corporate-sustai_b_7156148.html.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459(012025), http://iopscience.iop.org/1742-6596/459/1/012025.

Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2013, August). Causal Rasch models. Frontiers in Psychology: Quantitative Psychology and Measurement, 4(536), 1-14 [doi: 10.3389/fpsyg.2013.00536].

Williamson, O. E. (1981, November). The economics of organization: The transaction cost approach. The American Journal of Sociology, 87(3), 548-577.

Williamson, O. E. (1991). Economic institutions: Spontaneous and intentional governance [Special issue]. Journal of Law, Economics, & Organization: Papers from the Conference on the New Science of Organization, 7, 159-187.

Williamson, O. E. (2005). The economics of governance. American Economic Review, 95(2), 1-18.

Wilson, M. (Ed.). (2004). National Society for the Study of Education Yearbooks. Vol. 103, Part II: Towards coherence between classroom assessment and accountability. Chicago, Illinois: University of Chicago Press.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Wilson, M. R. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.

Wilson, M., Mari, L., Maul, A., & Torres Irribarra, D. (2015). A comparison of measurement concepts across physical science and social science domains: Instrument design, calibration, and measurement. Journal of Physics: Conference Series, 588(012034), http://iopscience.iop.org/1742-6596/588/1/012034.

Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14(2), 97-116 [http://www.rasch.org/memo42.htm].

Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every educator and psychologist should know (pp. 65-104 [http://www.rasch.org/memo64.htm]). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Wright, B. D., & Stone, M. H. (1999). Measurement essentials. Wilmington, DE: Wide Range, Inc. [http://www.rasch.org/measess/me-all.pdf].

Young, J. J., & Williams, P. F. (2010, August). Sorting and comparing: Standard-setting and “ethical” categories. Critical Perspectives on Accounting, 21(6), 509-521.

Professional capital as product of human, social, and decisional capitals

April 18, 2014

Leslie Pendrill gave me a tip on a very interesting book, Professional Capital, by Michael Fullan. The author’s distinction between business capital and professional capital is somewhat akin to my distinction (Fisher, 2011) between dead and living capital. The primary point of contact between Fullan’s sense of capital and mine stems from his inclusion of social and decisional capital as crucial enhancements of human capital.

Of course, defining human capital as talent, as Fullan does, is not going to go very far toward supporting generalized management of it. Efficient markets require that capital be represented in transparent and universally available instruments (common currencies or metrics). Transparent, systematic representation makes it possible to act on capital abstractly, in laboratories, courts, and banks, without having to do anything at all with the physical resource itself. (Contrast this with socialism’s focus on controlling the actual concrete resources, and the resulting empty store shelves, unfulfilled five-year plans, pogroms and purges, and overall failure.) Universally accessible transparent representations make capital additive (amounts can be accrued), divisible (it can be divided into shares), and mobile (it can be moved around in networks accepting the currency/metric). (See references below for more information.)

Fullan cites research by Carrie Leanna at the U of Pittsburgh showing that teachers with high social capital increased their students math scores by 5.7% more than teachers with low social capital. The teachers with the highest skill levels (most human capital) and high social capital did the overall best. Low-ability teachers in schools with high social capital did as well as average teachers.

This is great, but the real cream of Fullan’s argument concerns the importance of what he calls decisional capital. I don’t think this will likely work out to be entirely separate from human capital, but his point is well taken: the capacity to consistently engage with students with competence, good judgment, insight, inspiration, creative improvisation, and openness to feedback in a context of shared responsibility is vital. All of this is quite consistent with recent work on collective intelligence (Fischer, Giaccardi, Eden, et al., 2005; Hutchins, 2010; Magnus, 2007; Nersessian, 2006; Woolley, Chabris, Pentland, et al., 2010; Woolley and Fuchs, 2011).

And, of course, you can see this coming: decisional capital is precisely what better measurement provides. Integrated formative and summative assessment informs decision making at the individual level in ways that are otherwise impossible. When those assessments are expressed in uniformly interpretable and applicable units of measurement, collective intelligence and social capital are boosted in the ways documented by Leanna as enhancing teacher performance and boosting student outcomes.

Anyway, just wanted to share that. It fits right in with the trading zone concept I presented at IOMW (the slides are available on my LinkedIn page).

Fischer, G., Giaccardi, E., Eden, H., Sugimoto, M., & Ye, Y. (2005). Beyond binary choices: Integrating individual and social creativity. International Journal of Human-Computer Studies, 63, 482-512.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [http://www.rasch.org/rmt/rmt154j.htm].

Fisher, W. P., Jr. (2003). Measurement and communities of inquiry. Rasch Measurement Transactions, 17(3), 936-938 [http://www.rasch.org/rmt/rmt173.pdf].

Fisher, W. P., Jr. (2004a, Thursday, January 22). Bringing capital to life via measurement: A contribution to the new economics. In R. Smith (Chair), Session 3.3B. Rasch Models in Economics and Marketing. Second International Conference on Measurement. Perth, Western Australia:  Murdoch University.

Fisher, W. P., Jr. (2004b, Friday, July 2). Relational networks and trust in the measurement of social capital. Twelfth International Objective Measurement Workshops. Cairns, Queensland, Australia: James Cook University.

Fisher, W. P., Jr. (2005a). Daredevil barnstorming to the tipping point: New aspirations for the human sciences. Journal of Applied Measurement, 6(3), 173-179.

Fisher, W. P., Jr. (2005b, August 1-3). Data standards for living human, social, and natural capital. In Session G: Concluding Discussion, Future Plans, Policy, etc. Conference on Entrepreneurship and Human Rights. Pope Auditorium, Lowenstein Bldg, Fordham University.

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2008a, 3-5 September). New metrological horizons: Invariant reference standards for instruments measuring human, social, and natural capital. 12th IMEKO TC1-TC7 Joint Symposium on Man, Science, and Measurement. Annecy, France: University of Savoie.

Fisher, W. P., Jr. (2008b, March 28). Rasch, Frisch, two Fishers and the prehistory of the Separability Theorem. In J. William P. Fisher (Ed.), Session 67.056. Reading Rasch Closely: The History and Future of Measurement. American Educational Research Association. New York City [Paper available at SSRN: http://ssrn.com/abstract=1698919%5D: Rasch Measurement SIG.

Fisher, W. P., Jr. (2009a, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009b). NIST Critical national need idea White Paper: Metrological infrastructure for human, social, and natural capital (http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC: National Institute for Standards and Technology (11 pages).

Fisher, W. P., Jr. (2010a, 22 November). Meaningfulness, measurement, value seeking, and the corporate objective function: An introduction to new possibilities. Sausalito, California: LivingCapitalMetrics.com (http://ssrn.com/abstract=1713467).

Fisher, W. P. J. (2010b). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital (p. http://ssrn.com/abstract=2340674). Bridge to Business Postdoctoral Certification, Freeman School of Business: Tulane University.

Fisher, W. P., Jr. (2010c). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics: Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2011b). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com [Online]. Available: http://ssrn.com/abstract=1739386 (Accessed 18 January 2011).

Fisher, W. P., Jr. (2012). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr., & Stenner, A. J. (2005, Tuesday, April 12). Creating a common market for the liberation of literacy capital. In R. E. Schumacker (Ed.), Rasch Measurement: Philosophical, Biological and Attitudinal Impacts. American Educational Research Association. Montreal, Canada: Rasch Measurement SIG.

Fisher, W. P., Jr., & Stenner, A. J. (2011a, January). Metrology for the social, behavioral, and economic sciences. Available: http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36 (Accessed 12 January 2014).

Fisher, W. P., Jr., & Stenner, A. J. (2011b, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium. Jena, Germany: http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf.

Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science, 2, 705-715.

Magnus, P. D. (2007). Distributed cognition and the task of science. Social Studies of Science, 37(2), 297-310.

Nersessian, N. J. (2006, December). Model-based reasoning in distributed cognitive systems. Philosophy of Science, pp. 699-709.

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010, 29 October). Evidence for a collective intelligence factor in the performance of human groups. Science, pp. 686-688.

Woolley, A. W., & Fuchs, E. (2011, September-October). Collective intelligence in the organization of science. Organization Science, pp. 1359-1367.

Six Classes of Results Supporting the Measurability of Human Functioning and Capability

April 12, 2014

Another example of high-level analysis that suffers from a lack of input from state of the art measurement arises in Nussbaum (1997, p. 1205), where the author remarks that it is now a matter of course, in development economics, “to recognize distinct domains of human functioning and capability that are not commensurable along a single metric, and with regard to which choice and liberty of agency play a fundamental structuring role.” Though Nussbaum (2011, pp. 58-62) has lately given a more nuanced account of the challenges of measurement relative to human capabilities, appreciation of the power and flexibility of contemporary measurement models, methods, and instruments remains lacking. For a detailed example of the complexities and challenges that must be addressed in the context of global human development, which is Nussbaum’s area of interest, see Fisher (2011).

Though there are indeed domains of human functioning and capability that are not commensurable along a single metric, they are not the ones referred to by Nussbaum or the texts she cites. On the contrary, six different approaches to establishing the measurability of human functioning and capability have been explored and proven as providing, especially in their composite aggregate, a substantial basis for theory and practice (modified from Fisher, 2009, pp. 1279-1281). These six classes of results speak to the abstract, mathematical side of the paradox noted by Ricoeur (see previous post here) concerning the need to simultaneously accept roles for abstract ideal global universals and concrete local historical contexts in strategic planning and thinking. The six classes of results are:

  1. Mathematical proofs of the necessity and sufficiency of test and survey scores for invariant measurement in the context of Rasch’s probabilistic models (Andersen, 1977, 1999; Fischer, 1981; Newby, Conner, Grant, and Bunderson, 2009; van der Linden, 1992).
  2. Reproduction of physical units of measurement (centimeters, grams, etc.) from ordinal observations (Choi, 1997; Moulton, 1993; Pelton and Bunderson, 2003; Stephanou and Fisher, 2013).
  3. The common mathematical form of the laws of nature and Rasch models (Rasch, 1960, pp. 110-115; Fisher, 2010; Fisher and Stenner, 2013).
  4. Multiple independent studies of the same constructs on different (and common) samples using different (and the same) instruments intended to measure the same thing converge on common units, defining the same objects, substantiating theory, and supporting the viability of standardized metrics (Fisher, 1997a, 1997b, 1999, etc.).
  5. Thousands of peer-reviewed publications in hundreds of scientific journals provide a wide-ranging and diverse array of supporting evidence and theory.
  6. Analogous causal attributions and theoretical explanatory power can be created in both natural and social science contexts (Stenner, Fisher, Stone, and Burdick, 2013).

What we have here, in sum, is a combination of Greek axiomatic and Babylonian empirical algorithms, in accord with Toulmin’s (1961, pp. 28-33) sense of the contrasting principled bases for scientific advancement. Feynman (1965, p. 46) called for less of a focus on the Greek chain of reasoning approach, as it is only as strong as its weakest link, whereas the Babylonian algorithms are akin to a platform with enough supporting legs that one or more might fail without compromising its overall stability. The variations in theory and evidence under these six headings provide ample support for the conceptual and practical viability of metrological systems of measurement in education, health care, human resource management, sociology, natural resource management, social services, and many other fields. The philosophical critique of any type of economics will inevitably be wide of the mark if uninformed about these accomplishments in the theory and practice of measurement.

References

Andersen, E. B. (1977). Sufficient statistics and latent trait models. Psychometrika, 42(1), 69-81.

Andersen, E. B. (1999). Sufficient statistics in educational measurement. In G. N. Masters & J. P. Keeves (Eds.), Advances in measurement in educational research and assessment (pp. 122-125). New York: Pergamon.

Choi, S. E. (1997). Rasch invents “ounces.” Rasch Measurement Transactions, 11(2), 557 [http://www.rasch.org/rmt/rmt112.htm#Ounces].

Feynman, R. (1965). The character of physical law. Cambridge, Massachusetts: MIT Press.

Fischer, G. H. (1981). On the existence and uniqueness of maximum-likelihood estimates in the Rasch model. Psychometrika, 46(1), 59-77.

Fisher, W. P., Jr. (1997). Physical disability construct convergence across instruments: Towards a universal metric. Journal of Outcome Measurement, 1(2), 87-113.

Fisher, W. P., Jr. (1997). What scale-free measurement means to health outcomes research. Physical Medicine & Rehabilitation State of the Art Reviews, 11(2), 357-373.

Fisher, W. P., Jr. (1999). Foundations for health status metrology: The stability of MOS SF-36 PF-10 calibrations across samples. Journal of the Louisiana State Medical Society, 151(11), 566-578.

Fisher, W. P., Jr. (2009). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2010). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics: Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr. (2011). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com. Retrieved 18 January 2011, from Social Science Research Network: http://ssrn.com/abstract=1739386.

Fisher, W. P., Jr., & Stenner, A. J. (2013). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Moulton, M. (1993). Probabilistic mapping. Rasch Measurement Transactions, 7(1), 268 [http://www.rasch.org/rmt/rmt71b.htm].

Newby, V. A., Conner, G. R., Grant, C. P., & Bunderson, C. V. (2009). The Rasch model and additive conjoint measurement. Journal of Applied Measurement, 107(4), 348-354.

Nussbaum, M. (1997). Flawed foundations: The philosophical critique of (a particular type of) economics. University of Chicago Law Review, 64, 1197-1214.

Nussbaum, M. (2011). Creating capabilities: The human development approach. Cambridge, MA: The Belknap Press.

Pelton, T., & Bunderson, V. (2003). The recovery of the density scale using a stochastic quasi-realization of additive conjoint measurement. Journal of Applied Measurement, 4(3), 269-281.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.

Rasch, G. (1977). On specific objectivity: An attempt at formalizing the request for generality and validity of scientific statements. Danish Yearbook of Philosophy, 14, 58-94.

Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2013). Causal Rasch models. Frontiers in Psychology: Quantitative Psychology and Measurement, 4(536), 1-14.

Stephanou, A., & Fisher, W. P., Jr. (2013). From concrete to abstract in the measurement of length. Journal of Physics Conference Series, 459, http://iopscience.iop.org/1742-6596/459/1/012026.

Toulmin, S. E. (1961). Foresight and understanding: An enquiry into the aims of science. London, England: Hutchinson.

van der Linden, W. J. (1992). Sufficient and necessary statistics. Rasch Measurement Transactions, 6(3), 231 [http://www.rasch.org/rmt/rmt63d.htm].

 

Revisiting the “Glocal” integration of universals and historical context

April 11, 2014

Integrated considerations of the universal and the local, the pure ideal parameters and the messy concrete observations, seem ever more ubiquitous in my reading lately. For instance, Ricoeur (1992, p. 289) takes up the problem of human rights imperfectly realized as a product of Western Europe’s cultural history that has nonetheless been adopted by nearly every country in the world. Ricoeur raises the notion of “universals in context or of potential or inchoate universals” that embody the paradox in which

“on the one hand, one must maintain the universal claim attached to a few values where the universal and the historical intersect, and on the other hand, one must submit this claim to discussion, not on a formal level, but on the level of the convictions incorporated in concrete forms of life.”

I could hardly come up with a better description of Rasch measurement theory and practice myself. Any given Rasch model data analysis provides many times more individual-level qualitative statistics on the concrete, substantive observations than on the global quantitative measures. The whole point of graphical displays of measurement information in kidmaps (Chien, Wang, Wang, & Lin, 2009; Masters, 1994), Wright maps (Wilson, 2011), construct maps and self-scoring forms (Best, 2008; Linacre, 1997), etc. is precisely to integrate concrete events as they happened with the abstract ideal of a shared measurement dimension.

It is such a shame that there are so few people thinking about these issues aware of the practical value of the state of the art in measurement, and who include all of the various implications of multifaceted, multilevel, and multi-uni-dimensional modeling, fit assessment, equating, construct mapping, standard setting, etc. in their critiques.

The problem falls squarely in the domain of recent work on the coproduction of social, scientific, and economic orders (such as Hutchins 2010, 2012; Nersessian, 2012). Systems of standards, from languages to metric units to dollars, prethink the world for us and simplify a lot of complex work. But then we’re stuck at the level of conceptual, social, economic, and scientific complexity implied by those standards, unless we can create new forms of social organization integrating more domains. Those who don’t know anything about the available tools can’t get any analytic traction, those who know about the tools but don’t connect with the practitioners can’t get any applied traction (see Wilson’s Psychometric Society Presidential Address on this; Wilson, 2013), analysts and practitioners who form alliances but fail to include accountants or administrators may lack financial or organizational traction, etc. etc.

There’s a real need to focus on the formation of alliances across domains of practice, building out the implications of Callon’s (1995, p. 58) observation that “”translation networks weave a socionature.” In other words, standards are translated into the languages of different levels and kinds of practice to the extent that people become so thoroughly habituated to them that they succumb to the illusion that the objects of interest are inherently natural in self-evident ways. (My 2014 IOMW talk took this up, though there wasn’t a lot of time for details.)

Those who are studying these networks have come to important insights that set the stage for better measurement and metrology for human, social, and natural capital. For instance, in a study of universalities in medicine, Berg and Timmermans (2000, pp. 55, 56) note:

“In order for a statistical logistics to enhance precise decision making, it has to incorporate imprecision; in order to be universal, it has to carefully select its locales. The parasite cannot be killed off slowly by gradually increasing the scope of the Order. Rather, an Order can thrive only when it nourishes its parasite—so that it can be nourished by it.”

“Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.”

Though Berg and Timmermans show no awareness at all of probabilistic and additive conjoint measurement theory and practice, their description of how a statistical logistics has to work to enhance precise decision making is right on target. This phenomenon of noise-induced order is a kind of social stochastic resonance (Fisher, 1992, 2011b) that provides another direction in which explanations of Rasch measurement’s potential role in establishing new metrological standards (Fisher, 2009, 2011a) have to be taken.

Berg, M., & Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Best, W. R. (2008). A construct map that Ben Wright would relish. Rasch Measurement Transactions, 22(3), 1169-70 [http://www.rasch.org/rmt/rmt223a.htm].

Callon, M. (1995). Four models for the dynamics of science. In S. Jasanoff, G. E. Markle, J. C. Petersen & T. Pinch (Eds.), Handbook of science and technology studies (pp. 29-63). Thousand Oaks, California: Sage Publications.

Chien, T.-W., Wang, W.-C., Wang, H.-Y., & Lin, H.-J. (2009). Online assessment of patients’ views on hospital performances using Rasch model’s KIDMAP diagram. BMC Health Services Research, 9, 135 [10.1186/1472-6963-9-135 or http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2727503/%5D.

Fisher, W. P., Jr. (1992, Spring). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [http://www.rasch.org/rmt/rmt54k.htm].

Fisher, W. P., Jr. (2009, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2011b). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science, 2, 705-715.

Hutchins, E. (2012). Concepts in practice as sources of order. Mind, Culture, and Activity, 19, 314-323.

Linacre, J. M. (1997). Instantaneous measurement and diagnosis. Physical Medicine and Rehabilitation State of the Art Reviews, 11(2), 315-324 [http://www.rasch.org/memo60.htm].

Masters, G. N. (1994). KIDMAP – a history. Rasch Measurement Transactions, 8(2), 366 [http://www.rasch.org/rmt/rmt82k.htm].

Nersessian, N. J. (2012). Engineering concepts: The interplay between concept formation and modeling practices in bioengineering sciences. Mind, Culture, and Activity, 19, 222-239.

Wilson, M. R. (2011). Some notes on the term: “Wright Map.” Rasch Measurement Transactions, 25(3), 1331 [http://www.rasch.org/rmt/rmt253.pdf].

Wilson, M. (2013, April). Seeking a balance between the statistical and scientific elements in psychometrics. Psychometrika, 78(2), 211-236.

Convergence, Divergence, and the Continuum of Field-Organizing Activities

March 29, 2014

So what are the possibilities for growing out green shoots from the seeds and roots of an ethical orientation to keeping the dialogue going? What kinds of fruits might be expected from cultivating a common ground for choosing discourse over violence? What are the consequences for practice of planting this seed in this ground?

The same participant in the conversation earlier this week at Convergence XV who spoke of the peace building processes taking place around the world also described a developmental context for these issues of mutual understanding. The work of Theo Dawson and her colleagues (Dawson, 2002a, 2002b, 2004; Dawson, Fischer, and Stein, 2006) is especially pertinent here. Their comparisons of multiple approaches to cognitive and moral development have provided clear and decisive theory, evidence, and instrumentation concerning the conceptual integrations that take place in the evolution of hierarchical complexity.

Conceptual integrations occur when previously tacit, unexamined, and assumed principles informing a sphere of operations are brought into conscious awareness and are transformed into explicit objects of new operations. Developmentally, this is the process of discovery that takes place from the earliest stages of life, in utero. Organisms of all kinds mature in a process of interaction with their environments. Young children at the “terrible two” stage, for instance, are realizing that anything they can detach from, whether by throwing or by denying (“No!”), is not part of them. Only a few months earlier, the same children will have been fascinated with their fingers and toes, realizing these are parts of their own bodies, often by putting them in their mouths.

There are as many opportunities for conceptual integrations between the ages of 21 to 99 as there are between birth and 21. Developmental differences in perspectives can make for riotously comic situations, and can also lead to conflicts, even when the participants agree on more than they disagree on. And so here we arrive at a position from which we can get a grip on how to integrate convergence and divergence in a common framework that follows from the prior post’s brief description of the ontological method’s three moments of reduction, application, and deconstruction.

Image

Woolley and colleagues (Woolley, et al., 2010; Woolley and Fuchs, 2011) describe a continuum of five field-organizing activities categorizing the types of information needed for effective collective intelligence (Figure 1). Four of these five activities (defining, bounding, opening, and bridging) vary in the convergent versus divergent processes they bring to bear in collective thinking. Defining and bounding are convergent processes that inform judgment and decision making. These activities are especially important in the emergence of a new field or organization, when the object of interest and the methods of recognizing and producing it are in contention. Opening and bridging activities, in contrast, diverge from accepted definitions and transgress boundaries in the creative process of pushing into new areas. Undergirding the continuum as a whole is the fifth activity, grounding, which serves as a theory- and evidence-informed connection to meaningful and useful results.

There are instances in which defining and bounding activities have progressed to the point that the explanatory power of theory enables the calibration of test items from knowledge of the component parts included in those items. The efficiencies and cost reductions gained from computer-based item generation and administration are significant. Research in this area takes a variety of approaches; for more information, see Daniel and Embretson (2010), DeBoeck and Wilson (2004), Stenner, et al. (2013), and others.

The value of clear definitions and boundaries in this context stems in large part from the capacity to identify exceptions that prove (test) the rules, and that then also provide opportunities for opening and bridging. Kuhn (1961, p. 180; 1977, p. 205) noted that

To the extent that measurement and quantitative technique play an especially significant role in scientific discovery, they do so precisely because, by displaying significant anomaly, they tell scientists when and where to look for a new qualitative phenomenon.

Rasch (1960, p. 124) similarly understood that “Once a law has been established within a certain field then the law itself may serve as a tool for deciding whether or not added stimuli and/or objects belong to the original group.” Rasch gives the example of mechanical force applied to various masses with resulting accelerations, introducing idea that one of the instruments might exert magnetic as well as mechanical force, with noticeable effects on steel masses, but not on wooden masses. Rasch suggests that exploration of these anomalies may result in the discovery of other similar instruments that vary in the extent to which they also exert the new force, with the possible consequence of discovering a law of magnetic attraction.

There has been an intense interest in the assessment of divergent inconsistencies in measurement research and practice following in the wake of Rasch’s early work in psychological and social measurement (examples from a very large literature in this area include Karabatsos and Ulrich, 2002, and Smith and Plackner, 2009). Andrich, for instance, makes explicit reference to Kuhn (1961), saying, “…the function of a model for measurement…is to disclose anomalies, not merely to describe data” (Andrich, 2002, p. 352; also see Andrich, 1996, 2004, 2011). Typical software for applying Rasch models (Andrich, et al., 2013; Linacre, 2011, 2013; Wu, et al., 2007) thus accordingly provides many more qualitative numbers evaluating potential anomalies than quantitative measuring numbers. These qualitative numbers (digits that do not stand for something substantive that adds up in a constant unit) include uncertainty and confidence indicators that vary with sample size; mean square and standardized model fit statistics; and principal components analysis factor loadings and eigenvalues.

The opportunities for divergent openings onto new qualitative phenomena provided by data consistency evaluations are complemented in Rasch measurement by a variety of bridging activities. Different instruments intended to measure the same or closely related constructs may often be equated or co-calibrated, so they measure in a common unit (among many publications in this area, see Dawson, 2002a, 2004; Fisher, 1997; Fisher, et al., 1995; Massof and Ahmadian, 2007; Smith and Taylor, 2004). Similarly, the same instrument calibrated on different samples from the same population may exhibit consistent properties across those samples, offering further evidence of a potential for defining a common unit (Fisher, 1999).

Other opening and bridging activities include capacities (a) to drop items or questions from a test or survey, or to add them; (b) to adaptively administer subsets of custom-selected items from a large bank; and (c) to adjust measures for the leniency or severity of judges assigning ratings, all of which can be done, within the limits of the relevant definitions and boundaries, without compromising the unit of comparison. For methodological overviews, see Bond and Fox (2007), Wilson (2005), and others.

The various field-organizing activities spanning the range from convergence to divergence are implicated not only in research on collective thinking, but also in the history and philosophy of science. Galison and colleagues (Galison, 1997, 1999; Galison and Stump, 1996) closely examine positivist and antipositivist perspectives on the unity of science, finding their conclusions inconsistent with the evidence of history. A postpositivist perspective (Galison, 1999, p. 138), in contrast, finds “distinct communities and incommensurable beliefs” between and often within the areas of theory, experiment, and instrument-making. But instead of finding these communities “utterly condemned to passing one another without any possibility of significant interaction,” Galison (1999, p. 138) observes that “two groups can agree on rules of exchange even if they ascribe utterly different significance to the objects being exchanged; they may even disagree on the meaning of the exchange process itself.” In practice, “trading partners can hammer out a local coordination despite vast global differences.”

In accord with Woolley and colleagues’ work on convergent and divergent field-organizing activities, Galison (1999, p. 137) concludes, then, that “science is disunified, and—against our first intuitions—it is precisely the disunification of science that underpins its strength and stability.” Galison (1997, pp. 843-844) concludes with a section entitled “Cables, Bricks, and Metaphysics” in which the postpositivist disunity of science is seen to provide its unexpected coherence from the simultaneously convergent and divergent ways theories, experiments, and instruments interact.

But as Galison recognizes, a metaphor based on the intertwined strands in a cable is too mechanical to support the dynamic processes by which order arises from particular kinds of noise and chaos. Not cited by Galison is a burgeoning literature on the phenomenon of noise-induced order termed stochastic resonance (Andò  and Graziani 2000, Benzi, et al., 1981; Dykman and McClintock, 1998; Fisher, 1992, 2011; Hess and Albano, 1998; Repperger and Farris, 2010). Where the metaphor of a cable’s strands breaks down, stochastic resonance provides multiple ways of illustrating how the disorder of finite and partially independent processes can give rise to an otherwise inaccessible order and structure.

Stochastic resonance involves small noisy signals that can be amplified to have very large effects. The noise has to be of a particular kind, and too much of it will drown out rather than amplify the effect. Examples include the interaction of neuronal ensembles in the brain (Chialvo, Lontin, and Müller-Gerking, 1996), speech recognition (Moskowitz and Dickinson, 2002), and perceptual interpretation (Rianni and Simonotto, 1994). Given that Rasch’s models for measurement are stochastic versions of Guttman’s deterministic models (Andrich, 1985), the question has been raised as to how Rasch’s seemingly weaker assumptions could lead to a measurement model that is stronger than Guttman’s (Duncan, 1984, p. 220). Stochastic resonance may provide an essential clue to this puzzle (Fisher, 1992, 2011).

Another description of what might be a manifestation of stochastic resonance akin to that brought up by Galison arises in Berg and Timmermans’ (2000, p. 56) study of the constitution of universalities in a medical network. They note that, “Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.” Much the same has been said about the behaviors of markets (Mandelbrot, 2004), bringing us back to the topic of the day at Convergence XV earlier this week. I’ll have more to say on this issue of universalities constituted via noise-induced order in due course.

References

Andò, B., & Graziani, S. (2000). Stochastic resonance theory and applications. New York: Kluwer Academic Publishers.

Andrich, D. (1985). An elaboration of Guttman scaling with Rasch models for measurement. In N. B. Tuma (Ed.), Sociological methodology 1985 (pp. 33-80). San Francisco, California: Jossey-Bass.

Andrich, D. (1996). Measurement criteria for choosing among models with graded responses. In A. von Eye & C. Clogg (Eds.), Categorical variables in developmental research: Methods of analysis (pp. 3-35). New York: Academic Press, Inc.

Andrich, D. (2002). Understanding resistance to the data-model relationship in Rasch’s paradigm: A reflection for the next generation. Journal of Applied Measurement, 3(3), 325-359.

Andrich, D. (2004, January). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1), I-7–I-16.

Andrich, D. (2011). Rating scales and Rasch measurement. Expert Reviews in Pharmacoeconomics Outcome Research, 11(5), 571-585.

Andrich, D., Lyne, A., Sheridan, B., & Luo, G. (2013). RUMM 2030: Rasch unidimensional models for measurement. Perth, Australia: RUMM Laboratory Pty Ltd [www.rummlab.com.au].

Benzi, R., Sutera, A., & Vulpiani, A. (1981). The mechanism of stochastic resonance. Journal of Physics. A. Mathematical and General, 14, L453-L457.

Berg, M., & Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Chialvo, D., Longtin, A., & Müller-Gerking, J. (1996). Stochastic resonance in models of neuronal ensembles revisited [Electronic version].

Daniel, R. C., & Embretson, S. E. (2010). Designing cognitive complexity in mathematical problem-solving items. Applied Psychological Measurement, 34(5), 348-364.

Dawson, T. L. (2002a, Summer). A comparison of three developmental stage scoring systems. Journal of Applied Measurement, 3(2), 146-89.

Dawson, T. L. (2002b, March). New tools, new insights: Kohlberg’s moral reasoning stages revisited. International Journal of Behavioral Development, 26(2), 154-66.

Dawson, T. L. (2004, April). Assessing intellectual development: Three approaches, one sequence. Journal of Adult Development, 11(2), 71-85.

Dawson, T. L., Fischer, K. W., & Stein, Z. (2006). Reconsidering qualitative and quantitative research approaches: A cognitive developmental perspective. New Ideas in Psychology, 24, 229-239.

De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. Statistics for Social and Behavioral Sciences). New York: Springer-Verlag.

Duncan, O. D. (1984). Notes on social measurement: Historical and critical. New York: Russell Sage Foundation.

Dykman, M. I., & McClintock, P. V. E. (1998, January 22). What can stochastic resonance do? Nature, 391(6665), 344.

Fisher, W. P., Jr. (1992, Spring). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [http://www.rasch.org/rmt/rmt54k.htm].

Fisher, W. P., Jr. (1997). Physical disability construct convergence across instruments: Towards a universal metric. Journal of Outcome Measurement, 1(2), 87-113.

Fisher, W. P., Jr. (1999). Foundations for health status metrology: The stability of MOS SF-36 PF-10 calibrations across samples. Journal of the Louisiana State Medical Society, 151(11), 566-578.

Fisher, W. P., Jr. (2011). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Fisher, W. P., Jr., Harvey, R. F., Taylor, P., Kilgore, K. M., & Kelly, C. K. (1995, February). Rehabits: A common language of functional assessment. Archives of Physical Medicine and Rehabilitation, 76(2), 113-122.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Galison, P. (1999). Trading zone: Coordinating action and belief. In M. Biagioli (Ed.), The science studies reader (pp. 137-160). New York: Routledge.

Galison, P., & Stump, D. J. (1996). The disunity of science: Boundaries, contexts, and power. Palo Alto, California: Stanford University Press.

Hess, S. M., & Albano, A. M. (1998, February). Minimum requirements for stochastic resonance in threshold systems. International Journal of Bifurcation and Chaos, 8(2), 395-400.

Karabatsos, G., & Ullrich, J. R. (2002). Enumerating and testing conjoint measurement models. Mathematical Social Sciences, 43, 487-505.

Kuhn, T. S. (1961). The function of measurement in modern physical science. Isis, 52(168), 161-193. (Rpt. in T. S. Kuhn, (Ed.). (1977). The essential tension: Selected studies in scientific tradition and change (pp. 178-224). Chicago: University of Chicago Press.)

Linacre, J. M. (2011). A user’s guide to WINSTEPS Rasch-Model computer program, v. 3.72.0. Chicago, Illinois: Winsteps.com.

Linacre, J. M. (2013). A user’s guide to FACETS Rasch-Model computer program, v. 3.71.0. Chicago, Illinois: Winsteps.com.

Mandelbrot, B. (2004). The misbehavior of markets. New York: Basic Books.

Massof, R. W., & Ahmadian, L. (2007, July). What do different visual function questionnaires measure? Ophthalmic Epidemiology, 14(4), 198-204.

Moskowitz, M. T., & Dickinson, B. W. (2002). Stochastic resonance in speech recognition: Differentiating between /b/ and /v/. Proceedings of the IEEE International Symposium on Circuits and Systems, 3, 855-858.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.

Repperger, D. W., & Farris, K. A. (2010, July). Stochastic resonance –a nonlinear control theory interpretation. International Journal of Systems Science, 41(7), 897-907.

Riani, M., & Simonotto, E. (1994). Stochastic resonance in the perceptual interpretation of ambiguous figures: A neural network model. Physical Review Letters, 72(19), 3120-3123.

Smith, R. M., & Plackner, C. (2009). The family approach to assessing fit in Rasch measurement. Journal of Applied Measurement, 10(4), 424-437.

Smith, R. M., & Taylor, P. (2004). Equating rehabilitation outcome scales: Developing common metrics. Journal of Applied Measurement, 5(3), 229-42.

Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2013, August). Causal Rasch models. Frontiers in Psychology: Quantitative Psychology and Measurement, 4(536), 1-14 [doi: 10.3389/fpsyg.2013.00536].

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010, 29 October). Evidence for a collective intelligence factor in the performance of human groups. Science, 330, 686-688.

Woolley, A. W., & Fuchs, E. (2011, September-October). Collective intelligence in the organization of science. Organization Science, 22(5), 1359-1367.

Wu, M. L., Adams, R. J., Wilson, M. R., Haldane, S.A. (2007). ACER ConQuest Version 2: Generalised item response modelling software. Camberwell: Australian Council for Educational Research.

On the Criterion Institute’s Leaders Shaping Markets initiative

November 14, 2013

The Criterion Institute’s Leaders Shaping Markets initiative is an encouraging development in large part because of its focus on systems level change. As the Institute recognizes, the questions being raised and the resources being invested are essential to overcoming recurrent problems of fragmentation and marginalization in efforts being made in more piecemeal fashion across a number of other arenas.

Of particular interest from the Institute’s second roundtable session is Joy Anderson’s list of Strategies for Shaping Market Systems. Anderson presents five strategies:

  1. reframing the issues, problems, and boundaries of the system;
  2. engaging systems of power, elegantly;
  3. continuously identifying leverage points in the system;
  4. building structures and leadership for sustained systems-level disruption; and
  5. attending to change over time and across context.

Reframing is the right place to start. As I’ve said elsewhere in this blog, the problem is the problem. At this level of complexity, problems cannot be solved from within the same paradigm they were born from. Conceiving ways of redefining problems that truly reframe the issues and boundaries of a system is hard enough, but implementing them is even harder.

From my point of view, philosophically, the central problem that makes everything so difficult has to do with our deeply ingrained Western habits of thought around not viewing problems and solutions as of a piece, as wholes in which each implies the other. As long as we keep defining problems and solutions in ways that separate them, as though the solution is in no way involved in perpetuating the problem, we are hopelessly stuck.

So we restrict our options for solving problems by the way we frame the issues. And when we misidentify the problem, as when we fail to properly frame it, then we will likely not only not solve it, we will make it worse. That seems to be exactly what’s been going on in the struggle for economic and social justice for decades and centuries.

So if we reframe the problem of shaping markets around the mutual implication of problems and solutions, how do we move to the next step, to engaging systems of power, elegantly? There are a lot of deep and complex philosophical concepts involved here, but we can cut to the chase and note that our language and tools embody problem-solution unities. Social ecologies of relationships define the meanings and uses of things and ideas.

One way of engaging systems of power elegantly to shape markets might then be to harness the power driving those markets in new, more efficient and meaningful ways. The question that then immediately arises concerns the next of Anderson’s five points: where do we find the leverage in the system that would enable the harnessing of its power?

There is likely no greater concentration of power in markets than the profit motive. How might it become the primary lever for engaging the power of the market? We might, for instance, deploy tools and ideas that co-opt the interests of the systems of power by enhancing the predictability of market forces and sustainability of profits. Concentrating now on dwelling within the problem-solution unity of how to shape markets, we can tap into a key factor that makes markets efficient: we manage what we measure, and management is facilitated when we can measure quality and quantity cheaply and easily.

Common currencies for the exchange of value are essential not just to trade and commerce, but also take shape as the standard metrics employed in science, engineering, music, and as the signs and symbols of basic communication. Money is such an easy to manage measure of value that the problems we are addressing here stem in large part from using it too exclusively as a proxy for the authentic wealth we really want. Engaging with systems of power elegantly also then requires us to think in terms of extending the power of standard units of measurement into the new domains of intangible assets: human, social, and natural capital.

This is where we arrive at the structures for sustained system-level disruption. Current economic models and financial spreadsheets focus on the three classic forms of capital: land, labor, and manufactured tools/commodities. (Money, as liquid capital, is fungible relative to all three.) Of these three, we have a metric system for measuring and managing only property and manufactured tools/commodities.

Green economics offers an alternative four-capitals model that adds social capital and reframes land as natural capital and labor as human capital. Both of the latter are found to be far more complex and valuable than their usual reductions to a piece of ground or “hands” would suggest. Human capital involves health, abilities, and motivations; natural capital includes the earth’s air and water purification systems, and food supplies. The addition of social capital is justified on the grounds that, without it, markets are impossible.

What we do not have is a metric system for three of the four forms of capital. Nor do we have the legal and financial systems needed to bring these forms of capital to life in efficient markets, to make them recognized and accepted in banks and courts of law. We further also do not have leaders aware of the need for these structures, and of the established basis in scientific research that makes them viable.

The science is complex and technical, but it brings to bear practical capacities for meaningful, individual level, qualitatively informative and quantitatively rigorous measurement. There is considerable elegance in this method of approaching engagement with the systems of power. There is mathematical beauty in the symmetry and harmony of instruments tuned to the same scales. There is exquisite grace in the way the program for shaping markets grows organically from the seeds of existing markets. The human value of enabling the realization of heretofore unreachable degrees of individual potentials would be enormous, as would be the social value of being able to make returns on investments in education, health care, social services, and the environment accountable.

Successful new markets harnessing the profit motive in the name of socially responsible and sustainable economies well ought to provoke a new cultural renaissance as the proven relationships between higher rates of educational attainment and health, community relations, and environmental quality are born out. The challenges are huge, but properly framing the problems and their solutions will unify our energies in common purpose like never before, bringing joy to the effort.

For further reading along these lines, see:

Fisher, W. P., Jr., & Stenner, A. J. (2011, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium, http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf, Jena, Germany.

Fisher, W. P., Jr. (2009). NIST Critical national need idea White Paper: metrological infrastructure for human, social, and natural capital. Washington, DC: National Institute for Standards and Technology, http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf

Fisher, W. P., Jr., & Stenner, A. J. (2011, January). Metrology for the social, behavioral, and economic sciences (Social, Behavioral, and Economic Sciences White Paper Series). Washington, DC: National Science Foundation, http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36

Fisher, W. P., Jr. (2012, May/June). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5, http://ses-standards.org/associations/3698/files/2011WSDthirdplacepaper.pdf or http://ssrn.com/abstract=2083975

Fisher, W. P., Jr. (2009, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287, http://doi:10.1016/j.measurement.2009.03.014

Fisher, W. P., Jr. (2011). Bringing human, social, and natural capital to life: Practical consequences and opportunities. Journal of Applied Measurement, 12(1), 49-66, http://ssrn.com/abstract=1698867

Fisher, W. P. J. (2010). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital. Qualifying Paper, Bridge to Business Postdoctoral Certification, Freeman School of Business, Tulane University, http://ssrn.com/abstract=2340674

https://livingcapitalmetrics.wordpress.com/2010/01/13/reinventing-capitalism/

Dispelling Myths about Measurement in Psychology and the Social Sciences

August 27, 2013

Seven common assumptions about measurement and method in psychology and the social sciences stand as inconsistent anomalies in the experience of those who have taken the trouble to challenge them. As evidence, theory, and instrumentation accumulate, will we see a revolutionary break and disruptive change across multiple social and economic levels and areas as a result? Will there be a slower, more gradual transition to a new paradigm? Or will the status quo simply roll on, oblivious to the potential for new questions and new directions? We shall see.

1. Myth: Qualitative data and methods cannot really be integrated with quantitative data and methods because of opposing philosophical assumptions.

Fact: Qualitative methods incorporate a critique of quantitative methods that leads to a more scientific theory and practice of measurement.

2. Myth: Statistics is the logic of measurement.

Fact: Statistics did not emerge as a discipline until the 19th century, while measurement, of course, has been around for millennia. Measurement is modeled at the individual level within a single variable whereas statistics model at the population level between variables. Data are fit to prescriptive measurement models using the Garbage-In, Garbage-Out (GIGO) Principle, while descriptive statistical models are fit to data.

3. Myth: Linear measurement from ordinal test and survey data is impossible.

Fact: Ordinal data have been used as a basis for invariant linear measures for decades.

4. Myth: Scientific laws like Newton’s laws of motion cannot be successfully formulated, tested, or validated in psychology and the social sciences.

Fact: Mathematical laws of human behavior and cognition in the same form as Newton’s laws are formulated, tested, and validated in numerous Rasch model applications.

5. Myth: Experimental manipulations of psychological and social phenomena are inherently impossible or unethical.

Fact: Decades of research across multiple fields have successfully shown how theory-informed interventions on items/indicators/questions can result in predictable, consistent, and substantively meaningful quantitative changes.

6. Myth: “Real” measurement is impossible in psychology and the social sciences.

Fact: Success in predictive theory, instrument calibration, and in maintaining stable units of comparison over time are all evidence supporting the viability of meaningful uniform units of measurement in psychology and the social sciences.

7. Myth: Efficient economic markets can incorporate only manufactured and liquid capital, and property. Human, social, and natural capital, being intangible, have permanent status as market externalities as they cannot be measured well enough to enable accountability, pricing, or transferable representations (common currency instruments).

Fact: The theory and methods necessary for establishing an Intangible Assets Metric System are in hand. What’s missing is the awareness of the scientific, human, social, and economic value that would be returned from the admittedly very large investments that would be required.

References and examples are available in other posts in this blog, in my publications, or on request.

The New Information Platform No One Sees Coming

December 6, 2012

I’d like to draw your attention to a fundamentally important area of disruptive innovations no one seems to see coming. The biggest thing rising in the world of science today that does not appear to be on anyone’s radar is measurement. Transformative potential beyond that of the Internet itself is available.

Realizing that potential will require an Intangible Assets Metric System. This system will connect together all the different ways any one thing is measured, bringing common languages for representing human, social, and economic value into play everywhere. We need these metrics on the front lines of education, health care, social services, and in human, reputation, and natural resource management, as well as in the economic models and financial spreadsheets informing policy, and in the scientific research conducted in dozens of fields.

All reading ability measures, for instance, should be transparently, inexpensively, and effortlessly expressed in a universally uniform metric, in the same way that standardized measures of weight and volume inform grocery store purchasing decisions. We have made starts at such systems for reading, writing, and math ability measures, and for health status, functionality, and chronic disease management measures. There oddly seems to be, however, little awareness of the full value that stands to be gained from uniform metrics in these areas, despite the overwhelming human, economic, and scientific value derived from standardized units in the existing economy. There has accordingly been virtually no leadership or investment in this area.

Measurement practice in business is woefully out of touch with the true paradigm shift that has been underway in psychometrics for years, even though the mantra “you manage what you measure” is repeated far and wide. In a fascinating twist, practically the only ones who notice the business world’s conceptual shortfall in measurement practice are the contrarians who observe that quantification can often be more of a distraction from management than the medium of its execution—but this is true only when measures are poorly conceived, designed, and implemented.

Demand for better measurement—measurement that reduces data volume not only with no loss of information but with the addition of otherwise unavailable interstitial information; that supports mass customized comparability for informed purchasing and quality improvement decisions; and that enables common product definitions for outcomes-based budgeting—is growing hand in hand with the spread of resilient, nimble, lean, and adaptive business models, and with the ongoing geometrical growth in data volume.

An even bigger source of demand for the features of advanced measurement is the increasing dependence of the economy on intangible assets, those forms of human, social, and natural capital that comprise 90% or more of the total capital under management. We will bring these now economically dead forms of capital to life by systematically standardizing representations of their quality and quantity. The Internet is the planetary nervous system through which basic information travels, and the Intangible Assets Metric System will be the global cerebrum, where higher order thinking takes place.

It will not be possible to realize the full potential of lean thinking in the information- and service-based economy without an Intangible Assets Metric System. Given the long-proven business value of standards and the role of measurement in management, it seems self-evident that our ongoing economic difficulties stem largely from our failure to develop and deploy an Intangible Assets Metric System providing common currencies for the exchange of authentic wealth. The future of sustainable and socially responsible business practices must surely depend extensively on universal access to flexible and practical uniform metrics for intangible assets.

Of course, for global intangible assets standards to be viable, they must be adaptable to local business demands and conditions without compromising their comparability. And that is just what is most powerfully disruptive about contemporary measurement methods: they make mass customization a reality. They’ve been doing so in computerized testing since the 1970s. Isn’t it time we started putting this technology to systematic use in a wide range of applications, from human and environmental resource management to education, health care, and social services?