Metrology, the Advancement of Science and New Horizons in Psychological Measurement

Metrology is the science of measurement as practiced in engineering, physics, chemistry, and biology. Its processes center on the formulation, maintenance, and improvement of unit standards. Of particular interest are methods for establishing and ensuring the traceability of local instruments’ units to standards accepted by consensus as global points of reference. Research in the history, philosophy, and social studies of science over the last 30 years has come to focus on metrology and calibrated instrumentation—along with empirical observation and predictive theory—for their combined value as factors vital to explaining the historical successes of science (Ackermann, 1985; Bud and Cozzens, 1992; Golinski, 2012; Harman, 2009; Hussenot and Missonier, 2010; Kuukkanen, 2011; Latour, 1987, 2005; O’Connell, 1993; van Helden and Hankins, 1994; Wise, 1995).

What makes metrology so vitally important to science? One way of answering this question is to reflect on the longstanding importance assigned to measurement in science. For instance, it is often held that measurement is science, that there could be no science without measurement. Most considerations of measurement, however, have focused on the empirical aspects of data gathering, the theoretical aspects of predictive control, and/or the properties of instruments.

The problem repeatedly encountered in this work involves assumptions of universal generality or of mathematical essences which, when articulated and investigated, cannot be sustained. As one looks more and more closely at what scientists in any given area of research actually do, it becomes increasingly difficult to separate the science from a variety of political, legal, economic, social, historical, aesthetic, moral, linguistic, cultural, and religious influences (Bijker, Hughes, and Pinch, 2012; Kuhn, 1970; Latour, 1999, 2013). It ultimately becomes meaningless to try to conceptualize science and measurement in the absence of these influences, since taking them away removes the intrinsic motivations and extrinsic rewards for doing science.

New variations on longstanding empiricist, theoretical, or instrumentalist perspectives have emerged, some independent of (Michell, 2005), and others in response to, these deconstructions (Bloom, 1987, p. 387; Delandshire and Petrosky, 1994, p. 16; Gross and Levitt, 1994, p. 76; Sokal and Bricmont, 1998). These new variations tend to simply dig more deeply into existing positions without realizing that the problem is the problem, which is to say that the way we frame the situation determines to a large extent the applicable solutions.

But it is also true, of course, that the problems encountered at the broad conceptual level are not within the immediate scope of the situations in which working technicians and scientists find themselves. The practical problems of experimental or instrument design, of theorizing, of data gathering and analysis, etc. are all effectively addressed from within each of the metrologically-informed disciplines. That said, training inevitably involves at least implicit lessons in the tacit social, economic, legal, etc. domains impacting research, and experience shows that different skill sets and resources brought to bear are rewarded in what may seem unfair or arbitrary ways. Furthermore, the larger problem is how to understand science well enough to broaden and deepen its scope of application to fields that have not yet achieved the levels of accomplishment obtained in the natural sciences, while being more aware of its limits and risks.

What makes the situation so difficult is that the classic modern separation of subject and object no longer provides a tenable basis for thinking about science. Every time we try to draw a boundary around one sphere of activity, we find that it necessarily entails other spheres, and that it cannot be separated from them without irreparable damage. But what kind of methodology can be adapted to the mutual implication of subject and object? What systematic approaches can be credibly validated when the subject’s marshalling and application of resources to supposedly separately constituted problematic objects itself becomes the problem? How does one enter into the playful flow of unified subjects and objects in a way that is itself recognizably scientific? The situation is fundamentally hermeneutic, in the root mercurial sense of Hermes, the mythological originator of writing, numbers, and dice (Fisher, 2003).

A variety of responses to these problems have emerged in recent years under the headings of one or another kind of realism, such as covariant realism (Crease, 2009), instrumental realism (Ihde, 1991), horizonal realism (Heelan, 1983), and speculative realism (Bryant, Srnicek, and Latour, 2011; also see Kuukkanen, 2011). What do these forms of realism share in common? First, they take an anti-foundationalist perspective that avoids assumptions concerning untestable and unstated mathematical essences. Second, they focus on one form or another of the material practices (the content of communications, behaviors, forms, etc.) enacted by the agents involved in any way in the processes studied. Third, they put all existing things (inanimate, living, and human) in dialogue and on the same footing as regards their capacity to assert their mode of being as having an objective place or role in the real world. Fourth, they recognize the roles played by communications, metrological, transportation, and other kinds of networks as the channels through which objects, communications, effects, resources, etc. travel and are managed.

What is the role of metrology here? As Latour (1987, p. 251) explains,

“Metrology is the name of this gigantic enterprise to make of the outside a world inside which facts and machines can survive.”

“Scientists build their enlightened networks by giving the outside the same paper form as that of their instruments inside. [They can thereby] travel very far without ever leaving home.”

“There is a continuous trail of readings, checklists, paper forms, telephone lines, that tie all the clocks together. As soon as you leave this trail, you start to be uncertain about what time it is, and the only way to regain certainty is to get in touch again with the metrological chains.”

And so, demonstrating or applying the universality of Ohm’s law requires a standard power source and calibrated instrumentation, repairing an engine requires the correct tool set, etc. But a point of vital importance becomes apparent here: the systematic packaging of these standardized parts and processes reduces the costs of manufacturing, implementing, and maintaining them, and of educating technicians about them. The end result of making technical effects universally available is that they are made to seem naturally built into the world around us.

Metrological networks are, then, in effect the primary means by which civilization advances, in the sense referred to by Whitehead (1911, p. 61) as being in opposition to:

“…a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them. Operations of thought are like cavalry charges in a battle—they are strictly limited in number, they require fresh horses, and must only be made at decisive moments.”

We accordingly can operate an automobile without understanding how internal combustion engines or disk brakes work, and we can read thermometers without knowing anything about thermodynamics. The question that arises, then, is how civilization might be advanced via psychology and the social sciences: how might we increase the number of important operations in these areas that we can perform without thinking of them?

Almost all state of the art measurement in education, health care, performance assessment, etc. plainly depends entirely on the active participation of people able to think about the important operations that must be performed. Absent skilled experts, state of the art measurement simply does not usually happen in psychology or social sciences research and practice. Even when experts are involved, the complications and expense of high quality measurement are often enough to prevent it from taking place.

Why? Might it be because, with a very limited number of exceptions (Fisher and Stenner, 2013; Stenner and Fisher, 2013) measurement in psychology and the social sciences lacks virtually any methods and traditions concerned at all with metrological traceability? With uniform unit standards? With consensus processes for determining standard product definitions? With the power of metrology for simplifying processes, for reducing costs, for streamlining communication, and for amplifying collective intelligence (Magnus, 2007; Pentland, 2007; Surowiecki, 2004; Woolley, Chabris, Pentland, Hashmi, and Malone, 2010; Woolley and Fuchs, 2011)? Little or no attention is being focused on metrology even in the wake of recent developments that would seem to make its relevance unavoidably evident: networked communications, item banking, instrument equating, adaptive instrument administration, and the predictive control needed for on-the-fly automated item generation. Inevitably, however, increasing pressure to put two and two together will be applied as the human, economic, legal, moral, social, etc. implications of advancing psychological and social measurement technologies become apparent.

Rephrasing the question in Latour’s (1987, 1999, 2005, 2013) terms, how can we create a world in which the facts of psychological and social measurement can survive? What kind of environment would be required to build networks in which the outside world has the same form as the instruments in the laboratory? What kinds of continuous trails can be created to tie all of the literacy measures together, all of the numeracy measures together, all of the relationship quality, physical functioning, and health status measures together? What opportunities for such networks can be envisioned, and are there any approximations of such networks already in place in education, psychology, or the social sciences? And quite importantly, how can the interests of each group of stakeholders in a given area be satisfied and represented? Can divergent and even conflicting interests be productively mediated within and between various organizations and institutions? Can the rules, roles, and responsibilities constituting efficient market economics (Fisher and Stenner, 2011; Hussenot and Missonier, 2010; Miller and O’Leary, 2007) be brought to bear on exchanges of human, social, and natural capital value?

These are difficult questions, and initial reactions to them might assume them to be ill-formed, ungrounded, meaningless, or simply impossible to achieve. Others will see that the potential returns on investments in answers to these questions are huge, and well worth thorough exploration. In any case, conclusive negative results will give clearer assurances about viable paths for productive science and economics than could be obtained if the questions had never been raised at all.

References

Ackermann, J. R. (1985). Data, instruments, and theory: A dialectical approach to understanding science. Princeton, New Jersey: Princeton University Press.

Beges, G., Drnovsek, J., & Pendrill, L. R. (2010). Optimising calibration and measurement capabilities in terms of economics in conformity assessment. Accreditation and Quality Assurance: Journal for Quality, Comparability and Reliability in Chemical Measurement, 15(3), 147-154.

Bijker, W. E., Hughes, T. P., & Pinch, T. (Eds.). (2012). The social construction of technological systems: New directions in the sociology and history of technology. (The Social Construction of Technological Systems). Cambridge, Massachusetts: MIT Press.

Bloom, A. (1987). The closing of the American mind: How higher education has failed democracy and impoverished the souls of today’s students. New York: Simon & Schuster.

Bryant, L., Srnicek, N., & Latour, B. (Eds.). (2011). The speculative turn: Continental materialism and realism. Prahran, Victoria, Australia: Re.press.

Bud, R., & Cozzens, S. E. (Editors). (1992). SPIE Institutes. Vol. 9: Invisible connections: Instruments, institutions, and science (R. F. Potter, Ed.). Bellingham, WA: SPIE Optical Engineering Press.

Crease, R. (2009). Covariant realism. Human Affairs, 19(2), 223-232.

Delandshere, G., & Petrosky, A. R. (1994). Capturing teachers’ knowledge. Educational Researcher, 23(5), 11-18.

Fisher, W. P., Jr. (2003). Mathematics, measurement, metaphor, metaphysics: Part I. Implications for method in postmodern science. Theory & Psychology, 13(6), 753-790.

Fisher, W. P., Jr., & Stenner, A. J. (2011, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium, http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf, Jena, Germany.

Fisher, W. P., Jr., & Stenner, A. J. (2013). Overcoming the invisibility of metrology: A reading measurement network for education and the social sciences. Journal of Physics: Conference Series, 459(012024), doi:10.1088/1742-6596/459/1/012024.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Gross, P. R., & Levitt, N. (1994). Higher superstition: The academic left and its quarrels with science. Baltimore, MD: Johns Hopkins University Press.

Harman, G. (2009). Prince of networks: Bruno Latour and metaphysics. Melbourne, Australia: Re.press.

Heelan, P. A. (1983). Space perception and the philosophy of science. Berkeley, California: University of California Press.

Hussenot, A., & Missonier, S. (2010). A deeper understanding of evolution of the role of the object in organizational process. The concept of ‘mediation object.’ Journal of Organizational Change Management, 23(3), 269-286.

Ihde, D. (1991). Instrumental realism: The interface between philosophy of science and philosophy of technology. Bloomington, Indiana: Indiana University Press.

Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago, Illinois: University of Chicago Press.

Kuukkanen, J.-M. (2011). I am knowledge. Get me out of here! On localism and the universality of science. Studies in History and Philosophy of Science Part A, 42(4), 590-601.

Latour, B. (1987). Science in action: How to follow scientists and engineers through society. New York: Cambridge University Press.

Latour, B. (1999). Pandora’s hope: Essays on the reality of science studies. Cambridge, Massachusetts: Harvard University Press.

Latour, B. (2005). Reassembling the social: An introduction to Actor-Network-Theory. (Clarendon Lectures in Management Studies). Oxford, England: Oxford University Press.

Latour, B. (2013). An inquiry into modes of existence (C. Porter, Trans.). Cambridge, Massachusetts: Harvard University Press.

Magnus, P. D. (2007). Distributed cognition and the task of science. Social Studies of Science, 37(2), 297-310.

Michell, J. (2005). The logic of measurement: A realist overview. Measurement, 38, 285-294.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-34.

O’Connell, J. (1993). Metrology: The creation of universality by the circulation of particulars. Social Studies of Science, 23, 129-173.

Pentland, A. (2007). On the collective nature of human intelligence. Adaptive Behavior, 15(2), 189-198.

Sokal, A., & Bricmont, J. (1998). Fashionable nonsense: Postmodern intellectuals’ abuse of science. New York: Picador, USA.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: An example from reading measurement. Journal of Physics: Conference Series, 459(012025), doi:10.1088/1742-6596/459/1/012025.

Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. New York: Doubleday.

van Helden, A., & Hankins, T. L. (Eds.). (1994). Instruments (Vol. 9). Osiris: a research journal devoted to the history of science and its cultural influences). Chicago, Illinois: University of Chicago Press.

Whitehead, A. N. (1911). An introduction to mathematics. New York: Henry Holt and Co.

Wise, M. N. (1995). Precision: Agent of unity and product of agreement. Part III–“Today Precision Must Be Commonplace.” In M. N. Wise (Ed.), The values of precision (pp. 352-61). Princeton, New Jersey: Princeton University Press.

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010, 29 October). Evidence for a collective intelligence factor in the performance of human groups. Science, 330, 686-688.

Woolley, A. W., & Fuchs, E. (2011). Collective intelligence in the organization of science. Organization Science, 22(5), 1359-1367.

Advertisements

One Response to “Metrology, the Advancement of Science and New Horizons in Psychological Measurement”

  1. With Reich in spirit, but with a different sense of the problem and its solution | Livingcapitalmetrics's Blog Says:

    […] we own, and we can price it. But despite the well-established scientific facts of decades of measurement science research and practice, none of us can say, “I own x number of shares of stock in intellectual, literacy, or community […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: