Archive for the ‘Education’ Category

Cartesian problems cannot be solved by Cartesian solutions, no matter where those solutions originate

April 13, 2019

Trying to persuade or educate individuals to change the way they think and act, by pointing to the facts or by making emotional or moral appeals, seems always and everywhere to be the default go-to solution for those interested in addressing social and environmental problems. I suppose that approach works to varying degrees for different issues, but behavior change never occurs on as massive a scale as when it is mediated by a technology that enables people to do something they value.

The meaning of McLuhan’s expression, “the medium is the message,” and the long history of the many ways in which technologies transform cultures, for better and for worse, all seem utterly lost and forgotten when it comes to efforts aimed at provoking culture change. The ongoing discourses of environmental and social justice inevitably always seem to come back to targeting individual decisions and behaviors as the only recourse for effecting change.

But history teaches us that, if we want to change our values, we have to figure out how to embed the new terms in virally communicable metaphors that enthrall imaginations and captivate people’s attention and interest. Cultures turn on shared meanings that make some behaviors more likely than others. Good metaphors (“love is a rose;” “God is love”) organize experience in ways that allow infinite creative variations on the theme while also lending just a bit of structure and predictability to how things play out. We need to root new metaphors embodying shared human values in information infrastructures that operationalize consensus standards as the common currencies in which those values circulate.

Though the ongoing culture wars seem to suggest wildly divergent values in play across communities, research in developmental psychology strongly indicates that these differences are not what they seem. No matter what their politics, people need to feel valued, to have stable identities, to be recognized as someone of worth, to have a place of dignity in a community, to be trusted, and to see that others enjoy all of these qualities as well. Experience shows that these conditions cannot be implemented by a simple decree or force of will. Broad general conditions have to be cultivated in ways that make the emergence of abundant social capital resources more likely.

A point of entry into thinking about how those conditions might be created is provided by a 2010 quote in the Miami Herald from Gus Speth, former Dean of the Yale School of Forestry and Environmental Studies (http://tinyurl.com/y7mqtzzn). Speth recounts his sense that scientific solutions to ecosystem and climate problems are insufficient because the actual causes of the problems are greed, selfishness, and apathy. So he appeals to religious leaders for help.

But Speth’s moral diagnosis is as misconceived and uninformed as his original scientific one. As has been the topic of multiple posts in this blog, many of today’s problems cannot be solved using the same kind of Cartesian dualist thinking that was used in creating those problems. Voluminous citations in those earlier posts tap a large literature in the philosophy, history, and social studies of science describing a diverse array of examples of nondualist ecosystem thinking and acting (for instance, see references below). These works show how technological media fuse, embody, distribute, and enact social, moral, aesthetic, economic, and scientific values in complex multilevel metasystems (systems of systems). Moral values of fairness, for instance, are embedded in the quantitative values of measurement technologies exported from laboratories into markets where they inform economic values in trade.

Our task is to learn from these examples so that we can develop and deploy new languages that resonate with new values in analogous ways across similarly diverse cultural domains. Beauty, meaning, and poetry have to be as important as logic, mathematics, and science. Readily available theory and evidence already show how all of these are playing their roles in the evolving cultural transformation.

And, fortunately for humanity as well as for the earth, the new nondualist noncartesian solutions will not and cannot be primarily an outcome of deliberate intentions and conscious willpower. On the contrary, these integrated problem-solution monads are living, organic, self-organizing embodiments of ideas that captivate imaginations and draw creative, entrepreneurial energies in productive directions.

Of course, this kind of thing has happened many times in the past, though it has not previously emerged as a result of the kind of cultivated orchestration occurring today. Williamson, North, Ostrom, Coase, and others describe the roles institutions have played in setting up the rules, roles, and responsibilities of efficient markets. Today, new institutions are arising in a context of reproducible scientific results supporting ownership of, investments in, and profits harvested from sustainable impacts measured and managed via virally communicable media spreading social contagions of love and care. This is coming about because we all seek and value meaning and beauty right along with the capacity to enjoy life, liberty, and prosperity. However differently we each define and experience meaning and beauty, caring for the unity and sameness of the objects of the conversations that we are enables us to balance harmonies and dissonances in endless variations performed by every imaginable kind of rhythmic and melodic musical ensemble.

So instead of expecting different results from repeated applications of the same dualistic thinking that got us into today’s problems, we need to think and act nondualistically. Instead of assuming that solutions do not themselves already presuppose and embody problems of a certain type, we need to think in terms of integrated problem-solution monads deployed throughout ecosystems like species in symbiotic relationships. This is precisely what’s happened historically with the oil-automobile-highway-plastics-engineering ecosystem, and with the germ-disease-pharmaceutical-public health-medicine ecosystem. In each case, financial, market, accounting, regulatory, legal, educational, and other institutions evolved in tandem with the emerging sociotechnical ecology.

Now we face urgent needs to think and act on previously unheard of scales and levels of complexity. We have to work together and coordinate efforts in social and psychological domains with no previous history of communications capable of functioning at the needed efficiencies.

But merely urging people to live differently will never result in the changes that must be brought about. No matter how compelling the facts, no matter how persuasive the emotional power, and no matter how inspirational the moral argument, individual people and small groups simply cannot create new shared standards of behavior out of thin air. We are all products of our times and our sociocultural environments. People cannot be expected to simply wake up one day and spontaneously transform their habits by an effort of will. Instead, the values of fairness, equity, inclusion, and justice we say we live by must be embedded within the very fabric of everyday life, the way hours, meters, liters, degrees, grams, and volts are now.

That is, measurements read off instruments calibrated in fair units of comparison—measurements mathematically equivalent to those made with the scales of justice, measurements expressed in the common metrics of a new international system of units, and measurements as adaptable to local individual improvisations as they are generally comparable and navigable—have to be built into every institution in just the same way existing units of measurement are. Education, health care, social services, human resource management, environmental solutions—all of these and more need to attend closely to ways in which the objects of conversation can be more systematically expressed in meaningful words. Ecosystem thinking demands that everyone and everything in a system of relationships must be consistently kept in proportionate contact, within ranges of reported uncertainty, instead of being disconnected off into separate incommensurable universes of discourse, as occurs in today’s institutions.

These are all monumentally huge challenges. But much of the hardest work has been underway for decades, with important results and resources spreading into widely used applications often taken for granted in the background of largely unexamined assumptions. These results are now well enough established, and the associated social and environmental problems are so serious, that more can and should be done to put them to use.

The need for new values is indeed urgent, but empty talk and doing more of the same is getting us nowhere, at best, and more often is worsening conditions. Conceptual determinations of reproducible mathematical values embodying people’s lived social and moral values in fungible economic values are not just theoretical possibilities or provisional experimental results. They are longstanding, widely available, and practical, as well as beautiful and meaningful. With attentive cultivation and nurturing, there are abundant reasons for believing in a safe, healthy, happy, and prosperous future for humanity and life on earth.

References

Akera, A. (2007). Constructing a representation for an ecology of knowledge. Social Studies of Science, 37(3), 413-441.

Barney, M., & Fisher, W. P., Jr. (2016, April). Adaptive measurement and assessment. Annual Review of Organizational Psychology and Organizational Behavior, 3, 469-490.

Blok, A., Nakazora, M., & Winthereik, B. R. (2016). Infrastructuring environments. Science as Culture, 25(1), 1-22.

Bowker, G. C. (2016). How knowledge infrastructures learn. In P. Harvey, C. B. Jensen & A. Morita (Eds.), Infrastructures and social complexity: A companion (pp. 391-403). New York: Routledge.

Bowker, G., Timmermans, S., Clarke, A. E., & Balka, E. (Eds). (2015). Boundary objects and beyond: Working with Leigh Star. Cambridge, MA: MIT Press.

Brain, R. (1998). Standards and semiotics. In T. Lenoir (Ed.), Inscribing science: Scientific texts and the materiality of communication (pp. 249-w84). Stanford, California: Stanford University Press.

Cano, S. J., & Hobart, J. C. (2011). The problem with health measurement. Patient Preference and Adherence, 5, 279-290.

Cano, S., Klassen, A. F., & Pusic, A. L. (2009). The science behind quality-of-life measurement: A primer for plastic surgeons. Plastic and Reconstructive Surgery, 123(3), 98e-106e.

Cano, S., Melin, J., Fisher, W. P., Jr., Stenner, A. J., Pendrill, L., & EMPIR NeuroMet 15HLT04 Consortium. (2018). Patient-centred cognition metrology. Journal of Physics: Conference Series, 1065, 072033.

Cano, S., Pendrill, L., Barbic, S., & Fisher, W. P., Jr. (2018). Patient-centred outcome metrology for healthcare decision-making. Journal of Physics: Conference Series, 1044, 012057.

Cano, S., Pendrill, L., Melin, J., & Fisher, W. P., Jr. (2019). Towards consensus measurement standards for patient-centered outcomes. Measurement, in press.

Dawson, T. L. (2002, Summer). A comparison of three developmental stage scoring systems. Journal of Applied Measurement, 3(2), 146-89.

Dawson, T. L. (2002, March). New tools, new insights: Kohlberg’s moral reasoning stages revisited. International Journal of Behavioral Development, 26(2), 154-66.

Dawson, T. L. (2004, April). Assessing intellectual development: Three approaches, one sequence. Journal of Adult Development, 11(2), 71-85.

Dawson, T. L., & Stein, Z. (2011). We are all learning here: Cycles of research and application in adult development. In C. Hoare (Ed.), The Oxford handbook of reciprocal adult development and learning, 2nd Ed. (pp. 447-460). Oxford, England: Oxford University Press.

Fisher, W. P., Jr. (2009, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement: Concerning Foundational Concepts of Measurement Special Issue Section, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009). NIST Critical national need idea White Paper: Metrological infrastructure for human, social, and natural capital (http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC:. National Institute for Standards and Technology.

Fisher, W. P., Jr. (2010, November 22). The birds and the bees of living meaning. LivingCapitalMetrics blog. https://livingcapitalmetrics.wordpress.com/2010/11/22/the-birds-and-the-bees-of-living-meaning/.

Fisher, W. P., Jr. (2010). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital, Bridge to Business Postdoctoral Certification, Freeman School of Business, Tulane University (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2340674).

Fisher, W. P., Jr. (2011). Metaphor as measurement, and vice versa: Convergence and separation of figure and meaning in a Mawri proverb [Modified version of a paper presented to the African Studies Association, 1996]. Social Science Research Network. http://ssrn.com/abstract=1747967

Fisher, W. P., Jr. (2012). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr. (2012, June 1). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].

Fisher, W. P., Jr. (2013). Imagining education tailored to assessment as, for, and of learning: Theory, standards, and quality improvement. Assessment and Learning, 2, 6-22.

Fisher, W. P., Jr. (2014). The central theoretical problem of the social sciences. Rasch Measurement Transactions, 28(2), 1464-1466. http://www.rasch.org/rmt/rmt282.pdf

Fisher, W. P., Jr. (2017, September). Metrology, psychometrics, and new horizons for innovation. 18th International Congress of Metrology, Paris, 09007 [https://cfmetrologie.edpsciences.org/articles/metrology/pdf/2017/01/metrology_metr2017_09007.pdf].

Fisher, W. P., Jr. (2017). A practical approach to modeling complex adaptive flows in psychology and social science. Procedia Computer Science, 114, 165-174. https://doi.org/10.1016/j.procs.2017.09.027

Fisher, W. P., Jr. (2019). How beauty teaches us to understand meaning, in revision.

Fisher, W. P., Jr. (2019). A nondualist social ethic: Fusing subject and object horizons in measurement. TMQ–Techniques, Methodologies, and Quality, in press.

Fisher, W. P., Jr., & Cavanagh, R. (2016). Measurement as a medium for communication and social action, I & II. In Q. Zhang & H. H. Yang (Eds.), Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings (pp. 153-182). Berlin: Springer-Verlag.

Fisher, W. P., Jr., & Oon, E. P.-T. (2019). Information coherence and complexity across contexts: Negotiating discontinuities in educational assessment infrastructures. Information Systems Research, in review.

Fisher, W. P., Jr., Oon, E. P.-T., & Benson, S. (2018). Applying Design Thinking to systemic problems in educational assessment information management. Journal of Physics Conference Series, 1044, 012012 [http://iopscience.iop.org/article/10.1088/1742-6596/1044/1/012012].

Fisher, W. P., Jr., Oon, E. P.-T., & Benson, S. (2019). Rethinking the role of educational assessment in classroom communities: How can design thinking address the problems of coherence and complexity? Measurement, in review.

Fisher, W. P., Jr., & Stenner, A. J. (2011, January 1). Metrology for the social, behavioral, and economic sciences (Social, Behavioral, and Economic Sciences White Paper Series). http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36

Fisher, W. P., Jr., & Stenner, A. J. (2013). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Fisher, W. P., Jr., & Stenner, A. J. (2016). Theory-based metrological traceability in education: A reading measurement network. Measurement, 92, 489-496.

Fisher, W. P., Jr., & Stenner, A. J. (2018). Ecologizing vs modernizing in measurement and metrology. Journal of Physics Conference Series, 1044, 012025.

Fisher, W. P., Jr., & Stenner, A. J. (2017, September 18). Towards an alignment of engineering and psychometric approaches to uncertainty in measurement: Consequences for the future. 18th International Congress of Metrology, 12004, 1-9. https://doi.org/10.1051/metrology/201712004

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Hutchins, E. (2014). The cultural ecosystem of human cognition. Philosophical Psychology, 27(1), 34-49.

Jasanoff, S. (2005). Designs on nature: Science and democracy in Europe and the United States. Princeton, NJ: Princeton University Press.

Jasanoff, S. (2015). Future imperfect: Science, technology, and the imaginations of modernity. In S. Jasanoff & S.-H. Kim (Eds.), Dreamscapes of modernity: Sociotechnical imaginaries and the fabrication of power (pp. 1-22). Chicago: University of Chicago Press.

Jasanoff, S., & Martello, M. L. (Eds.) (2004). Earthly politics: Local and global in environmental governance. (Politics, Science, and the Environment). Cambridge, MA: MIT Press.

Kjellberg, H., & Helgesson, C.-F. (2006). Multiple versions of markets: Multiplicity and performativity in marketing practice. Industrial Marketing Management, 35, 839-855.

Lampland, M., & Star, S. L. (Eds.). (2008). Standards and their stories: How quantifying, classifying, and formalizing practices shape everyday life. Ithaca, NY: Cornell University Press.

Latour, B. (1990). Postmodern? No, simply amodern: Steps towards an anthropology of science. Studies in History and Philosophy of Science, 21(1), 145-171.

Latour, B. (1991). The impact of science studies on political philosophy. Science, Technology, & Human Values, 16(1), 3-19.

Latour, B. (1993). We have never been modern. Cambridge, Massachusetts: Harvard University Press.

Latour, B. (1998). To modernise or ecologise? That is the question. In B. Braun & N. Castree (Eds.), Remaking reality: Nature at the millennium (pp. 221-242). London: Routledge.

Latour, B. (2005). Reassembling the social: An introduction to Actor-Network-Theory. (Clarendon Lectures in Management Studies). Oxford, England: Oxford University Press.

Latour, B. (2009). On the modern cult of the factish gods (H. MacLean & C. Porter, Trans.). Durham, NC: Duke University Press.

Latour, B. (2010). Tarde’s idea of quantification. In M. Candea (Ed.), The social after Gabriel Tarde: Debates and assessments (pp. 145-162). London: Routledge.

Latour, B. (2011). Love your monsters: Why we must care for our technologies as we do our children. Breakthrough Journal, 2, 21-28. http://thebreakthrough.org/index.php/journal/past-issues/issue-2/love-your-monsters

Latour, B. (2014, February 26). On some of the affects of capitalism. Lecture given at the Royal Academy, Copenhagen, Denmark. Retrieved from http://www.bruno-latour.fr/sites/default/files/136-AFFECTS-OF-K-COPENHAGUE.pdf.

Latour, B., & Callon, M. (2011). “Thou shall not calculate!” or how to symmetricalize gift and capital. Revista De Pensamiento e Investifation Social, 11(1), 171-192.

Latour, B., & Lépinay, V. A. (2010). The science of passionate interests: An introduction to Gabriel Tarde’s economic anthropology. Chicago: Prickly Paradigm Press.

Lenoir, T. (Ed.). (1997). Instituting science: The cultural production of scientific disciplines (T. Lenoir & H. U. Gumbrecht, Eds.). Writing Science. Stanford, CA: Stanford University Press.

Lenoir, T. (1998). Inscribing science: Scientific texts and the materiality of communication. Stanford, California: Stanford University Press.

Li, E. Y., Commons, M. L., Miller, J. G., Robbinet, T. L., Marchand, H., Ost, C. M. et al. (2014, September). Relationship among measures within the social and moral development domain. Behavioral Development Bulletin, 19(3), 106-113.

Nersessian, N. J. (2012). Engineering concepts: The interplay between concept formation and modeling practices in bioengineering sciences. Mind, Culture, and Activity, 19, 222-239.

Nersessian, N. J. (2015). Conceptual innovation on the frontiers of science. In E. Margolis & S. Laurence (Eds.), The conceptual mind: New directions in the study of concepts (pp. 455-474). Cambridge, MA: MIT Press.

Nespor, J. (2011). Devices and educational change. Educational Philosophy and Theory, 43(S1).

Overton, W. F. (2015). Processes, relations and Relational-Developmental-Systems. In W. F. Overton & P. C. M. Molenaar (Eds.), Theory and Method. Volume 1 of the Handbook of child psychology and developmental science (7th Ed.) (pp. 9-62). Hoboken, NJ: Wiley.

Schaffer, S. (1992). Late Victorian metrology and its instrumentation: A manufactory of Ohms. In R. Bud & S. E. Cozzens (Eds.), Invisible connections: Instruments, institutions, and science (pp. 23-56). Bellingham, WA: SPIE Optical Engineering Press.

Schaffer, S. (1997). Metrology, metrication, and Victorian values. In B. Lightman (Ed.), Victorian science in context (pp. 438-474). Chicago: University of Chicago Press.

Shapin, S. (1994). A social history of truth: Civility and science in seventeenth-century England. Chicago, Illinois: University of Chicago Press.

Shapin, S., & Schaffer, S. (1985). Leviathan and the air-pump: Hobbes, Boyle, and the experimental life. Princeton, NJ: Princeton University Press.

 

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.

Evaluating Questionnaires as Measuring Instruments

June 23, 2018

An email came in today asking whether three different short (4- and 5-item) questionnaires could be expected to provide reasonable quality measurement. Here’s my response.

—–

Thanks for raising this question. The questionnaire plainly was not designed to provide data suitable for measurement. Though much can be learned about making constructs measurable from data produced by this kind of questionnaire, “Rasch analysis” cannot magically create a silk purse from a sow’s ear (as the old expression goes). Use Linacre’s (1993) generalizability theory nomograph to see what reliabilities are expected for each subscale, given the numbers of items and rating categories, and applying a conservative estimate of the adjusted standard deviations (1.0 logit, for instance). Convert the reliability coefficients into strata (Fisher, 1992, 2008; Wright & Masters, 1982, pp. 92, 105-106) to make the practical meaning of the precision obtained obvious.

So if you have data, analyze it and compare the expected and observed reliabilities. If the uncertainties are quite different, is that because of targeting issues? But before you do that, ask experts in the area to rank order:

  • the courses by relevance to the job;
  • the evaluation criteria from easy to hard; and
  • the skills/competencies in order of importance to job performance.

Then study the correspondence between the rankings and the calibration results. Where do they converge and diverge? Why? What’s unexpected? What can be learned?

Analyze all of the items in each area (student, employer, instructor) together in Winsteps and study each of the three tables 23.x, setting PRCOMP=S. Remember that the total variance explained is not interpreted simply in terms of “more is better” and that the total variance explained is not as important as the ratio of that variance to the variance in the first contrast (see Linacre, 2006, 2008). If the ratio is greater than 3, the scale is essentially unidimensional (though significant problems may remain to be diagnosed and corrected).

Common practice holds that unexplained variance eigenvalues should be less than 1.5, but this overly simplistic rule of thumb (Chou & Wang, 2010; Raîche, 2005) has been contradicted in practice many times, since, even if one or more eigenvalues are over 1.5, theory may say the items belong to the same construct, and the disattenuated correlations of the measures implied by the separate groups of items (provided in tables 23.x) may still approach 1.00, indicating that the same measures are produced across subscales. See Green (1996) and Smith (1996), among others, for more on this.

If subscales within each of the three groups of items are markedly different in the measures they produce, then separate them in different analyses. If these further analyses reveal still more multidimensionalities, it’s time to go back to the drawing board, given how short these scales are. If you define a plausible scale, study the item difficulty orders closely with one or more experts in the area. If there is serious interest in precision measurement and its application to improved management, and not just a bureaucratic need for data to satisfy empty demands for a mere appearance of quality assessment, then trace the evolution of the construct as it changes from less to more across the items.

What, for instance, is the common theme addressed across the courses that makes them all relevant to job performance? The courses were each created with an intention and they were brought together into a curriculum for a purpose. These intentions and purposes are the raw material of a construct theory. Spell out the details of how the courses build competency in translation.

Furthermore, I imagine that this curriculum, by definition, was set up to be effective in training students no matter who is in the courses (within the constraints of the admission criteria), and no matter which particular challenges relevant to job performance are sampled from the universe of all possible challenges. You will recognize these unexamined and unarticulated assumptions as what need to be explicitly stated as hypotheses informing a model of the educational enterprise. This model transforms implicit assumptions into requirements that are never fully satisfied but can be very usefully approximated.

As I’ve been saying for a long time (Fisher, 1989), please do not accept the shorthand language of references to “the Rasch model”, “Rasch scaling”, “Rasch analysis”, etc. Rasch did not invent the form of these models, which are at least as old as Plato. And measurement is not a function of data analysis. Data provide experimental evidence testing model-based hypotheses concerning construct theories. When explanatory theory corroborates and validates data in calibrated instrumentation, the instrument can be applied at the point of use with no need for data analysis, to produce measures, uncertainty (error) estimates, and graphical fit assessments (Connolly, Nachtman, & Pritchett, 1971; Davis, et al., 2008; Fisher, 2006; Fisher, Kilgore, & Harvey, 1995; Linacre, 1997; many others).

So instead of using those common shorthand phrases, please speak directly to the problem of modeling the situation in order to produce a practical tool for managing it.

Further information is available in the references below.

 

Aryadoust, S. V. (2009). Mapping Rasch-based measurement onto the argument-based validity framework. Rasch Measurement Transactions, 23(1), 1192-3 [http://www.rasch.org/rmt/rmt231.pdf].

Chang, C.-H. (1996). Finding two dimensions in MMPI-2 depression. Structural Equation Modeling, 3(1), 41-49.

Chou, Y. T., & Wang, W. C. (2010). Checking dimensionality in item response models with principal component analysis on standardized residuals. Educational and Psychological Measurement, 70, 717-731.

Connolly, A. J., Nachtman, W., & Pritchett, E. M. (1971). Keymath: Diagnostic Arithmetic Test. Circle Pines, Minnesota: American Guidance Service. Retrieved 23 June 2018 from https://images.pearsonclinical.com/images/pa/products/keymath3_da/km3-da-pub-summary.pdf

Davis, A. M., Perruccio, A. V., Canizares, M., Tennant, A., Hawker, G. A., Conaghan, P. G. et al. (2008, May). The development of a short measure of physical function for hip OA HOOS-Physical Function Shortform (HOOS-PS): An OARSI/OMERACT initiative. Osteoarthritis Cartilage, 16(5), 551-559.

Fisher, W. P., Jr. (1989). What we have to offer. Rasch Measurement Transactions, 3(3), 72 [http://www.rasch.org/rmt/rmt33d.htm].

Fisher, W. P., Jr. (1992). Reliability statistics. Rasch Measurement Transactions, 6(3), 238  [http://www.rasch.org/rmt/rmt63i.htm].

Fisher, W. P., Jr. (2006). Survey design recommendations [expanded from Fisher, W. P. Jr. (2000) Popular Measurement, 3(1), pp. 58-59]. Rasch Measurement Transactions, 20(3), 1072-1074 [http://www.rasch.org/rmt/rmt203.pdf].

Fisher, W. P., Jr. (2008). The cash value of reliability. Rasch Measurement Transactions, 22(1), 1160-1163 [http://www.rasch.org/rmt/rmt221.pdf].

Fisher, W. P., Jr., Harvey, R. F., & Kilgore, K. M. (1995). New developments in functional assessment: Probabilistic models for gold standards. NeuroRehabilitation, 5(1), 3-25.

Green, K. E. (1996). Dimensional analyses of complex data. Structural Equation Modeling, 3(1), 50-61.

Linacre, J. M. (1993). Rasch-based generalizability theory. Rasch Measurement Transactions, 7(1), 283-284; [http://www.rasch.org/rmt/rmt71h.htm].

Linacre, J. M. (1997). Instantaneous measurement and diagnosis. Physical Medicine and Rehabilitation State of the Art Reviews, 11(2), 315-324 [http://www.rasch.org/memo60.htm].

Linacre, J. M. (1998). Detecting multidimensionality: Which residual data-type works best? Journal of Outcome Measurement, 2(3), 266-83.

Linacre, J. M. (1998). Structure in Rasch residuals: Why principal components analysis? Rasch Measurement Transactions, 12(2), 636 [http://www.rasch.org/rmt/rmt122m.htm].

Linacre, J. M. (2003). PCA: Data variance: Explained, modeled and empirical. Rasch Measurement Transactions, 17(3), 942-943 [http://www.rasch.org/rmt/rmt173g.htm].

Linacre, J. M. (2006). Data variance explained by Rasch measures. Rasch Measurement Transactions, 20(1), 1045 [http://www.rasch.org/rmt/rmt201a.htm].

Linacre, J. M. (2008). PCA: Variance in data explained by Rasch measures. Rasch Measurement Transactions, 22(1), 1164 [http://www.rasch.org/rmt/rmt221j.htm].

Raîche, G. (2005). Critical eigenvalue sizes in standardized residual Principal Components Analysis. Rasch Measurement Transactions, 19(1), 1012 [http://www.rasch.org/rmt/rmt191h.htm].

Schumacker, R. E., & Linacre, J. M. (1996). Factor analysis and Rasch. Rasch Measurement Transactions, 9(4), 470 [http://www.rasch.org/rmt/rmt94k.htm].

Smith, E. V., Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3(2), 205-31.

Smith, R. M. (1996). A comparison of methods for determining dimensionality in Rasch measurement. Structural Equation Modeling, 3(1), 25-40.

Wright, B. D. (1996). Comparing Rasch measurement and factor analysis. Structural Equation Modeling, 3(1), 3-24.

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Chicago, Illinois: MESA Press.

Measuring Instruments as Media for the Expression of Creative Passions in Education

June 26, 2015

Measurement is often viewed as a reduction of complex phenomena to numbers. It is accordingly also often conceived as mechanical, and disconnected from the world of life. Educational examinations are seen by many as an especially egregious form of inappropriate reduction. This perspective is contradicted, however, by a perspective that sees an analogy between educational assessment and music. Calibrated instruments, mathematical scales, and high technology play key roles in the production of music, which, ironically, is widely considered the most alive, captivating and emotionally powerful of the arts. Though behavioral psychology has indeed learned how to use music to manipulate consumer purchasing decisions, music is unabashedly accepted nonetheless as the highest expression of passion in art.

The question then arises as to if and how measurement in other areas, such as in education, might be conceived, designed, and practiced as a medium for the expression and fulfillment of creative passions. Key issues involved in substantively realizing a musical metaphor in human and social measurement include capacities to tune instruments, to define common scales, to score performances, to orchestrate harmonious relationships, to enhance choral grace note effects, and to combine elements in unique but pleasing and recognizable rhythmic arrangements.

Practical methods for making educational measurement the medium for the expression of creative passions for learning are in place in thousands of schools nationally and internationally. With such tools in hand, formative applications of integrated instruction and assessment could be conceived as intuitive media for composing and conducting expressions of creative passions. Student outcomes in reading, mathematics, and other domains may then come to be seen in terms of portfolios of works akin to those produced by musicians, sculptors, film makers, or painters.

Hundreds of thousands of books and millions of articles tuned to the same text complexity scale, for instance, provide readers an extensive palette of colorful tones and timbres for expressing their desires and capacities for learning. Graphical presentations of individual students’ outcomes, as well as outcomes aggregated by classroom, school, district, etc., could be presented, interpreted and experienced as public performances of artful developmental narratives enabling dramatic performances of personal uniqueness and social generality.

Measurement instrumentation in education is able to capture, aggregate, and organize literacy, numeracy, socio-emotional intelligence, and other performances into special portfolios documenting the play and dance of emerging new understandings. As in any creative process, accidents, errors, and idiosyncratic patterns of strengths and weaknesses may evoke powerful and dramatic expressions of beauty, and human and social value. And just as members of musical ensembles may complement one another’s skills, using rhythm and harmony to improve each others’ playing abilities in practice, so, too, instruments of formative assessment tuned to the same scale can be used to coordinate and enhance individual student and teacher skill levels.

Possibilities for orchestrating such performances across educational, health care, social service, environmental management, and other fields could similarly take advantage of existing instrument calibration and measurement technologies.

Feminist Diffractions, Stochastic Resonance, and Education, Revisited

May 25, 2015

Lehrer (2015) offers an insightful commentary on Saxe et al’s (2015) recent article in Human Development that prompts some observations.

Two areas for questions and comments come to mind. The first has to do with construing the development and revision of new ways of understanding as contested, which implicitly aligns with Latour’s (1987, pp. 89, 93) sense of the way new constructs are subjected to tests of strength. Haraway (1996) makes an important point in her critique of what she sees as the overly masculinist metaphors of heroic competition and (perhaps not so) sublimated violence in these contests. Her sense of “feminist diffractions” stops short of what I have in mind, but opens the door to an alternative approach to what Lehrer calls the “close coupling of definitions with the development and revision of new concepts and ways of understanding.”

Galison (1997, pp. 843-844), for instance, seeks a metaphor capable of expressing what happens in the conceptual, practical, and argumentative contests between different communities of scientists (instrumentalist technicians, theoreticians, and experimentalists). He wants a metaphor that does justice to the disunified chaos and disorder one finds in the relationships between these different groups, which paradoxically results in such productive and coherent innovations. He recalls Peirce’s and Wittgenstein’s metaphors of cables and threads that take their strength from being intertwined from smaller wires and bits of fiber but finds these images too mechanical for his purposes. He wants something more akin to amorphous semiconductors or laminated materials that can fail microscopically but hold macroscopically better than more structurally homogenous materials.

Berg and Timmermans (2000, pp. 55-56) make a similar observation in their study of the constitution of universalities in medical fields:

“In order for a statistical logistics to enhance precise decision making, it has to incorporate imprecision; in order to be universal, it has to carefully select its locales. … Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.”

The general problem is taken up by Ricoeur (1992, p. 289), who raises the notion of “universals in context or of potential or inchoate universals” that embody the paradox in which

“on the one hand, one must maintain the universal claim attached to a few values where the universal and the historical intersect, and on the other hand, one must submit this claim to discussion, not on a formal level, but on the level of the convictions incorporated in concrete forms of life.”

To repeat another theme that comes up again and again in this blog, this kind of noise-induced order sounds like the phenomenon of stochastic resonance (Fisher, 1992, 2011). The importance of stochastic resonance is that it opens up a way to connect the phenomena of emergent understanding with measurement, both at the local individual and general systemic levels.

This is the crux of some very important issues in the philosophy of science and in philosophy generally. Haraway (1996, pp. 439-440), for instance, points out that “embedded relationality is the prophylaxis for both relativism and transcendence.” And Golinski (2012, p. 35) similarly says, “Practices of translation, replication, and metrology have taken the place of the universality that used to be assumed as an attribute of singular science.”

A start in the direction of embedded relationality, translation, replication, and metrology in education is apparent, for instance, in work that enables teachers to usefully relate individual student performances to general learning progressions, connecting instructional applications with accountability (Fisher & Wilson, 2015; Lehrer, 2013; Lehrer & Jones, 2014; Wilson, 2004). As Lehrer (2015, p. 49) says about the Saxe et al. work, “Recurrent forms of mathematical practice enabled the authors to create compelling trajectories of collective activity and learning over time while preserving the contributions of individual development.”

The second of the two topics I’d like to address comes up here in the closing paragraph of his short commentary, where Lehrer says a “hoped-for future innovation would make it possible to visualize individual and collective trajectories simultaneously.” Though future improvements can certainlty be expected, visualizations of individual and collective trajectories for growth in reading are already being recognized in both educational and metrological contexts (Stenner, Swartz, Hanlon, & Emerson, 2012; Stenner & Fisher, 2013, p. 4) for their potential to serve as the media of an embedded relationality capable of undercutting both the relativism of uncontrolled local variation and the universalist pretensions often built into accountability programs.

With emerging recognition of the potential Rasch’s stochastic approaches to construct mapping (Bond & Fox, 2007; Wilson, 2005) offer in the way of metrological translation networks (Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2015; Fisher & Wilson, 2015; Stenner & Fisher, 2013; Wilson, 2013; Wilson, Mari, Maul, & Torres Irribarra 2015), there are good reasons to expect significant new kinds of progress in fields that rely on assessments and surveys for outcome measurement and management.

References

Berg, M.,& Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Fisher, W. P., Jr. (1992). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [http://www.rasch.org/rmt/rmt54k.htm].

Fisher, W. P., Jr. (2011). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Fisher, W. P., Jr., & Stenner, A. J. (2015). The role of metrology in mobilizing and mediating the language and culture of scientific facts. Journal of Physics Conference Series, 588(012043).

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo, in review.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Haraway, D. J. (1996). Modest witness: Feminist diffractions in science studies. In P. Galison & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts, and power (pp. 428-441). Stanford, California: Stanford University Press.

Latour, B. (1987). Science in action: How to follow scientists and engineers through society. New York: Harvard University Press.

Lehrer, R. (2013, April 29). (Chair). In A learning progression emerges in a trading zone of professional community and identity. American Educational Research Association, Division C on Learning and Instruction, Section 2b on Learning and Motivation in Social and Cultural Contexts, San Francisco, CA.

Lehrer, R., & Jones, S. (2014, 2 April). Construct maps as boundary objects in the trading zone. In W. P. Fisher Jr. (Chair), Session 3-A: Rating Scales and Partial Credit, Theory and Applied. International Objective Measurement Workshop, Philadelphia, PA.

Lehrer, R. (2015). Designing for development: Commentary on Saxe, de Kirby, Kang, Le and Schneider. Human Development, 58(1), 45-49.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012002/pdf/1742-6596_459_1_012002.pdf.

Pendrill, L. (2014). Man as a measurement instrument [Special Feature]. NCSLi Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46-55.

Ricoeur, P. (1992). Oneself as another. Chicago, Illinois: University of Chicago Press.

Saxe, G. B., de Kirby, K., Kang, B., Le, M., & Schneider, A. (2015). Studying cognition through time in a classroom community: The interplay between “everyday” and “scientific” concepts. Human Development, 58(1), 5-44.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459(012025), http://iopscience.iop.org/1742-6596/459/1/012025.

Stenner, A. J., Swartz, C., Hanlon, S., & Emerson, C. (2012, February). Personalized learning platforms. Presented at the Pearson Global Research Conference, Fremantle, Western Australia.

Wilson, M. (Ed.). (2004). National Society for the Study of Education Yearbooks. Vol. 103, Part II: Towards coherence between classroom assessment and accountability. Chicago, Illinois: University of Chicago Press.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Wilson, M. R. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.

 

An Entrepreneurial Investment Model Alternative to Picketty’s Taxation Approach to Eliminating Wealth Disparities

May 14, 2014

Is taxation the only or the best solution to inequality? The way discussions of wealth disparities inevitably focus on variations in how, whom or what to tax, it is easy to assume there are no viable alternatives to taxation. But if the point is to invest in those with the most potential for making significant gains in productivity, so as to maximize the returns we realize, do we not wrongly constrain the domain of possible solutions when we misconceive an entrepreneurial problem in welfare terms?

Why can’t we require minimum levels of investment in social capital stocks and bonds offered by schools, hospitals, NGOs, etc? In human capital instruments offered by individuals? Why should not we expect those investments to be used to create new value? What supposed law of nature says it is impossible to associate new human, social and environmental value with stable and meaningful prices? And if there is such a law (such as Kenneth Arrow (1963) proposed), how can we break it? Why can’t we reconceive human and social capital stocks and flows in new ways?

There is one very good reason why we cannot now make such requirements, and it is the same reason why liberals (including me) had better become accustomed to accepting the failure of their agenda. That reason is this: social and environmental externalities. Inequality is inevitable only as long as we do not change the ways we deal with externalities. They can no longer be measured and managed in the same ways. They must be put on the books, brought into the models, measured scientifically, and traded in efficient markets. We have to invent accountability and accounting systems that harness the energy of the profit motive for the greater good—that actually grow authentic wealth and not mere money—and we have to do this far more effectively than has ever been done before.

It’s a tall order. But there are resources available to us that have not yet been introduced into the larger conversation. There are options to consider that need close study and creative experimentation. Proceeding toward the twin futilities of premature despair or unrealistic taxation will only set up another round of self-fulfilling prophecies inexorably grinding to yet another unforeseen but fully foretold disaster. Conversations about how to shape the roles, rules and institutions that make markets what they are (Miller and O’Leary, 2007) need to take place for human, social, and natural capital (Fisher and Stenner, 2011b). Indeed, those conversations are already well underway, as can be seen in the prior entries in this blog and in the sources listed below.

Arrow, K. J. (1963). Uncertainty and the welfare economics of medical care. American Economic Review, 53, 941-973.

Fisher, W. P., Jr. (2007). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009a). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009b). NIST Critical national need idea White Paper: Metrological infrastructure for human, social, and natural capital (http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC: National Institute for Standards and Technology (11 pages).

Fisher, W. P., Jr. (2010a, 22 November). Meaningfulness, measurement, value seeking, and the corporate objective function: An introduction to new possibilities. Sausalito, California: LivingCapitalMetrics.com (http://ssrn.com/abstract=1713467).

Fisher, W. P. J. (2010b). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital (http://ssrn.com/abstract=2340674). Bridge to Business Postdoctoral Certification, Freeman School of Business: Tulane University.

Fisher, W. P., Jr. (2010c, June 13-16). Rasch, Maxwell’s method of analogy, and the Chicago tradition. In G. Cooper (Ed.), https://conference.cbs.dk/index.php/rasch/Rasch2010/paper/view/824. Probabilistic models for measurement in education, psychology, social science and health: Celebrating 50 years since the publication of Rasch’s Probabilistic Models. FUHU Conference Centre, Copenhagen, Denmark: University of Copenhagen School of Business.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. Journal of Applied Measurement, 12(1), 49-66.

Fisher, W. P., Jr. (2011b, Thursday, September 1). Measurement, metrology and the coordination of sociotechnical networks. In S. Bercea (Ed.), New Education and Training Methods. International Measurement Confederation (IMEKO). Jena, Germany: http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24491/ilm1-2011imeko-017.pdf.

Fisher, W. P., Jr. (2012a). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr. (2012b, May/June). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].

Fisher, W. P., Jr., & Stenner, A. J. (2011a, January). Metrology for the social, behavioral, and economic sciences. http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36.

Fisher, W. P., Jr., & Stenner, A. J. (2011b, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium. Jena, Germany: http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf.

Fisher, W. P., Jr., & Stenner, A. J. (2013a). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Fisher, W. P., Jr., & Stenner, A. J. (2013b). Overcoming the invisibility of metrology: A reading measurement network for education and the social sciences. Journal of Physics: Conference Series, 459(012024), http://iopscience.iop.org/1742-6596/459/1/012024.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Professional capital as product of human, social, and decisional capitals

April 18, 2014

Leslie Pendrill gave me a tip on a very interesting book, Professional Capital, by Michael Fullan. The author’s distinction between business capital and professional capital is somewhat akin to my distinction (Fisher, 2011) between dead and living capital. The primary point of contact between Fullan’s sense of capital and mine stems from his inclusion of social and decisional capital as crucial enhancements of human capital.

Of course, defining human capital as talent, as Fullan does, is not going to go very far toward supporting generalized management of it. Efficient markets require that capital be represented in transparent and universally available instruments (common currencies or metrics). Transparent, systematic representation makes it possible to act on capital abstractly, in laboratories, courts, and banks, without having to do anything at all with the physical resource itself. (Contrast this with socialism’s focus on controlling the actual concrete resources, and the resulting empty store shelves, unfulfilled five-year plans, pogroms and purges, and overall failure.) Universally accessible transparent representations make capital additive (amounts can be accrued), divisible (it can be divided into shares), and mobile (it can be moved around in networks accepting the currency/metric). (See references below for more information.)

Fullan cites research by Carrie Leanna at the U of Pittsburgh showing that teachers with high social capital increased their students math scores by 5.7% more than teachers with low social capital. The teachers with the highest skill levels (most human capital) and high social capital did the overall best. Low-ability teachers in schools with high social capital did as well as average teachers.

This is great, but the real cream of Fullan’s argument concerns the importance of what he calls decisional capital. I don’t think this will likely work out to be entirely separate from human capital, but his point is well taken: the capacity to consistently engage with students with competence, good judgment, insight, inspiration, creative improvisation, and openness to feedback in a context of shared responsibility is vital. All of this is quite consistent with recent work on collective intelligence (Fischer, Giaccardi, Eden, et al., 2005; Hutchins, 2010; Magnus, 2007; Nersessian, 2006; Woolley, Chabris, Pentland, et al., 2010; Woolley and Fuchs, 2011).

And, of course, you can see this coming: decisional capital is precisely what better measurement provides. Integrated formative and summative assessment informs decision making at the individual level in ways that are otherwise impossible. When those assessments are expressed in uniformly interpretable and applicable units of measurement, collective intelligence and social capital are boosted in the ways documented by Leanna as enhancing teacher performance and boosting student outcomes.

Anyway, just wanted to share that. It fits right in with the trading zone concept I presented at IOMW (the slides are available on my LinkedIn page).

Fischer, G., Giaccardi, E., Eden, H., Sugimoto, M., & Ye, Y. (2005). Beyond binary choices: Integrating individual and social creativity. International Journal of Human-Computer Studies, 63, 482-512.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [http://www.rasch.org/rmt/rmt154j.htm].

Fisher, W. P., Jr. (2003). Measurement and communities of inquiry. Rasch Measurement Transactions, 17(3), 936-938 [http://www.rasch.org/rmt/rmt173.pdf].

Fisher, W. P., Jr. (2004a, Thursday, January 22). Bringing capital to life via measurement: A contribution to the new economics. In R. Smith (Chair), Session 3.3B. Rasch Models in Economics and Marketing. Second International Conference on Measurement. Perth, Western Australia:  Murdoch University.

Fisher, W. P., Jr. (2004b, Friday, July 2). Relational networks and trust in the measurement of social capital. Twelfth International Objective Measurement Workshops. Cairns, Queensland, Australia: James Cook University.

Fisher, W. P., Jr. (2005a). Daredevil barnstorming to the tipping point: New aspirations for the human sciences. Journal of Applied Measurement, 6(3), 173-179.

Fisher, W. P., Jr. (2005b, August 1-3). Data standards for living human, social, and natural capital. In Session G: Concluding Discussion, Future Plans, Policy, etc. Conference on Entrepreneurship and Human Rights. Pope Auditorium, Lowenstein Bldg, Fordham University.

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2008a, 3-5 September). New metrological horizons: Invariant reference standards for instruments measuring human, social, and natural capital. 12th IMEKO TC1-TC7 Joint Symposium on Man, Science, and Measurement. Annecy, France: University of Savoie.

Fisher, W. P., Jr. (2008b, March 28). Rasch, Frisch, two Fishers and the prehistory of the Separability Theorem. In J. William P. Fisher (Ed.), Session 67.056. Reading Rasch Closely: The History and Future of Measurement. American Educational Research Association. New York City [Paper available at SSRN: http://ssrn.com/abstract=1698919%5D: Rasch Measurement SIG.

Fisher, W. P., Jr. (2009a, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009b). NIST Critical national need idea White Paper: Metrological infrastructure for human, social, and natural capital (http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC: National Institute for Standards and Technology (11 pages).

Fisher, W. P., Jr. (2010a, 22 November). Meaningfulness, measurement, value seeking, and the corporate objective function: An introduction to new possibilities. Sausalito, California: LivingCapitalMetrics.com (http://ssrn.com/abstract=1713467).

Fisher, W. P. J. (2010b). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital (p. http://ssrn.com/abstract=2340674). Bridge to Business Postdoctoral Certification, Freeman School of Business: Tulane University.

Fisher, W. P., Jr. (2010c). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics: Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2011b). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com [Online]. Available: http://ssrn.com/abstract=1739386 (Accessed 18 January 2011).

Fisher, W. P., Jr. (2012). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr., & Stenner, A. J. (2005, Tuesday, April 12). Creating a common market for the liberation of literacy capital. In R. E. Schumacker (Ed.), Rasch Measurement: Philosophical, Biological and Attitudinal Impacts. American Educational Research Association. Montreal, Canada: Rasch Measurement SIG.

Fisher, W. P., Jr., & Stenner, A. J. (2011a, January). Metrology for the social, behavioral, and economic sciences. Available: http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36 (Accessed 12 January 2014).

Fisher, W. P., Jr., & Stenner, A. J. (2011b, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium. Jena, Germany: http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf.

Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science, 2, 705-715.

Magnus, P. D. (2007). Distributed cognition and the task of science. Social Studies of Science, 37(2), 297-310.

Nersessian, N. J. (2006, December). Model-based reasoning in distributed cognitive systems. Philosophy of Science, pp. 699-709.

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010, 29 October). Evidence for a collective intelligence factor in the performance of human groups. Science, pp. 686-688.

Woolley, A. W., & Fuchs, E. (2011, September-October). Collective intelligence in the organization of science. Organization Science, pp. 1359-1367.

Comment on Kerrey and Leeds in WSJ

November 20, 2013

Writing in today’s Wall Street Journal, Bob Kerrey and Jeffery T. Leeds note the unintended consequences likely to follow from new higher education regulations proposed by the U.S. Department of Education. Cutting to the chase, Kerrey and Leeds’ key points (emphases added) are that:

  • “Absent innovative, competitive—and, yes, disruptive—pressure to raise quality and lower costs, all the well-intentioned federal regulation in the world will not make college more accessible.”
  • “He [Secretary of Education, Arne Duncan] should insist on real and significant disclosure. Colleges should be required to post their graduation rates, job-placement rates, the average debt of their students upon graduation, their tax status and any and all information that will enable Americans to make informed decisions when choosing a school.”
  • “The department should also work with schools and colleges to address the fundamental causes of rising tuition, and hold schools accountable for student outcomes instead of their debt.”

These are, of course, exactly the themes repeatedly raised in this blog. Measurement quality is unavoidably implicated in holding schools accountable for student outcomes, in enabling consumers to make informed purchasing decisions, and in raising quality and lowering costs.

To meet the challenges we face, measurement quality must be far more than just a matter of precision and rigor. Quality must also speak to relevance, efficiency, and meaningfulness. Recent history has brought home the lesson that annual tests used solely for accountability purposes will not enable rebalanced quality/cost equations, informed consumer decisions, or fair accountability results. But how might these disparate purposes be efficiently and meaningfully realized?

It is essential that, if teachers are to be responsible for student outcomes and for raising the overall quality of education, formative measuring tools must provide the qualitative and quantitative information they need to be able to act responsibly. The irony is, of course, that the way to overcome the problems of a purely summative focus for educational measurement is to measure more! Now, measuring more need not involve devoting more time exclusively to taking tests. Instead, computerized and online assessments are increasingly integrated into instruction so that measures are made in the course of studying (Cheng and Mok, 2007; Wilson, 2004). Measures are thereby continuously updated, and are plotted in growth charts relative to long range outcome goals.

Furthermore, the qualitative information provided by the measurement process is used to inform teachers and students about what comes next in the individualized curriculum, as well as about special strengths and weaknesses. This information has been shown to be unparalleled in its value for advancing learning in the classroom (Black and Wiliam, 1998, 2009; Hattie, 2008).

But formative assessment alone will not be sufficient to the larger tasks of raising quality and lowering costs. For that, systematic quality improvement methods in schools will need to be joined with comparable outcome measures parents and students can use to inform school choice decisions (Fisher, 2013; Lunenberg, 2010).

Kerrey and Leeds rightly seek an infrastructure capable of disruptive effects, of transforming the inflationary economy of education (and health care). To state again a recurring theme in this blog, the command and control hierarchies of regulatory systems can and should be replaced with a metrological infrastructure of common metrics with the scientific, legal, and financial status of common currencies for the exchange of value. Only when such currencies are in place will we be able to set out clear paths for the informed decisions, improved quality, lower costs, and accountability for outcomes that we seek.

References

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74.

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21, 5-31.

Cheng, Y. C., & Mok, M. M. C. (2007). School-based management and paradigm shift in education: An empirical study. International Journal of Educational Management, 21(6), 517-542.

Fisher, W. P., Jr. (2013). Imagining education tailored to assessment as, for, and of learning: Theory, standards, and quality improvement. Assessment and Learning, 2, in press.

Hattie, J. (2008). Visible learning. New York: Routledge.

Lunenberg, F. C. (2010). Total Quality Management applied to schools. Schooling, 1(1), 1-6.

Wilson, M. (Ed.). (2004). Towards coherence between classroom assessment and accountability. (Vol. 103, Part II, National Society for the Study of Education Yearbooks). Chicago, Illinois: University of Chicago Press.

Survey: Technology Roadmap for Education

June 12, 2013

To whom it may concern:

The methods and techniques used in measuring reading ability in education have evolved as the practical advantages of mathematical modeling and advanced computational resources have become more widely implemented. Tests and assessments have become more complex and resources have shifted from analog atoms to digital bytes, as they have in so many other areas of life. However, not much information is available about what methods are being used, how fast new methods are coming online, or on what the associated costs are. To begin to formulate some answers to questions like these, MetaMetrics, Inc. has sponsored a “Technology Roadmap for Education” survey.

This survey was developed on the basis of ideas presented in 2011 at a meeting of the International Measurement Confederation (IMEKO) in Jena, Germany, and in 2013 at the National Council on Measurement in Education annual meeting in San Francisco, California. The survey items are suggested by Table 1 in the paper available at http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf.

We welcome comments and feedback.

This survey is sponsored by MetaMetrics, Inc., which is solely responsible for its content. MetaMetrics is a research firm in the education industry; it could be perceived that there is a conflict of interest in their support of this research, as its business interests would be served by knowledge of the results.  To level the playing field, information collected will be summarized in a publicly available report which can be used to guide future research and development in other companies and organizations, in addition to MetaMetrics.

The survey data are not individually identifiable, but all responses will be treated as confidential and will be reported only in aggregate form.

This study has been reviewed  by the MetaMetrics IRB, which determined that this project does not constitute “research with human subjects” as defined under federal regulations because the data will not be individually identifiable [45 CFR 46.102(f)].

The survey is in English, and should take approximately 20-40 minutes. Click here to take the survey, or cut and paste this web address:
https://www.surveymonkey.com/s/TechRoadMapEducation .

We would particularly like to hear from anyone who:

  • ·  has extensive experience managing educational testing and measurement projects; and
  • ·  has detailed knowledge of past and present educational measurement item development costs and technologies.

Please complete the survey by June 30, as we intend to submit a summary of the results by July 22 for presentation at the 2014 AERA meeting in Philadelphia.  If you would like a complimentary report summarizing the survey findings, please let me know. All responses are confidential.

Collegial regards,

William P. Fisher, Jr., Ph.D.

LivingCapitalMetrics.com

Consultant to MetaMetrics, Inc.

wfisher@livingcapitalmetrics.com

What the Economy Needs?

September 5, 2012

Expanding on remarks made by Thomas Friedman in the course of an interview with Charlie Rose broadcast on August 31, 2012…

Friedman broke the problem down to three key points. We have to have 1) a plan, 2) a fair tax contribution from the rich, and 3) aspirations for improving the overall quality of life, economically and  democratically.

The plan outlined from various points of view in this blog is to create a scientific and market infrastructure for intangible assets (human, social and natural capital), assets amounting to at least 90%of the capital under management.

The plan is fair in its advancement of equal opportunity to invest in and realize returns from one’s skills, motivations, health and trustworthiness. Everyone will be able to invest in, and receive their share of the profits from, the human, social, and natural capital stocks of individuals, communities, schools, hospitals, social service agencies, firms, etc. The rich will then both contribute to the advancement of the greater good at the same time they are able to profit from the growth in the authentic wealth created by improvements to human, community, and environmental value.

The plan aspires to great accomplishments in the depth and breadth of the innovation it will facilitate, its fulfillment of democratic principles, and the new economic growth it promises.

And so I would now like to raise a couple of sets of questions. What if all the money put into Medicare, Medicaid, education, HUD, food stamps, the EPA, etc. was instead invested in an infrastructure for intangible assets metrology and HSN capital stocks (individual, organizational–school, hospital, nonprofit, NGO, firm–and community)? Usually, talk of letting the market solve social and environmental problems is nothing but a self-serving excuse for allowing greed to rule at the expense of the greater good. Those so-called market solutions do nothing to actually shape the institutions, rules, and roles by which markets are created, and so the end result would be catastrophic. But there is an essential and unnoticed inconsistency in previously proposed approaches that involves the double standards used in defining and actualizing the various forms of capital.

As previous posts (like this one or this one) in this blog, and several of my publications, have argued, manufactured capital and property have long since been brought to life by transferable representations (titles, deeds, precision quantity measures, etc.) and the various legal, financial, educational, and scientific institutions built up around them. Human, social, and natural capital have not been brought to life and so we remain unable to take proper possession of our own properties, the ones that we most value and on which life, liberty, and happiness are most dependent.

But what if we created the needed market institutions, rules, and roles? What if everyone knew how many shares of community capital they owned, and what the current price of those shares in the market was? What if tuition for an advanced degree was denominated in the shares of literacy capital one obtained, as evident in the increased literacy measures achieved? What if taxes were abolished and minimum investments in human, social, and natural capital stocks were required? What if real, efficient, functional markets in intangible assets were created, and the associated governmental programs and departments were abolished? How much would the federal budget decrease? How much would government shrink? How much might the economy grow if that much money was invested in human, social, and natural capital stocks paying even a minimal reasonable profit?

Another round of questions asks whether we have the optimal social safety net in the current institutional context, or if perhaps that safety net could be significantly improved by following through on the concepts of impact investing and outcome-based budgeting to create a truly sustainable and socially responsible economic system? What if everyone held known numbers of tradable shares of their intangible assets (their skills, motivation, health, trust)? What if the value of those shares was common public knowledge? What if the investment paths to increasing the number and value of shares held were all well known? What if monetary profit could be derived–and could only be derived–by increasing the value of human, social, and natural capital shares? What if groups of people joined together in various kinds of organizations (schools, hospitals, businesses) to collectively grow the value of their authentic wealth? What if lean thinking was applied to the 90% of the capital under management (the human, social, and natural capital) that is currently nearly unmanageable because it is not measured in universally uniform scientific units?

The balance scale is a common symbol of justice. We do not usually aspire to take that symbol as seriously as we could. We ought to have a plan for economic justice that does not have to coerce anyone to acknowledge, pay back, and re-invest in the broad support they received en route to becoming successful. And we ought to have a plan that reinvigorates the aspirations for equal opportunity and freedom that have become a model for people all over the world. Friedman got the broad strokes right. Now’s the time to start filling in the details.

Reimagining Capitalism Again, Part III: Reflections on Greider’s “Bold Ideas” in The Nation

September 10, 2011

And so, The Nation’s “Bold Ideas for a New Economy” is disappointing for not doing more to start from the beginning identified by its own writer, William Greider. The soul of capitalism needs to be celebrated and nourished, if we are to make our economy “less destructive and domineering,” and “more focused on what people really need for fulfilling lives.” The only real alternative to celebrating and nourishing the soul of capitalism is to kill it, in the manner of the Soviet Union’s failed experiments in socialism and communism.

The article speaks the truth, though, when it says there is no point in trying to persuade the powers that be to make the needed changes. Republicans see the market as it exists as a one-size-fits-all economic panacea, when all it can accomplish in its current incomplete state is the continuing externalization of anything and everything important about human, social, and environmental decency. For their part, Democrats do indeed “insist that regulation will somehow fix whatever is broken,” in an ever-expanding socialistic micromanagement of every possible exception to the rules that emerges.

To date, the president’s efforts at a nonpartisan third way amount only to vacillations between these opposing poles. The leadership that is needed, however, is something else altogether. Yes, as The Nation article says, capitalism needs to be made to serve the interests of society, and this will require deep structural change, not just new policies. But none of the contributors of the “bold ideas” presented propose deep structural changes of a kind that actually gets at the soul of capitalism. All of the suggestions are ultimately just new policies tweaking superficial aspects of the economy in mechanical, static, and very limited ways.

The article calls for “Democratizing reforms that will compel business and finance to share decision-making and distribute rewards more fairly.” It says the vision has different names but “the essence is a fundamental redistribution of power and money.” But corporate distortions of liability law, the introduction of boardroom watchdogs, and a tax on financial speculation do not by any stretch of the imagination address the root causes of social and environmental irresponsibility in business. They “sound like obscure technical fixes” because that’s what they are. The same thing goes for low-cost lending from public banks, the double or triple bottom lines of Benefit Corporations, new anti-trust laws, calls for “open information” policies, added personal stakes for big-time CEOs, employee ownership plans, the elimination of tax subsidies for, new standards for sound investing, new measures of GDP, and government guarantees of full employment.

All of these proposals sound like what ought to be the effects and outcomes of efforts addressing the root causes of capitalisms’ shortcomings. Instead, they are band aids applied to scratched fingers and arms when multiple by-pass surgery is called for. That is, what we need is to understand how to bring the spirit of capitalism to life in the new domains of human, social, and environmental interests, but what we’re getting are nothing but more of the same piecemeal ways of moving around the deck chairs on the Titanic.

There is some truth in the assertion that what really needs reinventing is our moral and spiritual imagination. As someone (Einstein or Edison?) is supposed to have put it, originality is simply a matter of having a source for an analogy no one else has considered. Ironically, the best model is often the one most taken for granted and nearest to hand. Such is the case with the two-sided scientific and economic effects of standardized units of measurement. The fundamental moral aspect here is nothing other than the Golden Rule, independently derived and offered in cultures throughout history, globally. Individualized social measurement is nothing if not a matter of determining whether others are being treated in the way you yourself would want to be treated.

And so, yes, to stress the major point of agreement with The Nation, “the new politics does not start in Washington.” Historically, at their best, governments work to keep pace with the social and technical innovations introduced by their peoples. Margaret Mead said it well a long time ago when she asserted that small groups of committed citizens are the only sources of real social change.

Not to be just one of many “advocates with bold imaginations” who wind up marginalized by the constraints of status quo politics, I claim my personal role in imagining a new economic future by tapping as deeply as I can into the positive, pre-existing structures needed for a transition into a new democratic capitalism. We learn through what we already know. Standards are well established as essential to commerce and innovation, but 90% of the capital under management in our economy—the human, social, and natural capital—lacks the standards needed for optimal market efficiency and effectiveness. An intangible assets metric system will be a vitally important way in which we extend what is right and good in the world today into new domains.

To conclude, what sets this proposal apart from those offered by The Nation and its readers hinges on our common agreement that “the most threatening challenge to capitalism is arguably the finite carrying capacity of the natural world.” The bold ideas proposed by The Nation’s readers respond to this challenge in ways that share an important feature in common: people have to understand the message and act on it. That fact dooms all of these ideas from the start. If we have to articulate and communicate a message that people then have to act on, we remain a part of the problem and not part of the solution.

As I argue in my “The Problem is the Problem” blog post of some months ago, this way of defining problems is itself the problem. That is, we can no longer think of ourselves as separate from the challenges we face. If we think we are not all implicated through and through as participants in the construction and maintenance of the problem, then we have not understood it. The bold ideas offered to date are all responses to the state of a broken system that seek to reform one or another element in the system when what we need is a whole new system.

What we need is a system that so fully embodies nature’s own ecological wisdom that the medium becomes the message. When the ground rules for economic success are put in place such that it is impossible to earn a profit without increasing stocks of human, social, and natural capital, there will be no need to spell out the details of a microregulatory structure of controlling new anti-trust laws, “open information” policies, personal stakes for big-time CEOs, employee ownership plans, the elimination of tax subsidies, etc. What we need is precisely what Greider reported from Innovest in his book: reliable, high quality information that makes human, social, and environmental issues matter financially. Situated in a context like that described by Bernstein in his 2004 The Birth of Plenty, with the relevant property rights, rule of law, scientific rationality, capital markets, and communications networks in place, it will be impossible to stop a new economic expansion of historic proportions.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.