Archive for September, 2009

Just posted on www.economist.com in response to Sept 26 Schumpeter article

September 29, 2009

Let’s cut through the Gordian Knot to the real issue. That we manage what we measure is as close to an absolute truth as there ever was. What got us into this mess was the inadequacy of the vast majority of our measures. So-called “measures” that only get in the way of management are a sign that new standards, criteria, and methods of measurement are needed. The core issue we face is how to transform socialized externalities into capitalized internalities. Transaction costs are the most important and largest costs in any economic exchange. We reduce and control these via measurement. Human, social, and natural capital transaction costs are virtually uncontrolled and unmeasured. We need a metric system for universally uniform measures of abilities and skills, health, motivation, loyalty and trust, and environmental quality. And we needed it yesterday. But who is working on it? Who is talking about it? Most importantly, who is taking advantage of the huge strides that have been made in measurement science over the last 50 years, strides that have made measurement far more rigorous, practical, and flexible than anyone in business seems to know. As to business being an art, so is music, but music is played on and reproduced by some of the highest technology and finest precision instrumentation around. What we need to do is tune the instruments of the management arts and sciences so that we can harmonize our relationships, get with the beat, and sing the melodies we feel in our hearts and souls. For more information, see http://www.livingcapitalmetrics.com, or my blog at https://livingcapitalmetrics.wordpress.com.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.

Three demands of meaningful measurement

September 28, 2009

The core issue in measurement is meaningfulness. There are three major aspects of meaningfulness to take into account in measurement. These have to do with the constancy of the unit, interpreting the size of differences in measures, and evaluating the coherence of the units and differences.

First, raw scores (counts of right answers or other events, sums of ratings, or rankings) do not stand for anything that adds up the way they do (see previous blogs for more on this). Any given raw score unit can be 4-5 times larger than another, depending on where they fall in the range. Meaningful measurement demands a constant unit. Instrument scaling methods provide it.

Second, meaningful measurement requires that we be able to say just what any quantitative amount of difference is supposed to represent. What does a difference between two measures stand for in the way of what is and isn’t done at those two levels? Is the difference within the range of error, and so random? Is the difference many times more than the error, and so repeatedly reproducible and constant? Meaningful measurement demands that we be able to make reliable distinctions.

Third, meaningful measurement demands that the items work together to measure the same thing. If reliable distinctions can be made between measures, what is the one thing that all of the items tap into? If the data exhibit a consistency that is shared across items and across persons, what is the nature of that consistency? Meaningful measurement posits a model of what data must look like to be interpretable and coherent, and then it evaluates data in light of that model.

When a constant unit is in hand, when the limits of randomness relative to stable differences are known, and when individual responses are consistent with one another, then, and only then, is measurement meaningful. Inconstant units, unknown amounts of random variation, and inconsistent data can never amount to the science we need for understanding and managing skills, abilities, health, motivations, social bonds, and environmental quality.

Managing our investments in human, social, and natural capital for positive returns demands that meaningful measurement be universalized in uniformly calibrated and accessible metrics. Scientifically rigorous, practical, and convenient methods for setting reference standards and making instruments traceable to them are readily available.

We have the means in hand for effecting order-of-magnitude improvements in the meaningfulness of the measures used in education, health care, human and environmental resource management, etc. It’s time we got to work on it.

We are what we measure. It’s time we measured what we want to be.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.

Response to John Carney’s comments on Sept 25th’s Marketplace with Kai Ryssdal on NPR

September 25, 2009

Way to go, John Carney!! You’re my new hero!

Finally someone has said it right out loud: Even after a year in the economic trough, we are far from a consensus on what went wrong. Everyone is still fighting over the core problem, and so it is impossible to formulate a way forward.

Yes, it has been incredibly frustrating to watch everyone go round and round the central issue without ever really seeing it. Talk about the blind men and the elephant!

The first part of what needs to be done is on the table.  There does seem to be a fair degree of consensus on the idea that, somehow or other, we need to bring ALL forms of capital–human, social, and natural–into the econometric models, the financial spreadsheets, and the Genuine Progress Indicators or Happiness Indexes, so that the profit motive can be harnessed in sustainable, socially responsible ways to build communities, the environment, and human potential. As some have said, we need to transform socialized externalities into capitalized internalities.

I am hardly the first to suggest that. But what has been missing in previous proposals was the means by which we would devise transparent, fungible representations of each significant form of capital. How do we create common currencies for trading in each form of capital? By calibrating the instruments and deploying the reference standard common metrics we will use as those currencies. MEASUREMENT is the core problem. “You manage what you measure” is repeated like a mantra everywhere, but the quality of our measures of risk, opportunity, outcomes, and impacts–in short, of all measures of intangible forms of capital (human, social, intellectual, etc.)–is terrible!

You would never know it from most current measurement practice in business and government, but huge advances have been made over the last 50 years in scaling technology. The mathematical rigor, meaningfulness, practicality, and convenience of measures based in ratings, ability tests, surveys, and assessments could be an order of magnitude better if we just took advantage of existing technologies. We need universally uniform measures of health, skills and abilities, motivation, community life, governance, risk, and environmental quality akin to the measures of weight, volume, time, kilowatts, etc. that we take for granted in economic exchanges everyday.

It is essential to realize that universal uniformity in no way requires universal acceptance of exactly the same instruments, observations, content, questions, items, etc. Any brand tool that verifiably produces measures of the desired outcome, impact, process, etc. in the reference standard metric can compete in the measurement market, just as is the case with clocks or thermometers. Further, far from reducing rich complexities to a meaningless number, new measurement technologies open up new horizons for meaningful relationships by improving our understandings of ourselves and others.

The way forward centers on building a scientifically rigorous metric system that will make our stocks of human, social, and natural capital fungible. We need universally uniform metrics for the outcomes and impacts of schools, hospitals, social services, human, organizational, and natural resources, etc. When you appreciate the extent to which any economy lives and dies on its measurement standards, the need for national and global investments in quantitatively rigorous and qualitatively meaningful metrics becomes all too painfully obvious.

For more information, see https://livingcapitalmetrics.wordpress.com, http://www.livingcapitalmetrics.com, http://www.linkedin.com/in/livingcapitalmetrics, and other sources, such as the Wikipedia entry on Georg Rasch, http://www.rasch.org, and elsewhere.

These comments were also posted at http://marketplace.publicradio.org/display/web/2009/09/25/pm-weekly-wrap-q/.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.

NIST Call for White Papers

September 22, 2009

As I’ve been preparing the statistics.com course and consulting on a couple of projects, it’s been difficult to make time for postings here. There’s no lack of things to say, that’s for sure! The following is an alert to an opportunity that should not be passed up….

NIST Call for White Papers

The National Institute for Standards and Technology has posted a new Call for White Papers (http://www.nist.gov/tip/call_for_white_papers_sept09.pdf) as part of its mission “to support, promote, and accelerate innovation in the United States through high-risk, high-reward research in areas of critical national need.”

The White Papers are NIST’s mechanism for collaborating with practitioners in the field in the development of new areas of research into fundamental measurement and metrological systems. NIST is specifically seeking out areas of measurement research that are not currently a priority with any federal funding agency and that have the potential for bringing about fundamental transformations in particular scientific areas.

As was evident in its celebration of World Metrology Day last May, NIST is well aware of the human, economic, and scientific value of technical standards. Mathematics becomes the language of science most fully when universally uniform common currencies provide a lingua franca for communicating experimental results, theoretical predictions, and for economic exchanges of quantitative value. When this truth is fully appreciated, it is obvious that metrological standards for human, social, and natural capital are an area of critical national need that could be highly rewarding. Given the decades of supporting research that are on the books, the risks of investing in this research are quite reasonable. This is especially so when considered relative to the rewards that could accrue from order-of-magnitude improvements in the meaningfulness, utility, and efficiency of measurement based in ordinal observations.

The Call for White Papers is not a funding opportunity but a chance to influence the substance of the areas to be focused on in future funding competitions. One might imagine that NIST would be very interested in supporting research exploring the potential for expanding any of a number of existing measurement systems and methodologies into publicly recognized reference standards.

Deadlines over the next year for White Papers are November 9, February 15, May 10, and July 12, though submissions will be accepted any time between November 9, 2009 and September 30, 2010.

A PDF of a White Paper that builds a case for Rasch-based metrological standards and that was submitted to NIST in its previous round is available at http://www.livingcapitalmetrics.com/images/FisherNISTWhitePaper2.pdf.

Further articulations of connections between Rasch measurement and the wider concerns of instruments traceable to reference standards within metrological networks are available in the following, among others:

Fisher, W. P., Jr. (1996, Winter). The Rasch alternative. Rasch Measurement Transactions, 9(4), 466-467 [http://www.rasch.org/rmt/rmt94.htm].

Fisher, W. P., Jr. (1997). Thurstone’s missed opportunity. Rasch20Measurement Transactions, 11(1), 554 [http://www.rasch.org/rmt/rmt111p.htm].

Fisher, W. P., Jr. (2000). Objectivity in psychosocial measurement: What, why, how. Journal of Outcome Measurement, 4(2), 527-563 [http://www.livingcapitalmetrics.com/images/WP_Fisher_Jr_2000.pdf].

Fisher, W. P., Jr. (2008). Vanishing tricks and intellectualist condescension: Measurement, metrology, and the advancement of science. Rasch Measurement Transactions, 21(3), 1118-1121 [http://www.rasch.org/rmt/rmt213c.htm].

Fisher, W. P., Jr. (2009, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement (Elsevier), 42(9), 1278-1287.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.