Archive for May, 2009

And Here It Is: The Next Major Technological Breakthrough

May 29, 2009

How It Will Transform Your Business and Your Life

We’ve all witnessed an amazing series of events in our lifetimes, and, hopefully, we’ve learned some important lessons over the years. In business, we’ve come to see that innovation is rarely the work of one person. When the crowd has the right tools and puts its mind to the task, nothing can stop it. We’re accordingly also learning the real truth of the fact that any firm’s greatest resource is its people—there is no more effective source of new efficiencies and whole new directions. Concern for social responsibility is no longer the exclusive domain of activists, since everyone is now attuned to the susceptibility of markets to unrestrained greed. And there are increasingly good reasons for thinking that perhaps we can reverse ongoing major environmental debacles and orient our systems to profits that are sustainable over the long term.

And in our personal lives, we’ve learned the vital importance of access to learning opportunities across the lifespan, access to health care, and caring relationships. Whether we call it spiritual or not, life is hardly worth living without a sense of wonder at the very existence of the universe and all the strange things inhabiting it.

We’ve learned a few things, then. Perhaps foremost among them is that we are going to have to adapt to the changes we ourselves bring about. And given the pace of change and the plain need to do better, we don’t hear anyone repeating Lord Kelvin’s famous opinion, from the end of the nineteenth century, that pretty much everything that can be discovered has been discovered. (Though isn’t there someone at Microsoft who could top the classics “No one will ever need more than 640k memory—or more than one browser tab”?) With everything that’s happened in the 100 years or so since Kelvin’s remark, one of the big lessons that has been learned is a certain humility, at least in that regard.

Change is in the air, that’s for sure, even if it doesn’t seem that there is any one particular form of it. But in fact there is an important new technology coming on line. It isn’t really new. Viewed narrowly, it has been taking shape for over 80 years, even though its root mathematical principles go back to Plato (like so many do). And, at least in retrospect, this new technology’s major features may seem very humdrum and mundane, they are so everyday.

So just what is going on? Speaking in Abu Dhabi on Monday, May 25, Nobel economist Paul Krugman suggested that economic recovery could come about in the wake of a new major technological breakthrough, one of the size and scope of the IT revolution of the 1990s. Other factors cited by Krugman as candidates for turning things around included more investment by major corporations, and new climate change regulations and policies.

Industry-wide systems of metrological reference standards for human, social, and natural capital fit the bill. They are a new technological breakthrough on the scale of the initial IT revolution. They would also be a natural outgrowth of existing IT systems and an extension of existing global trade standards. Such systems would also require large investments from major corporations, and would facilitate highly significant moves on climate change.

In addition, stepping beyond the solutions suggested by Krugman, systematic and objective methods of measuring living capital would help meet the widely recognized need for socially responsible and sustainable business practices. Better measurement will play a vital role in reducing transaction costs and making human, social, and natural capital markets more efficient. It will also be essential to fostering new forms of innovation, as the shared standards and common product definitions made possible by advanced measurement systems enable people to think and act together collectively in common languages.

Striking advances have been made in measurement practice in recent years. It is easy to assume that the assignment of numbers to observations suffices as measurement, and that there have been no developments worthy of note in measurement theory or practice for decades. Nothing could be further from the truth. You don’t know the first thing about what you don’t know about measurement.

I came into the study and use of mathematically rigorous measurement and instrument calibration methods from the history and philosophy of science. The principles that make rulers, weight scales, clocks, and thermometers as meaningful, convenient and practical as they are, and that drive engineering practices in high tech, for instance, are pretty well understood. What’s more, those principles have been successfully applied to tests, rating scales, and assessments for decades, primarily in high stakes graduation, admissions, and certification/licensure testing. Increasingly these principles are finding their way into health care and business.

The general public doesn’t know much about all of this because the math is pretty intense, the software is hard to use, and we have an ingrained cultural prejudice that says all we have to do is come up with numbers of some kind, and–voila!– we have measurement. Nothing could be further from the truth.

My goal in all of this is to figure out how to put tools that work in the hands of the people who need them. You don’t need a PhD in thermodynamics to read a thermometer, so we ought to be able to calibrate similar instruments for other things we want to measure. And the way transparency and accountability demands are converging with economics and technology, I think the time is ripe for new ideas properly presented.

A quick way to see the point is to recognize that fair and just measures have to represent something that adds up the way the numbers do. Numbers don’t just automatically do that. We invest huge resources in crafting good instruments in the natural sciences, but we assume anyone at all can put together a measure using counts of right answers or sums of ratings or percents of the time some event occurs. But none of these are measures. Numbers certainly always add up in the same way, but whether they are meaningful or not is a question that is rarely asked. The numbers we often take as measures of outcomes or results or processes almost never stand for something that adds up the way everyone thinks they do.

So, yes, I know we need metrics that are manageable, understandable, and relevant. And I know how quickly people’s eyes glaze over in face of what they think are irrelevant technicalities. But eyes also tend to glaze over when something unexpected and completely different is offered. True originality is not easily categorized or recognized for what it is. And when something is fundamentally different from what people are used to, it can be rejected just because it is more trouble to to make the transition to a new system than it is to remain with the existing system, no matter how dysfunctional it is.

And boy is the current way of developing and deploying business metrics dysfunctional! Do you know that the difference between 1 percent and 2 percent can represent 4-8 times the difference between 49 percent and 50 percent? Did you know that sometimes a 15% difference can stand for as much as or even a lot more than a 39% difference? Did you know that three markedly different percentage values—differences that vary by more than a standard error or even five—might actually stand for the same measured amount?

In my 25 years of experience in measurement, people often turn out to not understand what they think they understand. And they then also turn out to be amazed at what they learn when they take the trouble to put some time and care into crafting an instrument that really measures what they’re after.

For instance, did you know that there are mathematical ways of reducing data volume that not only involve no loss of information but that actually increase the amount of actionable value? Given the way we are swimming in seas of data that do not usually mean what we think they mean, being able to experimentally make sure things add up properly at the same time we reduce the volume of numbers we have to deal with seems to me to be an eminently practical aid to understanding and manageability.

Did you know that different sets of indicators or items can measure in a common metric? Or that a large bank of items can be adaptively administered, with the instrument individually tailored and customized for each respondent, organization, or situation, all without compromising the comparability of the measures?

These are highly practical things to be able to do. Markets live and die on shared product definitions and shared metrics. Innovation almost never happens as a result of one person’s efforts; it is almost always a result of activities coordinated through a network structured by a common language of reference standards. We are very far from having the markets and levels of innovation we need in large part because the quality of measurement in so many business applications is so poor. But that is going to change in very short order as those most banal of subjects, measurement and metrological systems, catch fire.

Advertisements

The Irony of the Situation

May 28, 2009

You can call it greed or the logical consequences of capitalist logic, but the truth of the matter is highly ironic. Any resolution of the economic situation that both effectively and efficiently prevents future excesses of the kind we’re suffering from now must inevitably extend the free play of market forces further into the domain of social capital. Far from socializing traditional capital, what must be accomplished is much more a kind of capitalization of the stock of social resources, such as trust, commitment, loyalty, and care.

Capitalizing social resources will require that they be represented in a way that makes them fungible. So how do we create a common currency that we can use as a medium of exchange for social capital? By calibrating measures of social capital and equating them to a shared metric, that’s how.

The process is not much different from what happened in Western Europe in the late 18th and early 19th centuries, when two major systems of measurement standards were devised and implemented. Social capital today is measured and exchanged today in units of measurement that vary markedly from time to time, place to place, and person to person, much as weights and measures in Medieval markets varied. Today’s unjust social capital market traders are just as unscrupulous as the Medieval merchants and nobles who took unfair advantage of the metric confusion in the markets of their day, buying with one set of measures and selling with another at their whim, depending on who the customer or taxpayer was.

What we need, of course, are publicly available and high quality expressions of the amounts of social capital possessed at any one time by any individual, organization, firm, or government. Scientifically rigorous methods of calibrating instruments providing precision measures of social capital have been available and in use for decades. Their commercial applications have been restricted to high stakes tests, for the most part, but they are increasingly used in research in health care and the social sciences.

We use the balance scale as a symbol for justice for the simple reason that it clearly represent the values of the Golden Rule. Far from trying to destroy opportunities for personal profit, what we need to do is figure out how to extend the long experience we already have in using the profit motive to build social capital. In actual fact, social capital has always been essential to making money, and there has almost always been some kind of positive social benefits from business profits, though they’re not always easy to find. What we need to do is focus on those benefits, and make them easier to find. So easy, in fact, that any socially responsible way of making money will produce them.

World Metrology Day (May 20)

May 27, 2009

World Metrology Day, Science and Commerce
How to Reinvigorate the Economy via Better Measurement

An Open Letter to the President’s
Economic Recovery Advisory Board

by

William P. Fisher, Jr.
Living Capital Metrics
5252 Annunciation Street
New Orleans, LA 70115
919.599.7245
William@livingcapitalmetrics.com
http://www.livingcapitalmetrics.com

“Measurements in Commerce: Metrology Underpinning Economic Development” is the name of the National Institute for Standards and Technology’s World Metrology Day educational event in Gaithersburg, MD, held on Wednesday, May 20. Similar events around the world celebrated the economic prosperity and scientific successes that have followed from the signing of the Metre Convention in 1875.

For those wondering what the noise is all about, there are two reasons why we need a World Metrology Day. The first one is what the speakers at the NIST educational event addressed. Despite the vitally important role measurement and technical standards play in the economy, we take them almost completely for granted. Their very invisibility indicates how well they are working, but also makes it important that the public be reminded about them from time to time.

The second reason why we need a World Metrology Day concerns the role measurement science can, ought, and ultimately must play in reinvigorating the economy, and in supporting green, socially responsible, and sustainable economic policies and practices. Better measurement is capable of enhancing the security of the existing economic pie, and in expanding both its size and the fairness with which it is divvied up. In order to understand how these expansions are coming about, we need to start from what metrology is and does in the first place.

Confidence in our rulers, weight scales, clocks, thermometers, volt meters, and so on—trust that they all read out the same value for the same amount measured—is what metrology gives us. As you might imagine, commerce and science were seriously impeded in those historical epochs when measures varied depending on who made them, who used them, or which instrument they were made with. Ensuring consistent price-value relationships, and the interoperability of various technologies, are highly significant ways in which metrological standards keep transaction costs low and lubricate the wheels of commerce.

The need for standardized product definitions makes metrology ubiquitous. Metrological standards are quite costly, as much as 20 percent of any nation’s GDP, making them much too expensive for any one business or industry to create for themselves. But those investments provide remarkably high returns, from 32% to over 400%, as shown by NIST studies. Small businesses benefit to an especially large degree from the efforts made by NIST and other standards groups around the world to ensure the smooth flow of products in global markets.

There is a human side to measurement, too. Beyond the market and the laboratory, fairness in measurement is a recurring theme in the Bible, the Torah, and the Q’uran, as well as in the Magna Carta and the constitutions of nations everywhere. The Golden Rule itself can be seen as demanding that the scales of justice be balanced in the hands of a judge blind to everything but the truth.

And so metrological standards not only provide cost-effective precision, they also embody our notions of fairness, justice, and right conduct. The French Revolution, for instance, very self-consciously understood the universal measures proclaimed in the metric system as symbolically representing the ideals of universal rights for all people.

These historical achievements provide us with a model for the future. How so? The current global crisis resounds with cries for accountability and transparency, with expanded human rights, social justice, and environmental quality. Activists and managers in every area, from education to health care to governance to philanthropy, deplore their metrics and wonder how to beg, buy, borrow, or steal better ones. Their needs are real, demand is huge, and, fortunately, the methods they need are readily available.

Demand for fair, universally uniform, comparable, and accessible measures of school, hospital, employment, community, and environmental quality sets the stage for a major expansion of the role of metrology and its effects on the economy. Standardized product definitions for tangible amounts of things sold by weight, volume, area, time, or kilowatts are essential to the efficiency and fairness of markets. They will be equally essential to the efficiency and fairness of human, social, and natural capital markets.

Some readers may at this point be wondering how measurement with the necessary mathematical rigor and scientific precision can be obtained for this purpose, if it can be obtained at all (see box, below). Although measurement in psychology and the social sciences is roundly disparaged by many unfamiliar with its technical achievements, it has come of age in recent decades. The wider world desperately needs to know more, both about the advances that have been made, and about what still needs to be done.

In a nutshell, tests, assessments, and surveys are routinely calibrated to be equivalent in principle with physical measures in their objectivity, mathematical rigor, practicality, and meaningfulness. The problem is not only that hardly anyone is aware this is being done; more importantly, even those doing the work are unaware of the need to create systems of metrological standards for each various form of human, social, and natural capital. The special value of being able to think together harmoniously, using instruments tuned to the same scale, is lost on those accustomed to dealing with one customer, student, or patient at a time, or with one test, survey, or data set at a time.

But being able to think together in a common language as consumers and producers is what makes a market efficient. Having different names for the same things, or the same names for different things, is confusing. When measuring units change in a variety of uncontrolled ways, communication is compromised and markets are bogged down in frictions that waste resources, add to costs, and can leave one or the other partner to a transaction feeling cheated.

Unfortunately, measures of the quality of schooling, health care, governance, environmental management, etc. almost always vary in uncontrolled ways. Assessment instruments are only rarely calibrated using the kinds of quality standards we take for granted when we weigh produce in the grocery store. And even when instruments are properly calibrated, and they increasingly are, they do not share a common metric.

In addition, because so few know that instruments can be calibrated and that they can be equated to a shared quantitative scale, we lack the systems, both vertical, within organizations, and horizontal, between them, through which the information might flow to each different place it is needed.

What we need today are

1. metrological systems designed to calibrate instruments, equate them, and maintain the equatings,

2. educational systems designed to inform researchers and students about the value of research designed to produce meaningful and stable measurement,

3. management systems designed to incorporate the metrics where they are needed, at every level and in every division, in production, quality improvement, accounting, marketing, human, social, and environmental resource management, governance, etc.; and

4. funding to support each of these areas of endeavor.

The funding is significant, both for the large amounts that will be needed, and for the quality of the investments to be made. Given the very large returns obtained from existing metrology systems, there will be intense interest in extending those systems to new domains. NIST plainly should have a new division focusing on measuring instruments and metrology systems for human, social, and natural capital. NIH, AHRQ, and other federal research arms should require all research proposals to address instrument calibration, how the resulting metrics will be maintained relative to reference standards, and how stakeholders’ various applications will employ it. Further down the road, new accounting standards will be needed for incorporating the new forms of capital into spreadsheets, and econometric models will include values for service products defined in common terms.

NIST’s World Metrology Day symposium emphasizes the the everyday and essential role that measurement science and standards play in virtually every economic transaction. New metrological horizons are upon us, however. The crisis we are currently experiencing is a prime opportunity for creating and investing in the infrastructure of new systems that will pay dividends for years to come.

———————————————————-
Box
Some may be suspicious of the claim that tests and surveys can measure with the same kind of objectivity as a clock or weight scale. Such suspicions are, however, easily overcome with a bit of history. Of particular interest are two electrical engineer/physicists turned psychologists and a mathematician, all of whom had close connections with the University of Chicago.

L. L. Thurstone was a former electrical engineer turned psychologist who became the first president of the Psychometric Society. Writing in 1928, while a Professor at the University of Chicago, Thurstone held that measurement was achieved only to the extent that an instrument behaving like a ruler could be calibrated. As Thurstone put it, if a ruler measured differently depending on whether it was a piece of paper or a rug that was being measured, then that instrument’s trustworthiness as a measuring device would be impaired. Thurstone, his colleagues, and his students made significant headway in constructing scales that lived up to this demanding criterion. His methods were deemed cumbersome by those less concerned with meaningfulness and science than with expeditious analytic productivity, however, so much of Thurstone’s best work was neglected as the years passed.

Further strides were made in the 1950s by Georg Rasch, a Danish mathematician who had studied with Ronald Fisher in London in 1934-5. Rasch was also strongly influenced by the Nobel economist, Ragnar Frisch, with whom he studied in Oslo. From Fisher, Rasch took an emphasis on statistical sufficiency, and from Frisch, the related understanding that generalizable results must be autonomous from the data representing them. The connections with Fisher and Frisch led Rasch to work with the Cowles Commission at the University of Chicago in 1947, where Rasch made the acquaintance of the statistician, Jimmy Savage. A few years later, Rasch showed decisively that, with the right test instrument and data, the reading ability of a child could be measured with the same kind of objectivity obtained in measuring her or his weight.

Rasch’s models have become widely used in high stakes and commercial testing globally, especially in advanced computerized examinations, largely due to the efforts of Benjamin D. Wright. Wright had worked under Nobel laureates Townes, Mulliken, and Feynman in a previous life as an electrical engineer and physicist. In 1960, as an Assistant Professor of Education at the University of Chicago, at the urging of his friend and colleague, Jimmy Savage, Wright hosted a seminar series given by Rasch. Wright felt that Rasch resolved some of his own dilemmas concerning the reconciliation of the scientific values he had learned in physics with the methods then popular in educational research. Wright went so far in adopting Rasch’s models as to develop improved methods for estimating their parameters, new statistics for evaluating data quality and instrument reliability, what was for many years the most advanced software for analyzing data, and professional societies and publications for sharing new contributions.

Toward the end of his career, Wright wrote that there is no methodological reason why measurement in education and psychology cannot be as stable, reproducible, and useful as measurement in physics. Over 50 years of experience with Rasch’s models, and 30 before that with Thurstone’s, provides evidence conclusively supporting Wright’s position.

Paul Krugman on new major technology needed to stimulate recovery

May 25, 2009

Speaking in Abu Dhabi on Monday, May 25, Nobel economist Paul Krugman suggested that economic recovery could come about in the wake of a new major technological breakthrough, one of the size and scope of the IT revolution of the 1990s. Other factors cited by Krugman as candidates for turning things around included more investment by major corporations, and new climate change regulations and policies.

Metrology systems for human, social, and natural capital fit the bill. They are a new technological breakthrough on the scale of the initial IT revolution. They would also be a natural outgrowth of existing IT systems and an extension of existing global trade standards. Such systems would also require large investments from major corporations, and would embody highly significant moves on climate change.

It is well known that precision measurement standards play a vital role in reducing transaction costs across a wide range of industries. What is less well known is the extent to which radically new degrees of precision and objectivity have become available for ratings data. The efficiency of human, social, and natural capital markets will be greatly enhanced by reducing the friction in individual exchanges of information.

The theory primarily involves calibrating instruments that express value in universally uniform metrics that function as common currencies. The new efficiencies come from the reduced friction in transactions, which are made meaningful and comparable via metrological networks not much different from the one connecting all the clocks. It appears that the intangibles of health care, education, social services, and human/natural resource management may not be forever doomed to locally dependent product definitions that defy pricing.

What we need are ways of extending the basic capitalist ethos into the domain of the intangibles. How can we set up  living capital markets so that the invisible hand efficiently promotes social and environmental ends unintended by individuals maximizing their own gains? How might we extend the free play of self-interest into more comprehensively determined returns for the global dividend?

Measurement will inevitably be of central concern in answering these questions. This is the kind of thing that, in previous epochs, was left to itself to evolve in its own time. We no longer have that luxury. Human suffering, sociopolitical discontent, and environmental degradation all demand that we take the bull by the horns and act deliberately, with purpose, now.

It is well known that precision measurement standards play a vital role in reducing transaction costs across a wide range of industries. What is less well known is the extent to which radically new degrees of precision and objectivity have become available for ratings data. The focus of my measurement research is, in effect, on enhancing the efficiency of human, social, and natural capital markets by reducing the friction in individual exchanges of information.

The theory primarily involves calibrating instruments that express value in universally uniform metrics that function as common currencies. The new efficiencies come from the reduced friction in transactions, which are made meaningful and comparable via metrological networks not much different from the one connecting all the clocks. It appears that the intangibles of health care, education, social services, and human/natural resource management may not be forever doomed to locally dependent product definitions that defy pricing.

What we need are ways of extending the basic capitalist ethos into the domain of the intangibles. How can we set up markets so that the invisible hand efficiently promotes social and environmental ends unintended by individuals maximizing their own gains? How might we extend the free play of self-interest into more comprehensively determined returns for the global dividend? Measurement will inevitably be of central concern in answering these questions.

Stimulating Excellence in Education

May 22, 2009

Comments on

Stimulating Excellence:

Unleashing the Power of Innovation in Education

published in May 2009 by

The Center for American Progress

The American Enterprise Institute

The Broad Foundation

New Profit Inc.

PublicImpact

and

The Annie E. Casey Foundation

Available at

http://www.americanprogress.org/issues/2009/05/entrepreneurs_event.html

Comments by

William P. Fisher, Jr., Ph.D.

7 May 2009

New Orleans

Submitted for publication to the Chronicle of Higher Education

I’ve just had a look at the new report, “Stimulating Excellence: Unleashing the Power of Innovation in Education.” I agree wholeheartedly with its focus on creating the conditions for entrepreneurial innovation and reward in education. I, too, deplore the lack of a quality improvement culture in education, and the general failure to recognize the vital importance of measuring performance for active management.

The specific recommendations concerning power metrics and drastic improvements in information quality really resonate with me. But two very significant shortcomings in the report demand attention.

First, only on page 34, in the report’s penultimate paragraph, do the authors briefly touch on what all educators know is absolutely the most important thing to understand about teaching and learning: it always starts from where the student is at, growing out of what is already known. This is doubly important in the context of the report’s focus, teaching and learning about how to institute a new culture of power metrics and innovation. To try to institute fundamental changes with little or no concern for what is already in place is a sure recipe for failure.

Second, there is one feature of the educational system as it currently exists that will be of particular value as we strive to improve the quality of the available information. That feature concerns tests and measurement. Many of the report’s recommendations would be quite different if its authors had integrated their entrepreneurial focus with the technical capacities of state-of-the-art educational measurement.

The obvious recommendation with which to start concerns the reason why public education in the United States is such a fragmented system: because outcome standards and product definitions are expressed (almost) entirely in terms of locally-determined content and expert opinion. Local content, standards, and opinions are essential, but to be meaningful, comparable, practical, and scientific they have to be brought into a common scale of comparison.

The technology for creating such scales is widely available. For over 40 years, commercial testing agencies, state departments of education, school districts, licensure and certification boards, and academic researchers have been developing and implementing stable metrics that transcend the local particulars of specific tests. The authors of the “Stimulating Excellence” report are right to stress the central importance of comparable measures in creating an entrepreneurial environment in education, but they did not do enough to identify existing measurement capabilities and how they could help create that environment.

For instance, all three of the recommendations made at the bottom of page 12 and top of page 13 address capabilities that are already in place in various states and districts around the country. The examples that come easiest to mind involve the Lexile Framework for Reading and Writing, and the Quantile Framework for Mathematics, developed by MetaMetrics, Inc., of Durham, NC (www.lexile.com).

The Lexile metric for reading ability and text readability unifies all major reading tests in a common scale, and is used to report measures for over 28 million students in all 50 states. Hundreds of publishers routinely obtain Lexile values for their texts, with over 115,000 books and 80 million articles (most available electronically) Lexiled to date.

Furthermore, though one would never know from reading the “Stimulating Excellence” report, materials on the MetaMetrics web site show that the report’s three recommendations concerning the maximization of data utility have already been recognized and acted on, since

*    many standardized assessments are already aligned with state learning standards,

*   available products already quickly incorporate assessment results into the process of teaching and learning (and a lot more quickly than “a day or two after testing”!), and

*   several states already have years of demonstrated commitment to keeping their standards and assessments relevant to the changing world’s demands on students.

That said, a larger issue concerns the need to create standards that remain invariant across local specifics. A national curriculum and national testing standards seem likely to fall into the trap of either dictating specific content or fostering continued fragmentation when states refuse to accept that content. But in the same way that computer-adaptive testing creates a unique examination for each examinee—without compromising comparability—so, too, must we invest resources in devising a national system of educational standards that both takes advantage of existing technical capabilities and sets the stage for improved educational outcomes.

That is what the report’s key recommendation ought to have been. An approximation of it comes on page 35, with the suggestion that now is the time for investment in what is referred to as “backbone platforms” like the Internet. Much more ought to have been said about this, and it should have been integrated with the previous recommendations, such as those concerning information quality and power metrics. For instance, on page 27, a recommendation is made to“build on the open-source concept.” Upon reading that, my immediate thought was that the authors were going to make an analogy with adaptively administered item banks, not literally recommend actual software implementation processes.

But they took the literal road and missed the analogical boat. That is, we ought to build on the open-source concept by creating what might be called crowd-sourced wikitests—exams that teachers and researchers everywhere can add to and draw from, with the qualification that the items work in practice to measure what they are supposed to measure, according to agreed-upon data quality and construct validity standards. This process would integrate local content standards with global construct standards in a universally uniform metric not much different from the reference standard units of comparison we take for granted in measuring time, temperature, distance, electrical current, or weight.

And this is where the real value of the “backbone platform” concept comes in. The Internet, like phones and faxes before it, and like alphabetic, phonetic and grammatical standards before them, provides the structure of common reference standards essential to communication and commerce. What we are evolving toward is a new level of complexity in the way we create the common unities of meaning through which we achieve varying degrees of mutual understanding and community.

In addition, measurement plays a fundamental role in the economy as the primary means of determining the relation of price to value. The never-ending spiral of increasing costs in education is surely deeply rooted in the lack of performance metrics and an improvement culture. We ought to take the global infrastructure of measurement standards as a model for what we need as a “backbone platform” in education. We ought to take the metaphor of transparency and the need for “clear metrics” much more literally. We really do need instruments that we can look right through, that bring the thing we want to see into focus, without having to be primarily concerned with which particular instrument it is we are using.

Finally, as no one will be surprised to learn, the existing measurement technology ignored in the “Stimulating Excellence” report is also unknown to those seeking to improve higher education, health care, social services, human resource management, government and governance, and environmental management. Decades of research show that the advances made in educational measurement transfer quite well from tests to surveys, rating scales, assessments, rankings, and checklists. A great deal needs to be done, and the challenges are huge, but taking them on will enable us to expand the domains in which we insist on fair dealing, and in which the balance scale applies as a symbol of justice.

When the entrepreneurial vision presented in the “Stimulating Excellence” report is situated in a context better informed by what educators are already doing and what they already know, the stage will be set for a new culture of performance improvement in education, a culture that explicitly articulates, tests, and acts on its educational values. At that point, we can expect great things!

What We Measure Matters

May 19, 2009

This comment was posted in reply to Chris Conley’s recent blog athttp://www.huffingtonpost.com/chip-conley/what-we-measure-matters_b_204676.html.

Right on, Chip!

The focus on meaning is essential. And unbeknownst to just about everyone but geeks like me, there is an extensive, longstanding, and mathematically rigorous scientific literature on meaningfulness in measurement.

We need to follow through from meaningful content to meaningful numbers, since survey and assessment ratings, scores, and response percentages are NOT measures in the everyday sense of what we mean when we deal with weight scales, clocks, thermometers, or rulers. That is, these numbers do not and cannot stand for something that adds up in the same way they do. The meaning of any given unit difference changes depending on where it falls in the measurement range, on who is measured, and/or on which item(s) are measuring.

For something we want to measure to be mapped onto a number line and to be truly and fully quantified, data have to have certain properties, like additivity, sufficiency, invariance, separable parameters, etc. When those properties are obtained, an instrument can be calibrated, data volume dramatically reduced, data quality assessed in terms of its internal consistency, and the measures made meaningfully interpretable.

Fortunately, scientific scaling methods have been applied in high stakes graduation, admissions, and professional certification/licensure testing for almost 40 years. Over the last 30 years, they have come to be applied in all kinds of survey research in health care and management consulting. Contact me for more information, see my web site at www.livingcapitalmetrics.com, or see www.rasch.org for full text articles.
More on Happiness
Read the Article at HuffingtonPost

The Ongoing Cultural Revolution

May 11, 2009

Jack Stenner sent a link to a recent piece by Volker Grassmuck at http://www.ip-watch.org/weblog/2009/05/11/the-world-is-going-flat-rate/#social, which includes the following comments:

VG: “It seems we are stumbling blindly into the future ahead of us, bumping against walls and into each other as we go along into the unfolding digital revolution. Our actions have more unintended and far-reaching consequences than we had thought, causing more collateral damage than good.”

WPF: Developmentally speaking, this is inevitable. It is the way in which we learn, and it is how we develop, in effect, eye-hand coordination. What we are up against right now is learning how to engage in this learning process safely, by bringing art and science into a context in which we can try things without hurting ourselves.

VG: “First of all, we need data, knowledge and understanding of the workings of the digital knowledge environment. It seems that we know more about the smallest particles and the largest galaxies than about ourselves as cultural animals. This requires systemic self-reflection and systematic research, developing a sensorium for the relevant factors and the dynamics in this space.”

WPF: A “sensorium”! Great term. What “systemic self-reflection and systematic research” amount to, in my terms, are an instrumentarium and associated metrological standards for each form of living human, social, and environmental capital. Instruments extend the senses, and so we could say that what we are doing is tuning the instruments of the human, social, and environmental sciences. Existing research is determining that the instruments are tunable, and that they can in principle be harmonized, but so far almost no one is yet concerned with setting up well-tempered scales as reference standards that all instruments can play in. The Lexile Framework for Reading (www.Lexile.com) is one exception.

VG: “…culture industries are as non-transparent as any industry. Disclosure and reporting requirements are needed beyond those that exist for public companies towards fiscal authorities, that the official statistics are based on. This should also include rules of access to data for scientific research so that public policy is not informed by industry’s self-reported numbers alone but also on independent scientific enquiry.”

WPF: Could not have said it better myself. Nothing to add.

VG: “Metadata are a key element for knowing and operating the knowledge environment. A registry of works and rights, ideally with rich metadata and fingerprints would be highly desirable. It is needed for trading rights, for knowing when a work comes into the public domain and for measuring and distributing levies, including the culture flat-rate. Parts are there…but none are comprehensive and they do not interoperate. This is a basic infrastructure. It should be shared and improved by all in the way free software is.”

WPF: Again, right on! See my LinkedIn page (www.linkedin.com/in/livingcapitalmetrics and my own web page (www.livingcapitalmetrics.com) for more on the metrics, the instruments, the scaling methods, and the research that has been going on in this domain for more than 50 years.

VG: “…whenever we decide upon in principle undecidable questions we are metaphysicians.”

WPF: Hence, my longstanding focus on issues of metaphysics and first philosophy. See my 2003 articles in Theory & Psychology, or for something shorter, seehttp://www.rasch.org/rmt/rmt184d.htm.

VG: “We cannot perceive ourselves to be apart from the knowledge environment, looking as through a peephole upon an unfolding universe. We are part of the knowledge environment. Whenever we act, we are changing ourselves and the universe as well. We are not citizens of an independent universe, whose regularities, rules and customs we may eventually discover, but participants of a conspiracy, whose customs, rules, and regulations we are now inventing.”

WPF: The shift to recognizing and accepting that we live in a participatory universe is a fundamental part of making our metaphysics explicit, as is elaborated in my Theory & Psychology articles.

VG: Mentions the UN Millenium Development Goals as an example of smart societal decision-making.

WPF: My analyses of the UN MDG data shows that the massively unmanageable data volume and the uninterpretable ordinal “metrics” (most people don’t even know they don’t add up) can be meaningfully scaled. The instruments are tuned so their measures harmonize, and data volume is reduced with no loss of information. Contact me for more information.

VG: “About us as homo sapiens we know (from Wikipedia) that what makes us special is “a highly developed brain, capable of abstract reasoning, language, introspection and problem-solving.” Globally networked computers give us the means for truly humanity-wide reasoning, introspection and problem solving. So let’s stick our brains together and work this out.”

WPF: Sticking our brains together requires that we use our global computer networks for distributed cognition. Though the network itself is in place, and the fundamentals of interactive communication are operable, we still do not have the common languages we need to think together. We need metrological standards and systems for tracing measures to reference standards. Instruments for measuring intangibles need to be calibrated to common metrics, not unlike the well- or equal-tempered musical scales, or time, temperature, distance, weight, kilowatts, etc.  We need to tune the instruments we need for arranging, orchestrating, and choreographing the beautiful music we could be making together. Similarly, markets for human, social, and natural capital, for literacy, numeracy, creativity, innovation, health, environmental quality, etc., need common currencies stable enough to trade on, and rule sets stable enough to guide decision making. Well said, Volker! You couldn’t have gotten much further than this in your articulation of the issues without expertise in measurement theory and practice.

Question of the day

May 6, 2009

The measurement of human, social, and natural capital has progressed to the point where we could have metrological reference standards for performance, ability, attitude, health, trust, and environmental quality metrics, if we cared to take the trouble to create them. My new article in Measurement (Elsevier) documents the science, and makes recommendations. (See http://dx.doi.org/10.1016/j.measurement.2009.03.014)

Having such standards would mean we’d be much better able to estimate the value of intangible forms of capital, to price them, and so to include them in econometric models and financial spreadsheets. Our investments in them would give measured returns, meaning we could include their value in stocks, bonds, other instruments, and in the Genuine Progress Index and the Happiness Index. Having this information would go a long way toward redefining profit as value for life.

In the days and weeks to come, I’ll spell out the technical issues involved. I’ll try to keep it jargon-free, and I hope you’ll help with that! Even without jargon, however, we’re moving into new territory, so new terms will need to be used. I’ll explain them the best I can, and I’ll also pay attention to new or special meanings of words that we might assume are already understood.

So here’s the first of many questions of the day:

Given the demonstrated viability of metrological reference standards for human, social, and natural capital, what would you say is the most important thing to do now to advance this cause?

I’ve been trying to answer this question from a fresh perspective everyday for years. It’s your turn.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.