Archive for the ‘measurement’ Category

Private Costs and Public Goods

December 4, 2017

Concerning the relation of private costs to public goods, I see two issues here that need to be spelled out in greater detail.

First, everyone is likely well aware, the concept of private property is unknown in many, most or all traditional cultures. Land in the sense of a bordered parcel carved out of the larger whole of interdependent relationships in an ecosystem is bizarrely dysfunctional and confused, from this point of view. The public goods of watershed, air purification, fishery, wildlife, etc. services cannot in any way be disentangled from the way private costs are thoroughly absorbed into the cyclical dynamics of symbiotic give and take, investment and return. Profit is defined entirely in terms of value for life. Overuse and misuse that imbalances relationships has tangible consequences that negatively impact quality of life.

Western culture broadened the scope of activity in ways that made it possible to expand the relationships in play beyond the constraints of local circumstances. Private costs incurred in cycles of investment and return could be sunk over longer time periods and across wider geographic ranges. Negative local consequences were balanced against positive returns accumulated elsewhere. Overall returns were negative as the resource base was destroyed, but the accounting methods in use and cultural values in play ignored this in favor of a narrower definition of personal profit that reinforced the extractive processes.

This system could continue only as long as new resources could be identified and converted to profit. As private costs increased and returns decreased new accounting methods and cultural values emerged and focused on rebalancing ecosystem interdependencies. Unfortunately, culturally ingrained habits of thought and institutionalized patterns of incentives and rewards often result in counterproductive conceptualizations of the situation. Too often, the linear destruction of the resource base is assumed to be the defining characteristic of a functional economy, and it is further assumed that this system must somehow be juxtaposed alongside or made externally congruent with ecosystems’ paradigmatically different circular interdependencies.

This, to me, is the background to your question about private costs and public goods. As long as we conceive and enact our relationships with social value and environmental services in terms of an either-or dichotomy, we are lost. But is it really true that our only alternatives are to subject externalities to extractive processes or to constrain those processes so as to allow ecosystem dynamics a wider scope of free reign at the expense of humanity? Should we think only in terms of (a) letting the current form of capitalism run rampant, (b) scaling back human activity to some kind of utopian low-tech, low-impact integration with a nature religion, or (c) the currently dominant assumption of a tense compromise between these two options of destroying or preserving the resource base?

Why are so few people talking about reconceiving economies as ecosystems of interdependent relationships that encompass not just human institutions but nonhuman forms of life and ecologies as well? Why should not we strive for an economy in which value for life is conceived as authentic wealth and market institutions make monetized profits contingent on nurturing genuine productivity? Why should not negative impacts on public goods be translated into private costs borne by individuals, communities, and businesses? Why should not everyone have the means to impact via their own behaviors, and to track in their own accounts, the quality and quantity of the personal stocks of shares in public goods they own? Why cannot we create legal and regulatory supports for entrepreneurs to be rewarded for wide commercial sales of their innovations reducing human suffering, social discontent, and environmental degradation? Why should not individuals be able to invest in privately owned shares of public goods in ways that efficiently move capital resources to where they are most effectively employed in the name of creating authentic wealth? How else could we ever amass the magnitude of focused effort it is going to take to rebalance the climate, get plastic microparticles out of the oceans, eliminate human suffering, and remove the sources of social discontent?

I am sure it seems counterintuitive but this problem is akin to a Chinese finger puzzle. The harder we pull, the more tightly trapped we become. We have to relax into the problem to be released from it. In jujitsu fashion, we have to use the energy of what we oppose to advance toward our goals. The profit motive is destructive because we have not integrated the genuine wealth we say we value into our accounting, financial, and economic systems. If we really do value human, social, and environmental riches over mere monetary riches as much as we say we do, why have we invested so little effort in finding qualitatively meaningful and mathematically rigorous ways of communicating, sharing, storing, and growing that value? Why haven’t we codified the legal structures and enforcement bodies that would monitor adherence to new institutional norms? Why don’t we have standards for the common languages and currencies we need if we are to be able to efficiently exchange meaningful expressions of real value?

If we would measure and manage investments in individual stocks of intangible assets the way we do for tangible assets, we could orchestrate a different balance of power. Efficient markets for investments in human, social, and natural capital would quickly channel flows away from businesses destroying genuine wealth toward those growing it. So efficient markets are not themselves the primary problem. The problem is that three of the four forms of capital comprising the economy have not been brought to life in the form of transferable representations serving as common currencies for the exchange of value. Just as land was once universally regarded a public good but came to be a privately owned source of costs and profits, so, too, will today’s public goods also be transformed.

There is a huge difference in the contexts of these transformations that must not be overlooked. Land became a private commodity in isolation with no system of checks and balances to constrain it. The earth appeared to be a bottomless resource well available for the taking. Issues of intangible market externalities were relevant only to the extent they negatively affected profits. Compassion for collateral damage was regarded either as foolish or as marketing opportunities for ostentatious public displays of prepackaged concern.

The transformation of the broader array of public goods into some form of privately owned and managed property is taking place in a qualitatively different context. Taking the trouble to articulate our values in ways that make us accountable for them will lead to a decisive “put up or shut up” moment for humanity. If people are as innately greedy and selfish as some think, the availability of legally binding and monetarily profitable accounting, financial, and economic systems for investing in and producing enhanced social value will mean nothing, and monetary profits will continue to be extracted, illegally, and perhaps at higher rates, only from privately owned tangible assets in such high demand that no one could get along without them.

On the other hand, a global movement spanning well more than half a century has been consistently seeking to devise ways of addressing these problems. Hundreds of millions of individuals, thousands of organizations, and hundreds of countries have conferred, invested, discussed, agreed, planned, and created across billions of ideas and possibilities. The will to do what needs to be done seems to me to exist in abundance. Those who think otherwise seem reconciled to the destructive scaling back of human activity, which only puts on display their own inability to think past today’s cultural assumptions to new expansions of the ecologically sound aspects of institutions of the past.

I think that we can indeed imagine ecosystems of interdependent relationships in which people will do what needs to be done not because it is the right or good thing to do but because they can see how it works to enrich the value obtained in their lives, for their families and communities. The ecosystem has to do the work of multiplying value indirectly into the larger community as an effect of each individual’s decisions and behaviors. This is not and cannot be a matter of individual concerns or intentions. The institutions have to support what Hayek called the true individual: not the falsely isolated and selfish individual but the authentic person whose identity and roles are shaped via relations of trust in thousands of unknown others, mediated by every exchange of information or value.

What we need to do is amplify and multiply the number and nature of these relations of trust. These amplifications and multiplications have been in process for decades in education, health care, social services, environmental resource management, and other areas. Technical advances in the quality of the information created and managed in these fields is quietly accumulating into new relationships between teachers and students, clinicians and patients, researchers and practitioners, consumers and producers, the social and natural sciences, markets and externalities, etc. The information transforming these relationships builds trust by revealing the day-to-day consistency and reliability of what people say and do in classrooms, clinics, and offices over time and across situations. New information systems show where students, teachers, clinicians, patients, managers, etc. stand relative to where they were in the past, relative to their goals, and relative to everyone else. The information is actionable in previously unavailable ways, as it shows what comes next in one’s self-defined learning trajectory or improvement goal sequence.

Technical issues concerning information complexity are being addressed to prevent the kinds of failures plaguing efforts in the past. Most importantly, in the manner of the advances in resilience and lean thinking informing manufacturing in recent decades, one of the consequences of increased information quality and the enhanced levels of trust obtained in the caring arts and sciences is that those on the front lines are empowered to act creatively in new ways. There is no reason to think that the innovations and advances that can be expected in these contexts will be less impressive or valuable than we have witnessed in technological areas in recent years. It seems highly likely that the inflationary spirals characteristic of these fields will be reversed into deflationary economies akin to that of microelectronics, where lower costs and higher quality drive increased profits.

———-

To express all this from another point of view, as I’ve said many times before, we keep assuming that our modern Cartesian methods are the only ways of defining problems and solutions, but the actual truth of the matter is that these methods are themselves the problem. As long as we keep assuming that solutions to our problems depend on finding the political or financial will to take them on, we will continue to fail. If we instead harness the energy of the profit motive to focus efforts productively in the direction of integrated solutions, we will successfully achieve our goals on a scale far exceeding what anyone so far has projected as possible or likely.

——-

Leveling the playing field and setting up equal opportunities for entry into a game played with the goal of keeping the game going.

Advertisements

Differences between today’s sustainability metrics and the ones needed for low cost social value transactions and efficient markets for intangible assets

November 16, 2017

Measurement is such a confusing topic! Everyone proclaims how important it is, but almost no one ever seeks out and implements the state of the art, despite the enormous advantages to be gained from doing so.

A key metric quality issue concerns the cumbersome and uninterpretable masses of data that well-intentioned people can hobble themselves with when they are interested in improving their business processes and outcomes. They focus on what they can easily count, and then they wrongly (at great but unrecognized cost) misinterpret the counts and percentages as measures.

For instance, today’s sustainability and social value indicators are each expressed in a different unit (dollars, hours, tons, joules, kilowatt hours, survey ratings, category percentages, etc.; see below for a sample list). Some of them may indeed be scientific measures of that individual aspect of the business. The problem is they are all being interpreted in an undefined and chaotic aggregate as a measure of something else (social value, sustainability, etc.). Technically speaking, if we want a scientific measure of that higher order construct, we need to model it, estimate it, calibrate it, and deploy it as a common language in a network of instruments all traceable to a common unit standard.

All of this is strictly parallel with what we do to make markets in bushels of corn, barrels of oil, and kilowatts of electricity. We don’t buy produce by count in the grocery store because unscrupulous merchants would charge the same amount for small fruits as for large. All of the scales in grocery store produce markets measure in the same unit, and all of the packages of food are similarly marked in standard units of weight and volume so we can compare prices and value.

There are a lot of advantages to taking the trouble to extend this system to social value. I suppose every one of these points could be a chapter in a book:

  • First, investing in scientific measurement reduces data volume to a tiny fraction of what we start with, not only with no loss of information but with the introduction of additional information telling us how confident we can be in the data and exactly what the data specifically mean (see below). That is, all the original information is recoverable from the calibrated measure, which is also qualified with an uncertainty range and a consistency statistic. Inconsistencies can be readily identified and acted on at individual levels.
  • Now the numbers represent something that adds up the way they do, instead of standing for the unknown, differing, and uncontrolled units used in the original counts and percentages.
  • We can take missing data into account, which means we can adapt the indicators used in different situations to specific circumstances without compromising comparability.
  • We know how to gauge the dependability of the data better, meaning that we will not be over-confident about unreliable data, and we won’t waste our time and resources obtaining data of greater precision than we actually need.
  • Furthermore, the indicators themselves are now scaled into a hierarchy that maps the continuum from low to high performance. This map points the way to improvement. The order of things on the scale shows what comes first and how more complex and difficult goals build on simpler and easier ones. The position of a measure on the scale shows what’s been accomplished, what remains to be done, and what to do next.
  • Finally, we have a single metric we can use to price value across the local particulars of individual providers. This is where it becomes possible to see who gives the most bang for the buck, to reward them, to scale up an expanded market for the product, and to monetize returns on investment.

The revolutionary network effects of efficient markets are produced by the common currencies for the exchange of value that emerge out of this context. Improvements rebalancing cost and quality foster deflationary economies that drive more profit from lower costs (think Moore’s law). We gain the efficiency of dramatic reductions in data volume, and the meaningfulness of numbers that stand for something substantively real in the world that we can act on. These combine to lower the cost of transactions, as it now becomes vastly less expensive to find out how much of the social good is available, and what quality it is. Instead of dozens or hundreds of indicators repeated for each company in an industry, and repeated for each division in each company, and all of these repeated for each year or quarter, we have access to all of that information properly contextualized in a succinct, meaningful, and interpretable format for different applications at individual, organizational, industry-wide, national, regional, or global levels of complexity.

That’s likely way too much to digest at once! But it seemed worth saying it all at once in once place, in case anyone might be motivated to get in touch or start efforts in this direction on their own.

Examples of the variety of units in a handy sustainability metrics spreadsheet can be found at the Hess web site (http://www.hess.com/sustainability/performance-data/key-sustainability-metrics): freshwater use in millions or thousands of cubic meters, solid waste and carbon emissions in thousands of tons, natural gas consumption in thousands of gigajoules, electricity consumption in thousands of kilowatt hours; employee union members, layoffs, and turnover as percentages; employee lost time incident rates in hundreds of thousands of hours worked, percentages of female or minority board members, dollars for business performance.

These indicators are chosen with good reasons for use within each specific area of interest. They comprise an intuitive observation model that has face validity. But this is only the start of the work that needs to be done to create the metrics we need if we are to radically multiply the efficiency of social value markets. For an example of how to work from today’s diverse arrays of social value indicators (where each one is presented in its own spreadsheet) toward more meaningful, adaptable, and precise measures, see:

Fisher, W. P., Jr. (2011). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com. Social Science Research Network: http://ssrn.com/abstract=1739386 .

Excellent articulation of the rationale for living capital metrics 

November 2, 2017

I just found the best analysis of today’s situation I’ve seen yet. And it explicitly articulates and substantiates all my reasons for doing the work I’m doing. Wonderful to have this independent source of validation.

The crux of the problem is spelled out at the end of the article, where the degree of polarizing opposition is so extreme that standards of truth and evidence are completely compromised. My point is that the fact will remain, however, that everyone still uses language, and language still requires certain connections between concepts, words, and things to function. Continuing to use language in everyday functions in ways that assume a common consensus on meaningful reference may eventually come to be unbearably inconsistent with the way language is used politically, creating a social vacuum that will be filled by a new language capable of restoring the balance of meaning in the word-concept-thing triangles.

As is repeatedly argued in this blog, my take is that what we are witnessing is language restructuring itself to incorporate new degrees of complexity at a general institutional, world historic level. The falsehoods of our contemporary institutional definitions of truth and fact are rooted in the insufficiencies of the decision making methods and tools widely used in education, health care, government, business, etc. The numbers called measures are identified using methods that almost universally ignore the gifts of self-organized meaning that offer themselves in the structure of test, assessment, survey, poll, and evaluation response data. Those shortcomings in our information infrastructure and communication systems are causing negative feedback loops of increasingly chaotic noise.

This is why it is so important that precision science is rooted in everyday language and thinking, per Nersessian’s (2002) treatment of Maxwell and Rasch’s (1960, pp. 110-115) adoption of Maxwell’s method of analogy (Fisher, 2010; Fisher & Stenner, 2013). The metric system (System International des Unites, or SI) is a natural language extension of intuitive and historical methods of bringing together words, concepts, and things, renamed instruments, theories, and data. A new SI for human, social, and natural capital built out into science and commerce will be one component of a multilevel and complex adaptive system that resolves today’s epistemic crisis by tapping deeper resources for the creation of meaning than are available in today’s institutions.

Everything is interrelated. The epistemic crisis will be resolved when our institutions base decisions not just on a potentially arbitrary collection of facts but on facts internally consistent enough to support instrument calibration and predictive theory. The facts have to be common sensical to everyday people, to employees, customers, teachers, students, patients, doctors, nurses, managers. People have to be able to see themselves and where they stand relative to their goals, their origins, and everyone else in the pictures drawn by the results of tests, surveys, and evaluations. That’s not possible in today’s systems. And in those systems, some people have systematically unfair advantages. That has to change, not through some kind of Brave New World hobbling of those with advantages but by leveling the playing field to allow everyone the same opportunities for self-improvement and the rewards that follow from it.

That’s it in a nutshell. Really good article:

America is facing an epistemic crisis – Vox

https://apple.news/A0alOElOQT5itYGPAJ3eYPQ

References

Fisher, W. P., Jr. (2010, June 13-16). Rasch, Maxwell’s method of analogy, and the Chicago tradition. In G. Cooper (Chair), Https://conference.cbs.dk/index.php/rasch/Rasch2010/paper/view/824. Probabilistic models for measurement in education, psychology, social science and health: Celebrating 50 years since the publication of Rasch’s Probabilistic Models, University of Copenhagen School of Business, FUHU Conference Centre, Copenhagen, Denmark.

Fisher, W. P., Jr. (2010). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr., & Stenner, A. J. (2013). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Nersessian, N. J. (2002). Maxwell and “the method of physical analogy”: Model-based reasoning, generic abstraction, and conceptual change. In D. Malament (Ed.), Reading natural philosophy: Essays in the history and philosophy of science and mathematics (pp. 129-166). Lasalle, Illinois: Open Court.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.

Thinking Systemically at the Global Level: Different Answers to Zelikow’s Three Strategic Questions

August 13, 2017

In a piece appearing in The Atlantic on 11 August 2017, Philip Zelikow asks, “Is the World Slouching Toward a Grave Systemic Crisis?” (https://apple.news/APcX6_AjzTXqTVDwU7Az78Q). He makes several well-stated observations that present a new opportunity for making the case that bringing human, social, and natural capital to life is strategically important to good public policy today.

Early in his essay, without calling it by name, Zelikow picks up on the important social phenomenon of self-organization, referred to by Lenoir (1997, p. 52) as “interactive system effects,” by Hayek (1948, p. 54) in terms of spontaneous economic order, by Magnus (2007) as distributed cognition, and, more popularly, by Surowiecki (2004) as the wisdom of crowds effect. As I’ve said repeatedly in this blog and in many publications (for instance, Fisher, 2007, 2009,2010, 2012a, 2012b; Fisher & Wilson, 2015; Fisher & Cavanagh, 2016), human, social, and natural capital will be brought to life when the tools and concepts we use for measuring and managing them are coordinated and aligned in terms of these kinds of bottom-up, autopoietic processes (Fisher, 2017).

So, in this context, Zelikow points out that

“The so-called ‘world order’ is really the accumulation of such local problem-solving. In this construct, power and persuasion comes mainly by example. Because people see what works—or what fails. Inspired or alarmed, they make their local choices, which accumulate.”

The accumulation that follows from local choices made when people see what works and what doesn’t is not a simple addition of effects. Instead, the social whole is greater than the sum of the parts. The point of the work cited above by Lenoir, Hayek, Magnus, and Surowiecki is that individuals do not design, implement, and impose their own visions so much as “interactive system effects” are projected via dynamics stemming from a capacity to think together in a common language. Shared symbol systems enable the coordination of decisions made by persons unknown to one another, creating an efficient information market.

Furthermore, when there is no clear evidence of what works and what does not, people still make choices that accumulate. The problem here is that what accumulates is indecision, confusion, and conflicting ideas based in local conditions and ideologies.  This, of course, is the state of things in education, health care, human resource management, social services, environmental management, etc., since we lack the information infrastructure we need to be able to coordinate cognitive ecosystems at the social level (Fisher & Stenner, 2017).

The viability, desirability, and feasibility of these kinds of cognitive ecosystems (Fisher, Oon, & Benson, 2017) leads to some specific answers to Zelikow’s concluding “checklist of three strategic questions:”

“1. Set priorities. What battleground issues or states are most likely to influence this generation’s global election about prospects for an open and civilized world?”

From where I sit, the most vitally important battleground issue obstructing our prospects for an open and civilized world concerns the schizophrenic disconnections that exist in our communications systems. Because of these disconnections, we fail to say what we mean and systematically confuse ourselves. Information infrastructures are designed as though language functions in a homogenous way from the highest to the lowest levels. Language does not function this way, and never has. Fortunately, we have the tools and methods we need to design information infrastructures that respect and adapt to the discontinuous shifts in context that currently defeat our best efforts at organizational transformation.

In a remarkably perceptive article, Star and Ruhleder (1996) describe the challenges we face in working through the inherent discontinuities embodied in the way language works. These challenges hinge on finding our way to information systems combining global standards and local customization. In our current world, we either impose irrelevant global standards from above, or allow local circumstances to foster widespread chaos. Similarly, Paul Ricoeur (1974, p. 166), the late French philosopher, observed that the social ethic we need demands resolution of a paradox, the seemingly irreconcilable opposition of human totality and human singularity. The potential for that resolution resides squarely in the accomplishments of research that has been underway throughout the course of the history of science, and that has culminated in recent years in theory, methods, evidence, and tools capable of meeting the need for improved communications.

Zelikow’s second strategic question is:

“2. Think outside-in. Out in those states, out in the world of those issues, are there catalytic possibilities? How do they see their situation? What (and who) are the critical variables in their choices?

Out in the world of those issues, catalytic possibilities for an open and civilized world are nowhere brighter than when approached from the perspective of research results emerging from psychology over the last 90 years, and from the history of science over the last 300 years (Mari & Wilson, 2015; Pendrill, 2014; Wright, 1997). The potential for a broad new consensus on the processes and outcomes of human development (Overton, 2015; Dawson, 2002) represents a possible basis for a shared self-understanding across global humanity that neither denies or homogenizes individual uniqueness, on the one hand, nor elevates the individual over society, on the other. The critical variables are presented in existing information systems that adaptively tailor information for individual relevance without relinquishing coherence with global standards (Barney & Fisher, 2016; Bergstrom, Lunz, & Gershon, 1994; Chien, Linacre, & Wang, 2011; Masters, 1994; Wilson, 2004). These systems provide models to follow across education, health care, government, social services, environmental resource management, etc.

Zelikow’s third question is:

“3. U.S. efficacy. In that context, where or how can the U.S. really make a strategic difference?”

A real strategic difference can be made by focusing the attention of policy makers, researchers, and practitioners in every field on creating coherent communications and information systems. The crux of the strategic issue is how to re-invent our culture, politics, and economics to focus on the creation of authentic wealth. The large number of works produced addressing this topic to date fail, however, to articulate the decisive issue: how to bring today’s lifeless expressions of human, social, and natural capital to life as fungible representations of genuine value in efficient markets (Fisher, 2007, 2010, 2011, 2012a).

The top immediate priority for an open and civilized world, then, ought to be establishing the property rights, scientific rationality, capital markets, and communications networks (Bernstein, 2004; Fisher, 2012b) needed to create viable cognitive and social ecosystems reconciling the apparent paradox of global and local in our information infrastructures. These technical demands must be complemented by associated articulations of the ethics, philosophy, and art of relational system effects if we are to realize humanity’s desire for fulfilling lives and communities.

References

Barney, M., & Fisher, W. P., Jr. (2016, April). Adaptive measurement and assessment. Annual Review of Organizational Psychology and Organizational Behavior, 3, 469-490.

Bernstein, W. J. (2004). The birth of plenty: How the prosperity of the modern world was created. New York: McGraw-Hill.

Chien, T.-W., Linacre, J. M., & Wang, W.-C. (2011). Examining student ability using KIDMAP fit statistics of Rasch analysis in Excel. In Communications in Computer and Information Science. Vol. 201: Advances in Information Technology and Education (pp. 578-585). Berlin: Springer Verlag.

Dawson, T. L. (2002). New tools, new insights: Kohlberg’s moral reasoning stages revisited. International Journal of Behavioral Development, 26(2), 154-66.

Fisher, W. P., Jr. (2007). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2010). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital., Bridge to Business Postdoctoral Certification, Freeman School of Business, Tulane University (p. http://ssrn.com/abstract=2340674).

Fisher, W. P., Jr. (2011). Bringing human, social, and natural capital to life: Practical consequences and opportunities. Journal of Applied Measurement, 12(1), 49-66.

Fisher, W. P., Jr. (2012a). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr. (2012b). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].

Fisher, W. P., Jr. (2017). A practical approach to modeling complex adaptive flows in psychology and social science. Procedia Computer Science, in press.

Fisher, W. P., Jr., & Cavanagh, R. (2016). Measurement as a medium for communication and social action, I & II. In Q. Zhang & H. H. Yang (Eds.), Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings (pp. 153-182). Berlin: Springer-Verlag.

Fisher, W. P., Jr., Oon, E. P.-T., & Benson, S. (2017). Applying Design Thinking to systemic problems in educational assessment information management. Journal of Physics Conference Series, in press.

Fisher, W. P., Jr., & Stenner, A. J. (2017). Ecologizing vs modernizing in measurement and metrology. Journal of Physics Conference Series, in press.

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo: Revista de Investigacion Educacional Latinoamericana, 52(2), 55-78.

Hayek, F. A. (1948). Individualism and economic order. Chicago: University of Chicago Press.

Lenoir, T. (Ed.). (1997). Instituting science: The cultural production of scientific disciplines (T. Lenoir & H. U. Gumbrecht, Eds.). Writing Science. Stanford, CA: Stanford University Press.

Lunz, M. E., Bergstrom, B. A., & Gershon, R. C. (1994). Computer adaptive testing. International Journal of Educational Research, 21(6), 623-634.

Magnus, P. D. (2007). Distributed cognition and the task of science. Social Studies of Science, 37(2), 297-310.

Mari, L., & Wilson, M. (2014). An introduction to the Rasch measurement approach for metrologists. Measurement, 51, 315-327.

Masters, G. N. (1994). KIDMAP – a history. Rasch Measurement Transactions, 8(2), 366 [http://www.rasch.org/rmt/rmt82k.htm].

Overton, W. F. (2015). Processes, relations and Relational-Developmental-Systems. In W. F. Overton & P. C. M. Molenaar (Eds.), Theory and Method. Volume 1 of the Handbook of child psychology and developmental science (7th Ed.) (pp. 9-62). Hoboken, NJ: Wiley.

Pendrill, L. (2014). Man as a measurement instrument [Special Feature]. NCSLi Measure: The Journal of Measurement Science, 9(4), 22-33.

Ricoeur, P. (1974). The project of a social ethic. In D. Stewart & J. Bien, (Eds.). Political and social essays by Paul Ricoeur (pp. 160-175). Athens, Ohio: Ohio University Press.

Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: Design and access for large information spaces. Information Systems Research, 7(1), 111-134.

Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. New York: Doubleday.

Wilson, M. (2004). Towards coherence between classroom assessment and accountability. Chicago: University of Chicago Press. 

Wright, B. D. (1997). A history of social science measurement. Educational Measurement: Issues and Practice, 16(4), 33-45, 52 [http://www.rasch.org/memo62.htm].

Measuring Values To Apply The Golden Rule

December 29, 2016

Paper presentation 45.20, American Educational Research Association

New Orleans, April 1994

 

Objective

Basing her comments on the writings of Michael Lerner in Tikkun magazine, “Hillary Rodham Clinton speaks appealingly of a political morality based on the Golden Rule,” says Chicago Tribune columnist Clarence Page.  Lerner and Clinton are correct in asserting that we need to rediscover and re-invigorate our spiritual values, though there is nothing new in this assertion, and Page is correct in his opinion that conservative columnists who say religion is spirituality, and that there is therefore nothing in need of re-invigoration, are wrong.  Research on the spiritual dimension of disability, for instance, shows that the quality of spiritual experience has little, if anything, to do with religious church attendance, bible reading, prayer, or the taking of sacraments (Fisher & Pugliese, 1989).

The purpose of this paper is to propose a research program that would begin to prepare the ground in which a political morality based on the Golden Rule might be cultivated.

Theoretical Framework

Implementing a “political morality based on the Golden Rule” requires some way of knowing that what I do unto others is the same as what I would have done unto me. To know this, I need a measuring system that keeps things in proportion by showing what counts as the same thing for different people.  A political morality based on the Golden Rule has got to have some way of identifying when a service or action done unto others is the same as the one done unto me.  In short, application of the Golden Rule requires an empirical basis of comparison, a measuring system that sets up analogies between people’s values and what is valued.  We must be able to say that my values are to one aspect of a situation what yours are to that or another aspect, and that proportions of this kind hold constant no matter which particular persons are addressed and no matter which aspects of the situation are involved.

Technique

Is it possible to measure what people value—politically, socially, economically, spiritually, and culturally—in a way that embodies the Golden Rule? If so, could such a measure be used for realizing the political morality Hillary Rodham Clinton has advocated?  L. L. Thurstone presented methods for successfully revealing the necessary proportions in the 1920s; these were improved upon by the Danish mathematician Georg Rasch in the 1950s.  Thurstone’s and Rasch’s ideas are researched and applied today by Benjamin D. Wright and J. Michael Linacre.  These and other thinkers hold that measurement takes place only when application of the Golden Rule is possible.  That is, measurement is achieved only if someone’s measure does not depend on who is in the group she is measured with, on the particular questions answered or not answered, on who made the measure, on the brand name of the instrument, or on where the measure took place.

Measurement of this high quality is called scale-free because its quantities do not vary according to the particular questions asked (as long as they pertain to the construct of interest); neither do they vary according to the structure or combination of the particular rating scheme(s) employed (rating scale, partial credit, correct/incorrect, true/false, present/absent, involvement of judges, paired comparisons, etc.), or the brand name of the instrument measuring.  All of these requirements must hold if I am to treat a person as I would like to be treated, because if they do not hold, I do not know enough about her values or mine to say whether she’s receiving the treatment I’d prefer in the same circumstance.

In order to make the Golden Rule the basis of a political morality, we need to improve the quality of measurement in every sphere of our lives; after all, politics is more than just what politicians do, it is a basic part of community life.  Even though the technology and methods for high quality measurement in education, sociology, and psychology have existed for decades, researchers have been indifferent to their use.

That indifference may be near an end.  If people get serious about applying the Golden Rule, they are going to come up against a need for rigorous quantitative measurement.  We need to let them know that the tools for the job are available.

Data sources

Miller’s Scale Battery of International Patterns and Norms (SBIPN) (Miller, 1968, 1970, 1973), described in Miller (1983, pp. 462-468), is an instrument that presents possibilities for investigating quantitative relations among value systems.  The instrument is composed of 20 six-point rating scale items involving such cultural norms and patterns as social acceptance, family solidarity, trustfulness, moral code, honesty, reciprocity, class structure, etc.  Each pair of rating scale points (1-2, 3-4, 5-6) is associated with a 15-30 word description; raters judge national values by assigning ratings, where 1 indicates the most acceptance, solidarity, trust, morality, etc., and 6 the least.  Miller (1983, p. 462) reports test-retest correlations of .74 to .97 for the original 15 items on the survey as testing in the United States and Peru.  Validity claims are based on the scale’s ability to distinguish between values of citizens of the United States and Peru, with supporting research comparing values in Argentina, Spain, England, and the United States.

The SBIPN could probably be improved in several ways.  First, individual countries contain so many diverse ethnic groups and subcultures whose value systems are often in conflict that ratings should probably be made of them and not of the entire population.  The geographical location of the ethnic group or subculture rated should also be tracked in order to study regional variations.  Second, Miller contends that raters must have a college degree to be qualified as a SBIPN judge; the complexity of his rating procedure justifies this claim.  In order to simplify the survey and broaden the base of qualified judges, the three groups of short phrases structuring each six-point rating scale should be used as individual items rated on a frequency continuum.

For instance, the following phrases appear in association with ratings of 1 and 2 under social acceptance:

high social acceptance. Social contacts open and nonrestrictive. Introductions not needed for social contacts.  Short acquaintance provides entry into the home and social organizations.

Similar descriptions are associated with the 3-4 (medium social acceptance) and 5-6 (low social acceptance) rating pairs; only one rating from the series of six is assigned, so that a rating of 1 or 2 is assigned only if the judgment is of high social acceptance.  Instead of asking the rater to assign one of two ratings to all six of these statements (breaking apart the two conjunctive phrases), and ignoring the 10-20 phrases associated with the other four rating scale points, each phrase presented on the six-point continuum should be rated separately for the frequency of the indicated pattern or norm.  A four-point rating scale (Almost Always, Frequently, Sometimes, Rarely) should suffice.

Linacre’s (1993, p. 284) graphical presentation of Rasch-based Generalizability Theory indicates that reliability and separation statistics of .92 and 3.4, respectively, can be expected for a 20-item, six-point rating scale survey (Miller’s original format), assuming a measurement standard deviation of one logit.  360 items will be produced if each of the original 20 six-point items can be transformed into 18 four-point items (following the above example’s derivation of six items from one of the three blocks of one item’s descriptive phrases).  If only 250 of these items work to support the measurement effort, Linacre’s graph shows that a reliability of .99 and separation of 10 might be obtained, again assuming a measurement standard deviation of one logit.  Since not all of the survey’s items would probably be administered at once, these estimates are probably high.  The increased number of items, however, would be advantageous for use as an item bank in a computer adapted administration of the survey.

Expected results

Miller’s applications of the SBIPN provide specific indications of what might be expected from the revised form of the survey.  Family solidarity tends to be low, labor assimilated into the prevailing economic system, class consciousness devalued, and moral conduct secularly defined in the United States, in opposition to Colombia and Peru, where family solidarity is high, labor is antagonistic to the prevailing economic system, class structure is rigidly defined, and moral conduct is religiously defined.  At the other extreme, civic participation, work and achievement, societal consensus, children’s independence, and democracy are highly valued in the United States, but considerably less so in Colombia and Peru.

Miller’s presentation of the survey results will be improved on in several ways.  First, construct validity will be examined in terms of the data’s internal consistency (fit analysis) and the conceptual structure delineated by the items.  Second, the definition of interval measurement continua for each ethnic group or subculture measured will facilitate quantitative and qualitative comparisons of each group’s self-image with its public image.  Differences in group perception can be used for critical self-evaluation as well as information crucial for rectifying unjust projections of prejudice.

Scientific importance

One of the most important benefits of this survey could be the opportunity to show that, although different value systems vary in their standards of what counts as acceptable behaviors and attitudes, the procedures by which values are calibrated and people’s personal values are measured do not vary.  That this should turn out to be the case will make it more difficult to justify and maintain hostile prejudices against others whose value systems differ from one’s own.  If people who do not share my values cannot immediately be categorized as godless, heathens, infidels, pagans, unwashed, etc., ie, in the category of the non-classifiable, then I should be less prone to disregard, hate, or fear them, and more able to build a cohesive, healthy, and integrated community with them.

The cultural prejudice structuring this proposal is that increased understanding of others’ values is good; that this prejudice needs to be made explicit and evaluated for its effect on those who do not share it is of great importance.  The possibility of pursuing a quantitative study of value systems may strike some as an area of research that could only be used to dominate and oppress those who do not have the power to defend themselves.  This observation implies that one reason why more rigorous scientific measurement procedures have failed to take hold in the social studies may be because we have unspoken, but nonetheless justifiable, reservations concerning our capacity to employ high quality information responsibly.  Knowledge is inherently dangerous, but a political morality based on the Golden Rule will require nothing less than taking another bite of the apple from the Tree of Knowledge.

 

References

Fisher, William P. & Karen Pugliese. 1989.  Measuring the importance of pastoral care in rehabilitation. Archives of Physical Medicine and Rehabilitation, 70, A-22 [Abstract].

Linacre, J. Michael. 1993. Rasch-based generalizability theory. Rasch Measurement, 7: 283-284.

Miller, Delbert C. 1968. The measurement of international patterns and norms: A tool for comparative research. Southwestern Social Science Quarterly, 48: 531-547.

Miller, Delbert C. 1970. International Community Power Structures: Comparative Studies of Four World Cities. Bloomington: Indiana University Press.

Miller, Delbert C. 1972. Measuring cross national norms: Methodological problems in identifying patterns in Latin America and Anglo-Saxon Cultures.  International Journal of Comparative Sociology, 13(3-4): 201-216.

Miller, Delbert C. 1983. Handbook of Research Design and Social Measurement. 4th ed. New York: Longman.

Excerpts and Notes from Goldberg’s “Billions of Drops…”

December 23, 2015

Goldberg, S. H. (2009). Billions of drops in millions of buckets: Why philanthropy doesn’t advance social progress. New York: Wiley.

p. 8:
Transaction costs: “…nonprofit financial markets are highly disorganized, with considerable duplication of effort, resource diversion, and processes that ‘take a fair amount of time to review grant applications and to make funding decisions’ [citing Harvard Business School Case No. 9-391-096, p. 7, Note on Starting a Nonprofit Venture, 11 Sept 1992]. It would be a major understatement to describe the resulting capital market as inefficient.”

A McKinsey study found that nonprofits spend 2.5 to 12 times more raising capital than for-profits do. When administrative costs are factored in, nonprofits spend 5.5 to 21.5 times more.

For-profit and nonprofit funding efforts contrasted on pages 8 and 9.

p. 10:
Balanced scorecard rating criteria

p. 11:
“Even at double-digit annual growth rates, it will take many years for social entrepreneurs and their funders to address even 10% of the populations in need.”

p. 12:
Exhibit 1.5 shows that the percentages of various needs served by leading social enterprises are barely drops in the respective buckets; they range from 0.07% to 3.30%.

pp. 14-16:
Nonprofit funding is not tied to performance. Even when a nonprofit makes the effort to show measured improvement in impact, it does little or nothing to change their funding picture. It appears that there is some kind of funding ceiling implicitly imposed by funders, since nonprofit growth and success seems to persuade capital sources that their work there is done. Mediocre and low performing nonprofits seem to be able to continue drawing funds indefinitely from sympathetic donors who don’t require evidence of effective use of their money.

p. 34:
“…meaningful reductions in poverty, illiteracy, violence, and hopelessness will require a fundamental restructuring of nonprofit capital markets. Such a restructuring would need to make it much easier for philanthropists of all stripes–large and small, public and private, institutional and individual–to fund nonprofit organizations that maximize social impact.”

p. 54:
Exhibit 2.3 is a chart showing that fewer people rose from poverty, and more remained in it or fell deeper into it, in the period of 1988-98 compared with 1969-1979.

pp. 70-71:
Kotter’s (1996) change cycle.

p. 75:
McKinsey’s seven elements of nonprofit capacity and capacity assessment grid.

pp. 94-95:
Exhibits 3.1 and 3.2 contrast the way financial markets reward for-profit performance with the way nonprofit markets reward fund raising efforts.

Financial markets
1. Market aggregates and disseminates standardized data
2. Analysts publish rigorous research reports
3. Investors proactively search for strong performers
4. Investors penalize weak performers
5. Market promotes performance
6. Strong performers grow

Nonprofit markets
1. Social performance is difficult to measure
2. NPOs don’t have resources or expertise to report results
3. Investors can’t get reliable or standardized results data
4. Strong and weak NPOs spend 40 to 60% of time fundraising
5. Market promotes fundraising
6. Investors can’t fund performance; NPOs can’t scale

p. 95:
“…nonprofits can’t possibly raise enough money to achieve transformative social impact within the constraints of the existing fundraising system. I submit that significant social progress cannot be achieved without what I’m going to call ‘third-stage funding,’ that is, funding that doesn’t suffer from disabling fragmentation. The existing nonprofit capital market is not capable of [p. 97] providing third-stage funding. Such funding can arise only when investors are sufficiently well informed to make big bets at understandable and manageable levels of risk. Existing nonprofit capital markets neither provide investors with the kinds of information needed–actionable information about nonprofit performance–nor provide the kinds of intermediation–active oversight by knowledgeable professionals–needed to mitigate risk. Absent third-stage funding, nonprofit capital will remain irreducibly fragmented, preventing the marshaling of resources that nonprofit organizations need to make meaningful and enduring progress against $100 million problems.”

pp. 99-114:
Text and diagrams on innovation, market adoption, transformative impact.

p. 140:
Exhibit 4.2: Capital distribution of nonprofits, highlighting mid-caps

pages 192-3 make the case for the difference between a regular market and the current state of philanthropic, social capital markets.

p. 192:
“So financial markets provide information investors can use to compare alternative investment opportunities based on their performance, and they provide a dynamic mechanism for moving money away from weak performers and toward strong performers. Just as water seeks its own level, markets continuously recalibrate prices until they achieve a roughly optimal equilibrium at which most companies receive the ‘right’ amount of investment. In this way, good companies thrive and bad ones improve or die.
“The social sector should work the same way. .. But philanthropic capital doesn’t flow toward effective nonprofits and away from ineffective nonprofits for a simple reason: contributors can’t tell the difference between the two. That is, philanthropists just don’t [p. 193] know what various nonprofits actually accomplish. Instead, they only know what nonprofits are trying to accomplish, and they only know that based on what the nonprofits themselves tell them.”

p. 193:
“The signs that the lack of social progress is linked to capital market dysfunctions are unmistakable: fundraising remains the number-one [p. 194] challenge of the sector despite the fact that nonprofit leaders divert some 40 to 60% of their time from productive work to chasing after money; donations raised are almost always too small, too short, and too restricted to enhance productive capacity; most mid-caps are ensnared in the ‘social entrepreneur’s trap’ of focusing on today and neglecting tomorrow; and so on. So any meaningful progress we could make in the direction of helping the nonprofit capital market allocate funds as effectively as the private capital market does could translate into tremendous advances in extending social and economic opportunity.
“Indeed, enhancing nonprofit capital allocation is likely to improve people’s lives much more than, say, further increasing the total amount of donations. Why? Because capital allocation has a multiplier effect.”

“If we want to materially improve the performance and increase the impact of the nonprofit sector, we need to understand what’s preventing [p. 195] it from doing a better job of allocating philanthropic capital. And figuring out why nonprofit capital markets don’t work very well requires us to understand why the financial markets do such a better job.”

p. 197:
“When all is said and done, securities prices are nothing more than convenient approximations that market participants accept as a way of simplifying their economic interactions, with a full understanding that market prices are useful even when they are way off the mark, as they so often are. In fact, that’s the whole point of markets: to aggregate the imperfect and incomplete knowledge held by vast numbers of traders about much various securities are worth and still make allocation choices that are better than we could without markets.
“Philanthropists face precisely the same problem: how to make better use of limited information to maximize output, in this case, social impact. Considering the dearth of useful tools available to donors today, the solution doesn’t have to be perfect or even all that good, at least at first. It just needs to improve the status quo and get better over time.
“Much of the solution, I believe, lies in finding useful adaptations of market mechanisms that will mitigate the effects of the same lack of reliable and comprehensive information about social sector performance. I would even go so far as to say that social enterprises can’t hope to realize their ‘one day, all children’ visions without a funding allociation system that acts more like a market.
“We can, and indeed do, make incremental improvements in nonprofit funding without market mechanisms. But without markets, I don’t see how we can fix the fragmentation problem or produce transformative social impact, such as ensuring that every child in America has a good education. The problems we face are too big and have too many moving parts to ignore the self-organizing dynamics of market economics. As Thomas Friedman said about the need to impose a carbon tax at a time of falling oil prices, ‘I’ve wracked my brain trying to think of ways to retool America around clean-power technologies without a price signal–i.e., a tax–and there are no effective ones.”

p. 199:
“Prices enable financial markets to work the way nonprofit capital markets should–by sending informative signals about the most effective organizations so that money will flow to them naturally..”

p. 200:
[Quotes Kurtzman citing De Soto on the mystery of capital. Also see p. 209, below.]
“‘Solve the mystery of capital and you solve many seemingly intractable problems along with it.'”
[That’s from page 69 in Kurtzman, 2002.]

p. 201:
[Goldberg says he’s quoting Daniel Yankelovich here, but the footnote does not appear to have anything to do with this quote:]
“‘The first step is to measure what can easily be measured. The second is to disregard what can’t be measured, or give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily isn’t very important. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.'”

Goldberg gives example here of $10,000 invested witha a 10% increase in value, compared with $10,000 put into a nonprofit. “But if the nonprofit makes good use of the money and, let’s say, brings the reading scores of 10 elementary school students up from below grade level to grade level, we can’t say how much my initial investment is ‘worth’ now. I could make the argument that the value has increased because the students have received a demonstrated educational benefit that is valuable to them. Since that’s the reason I made the donation, the achievement of higher scores must have value to me, as well.”

p. 202:
Goldberg wonders whether donations to nonprofits would be better conceived as purchases than investments.

p. 207:
Goldberg quotes Jon Gertner from the March 9, 2008, issue of the New York Times Magazine devoted to philanthropy:

“‘Why shouldn’t the world’s smartest capitalists be able to figure out more effective ways to give out money now? And why shouldn’t they want to make sure their philanthropy has significant social impact? If they can measure impact, couldn’t they get past the resistance that [Warren] Buffet highlighted and finally separate what works from what doesn’t?'”

p. 208:
“Once we abandon the false notions that financial markets are precision instruments for measuring unambiguous phenomena, and that the business and nonproft sectors are based in mutually exclusive principles of value, we can deconstruct the true nature of the problems we need to address and adapt market-like mechanisms that are suited to the particulars of the social sector.
“All of this is a long way (okay, a very long way) of saying that even ordinal rankings of nonprofit investments can have tremendous value in choosing among competing donation opportunities, especially when the choices are so numerous and varied. If I’m a social investor, I’d really like to know which nonprofits are likely to produce ‘more’ impact and which ones are likely to produce ‘less.'”

“It isn’t necessary to replicate the complex working of the modern stock markets to fashion an intelligent and useful nonprofit capital allocation mechanism. All we’re looking for is some kind of functional indication that would (1) isolate promising nonprofit investments from among the confusing swarm of too many seemingly worthy social-purpose organizations and (2) roughly differentiate among them based on the likelihood of ‘more’ or ‘less’ impact. This is what I meant earlier by increasing [p. 209] signals and decreasing noise.”

p. 209:
Goldberg apparently didn’t read De Soto, as he says that the mystery of capital is posed by Kurtzman and says it is solved via the collective intelligence and wisdom of crowds. This completely misses the point of the crucial value that transparent representations of structural invariance hold in market functionality. Goldberg is apparently offering a loose kind of market for which there is an aggregate index of stocks for nonprofits that are built up from their various ordinal performance measures. I think I find a better way in my work, building more closely from De Soto (Fisher, 2002, 2003, 2005, 2007, 2009a, 2009b).

p. 231:
Goldberg quotes Harvard’s Allen Grossman (1999) on the cost-benefit boundaries of more effective nonprofit capital allocation:

“‘Is there a significant downside risk in restructuring some portion of the philanthropic capital markets to test the effectiveness of performance driven philanthropy? The short answer is, ‘No.’ The current reality is that most broad-based solutions to social problems have eluded the conventional and fragmented approaches to philanthropy. It is hard to imagine that experiments to change the system to a more performance driven and rational market would negatively impact the effectiveness of the current funding flows–and could have dramatic upside potential.'”

p. 232:
Quotes Douglas Hubbard’s How to Measure Anything book that Stenner endorsed, and Linacre and I didn’t.

p. 233:
Cites Stevens on the four levels of measurement and uses it to justify his position concerning ordinal rankings, recognizing that “we can’t add or subtract ordinals.”

pp. 233-5:
Justifies ordinal measures via example of Google’s PageRank algorithm. [I could connect from here using Mary Garner’s (2009) comparison of PageRank with Rasch.]

p. 236:
Goldberg tries to justify the use of ordinal measures by citing their widespread use in social science and health care. He conveniently ignores the fact that virtually all of the same problems and criticisms that apply to philanthropic capital markets also apply in these areas. In not grasping the fundamental value of De Soto’s concept of transferable and transparent representations, and in knowing nothing of Rasch measurement, he was unable to properly evaluate to potential of ordinal data’s role in the formation of philanthropic capital markets. Ordinal measures aren’t just not good enough, they represent a dangerous diversion of resources that will be put into systems that take on lives of their own, creating a new layer of dysfunctional relationships that will be hard to overcome.

p. 261 [Goldberg shows here his complete ignorance about measurement. He is apparently totally unaware of the work that is in fact most relevant to his cause, going back to Thurstone in 1920s, Rasch in the 1950s-1970s, and Wright in the 1960s to 2000. Both of the problems he identifies have long since been solved in theory and in practice in a wide range of domains in education, psychology, health care, etc.]:
“Having first studied performance evaluation some 30 years ago, I feel confident in saying that all the foundational work has been done. There won’t be a ‘eureka!’ breakthrough where someone finally figures out the one true way to guage nonprofit effectiveness.
“Indeed, I would venture to say that we know virtually everything there is to know about measuring the performance of nonprofit organizations with only two exceptions: (1) How can we compare nonprofits with different missions or approaches, and (2) how can we make actionable performance assessments common practice for growth-ready mid-caps and readily available to all prospective donors?”

p. 263:
“Why would a social entrepreneur divert limited resources to impact assessment if there were no prospects it would increase funding? How could an investor who wanted to maximize the impact of her giving possibly put more golden eggs in fewer impact-producing baskets if she had no way to distinguish one basket from another? The result: there’s no performance data to attract growth capital, and there’s no growth capital to induce performance measurement. Until we fix that Catch-22, performance evaluation will not become an integral part of social enterprise.”

pp. 264-5:
Long quotation from Ken Berger at Charity Navigator on their ongoing efforts at developing an outcome measurement system. [wpf, 8 Nov 2009: I read the passage quoted by Goldberg in Berger’s blog when it came out and have been watching and waiting ever since for the new system. wpf, 8 Feb 2012: The new system has been online for some time but still does not include anything on impacts or outcomes. It has expanded from a sole focus on financials to also include accountability and transparency. But it does not yet address Goldberg’s concerns as there still is no way to tell what works from what doesn’t.]

p. 265:
“The failure of the social sector to coordinate independent assets and create a whole that exceeds the sum of its parts results from an absence of.. platform leadership’: ‘the ability of a company to drive innovation around a particular platform technology at the broad industry level.’ The object is to multiply value by working together: ‘the more people who use the platform products, the more incentives there are for complement producers to introduce more complementary products, causing a virtuous cycle.'” [Quotes here from Cusumano & Gawer (2002). The concept of platform leadership speaks directly to the system of issues raised by Miller & O’Leary (2007) that must be addressed to form effective HSN capital markets.]

p. 266:
“…the nonprofit sector has a great deal of both money and innovation, but too little available information about too many organizations. The result is capital fragmentation that squelches growth. None of the stakeholders has enough horsepower on its own to impose order on this chaos, but some kind of realignment could release all of that pent-up potential energy. While command-and-control authority is neither feasible nor desirable, the conditions are ripe for platform leadership.”

“It is doubtful that the IMPEX could amass all of the resources internally needed to build and grow a virtual nonprofit stock market that could connect large numbers of growth-capital investors with large numbers of [p. 267] growth-ready mid-caps. But it might be able to convene a powerful coalition of complementary actors that could achieve a critical mass of support for performance-based philanthropy. The challenge would be to develop an organization focused on filling the gaps rather than encroaching on the turf of established firms whose participation and innovation would be required to build a platform for nurturing growth of social enterprise..”

p. 268-9:
Intermediated nonprofit capital market shifts fundraising burden from grantees to intermediaries.

p. 271:
“The surging growth of national donor-advised funds, which simplify and reduce the transaction costs of methodical giving, exemplifies the kind of financial innovation that is poised to leverage market-based investment guidance.” [President of Schwab Charitable quoted as wanting to make charitable giving information- and results-driven.]

p. 272:
Rating agencies and organizations: Charity Navigator, Guidestar, Wise Giving Alliance.
Online donor rankings: GlobalGiving, GreatNonprofits, SocialMarkets
Evaluation consultants: Mathematica

Google’s mission statement: “to organize the world’s information and make it universally accessible and useful.”

p. 273:
Exhibit 9.4 Impact Index Whole Product
Image of stakeholders circling IMPEX:
Trading engine
Listed nonprofits
Data producers and aggregators
Trading community
Researchers and analysts
Investors and advisors
Government and business supporters

p. 275:
“That’s the starting point for replication [of social innovations that work]: finding and funding; matching money with performance.”

[WPF bottom line: Because Goldberg misses De Soto’s point about transparent representations resolving the mystery of capital, he is unable to see his way toward making the nonprofit capital markets function more like financial capital markets, with the difference being the focus on the growth of human, social, and natural capital. Though Goldberg intuits good points about the wisdom of crowds, he doesn’t know enough about the flaws of ordinal measurement relative to interval measurement, or about the relatively easy access to interval measures that can be had, to do the job.]

References

Cusumano, M. A., & Gawer, A. (2002, Spring). The elements of platform leadership. MIT Sloan Management Review, 43(3), 58.

De Soto, H. (2000). The mystery of capital: Why capitalism triumphs in the West and fails everywhere else. New York: Basic Books.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [http://www.rasch.org/rmt/rmt154j.htm].

Fisher, W. P., Jr. (2003). Measurement and communities of inquiry. Rasch Measurement Transactions, 17(3), 936-8 [http://www.rasch.org/rmt/rmt173.pdf].

Fisher, W. P., Jr. (2005). Daredevil barnstorming to the tipping point: New aspirations for the human sciences. Journal of Applied Measurement, 6(3), 173-9 [http://www.livingcapitalmetrics.com/images/FisherJAM05.pdf].

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-3 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In M. Wilson, K. Draney, N. Brown & B. Duckor (Eds.), Advances in Rasch Measurement, Vol. Two (p. in press [http://www.livingcapitalmetrics.com/images/BringingHSN_FisherARMII.pdf]). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2009b, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement (Elsevier), 42(9), 1278-1287.

Garner, M. (2009, Autumn). Google’s PageRank algorithm and the Rasch measurement model. Rasch Measurement Transactions, 23(2), 1201-2 [http://www.rasch.org/rmt/rmt232.pdf].

Grossman, A. (1999). Philanthropic social capital markets: Performance driven philanthropy (Social Enterprise Series 12 No. 00-002). Harvard Business School Working Paper.

Kotter, J. (1996). Leading change. Cambridge, Massachusetts: Harvard Business School Press.

Kurtzman, J. (2002). How the markets really work. New York: Crown Business.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-34.

A truly ambitious plan to tackle climate change 

December 3, 2015

A recent story in the NY Times asks just what a truly ambitious plan to tackle climate change would look like. Pledges of emissions cuts being made in Paris this month are projected to fall short of what is needed to solve the problem of climate change. Calls for mass mobilization on the scale of the U.S.’s entry into WWII are met with skepticism at the same time that leaders are signing on for stronger terms in the Paris agreement than their countries have agreed to.

One crucial assumption is made across the full range of the proposals for more stringent standards and innovative technologies. That assumption is that solving the problem of climate change is a matter of marshaling the will to get the job done. On the face of it, of course, it seems inane to consider something as important as will power to be part of the problem. If people don’t want to do something, how could it possibly ever get done?

But as I’ve pointed out in a number of previous posts in this blog, complex problems sometimes cannot be solved from within the conceptual framework that engendered them. We are in this situation in large part because our overall relation to the earth is based on assuming it to be a bottomless well of resources, with the only limitation being the creativity we bring to bear in tapping those resources. Though many of us, perhaps a majority, are seriously committed to reconceiving our relation to the earth in sustainable terms, practical results are nearly impossible to produce within the existing institutional framework. Our economic, legal, accounting, education, etc. systems are all set up to support a consumer ethos that hobbles and undercuts almost all efforts intended to support an alternative sustainability ethos. It is both ironic and counterproductive to formulate solutions to the problem of climate change without first changing the institutional background assumptions informing the rules, roles and responsibilities through which we conceptualize and implement those solutions.

Insight into this problem is provided by recent work on standards for sustainability accounting. It shows that, by definition, efforts targeting change in economic externalities like environmental concerns cannot be scaled up in ways that are needed. This happens simply because balancing mission and margin demands maintenance of the bottom line. Giving away the business in the name of saving the planet might be a noble gesture but it is the opposite of sustainable and more importantly does not provide a viable model for the future.

So how do we model a new kind of bottom line that balances mission and margin in a new way? A way in which institutional rules, roles and responsibilities are themselves configured into the sustainable ecological relations we need? A way in which means and ends are unified? How do we become the change we want to see? How can we mobilize an international mass movement focused on doing what needs to be done to solve the problem of climate change? What possibilities do we have for catalyzing the increasingly saturated solution of global discontent and desire for a new relation to the earth? Can natural social processes of leaderless self organizing systems be seeded and guided to fruition? What would that seeding and guidance look like?

For proposed answers to these questions and more on what a model of a truly ambitious plan to tackle climate change might look like, see other posts in this blog here, here, here, and here.

On Deaton and Deserving

October 13, 2015

In Angus Deaton’s “view, theory is a complement to measurement, and generalizable insights arise only when the underlying economic mechanisms are elucidated and tested.” This appears in an article in today’s New York Times 
(http://mobile.nytimes.com/2015/10/13/upshot/why-angus-deaton-deserved-the-economics-nobel-prize.html).

But that’s hardly even half the story. Past this, theory and data should together be embodied in portable instrumentation measuring in a common language and distributed throughout interconnected stakeholder networks. Then economics ceases to be merely the study of economic behaviors and phenomena and becomes a living instantiation and ontological management embodying those behaviors and phenomena. For instance, theory-informed data on the economics of education may support robustly generalizable policies but can do nothing to support formation of common currencies for the exchange of literacy capital. For that, the rules, roles and responsibilities of market institutions must be structured with instruments facilitating the exchange of intangible assets. 

That is, to inform teachers’ instructional decisions for individual students as well as their pedagogical methods, and so to make teachers into effective cultivators of literacy and other forms of living capital, the theory-data-instrument package has to be systematically incorporated into the curriculum, assessment, and teachers’ continuing education and professional development. A start in this direction can be found in, for instance, the uses made of the Lexile Framework for Reading by various educational publishers and assessment agencies. 

Deaton’s “method of careful analysis of data from household surveys has transformed four large swaths of the dismal science: microeconomics, econometrics, macroeconomics and development economics.”

“This focus on empirics has been a boon for the field of econometrics, which is the application of statistical methods to economic problems. Mr. Deaton’s signature achievement in this area has been in forcing empirical researchers to pay closer attention to questions of measurement.”

“As the Nobel committee put it, Mr. Deaton’s ‘work covers a wide spectrum, from the deepest implications of theory to the grittiest detail of measurement.'”

Here we see a common but very shortsighted automatic connection between statistical data analysis and measurement. Statistical analyses usually do nothing to investigate, establish or deploy the three key features of measurement: a) the existence of a meaningful, invariant unit for expressing theoretically explained and empirically reproducible comparisons, b) the calibration, universal distribution and maintenance of instruments measuring in that unit, and c) the systematic incorporation of that unit in research and practice as the legally required expression of quantities exchanged in accord with financial, accounting, and regulatory standards. 

Had Deaton actually covered the full econometric spectrum the way the Nobel committee said, “from the deepest implications of theory to the grittiest detail of measurement,” his Nobel prize would celebrate the accomplishment of a whole new science of economics. That new science would not be, as today’s economics remains, primarily focused on centralized statistical analyses of data incorporating uncontrolled, unexamined, instrument- and sample-dependent variations in unit size. Hayek’s critique of socialism’s “fatal conceit” of central planning suggests that a more thoroughly capitalist economics would instead seek to form efficient markets for low friction exchanges of high quality information owned and controlled by individuals. 

Most readers first reaction to the preceding statements will be that these expectations are patently unrealistic and impossible for measurement in the economics of a wide range of sectors, from education to health care to social services to environmental resource management. But that reaction is based in ignorance of the decades of research and practice that have already put measurement in the hands of teachers, clinicians and others on the front lines of these fields. 

It is encouraging that “Deaton has turned his attention to measures of subjective well-being, including happiness,” and that he has “highlighted the problems in constructing coherent measures of global poverty.” But again, examination of his work shows no use of the well-established viability and value of the three key features of measurement, despite the large and readily available literature on the relevant theories, data, instruments, models, methods, software, and studies. 

How long will we continue rewarding and celebrating only ideas that fit our preconceptions concerning what’s possible? What will it take for truly new ideas to break out of our cultural blind spots and start informing explorations of alternative methods and possibilities? When will we start systematically testing more of our assumptions about what’s possible, instead of blithely chaining ourselves to perspectives that prevent us from fulfilling our dreams of a fairer, more equitable world? 

Deaton deserved a Nobel prize, as others have said, because the prize rewards particular ways of satisfying unspoken norms more than it encourages truly breakout disruptions of our expectations. 

With Reich in spirit, but with a different sense of the problem and its solution

October 4, 2015

In today’s editorial in the San Francisco Chronicle, Robert Reich seeks some way of defining a solution to the pressing problems of how globalization and technological changes have made American workers less competitive. He rightly says that “reversing the scourge of widening inequality requires reversing the upward distributions [of income] within the rules of the market, and giving average people the bargaining power they need to get a larger share of the gains from growth.”

But Reich then says that the answer to this problem lies in politics, not economics. As I’ve pointed out before in this blog, focusing on marshaling political will is part of the problem, not part of the solution. Historically, politicians do not lead, they follow. As is demonstrated across events as diverse as the Arab Spring and the Preemption Act of 1841, mass movements of people have repeatedly demanded ways of cutting through the Gordian knots of injustice. And just as the political “leadership” across the Middle East and in the early U.S. dragged its feet, obstructed, and violently opposed change until it was already well underway, so, too, will that pattern repeat itself again in the current situation of inequitable income distribution.

The crux of the problem is that no one can give average people anything, not freedom (contra Dylan’s line in Blowin’ in the Wind about “allowing” people to be free) and certainly not a larger share of the gains from growth. As the old saying goes, you can lead a horse to water, but you can’t make it drink. People have to take what’s theirs. They have to want it, they have to struggle for it, and they have to pay for it, or they cannot own it and it will never be worth anything to them.

It is well known that a lack of individual property rights doomed communism and socialism because when everything is owned collectively by everyone, no one takes responsibility for it. The profit motive has the capacity to drive people to change things. The problem is not in profit itself. If birds and bees and trees and grasses did not profit from the sun, soil, and rain, there would be no life. The problem is in finding how to get a functional, self-sustaining economic ecology off the ground, not in unrealistically trying to manipulate and micromanage every detail.

The fundamental relevant characteristic of the profits being made today from intellectual property rights is that our individual rights to our own human and social capital are counter-productively restricted and undeveloped. How can it be that no one has any idea how much literacy or health capital they have, or what it is worth?! We have a metric system that tells us how much real estate and manufactured capital we own, and we can price it. But despite the well-established scientific facts of decades of measurement science research and practice, none of us can say, “I own x number of shares of stock in intellectual, literacy, or community capital, that have a value of x dollars in today’s market.” We desperately need an Intangible Assets Metric System, and the market rules, roles, and responsibilities that will make it impossible to make a profit while destroying human, social, and natural capital.

In this vein, what Reich gets absolutely correct is hidden inside his phrase, “within the rules of the market.” As I’ve so often repeated in this blog, capitalism is not inherently evil; it is, rather, unfinished. The real evil is in prolonging the time it takes to complete it. As was so eloquently stated by Miller and O’Leary (2007, p. 710):

“Markets are not spontaneously generated by the exchange activity of buyers and sellers. Rather, skilled actors produce institutional arrangements, the rules, roles and relationships that make market exchange possible. The institutions define the market, rather than the reverse.”

We have failed to set up the institutional arrangements needed to define human, social, and natural capital markets. The problem is that we cannot properly manage three of the four major forms of capital (human, social, and natural, with the fourth being manufactured/property) because we do not measure them in a common language built into scientifically, economically, legally and financially accountable titles, deeds, and other instruments.

And so, to repeat another one of my ad nauseum broken record nostrums, the problem is the problem. As long as we keep defining problems in the way we always have, as matters of marshalling political will, we will inadvertently find ourselves contributing more to prolonging tragic and needless human suffering, social discontent, and environmental degradation.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

The Counterproductive Consequences of Common Study Designs and Statistical Methods

May 21, 2015

Because of the ways studies are designed and the ways data are analyzed, research results in psychology and the social sciences often appear to be nonlinear, sample- and instrument-dependent, and incommensurable, even when they need not be. In contrast with what are common assumptions about the nature of the constructs involved, invariant relations may be more obscured than clarified by typically employed research designs and statistical methods.

To take a particularly salient example, the number of small factors with Eigenvalues greater than 1.0 identified via factor analysis increases as the number of modes in a multi-modal distribution also increases, and the interpretation of results is further complicated by the fact that the number of factors identified decreases as sample size increases (Smith, 1996).

Similarly, variation in employment test validity across settings was established as a basic assumption by the 1970s, after 50 years of studies observing the situational specificity of results. But then Schmidt and Hunter (1977) identified sampling error, measurement error, and range restriction as major sources of what was only the appearance of incommensurable variation in employment test validity. In other words, for most of the 20th century, the identification of constructs and comparisons of results across studies were pointlessly confused by mixed populations, uncontrolled variation in reliability, and unnoted floor and/or ceiling effects. Though they do nothing to establish information systems deploying common languages structured by standard units of measurement (Feinstein, 1995), meta-analysis techniques are a step forward in equating effect sizes (Hunter & Schmidt, 2004).

Wright and Stone’s (1979) Best Test Design, in contrast, takes up each of these problems in an explicit way. Sampling error is addressed in that both the sample’s and the items’ representations of the same populations of persons and expressions of a construct are evaluated. The evaluation of reliability is foregrounded and clarified by taking advantage of the availability of individualized measurement uncertainty (error) estimates (following Andrich, 1982, presented at AERA in 1977). And range restriction becomes manageable in terms of equating and linking instruments measuring in different ranges of the same construct. As was demonstrated by Duncan (1985; Allerup, Bech, Loldrup, et al., 1994; Andrich & Styles, 1998), for instance, the restricted ranges of various studies assessing relationships between measures of attitudes and behaviors led to the mistaken conclusion that these were separate constructs. When the entire range of variation was explicitly modeled and studied, a consistent relationship was found.

Statistical and correlational methods have long histories of preventing the discovery, assessment, and practical application of invariant relations because they fail to test for invariant units of measurement, do not define standard metrics, never calibrate all instruments measuring the same thing in common units, and have no concept of formal measurement systems of interconnected instruments. Wider appreciation of the distinction between statistics and measurement (Duncan & Stenbeck, 1988; Fisher, 2010; Wilson, 2013a), and of the potential for metrological traceability we have within our reach (Fisher, 2009, 2012; Fisher & Stenner, 2013; Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2015; Wilson, 2013b; Wilson, Mari, Maul, & Torres Irribarra, 2015), are demonstrably fundamental to the advancement of a wide range of fields.

References

Allerup, P., Bech, P., Loldrup, D., Alvarez, P., Banegil, T., Styles, I., & Tenenbaum, G. (1994). Psychiatric, business, and psychological applications of fundamental measurement models. International Journal of Educational Research, 21(6), 611-622.

Andrich, D. (1982). An index of person separation in Latent Trait Theory, the traditional KR-20 index, and the Guttman scale response pattern. Education Research and Perspectives, 9(1), 95-104 [http://www.rasch.org/erp7.htm].

Andrich, D., & Styles, I. M. (1998). The structural relationship between attitude and behavior statements from the unfolding perspective. Psychological Methods, 3(4), 454-469.

Duncan, O. D. (1985). Probability, disposition and the inconsistency of attitudes and behaviour. Synthese, 42, 21-34.

Duncan, O. D., & Stenbeck, M. (1988). Panels and cohorts: Design and model in the study of voting turnout. In C. C. Clogg (Ed.), Sociological Methodology 1988 (pp. 1-35). Washington, DC: American Sociological Association.

Feinstein, A. R. (1995). Meta-analysis: Statistical alchemy for the 21st century. Journal of Clinical Epidemiology, 48(1), 71-79.

Fisher, W. P., Jr. (2009). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2010). Statistics and measurement: Clarifying the differences. Rasch Measurement Transactions, 23(4), 1229-1230.

Fisher, W. P., Jr. (2012, May/June). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5.

Fisher, W. P., Jr., & Stenner, A. J. (2013). Overcoming the invisibility of metrology: A reading measurement network for education and the social sciences. Journal of Physics: Conference Series, 459(012024), http://iopscience.iop.org/1742-6596/459/1/012024.

Hunter, J. E., & Schmidt, F. L. (Eds.). (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012002/pdf/1742-6596_459_1_012002.pdf.

Pendrill, L. (2014). Man as a measurement instrument [Special Feature]. NCSLi Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46-55. doi: http://dx.doi.org/10.1016/j.measurement.2015.04.010

Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62(5), 529-540.

Smith, R. M. (1996). A comparison of methods for determining dimensionality in Rasch measurement. Structural Equation Modeling, 3(1), 25-40.

Wilson, M. R. (2013a). Seeking a balance between the statistical and scientific elements in psychometrics. Psychometrika, 78(2), 211-236.

Wilson, M. R. (2013b). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.

Wilson, M., Mari, L., Maul, A., & Torres Irribarra, D. (2015). A comparison of measurement concepts across physical science and social science domains: Instrument design, calibration, and measurement. Journal of Physics: Conference Series, 588(012034), http://iopscience.iop.org/1742-6596/588/1/012034.

Wright, B. D., & Stone, M. H. (1979). Best test design: Rasch measurement. Chicago, Illinois: MESA Press.