Archive for the ‘metrology’ Category

Excerpts and Notes from Goldberg’s “Billions of Drops…”

December 23, 2015

Goldberg, S. H. (2009). Billions of drops in millions of buckets: Why philanthropy doesn’t advance social progress. New York: Wiley.

p. 8:
Transaction costs: “…nonprofit financial markets are highly disorganized, with considerable duplication of effort, resource diversion, and processes that ‘take a fair amount of time to review grant applications and to make funding decisions’ [citing Harvard Business School Case No. 9-391-096, p. 7, Note on Starting a Nonprofit Venture, 11 Sept 1992]. It would be a major understatement to describe the resulting capital market as inefficient.”

A McKinsey study found that nonprofits spend 2.5 to 12 times more raising capital than for-profits do. When administrative costs are factored in, nonprofits spend 5.5 to 21.5 times more.

For-profit and nonprofit funding efforts contrasted on pages 8 and 9.

p. 10:
Balanced scorecard rating criteria

p. 11:
“Even at double-digit annual growth rates, it will take many years for social entrepreneurs and their funders to address even 10% of the populations in need.”

p. 12:
Exhibit 1.5 shows that the percentages of various needs served by leading social enterprises are barely drops in the respective buckets; they range from 0.07% to 3.30%.

pp. 14-16:
Nonprofit funding is not tied to performance. Even when a nonprofit makes the effort to show measured improvement in impact, it does little or nothing to change their funding picture. It appears that there is some kind of funding ceiling implicitly imposed by funders, since nonprofit growth and success seems to persuade capital sources that their work there is done. Mediocre and low performing nonprofits seem to be able to continue drawing funds indefinitely from sympathetic donors who don’t require evidence of effective use of their money.

p. 34:
“…meaningful reductions in poverty, illiteracy, violence, and hopelessness will require a fundamental restructuring of nonprofit capital markets. Such a restructuring would need to make it much easier for philanthropists of all stripes–large and small, public and private, institutional and individual–to fund nonprofit organizations that maximize social impact.”

p. 54:
Exhibit 2.3 is a chart showing that fewer people rose from poverty, and more remained in it or fell deeper into it, in the period of 1988-98 compared with 1969-1979.

pp. 70-71:
Kotter’s (1996) change cycle.

p. 75:
McKinsey’s seven elements of nonprofit capacity and capacity assessment grid.

pp. 94-95:
Exhibits 3.1 and 3.2 contrast the way financial markets reward for-profit performance with the way nonprofit markets reward fund raising efforts.

Financial markets
1. Market aggregates and disseminates standardized data
2. Analysts publish rigorous research reports
3. Investors proactively search for strong performers
4. Investors penalize weak performers
5. Market promotes performance
6. Strong performers grow

Nonprofit markets
1. Social performance is difficult to measure
2. NPOs don’t have resources or expertise to report results
3. Investors can’t get reliable or standardized results data
4. Strong and weak NPOs spend 40 to 60% of time fundraising
5. Market promotes fundraising
6. Investors can’t fund performance; NPOs can’t scale

p. 95:
“…nonprofits can’t possibly raise enough money to achieve transformative social impact within the constraints of the existing fundraising system. I submit that significant social progress cannot be achieved without what I’m going to call ‘third-stage funding,’ that is, funding that doesn’t suffer from disabling fragmentation. The existing nonprofit capital market is not capable of [p. 97] providing third-stage funding. Such funding can arise only when investors are sufficiently well informed to make big bets at understandable and manageable levels of risk. Existing nonprofit capital markets neither provide investors with the kinds of information needed–actionable information about nonprofit performance–nor provide the kinds of intermediation–active oversight by knowledgeable professionals–needed to mitigate risk. Absent third-stage funding, nonprofit capital will remain irreducibly fragmented, preventing the marshaling of resources that nonprofit organizations need to make meaningful and enduring progress against $100 million problems.”

pp. 99-114:
Text and diagrams on innovation, market adoption, transformative impact.

p. 140:
Exhibit 4.2: Capital distribution of nonprofits, highlighting mid-caps

pages 192-3 make the case for the difference between a regular market and the current state of philanthropic, social capital markets.

p. 192:
“So financial markets provide information investors can use to compare alternative investment opportunities based on their performance, and they provide a dynamic mechanism for moving money away from weak performers and toward strong performers. Just as water seeks its own level, markets continuously recalibrate prices until they achieve a roughly optimal equilibrium at which most companies receive the ‘right’ amount of investment. In this way, good companies thrive and bad ones improve or die.
“The social sector should work the same way. .. But philanthropic capital doesn’t flow toward effective nonprofits and away from ineffective nonprofits for a simple reason: contributors can’t tell the difference between the two. That is, philanthropists just don’t [p. 193] know what various nonprofits actually accomplish. Instead, they only know what nonprofits are trying to accomplish, and they only know that based on what the nonprofits themselves tell them.”

p. 193:
“The signs that the lack of social progress is linked to capital market dysfunctions are unmistakable: fundraising remains the number-one [p. 194] challenge of the sector despite the fact that nonprofit leaders divert some 40 to 60% of their time from productive work to chasing after money; donations raised are almost always too small, too short, and too restricted to enhance productive capacity; most mid-caps are ensnared in the ‘social entrepreneur’s trap’ of focusing on today and neglecting tomorrow; and so on. So any meaningful progress we could make in the direction of helping the nonprofit capital market allocate funds as effectively as the private capital market does could translate into tremendous advances in extending social and economic opportunity.
“Indeed, enhancing nonprofit capital allocation is likely to improve people’s lives much more than, say, further increasing the total amount of donations. Why? Because capital allocation has a multiplier effect.”

“If we want to materially improve the performance and increase the impact of the nonprofit sector, we need to understand what’s preventing [p. 195] it from doing a better job of allocating philanthropic capital. And figuring out why nonprofit capital markets don’t work very well requires us to understand why the financial markets do such a better job.”

p. 197:
“When all is said and done, securities prices are nothing more than convenient approximations that market participants accept as a way of simplifying their economic interactions, with a full understanding that market prices are useful even when they are way off the mark, as they so often are. In fact, that’s the whole point of markets: to aggregate the imperfect and incomplete knowledge held by vast numbers of traders about much various securities are worth and still make allocation choices that are better than we could without markets.
“Philanthropists face precisely the same problem: how to make better use of limited information to maximize output, in this case, social impact. Considering the dearth of useful tools available to donors today, the solution doesn’t have to be perfect or even all that good, at least at first. It just needs to improve the status quo and get better over time.
“Much of the solution, I believe, lies in finding useful adaptations of market mechanisms that will mitigate the effects of the same lack of reliable and comprehensive information about social sector performance. I would even go so far as to say that social enterprises can’t hope to realize their ‘one day, all children’ visions without a funding allociation system that acts more like a market.
“We can, and indeed do, make incremental improvements in nonprofit funding without market mechanisms. But without markets, I don’t see how we can fix the fragmentation problem or produce transformative social impact, such as ensuring that every child in America has a good education. The problems we face are too big and have too many moving parts to ignore the self-organizing dynamics of market economics. As Thomas Friedman said about the need to impose a carbon tax at a time of falling oil prices, ‘I’ve wracked my brain trying to think of ways to retool America around clean-power technologies without a price signal–i.e., a tax–and there are no effective ones.”

p. 199:
“Prices enable financial markets to work the way nonprofit capital markets should–by sending informative signals about the most effective organizations so that money will flow to them naturally..”

p. 200:
[Quotes Kurtzman citing De Soto on the mystery of capital. Also see p. 209, below.]
“‘Solve the mystery of capital and you solve many seemingly intractable problems along with it.'”
[That’s from page 69 in Kurtzman, 2002.]

p. 201:
[Goldberg says he’s quoting Daniel Yankelovich here, but the footnote does not appear to have anything to do with this quote:]
“‘The first step is to measure what can easily be measured. The second is to disregard what can’t be measured, or give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily isn’t very important. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.'”

Goldberg gives example here of $10,000 invested witha a 10% increase in value, compared with $10,000 put into a nonprofit. “But if the nonprofit makes good use of the money and, let’s say, brings the reading scores of 10 elementary school students up from below grade level to grade level, we can’t say how much my initial investment is ‘worth’ now. I could make the argument that the value has increased because the students have received a demonstrated educational benefit that is valuable to them. Since that’s the reason I made the donation, the achievement of higher scores must have value to me, as well.”

p. 202:
Goldberg wonders whether donations to nonprofits would be better conceived as purchases than investments.

p. 207:
Goldberg quotes Jon Gertner from the March 9, 2008, issue of the New York Times Magazine devoted to philanthropy:

“‘Why shouldn’t the world’s smartest capitalists be able to figure out more effective ways to give out money now? And why shouldn’t they want to make sure their philanthropy has significant social impact? If they can measure impact, couldn’t they get past the resistance that [Warren] Buffet highlighted and finally separate what works from what doesn’t?'”

p. 208:
“Once we abandon the false notions that financial markets are precision instruments for measuring unambiguous phenomena, and that the business and nonproft sectors are based in mutually exclusive principles of value, we can deconstruct the true nature of the problems we need to address and adapt market-like mechanisms that are suited to the particulars of the social sector.
“All of this is a long way (okay, a very long way) of saying that even ordinal rankings of nonprofit investments can have tremendous value in choosing among competing donation opportunities, especially when the choices are so numerous and varied. If I’m a social investor, I’d really like to know which nonprofits are likely to produce ‘more’ impact and which ones are likely to produce ‘less.'”

“It isn’t necessary to replicate the complex working of the modern stock markets to fashion an intelligent and useful nonprofit capital allocation mechanism. All we’re looking for is some kind of functional indication that would (1) isolate promising nonprofit investments from among the confusing swarm of too many seemingly worthy social-purpose organizations and (2) roughly differentiate among them based on the likelihood of ‘more’ or ‘less’ impact. This is what I meant earlier by increasing [p. 209] signals and decreasing noise.”

p. 209:
Goldberg apparently didn’t read De Soto, as he says that the mystery of capital is posed by Kurtzman and says it is solved via the collective intelligence and wisdom of crowds. This completely misses the point of the crucial value that transparent representations of structural invariance hold in market functionality. Goldberg is apparently offering a loose kind of market for which there is an aggregate index of stocks for nonprofits that are built up from their various ordinal performance measures. I think I find a better way in my work, building more closely from De Soto (Fisher, 2002, 2003, 2005, 2007, 2009a, 2009b).

p. 231:
Goldberg quotes Harvard’s Allen Grossman (1999) on the cost-benefit boundaries of more effective nonprofit capital allocation:

“‘Is there a significant downside risk in restructuring some portion of the philanthropic capital markets to test the effectiveness of performance driven philanthropy? The short answer is, ‘No.’ The current reality is that most broad-based solutions to social problems have eluded the conventional and fragmented approaches to philanthropy. It is hard to imagine that experiments to change the system to a more performance driven and rational market would negatively impact the effectiveness of the current funding flows–and could have dramatic upside potential.'”

p. 232:
Quotes Douglas Hubbard’s How to Measure Anything book that Stenner endorsed, and Linacre and I didn’t.

p. 233:
Cites Stevens on the four levels of measurement and uses it to justify his position concerning ordinal rankings, recognizing that “we can’t add or subtract ordinals.”

pp. 233-5:
Justifies ordinal measures via example of Google’s PageRank algorithm. [I could connect from here using Mary Garner’s (2009) comparison of PageRank with Rasch.]

p. 236:
Goldberg tries to justify the use of ordinal measures by citing their widespread use in social science and health care. He conveniently ignores the fact that virtually all of the same problems and criticisms that apply to philanthropic capital markets also apply in these areas. In not grasping the fundamental value of De Soto’s concept of transferable and transparent representations, and in knowing nothing of Rasch measurement, he was unable to properly evaluate to potential of ordinal data’s role in the formation of philanthropic capital markets. Ordinal measures aren’t just not good enough, they represent a dangerous diversion of resources that will be put into systems that take on lives of their own, creating a new layer of dysfunctional relationships that will be hard to overcome.

p. 261 [Goldberg shows here his complete ignorance about measurement. He is apparently totally unaware of the work that is in fact most relevant to his cause, going back to Thurstone in 1920s, Rasch in the 1950s-1970s, and Wright in the 1960s to 2000. Both of the problems he identifies have long since been solved in theory and in practice in a wide range of domains in education, psychology, health care, etc.]:
“Having first studied performance evaluation some 30 years ago, I feel confident in saying that all the foundational work has been done. There won’t be a ‘eureka!’ breakthrough where someone finally figures out the one true way to guage nonprofit effectiveness.
“Indeed, I would venture to say that we know virtually everything there is to know about measuring the performance of nonprofit organizations with only two exceptions: (1) How can we compare nonprofits with different missions or approaches, and (2) how can we make actionable performance assessments common practice for growth-ready mid-caps and readily available to all prospective donors?”

p. 263:
“Why would a social entrepreneur divert limited resources to impact assessment if there were no prospects it would increase funding? How could an investor who wanted to maximize the impact of her giving possibly put more golden eggs in fewer impact-producing baskets if she had no way to distinguish one basket from another? The result: there’s no performance data to attract growth capital, and there’s no growth capital to induce performance measurement. Until we fix that Catch-22, performance evaluation will not become an integral part of social enterprise.”

pp. 264-5:
Long quotation from Ken Berger at Charity Navigator on their ongoing efforts at developing an outcome measurement system. [wpf, 8 Nov 2009: I read the passage quoted by Goldberg in Berger’s blog when it came out and have been watching and waiting ever since for the new system. wpf, 8 Feb 2012: The new system has been online for some time but still does not include anything on impacts or outcomes. It has expanded from a sole focus on financials to also include accountability and transparency. But it does not yet address Goldberg’s concerns as there still is no way to tell what works from what doesn’t.]

p. 265:
“The failure of the social sector to coordinate independent assets and create a whole that exceeds the sum of its parts results from an absence of.. platform leadership’: ‘the ability of a company to drive innovation around a particular platform technology at the broad industry level.’ The object is to multiply value by working together: ‘the more people who use the platform products, the more incentives there are for complement producers to introduce more complementary products, causing a virtuous cycle.'” [Quotes here from Cusumano & Gawer (2002). The concept of platform leadership speaks directly to the system of issues raised by Miller & O’Leary (2007) that must be addressed to form effective HSN capital markets.]

p. 266:
“…the nonprofit sector has a great deal of both money and innovation, but too little available information about too many organizations. The result is capital fragmentation that squelches growth. None of the stakeholders has enough horsepower on its own to impose order on this chaos, but some kind of realignment could release all of that pent-up potential energy. While command-and-control authority is neither feasible nor desirable, the conditions are ripe for platform leadership.”

“It is doubtful that the IMPEX could amass all of the resources internally needed to build and grow a virtual nonprofit stock market that could connect large numbers of growth-capital investors with large numbers of [p. 267] growth-ready mid-caps. But it might be able to convene a powerful coalition of complementary actors that could achieve a critical mass of support for performance-based philanthropy. The challenge would be to develop an organization focused on filling the gaps rather than encroaching on the turf of established firms whose participation and innovation would be required to build a platform for nurturing growth of social enterprise..”

p. 268-9:
Intermediated nonprofit capital market shifts fundraising burden from grantees to intermediaries.

p. 271:
“The surging growth of national donor-advised funds, which simplify and reduce the transaction costs of methodical giving, exemplifies the kind of financial innovation that is poised to leverage market-based investment guidance.” [President of Schwab Charitable quoted as wanting to make charitable giving information- and results-driven.]

p. 272:
Rating agencies and organizations: Charity Navigator, Guidestar, Wise Giving Alliance.
Online donor rankings: GlobalGiving, GreatNonprofits, SocialMarkets
Evaluation consultants: Mathematica

Google’s mission statement: “to organize the world’s information and make it universally accessible and useful.”

p. 273:
Exhibit 9.4 Impact Index Whole Product
Image of stakeholders circling IMPEX:
Trading engine
Listed nonprofits
Data producers and aggregators
Trading community
Researchers and analysts
Investors and advisors
Government and business supporters

p. 275:
“That’s the starting point for replication [of social innovations that work]: finding and funding; matching money with performance.”

[WPF bottom line: Because Goldberg misses De Soto’s point about transparent representations resolving the mystery of capital, he is unable to see his way toward making the nonprofit capital markets function more like financial capital markets, with the difference being the focus on the growth of human, social, and natural capital. Though Goldberg intuits good points about the wisdom of crowds, he doesn’t know enough about the flaws of ordinal measurement relative to interval measurement, or about the relatively easy access to interval measures that can be had, to do the job.]

References

Cusumano, M. A., & Gawer, A. (2002, Spring). The elements of platform leadership. MIT Sloan Management Review, 43(3), 58.

De Soto, H. (2000). The mystery of capital: Why capitalism triumphs in the West and fails everywhere else. New York: Basic Books.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [http://www.rasch.org/rmt/rmt154j.htm].

Fisher, W. P., Jr. (2003). Measurement and communities of inquiry. Rasch Measurement Transactions, 17(3), 936-8 [http://www.rasch.org/rmt/rmt173.pdf].

Fisher, W. P., Jr. (2005). Daredevil barnstorming to the tipping point: New aspirations for the human sciences. Journal of Applied Measurement, 6(3), 173-9 [http://www.livingcapitalmetrics.com/images/FisherJAM05.pdf].

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-3 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In M. Wilson, K. Draney, N. Brown & B. Duckor (Eds.), Advances in Rasch Measurement, Vol. Two (p. in press [http://www.livingcapitalmetrics.com/images/BringingHSN_FisherARMII.pdf]). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2009b, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement (Elsevier), 42(9), 1278-1287.

Garner, M. (2009, Autumn). Google’s PageRank algorithm and the Rasch measurement model. Rasch Measurement Transactions, 23(2), 1201-2 [http://www.rasch.org/rmt/rmt232.pdf].

Grossman, A. (1999). Philanthropic social capital markets: Performance driven philanthropy (Social Enterprise Series 12 No. 00-002). Harvard Business School Working Paper.

Kotter, J. (1996). Leading change. Cambridge, Massachusetts: Harvard Business School Press.

Kurtzman, J. (2002). How the markets really work. New York: Crown Business.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-34.

Advertisements

Feminist Diffractions, Stochastic Resonance, and Education, Revisited

May 25, 2015

Lehrer (2015) offers an insightful commentary on Saxe et al’s (2015) recent article in Human Development that prompts some observations.

Two areas for questions and comments come to mind. The first has to do with construing the development and revision of new ways of understanding as contested, which implicitly aligns with Latour’s (1987, pp. 89, 93) sense of the way new constructs are subjected to tests of strength. Haraway (1996) makes an important point in her critique of what she sees as the overly masculinist metaphors of heroic competition and (perhaps not so) sublimated violence in these contests. Her sense of “feminist diffractions” stops short of what I have in mind, but opens the door to an alternative approach to what Lehrer calls the “close coupling of definitions with the development and revision of new concepts and ways of understanding.”

Galison (1997, pp. 843-844), for instance, seeks a metaphor capable of expressing what happens in the conceptual, practical, and argumentative contests between different communities of scientists (instrumentalist technicians, theoreticians, and experimentalists). He wants a metaphor that does justice to the disunified chaos and disorder one finds in the relationships between these different groups, which paradoxically results in such productive and coherent innovations. He recalls Peirce’s and Wittgenstein’s metaphors of cables and threads that take their strength from being intertwined from smaller wires and bits of fiber but finds these images too mechanical for his purposes. He wants something more akin to amorphous semiconductors or laminated materials that can fail microscopically but hold macroscopically better than more structurally homogenous materials.

Berg and Timmermans (2000, pp. 55-56) make a similar observation in their study of the constitution of universalities in medical fields:

“In order for a statistical logistics to enhance precise decision making, it has to incorporate imprecision; in order to be universal, it has to carefully select its locales. … Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.”

The general problem is taken up by Ricoeur (1992, p. 289), who raises the notion of “universals in context or of potential or inchoate universals” that embody the paradox in which

“on the one hand, one must maintain the universal claim attached to a few values where the universal and the historical intersect, and on the other hand, one must submit this claim to discussion, not on a formal level, but on the level of the convictions incorporated in concrete forms of life.”

To repeat another theme that comes up again and again in this blog, this kind of noise-induced order sounds like the phenomenon of stochastic resonance (Fisher, 1992, 2011). The importance of stochastic resonance is that it opens up a way to connect the phenomena of emergent understanding with measurement, both at the local individual and general systemic levels.

This is the crux of some very important issues in the philosophy of science and in philosophy generally. Haraway (1996, pp. 439-440), for instance, points out that “embedded relationality is the prophylaxis for both relativism and transcendence.” And Golinski (2012, p. 35) similarly says, “Practices of translation, replication, and metrology have taken the place of the universality that used to be assumed as an attribute of singular science.”

A start in the direction of embedded relationality, translation, replication, and metrology in education is apparent, for instance, in work that enables teachers to usefully relate individual student performances to general learning progressions, connecting instructional applications with accountability (Fisher & Wilson, 2015; Lehrer, 2013; Lehrer & Jones, 2014; Wilson, 2004). As Lehrer (2015, p. 49) says about the Saxe et al. work, “Recurrent forms of mathematical practice enabled the authors to create compelling trajectories of collective activity and learning over time while preserving the contributions of individual development.”

The second of the two topics I’d like to address comes up here in the closing paragraph of his short commentary, where Lehrer says a “hoped-for future innovation would make it possible to visualize individual and collective trajectories simultaneously.” Though future improvements can certainlty be expected, visualizations of individual and collective trajectories for growth in reading are already being recognized in both educational and metrological contexts (Stenner, Swartz, Hanlon, & Emerson, 2012; Stenner & Fisher, 2013, p. 4) for their potential to serve as the media of an embedded relationality capable of undercutting both the relativism of uncontrolled local variation and the universalist pretensions often built into accountability programs.

With emerging recognition of the potential Rasch’s stochastic approaches to construct mapping (Bond & Fox, 2007; Wilson, 2005) offer in the way of metrological translation networks (Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2015; Fisher & Wilson, 2015; Stenner & Fisher, 2013; Wilson, 2013; Wilson, Mari, Maul, & Torres Irribarra 2015), there are good reasons to expect significant new kinds of progress in fields that rely on assessments and surveys for outcome measurement and management.

References

Berg, M.,& Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Fisher, W. P., Jr. (1992). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [http://www.rasch.org/rmt/rmt54k.htm].

Fisher, W. P., Jr. (2011). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Fisher, W. P., Jr., & Stenner, A. J. (2015). The role of metrology in mobilizing and mediating the language and culture of scientific facts. Journal of Physics Conference Series, 588(012043).

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo, in review.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Haraway, D. J. (1996). Modest witness: Feminist diffractions in science studies. In P. Galison & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts, and power (pp. 428-441). Stanford, California: Stanford University Press.

Latour, B. (1987). Science in action: How to follow scientists and engineers through society. New York: Harvard University Press.

Lehrer, R. (2013, April 29). (Chair). In A learning progression emerges in a trading zone of professional community and identity. American Educational Research Association, Division C on Learning and Instruction, Section 2b on Learning and Motivation in Social and Cultural Contexts, San Francisco, CA.

Lehrer, R., & Jones, S. (2014, 2 April). Construct maps as boundary objects in the trading zone. In W. P. Fisher Jr. (Chair), Session 3-A: Rating Scales and Partial Credit, Theory and Applied. International Objective Measurement Workshop, Philadelphia, PA.

Lehrer, R. (2015). Designing for development: Commentary on Saxe, de Kirby, Kang, Le and Schneider. Human Development, 58(1), 45-49.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012002/pdf/1742-6596_459_1_012002.pdf.

Pendrill, L. (2014). Man as a measurement instrument [Special Feature]. NCSLi Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46-55.

Ricoeur, P. (1992). Oneself as another. Chicago, Illinois: University of Chicago Press.

Saxe, G. B., de Kirby, K., Kang, B., Le, M., & Schneider, A. (2015). Studying cognition through time in a classroom community: The interplay between “everyday” and “scientific” concepts. Human Development, 58(1), 5-44.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459(012025), http://iopscience.iop.org/1742-6596/459/1/012025.

Stenner, A. J., Swartz, C., Hanlon, S., & Emerson, C. (2012, February). Personalized learning platforms. Presented at the Pearson Global Research Conference, Fremantle, Western Australia.

Wilson, M. (Ed.). (2004). National Society for the Study of Education Yearbooks. Vol. 103, Part II: Towards coherence between classroom assessment and accountability. Chicago, Illinois: University of Chicago Press.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Wilson, M. R. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.

 

Moore’s Law at 50

May 13, 2015

Thomas Friedman interviewed Gordon Moore on the occasion of the 50th anniversary of Moore’s 1965 article predicting that computing power would exponentially increase at little additional cost. Moore’s ten-year prediction for the doubling rate of the numbers of transistors on microchips held up, and has now, with small adjustments, guided investments and expectations in electronics for five decades.

Friedman makes an especially important point, saying:

But let’s remember that it [Moore’s Law] was enabled by a group of remarkable scientists and engineers, in an America that did not just brag about being exceptional, but invested in the infrastructure and basic scientific research, and set the audacious goals, to make it so. If we want to create more Moore’s Law-like technologies, we need to invest in the building blocks that produced that America.”

These kinds of calls for investments in infrastructure and basic research, for new audacious goals, and for more Moore’s Law-like technologies are, of course, some of the primary and recurring themes of this blog (here, here, here, and here) and presentations and publications of the last several years. For instance, Miller and O’Leary’s (2007) close study of how Moore’s Law has aligned and coordinated investments in the electronics industry has been extrapolated into the education context (Fisher, 2012; Fisher & Stenner, 2011).

Education already has had over 60 years experience with a close parallel to Moore’s Law in reading measurement. Stenner’s Law retrospectively predicts exactly the same doubling period for the increasing numbers from 1960 to 2010 of children’s reading abilities measured in a common (or equatable) unit with known uncertainty and personalized consistency indicators. Knowledge of this kind has enabled manufacturers, suppliers, marketers, customers, and other stakeholders in the electronics industry to plan five and ten years into the future, preparing products and markets to take advantage of increased power and speed at the same or lower cost. Similarly, that same kind of knowledge could be used in education, health care, social services, and natural resource management to define the rules, roles, and responsibilities of actors and institutions involved in literacy, health, community, and natural capital markets.

Reading instruction, for example, requires text complexities to be matched to reader abilities at a comprehension rate that challenges but does not discourage the reader. Uniform grade-level textbooks are often too easy for a third of a given classroom, and too hard for another third. Individualized instruction by teachers in classrooms of 25 and more students is too cumbersome to implement. Connecting classroom reading assessments with known text complexity measures informed by judicious teacher input sets the stage for the realization of new potentials in educational outcomes. Electronic resources tapping existing text complexity measures for millions of articles and books connect individual students’ high stakes and classroom assessments in a common instructional framework (for instance, see here for an offering from Pearson). As the numbers of student reading measures made in a common unit continues to grow exponentially, capacities for connecting readers to texts, and for communicating about what works and what doesn’t in education, will grow as well.

This model is exactly the kind of infrastructure, basic scientific research, and audacious goal setting that’s needed if we are to succeed in creating more Moore’s Law-like technologies. If we as a society made the decision to invest deliberately, intentionally, and massively in infrastructure of this kind across education, health care, social services, and natural resource management, who knows what kinds of powerful results might be attained?

References

Fisher, W. P., Jr. (2012). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr., & Stenner, A. J. (2011, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium, http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf, Jena, Germany.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Living Capital Metrics for Financial and Sustainability Accounting Standards

May 1, 2015

I was very happy a few days ago to come across Jane Gleeson-White’s new book, Six Capitals, or Can Accountants Save the Planet? Rethinking Capitalism for the 21st Century. The special value for me in this book comes in the form of an accessible update on what’s been going on in the world of financial accounting standards. Happily, there’s been a lot of activity (check out, for instance, Amato & White, 2013; Rogers & White, 2015). Less fortunately, the activity seems to be continuing to occur in the same measurement vacuum it always has, despite my efforts in this blog to broaden the conversation to include rigorous measurement theory and practice.

But to back up a bit, recent events around sustainability metric standards don’t seem to be connected to previous controversies around financial standards and economic modeling, which were more academically oriented to problems of defining and expressing value. Gleeson-White doesn’t cite any of the extensive literature in those areas (for instance, Anielski, 2007; Baxter, 1979; Economist, 2010; Ekins, 1992, 1999; Ekins, Dresner, & Dahlstrom, 2008; Ekins, Hillman, & Hutchins, 1992; Ekins & Voituriez, 2009; Fisher, 2009b, 2009c, 2011; Young & Williams, 2010). Valuation is still a problem, of course, as is the analogy between accounting standards and scientific standards (Baxter, 1979). But much of the sensitivity of the older academic debate over accounting standards seems to have been lost in the mad, though well-intentioned, rush to devise metrics for the traditionally externalized nontraditional forms of capital.

Before addressing the thousands of metrics in circulation and the science that needs to be brought to bear on them (the ongoing theme of posts in this blog), some attention to terminology is important. Gleeson-White refers to six capitals (manufactured, liquid, intellectual, human, social, and natural), in contrast with Ekins (1992; Ekins, et al., 2008), who describes four (manufactured, human, social, and natural). Gleeson-White’s liquid capital is cash money, which can be invested in capital (a means of producing value via ongoing services) and which can be extracted as a return on capital, but is not itself capital, as is shown by the repeated historical experience in many countries of printing money without stimulating economic growth and producing value. Of her remaining five forms of capital, intellectual capital is a form of social capital that can satisfactorily be categorized alongside the other forms of organization-level properties and systems involving credibility and trust.

On pages 209-227, Gleeson-White takes up questions relevant to the measurement and information quality topics of this blog. The context here is informed by the International Integrated Reporting Council’s (IIRC) December 2013 framework for accounting reports integrating all forms of capital (Amato & White, 2013), and by related efforts of the Sustainability Accounting Standards Board (SASB) (Rogers & White, 2015). Following the IIRC, Gleeson-White asserts that

“Not all the new capitals can be quantified, yet or perhaps ever–for example, intellectual, human and social capital, much of natural capital–and so integrated reports are not expected to provide quantitative measures of each of the capitals.”

Of course, this opinion flies in the face of established evidence and theory accepted by both metrologists (weights and measures standards engineers and physicists) and psychometricians as to the viability of rigorous measurement standards for the outcomes of education, health care, social services, natural resource management, etc. (Fisher, 2009b, 2011, 2012a, 2012b; Fisher & Stenner, 2011a, 2013, 2015; Fisher & Wilson, 2015; Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2013, 2015; Wilson, 2013; Wilson, Mari, Maul, & Torres Irribarra, 2015). Pendrill (2014, p. 26), an engineer, physicist, and past president of the European Association of National Metrology Institutes, for instance, states that “The Rasch approach…is not simply a mathematical or statistical approach, but instead [is] a specifically metrological approach to human-based measurement.” As is repeatedly shown in this blog, access to scientific measures sets the stage for a dramatic transformation of the potential for succeeding in the goal of rethinking capitalism.

Next, Gleeson-White’s references to several of the six capitals as the “living” capitals (p. 193) is a literal reference to the fact that human, social, and natural capital are all carried by people, organizations/communities, and ecosystems. The distinction between dead and living capital elaborated by De Soto (2000) and Fisher (2002, 2007, 2010b, 2011), which involves making any form of capital fungible by representing it in abstract forms negotiable in banks and courts of law, is not taken into account, though this would seem to be a basic requirement that must be fulfilled before the rethinking of capitalism could said to have been accomplished.

Gleeson-White raises the pointed question as to exactly how integrated reporting is supposed to provoke positive growth in the nontraditional forms of capital. The concept of an economic framework integrating all forms of capital relative to the profit motive, as described in Ekins’ work, for instance, and as is elaborated elsewhere in this blog, seems just over the horizon, though repeated mention is made of natural capitalism (Hawken, Lovins, & Lovins, 1999). The posing of the questions provided by Gleeson-White (pp. 216-217) is priceless, however:

“…given integrated reporting’s purported promise to contribute to sustainable development by encouraging more efficient resource allocation, how might it actually achieve this for natural and social capitals on their own terms? It seems integrated reporting does nothing to address a larger question of resource allocation….”

“To me the fact that integrated reporting cannot address such questions suggests that as with the example of human capital, its promise to foster efficient resource allocation pertains only to financial capital and not to the other capitals. If we accept that the only way to save our societies and planet is to reconceive them in terms of capital, surely the efficient valuing and allocation of all six capitals must lie at the heart of any economics and accounting for the planet’s scarce resources in the twenty-first century.
“There is a logical inconsistency here: integrated reporting might be the beginning of a new accounting paradigm, but for the moment it is being practiced by an old-paradigm corporation: essentially, one obliged to make a return on financial capital at the cost of the other capitals.”

The goal requires all forms of capital to be integrated into the financial bottom line. Where accounting for manufactured capital alone burns living capital resources for profit, a comprehensive capital accounting framework defines profit in terms of reduced waste. This is a powerful basis for economics, as waste is the common root cause of human suffering, social discontent and environmental degradation (Hawken, Lovins, & Lovins, 1999).

Multiple bottom lines are counter-productive, as they allow managers the option of choosing which stakeholder group to satisfy, often at the expense of the financial viability of the firm (Jensen, 2001; Fisher, 2010a). Economic sustainability requires that profits be legally, morally, and scientifically contingent on a balance of powers distributed across all forms of capital. Though the devil will no doubt lurk in the details, there is increasing evidence that such a balance of powers can be negotiated.

A key point here not brought up by Gleeson-White concerns the fact that markets are not created by exchange activity, but rather by institutionalized rules, roles, and responsibilities (Miller & O’Leary, 2007) codified in laws, mores, technologies, and expectations. Translating historical market-making activities as they have played out relative to manufactured capital in the new domains of human, social, and natural capital faces a number of significant challenges, adapting to a new way of thinking about tests, assessments, and surveys foremost among them (Fisher & Stenner, 2011b).

One of the most important contributions advanced measurement theory and practice (Rasch, 1960; Wright, 1977; Andrich, 1988, 2004; Fisher & Wright, 1994; Wright & Stone, 1999; Bond & Fox, 2007; Wilson, 2005; Engelhard, 2012; Stenner, Fisher, Stone, & Burdick, 2013) can make to the process of rethinking capitalism involves the sorting out of the myriad metrics that have erupted in the last several years. Gleeson-White (p. 223) reports, for instance, that the Bloomberg financial information network now has over 750 ESG (Environmental, Social, Governance) data fields, which were extracted from reports provided by over 5,000 companies in 52 countries.  Similarly, Rogers and White (2015) say that

“…today there are more than 100 organizations offering more than 400 corporate sustainability ratings products that assess some 50,000 companies on more than 8,000 metrics of environmental, social and governance (ESG) performance.”

As is also the case with the UN Millennium Development Goals (Fisher, 2011b), the typical use of these metrics as single-item “quantities” is based in counts of relevant events. This procedure misses the basic point that counts of concrete things in the world are not measures. Is it not obvious that I can have ten rocks to your two, and you can still have more rock than I do? The same thing applies to any kind of performance ratings, survey responses, or test scores. We assign the same numeric increase to every addition of one more count, but hardly anyone experimentally tests the hypothesis that the counts all work together to measure the same thing. Those who think there’s no need for precision science in this context are ignoring the decades of successful and widespread technical work in this area, at their own risk.

The repetition of history here is fascinating. As Ashworth (2004, p. 1,314) put it, historically, “The requirements of increased trade and the fiscal demands of the state fuelled the march toward a regular form of metrology.” For instance, in 1875 it was noted that “the existence of quantitative correlations between the various forms of energy, imposes upon men of science the duty of bringing all kinds of physical quantity to one common scale of comparison” (Everett, 1875, p. 9). The moral and economic  value of common scales was recognized during the French revolution, when, Alder (2002, p. 32) documents, it was asked:

“Ought not a single nation have a uniform set of measures, just as a soldier fought for a single patrie? Had not the Revolution promised equality and fraternity, not just for France, but for all the people of the world? By the same token, should not all of the world’s people use a single set of weights and measures to encourage peaceable commerce, mutual understanding, and the exchange of knowledge? That was the purpose of measuring the world.”

The value of rigorously measuring human, social and natural capital includes meaningfully integrating qualitative substance with quantitative convenience, reduced data volume, augmenting measures with uncertainty and consistency indexes, and the capacity to take missing data into account (making possible instrument equating, item banking, etc.)  In contrast with the usual methods, rigorous science demands that experiments determine which indicators cohere to measure the same thing by repeatedly giving the same values across samples, over time and space, and across subsets of indicators. Beyond such data-based results, advanced theory makes it possible to arrive at explanatory, predictive methods that add a whole new layer of efficiency to the generation of indicators (de Boeck & Wilson, 2004; Stenner, et al., 2013).

Finally, Gleeson-White (pp. 220-221) reports that “In July 2011, the SASB [Sustainability Accounting Standards Board] was launched in the United States to create standardized measures for the new capitals.” “Founded by environmental engineer and sustainability expert Jean Rogers in San Francisco, SASB is creating a full set of industry-specific standards for sustainability accounting, with the aim of making this information more consistent and comparable.” As of May 2014, the SASB vice chair is Mary Schapiro, former SEC chair, and the chairman of SASB is Michael Bloomfield, former mayor of NYC and founder of the financial information empire. The “SASB is developing nonfinancial standards for eighty-nine industries grouped in ten different sectors and aims to have completed this grueling task by February 2015. It is releasing each set of metrics as they are completed.”

Like the SASB and other groups, Gleeson-White (p. 222) reports, Bloomberg

“aims to use its metrics to start ‘standardizing the discourse around sustainability, so we’re all talking about the same things in the same way,’ as Bloomberg’s senior sustainability strategist Andrew Park put it. What companies ‘desperately want,’ he says, is ‘a legitimate voice’ to tell them: ‘This is what you need to do. You exist in this particular sector. Here are the metrics that you need to be reporting out on. So SASB will provide that. And we think that’s important, because that will help clean up the metrics that ultimately the finance community will start using.’
“Bloomberg wants to price environmental, social and governance externalities to legitimize them in the eyes of financial capital.”

Gleeson-White (p. 225) continues, saying

“Bloomberg wants to do more generally what Trucost did for Puma’s natural capital inputs: create standardized measures for the new capitals–such as ecosystem services and social impacts–so that this information can be aggregated and used by investors. Park and Ravenel call the failure to value clean air, water, stable coastlines and other environmental goods ‘as much a failure to measure as it is a market failure per se–one that could be addressed in part by providing these ‘unpriced’ resources with quantitative parameters that would enable their incorporation into market mechanisms. Such mechanisms could then appropriately ‘regulate’ the consumption of those resources.'”

Integrating well-measured living capitals into the context of appropriately configured institutional rules, roles, and responsibilities for efficient markets (Fisher, 2010b) should indeed involve a capacity to price these resources quantitatively, though this capacity alone would likely prove insufficient to the task of creating the markets (Miller & O’Leary, 2007; Williamson, 1981, 1991, 2005). Rasch’s (1960, pp. 110-115) deliberate patterning of his measurement models on the form of Maxwell’s equations for Newton’s Second Law provides a mathematical basis for connecting psychometrics with both geometry and natural laws, as well as with the law of supply and demand (Fisher, 2010c, 2015; Fisher & Stenner, 2013a).

This perspective on measurement is informed by an unmodern or amodern, post-positivist philosophy (Dewey, 2012; Latour, 1990, 1993), as opposed to a modern and positivist, or postmodern and anti-positivist, philosophy (Galison, 1997). The essential difference is that neither a universalist nor a relativist perspective is necessary to the adoption of practices of traceability to metrological standards. Rather, focusing on local, situated, human relationships, as described by Wilson (2004) in education, for instance, offers a way of resolving the false dilemma of that dichotomous contrast. As Golinski (2012, p. 35) puts it, “Practices of translation, replication, and metrology have taken the place of the universality that used to be assumed as an attribute of singular science.” Haraway (1996, pp. 439-440) harmonizes, saying “…embedded relationality is the prophylaxis for both relativism and transcendance.” Latour (2005, pp. 228-229) elaborates, saying:

“Standards and metrology solve practically the question of relativity that seems to intimidate so many people: Can we obtain some sort of universal agreement? Of course we can! Provided you find a way to hook up your local instrument to one of the many metrological chains whose material network can be fully described, and whose cost can be fully determined. Provided there is also no interruption, no break, no gap, and no uncertainty along any point of the transmission. Indeed, traceability is precisely what the whole of metrology is about! No discontinuity allowed, which is just what ANT [Actor Network Theory] needs for tracing social topography. Ours is the social theory that has taken metrology as the paramount example of what it is to expand locally everywhere, all while bypassing the local as well as the universal. The practical conditions for the expansion of universality have been opened to empirical inquiries. It’s not by accident that so much work has been done by historians of science into the situated and material extension of universals. Given how much modernizers have invested into universality, this is no small feat.
“As soon as you take the example of scientific metrology and standardization as your benchmark to follow the circulation of universals, you can do the same operation for other less traceable, less materialized circulations: most coordination among agents is achieved through the dissemination of quasi-standards.”

As Rasch (1980: xx) understood, “this is a huge challenge, but once the problem has been formulated it does seem possible to meet it.” Though some metrologically informed traceability networks have begun to emerge in education and health care (for instance, Fisher & Stenner, 2013, 2015; Stenner & Fisher, 2013), virtually everything remains to be done to make the coordination across stakeholders as fully elaborated as the standards in the natural sciences.

References

Alder, K. (2002). The measure of all things: The seven-year odyssey and hidden error that transformed the world. New York: The Free Press.

Amato, N., & White, S. (2013, December 7). IIRC releases International Integrated Reporting Framework. Journal of Accountancy. Retrieved from http://www.journalofaccountancy.com/news/2013/dec/20139207.html

Andrich, D. (1988). Sage University Paper Series on Quantitative Applications in the Social Sciences. Vol. series no. 07-068: Rasch models for measurement. Beverly Hills, California: Sage Publications.

Andrich, D. (2004, January). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1), I-7–I-16.

Andrich, D. (2010). Sufficiency and conditional estimation of person parameters in the polytomous Rasch model. Psychometrika, 75(2), 292-308.

Anielski, M. (2007). The economics of happiness: Building genuine wealth. Gabriola, British Columbia: New Society Publishers.

Ashworth, W. J. (2004, 19 November). Metrology and the state: Science, revenue, and commerce. Science, 306(5700), 1314-1317.

Baxter, W. T. (1979). Accounting standards: Boon or curse? In The Emmanuel Saxe distinguished lectures in accounting. http://newman.baruch.cuny.edu/digital/saxe/saxe_1978/baxter_79.htm.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. Statistics for Social and Behavioral Sciences). New York: Springer-Verlag.

De Soto, H. (2000). The mystery of capital: Why capitalism triumphs in the West and fails everywhere else. New York: Basic Books.

Dewey, J. (2012). Unmodern philosophy and modern philosophy (P. Deen, Ed.). Carbondale, Illinois: Southern Illinois University Press.

Editorial. (2010, 10 June). Accounting standards: To FASB or not to FASB? The Economist, http://www.economist.com/node/16319655.

Ekins, P. (1992). A four-capital model of wealth creation. In P. Ekins & M. Max-Neef (Eds.), Real-life economics: Understanding wealth creation (pp. 147-155). London: Routledge.

Ekins, P. (1999). Economic growth and environmental sustainability: The prospects for green growth. New York: Routledge.

Ekins, P., Dresner, S., & Dahlstrom, K. (2008, March/April). The four-capital method of sustainable development evaluation. European Environment, 18(2), 63-80.

Ekins, P., Hillman, M., & Hutchison, R. (1992). The Gaia atlas of green economics (Foreword by Robert Heilbroner). New York: Anchor Books.

Ekins, P., & Voituriez, T. (2009). Trade, globalization and sustainability impact assessment: A critical look at methods and outcomes. London, England: Earthscan Publications Ltd.

Engelhard, G., Jr. (2012). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. New York: Routledge Academic.

Everett, J. D. (1875). Illustrations of the C. G. S. system of units. London, England: Taylor & Francis.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [http://www.rasch.org/rmt/rmt154j.htm].

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009a, November 19). Draft legislation on development and adoption of an intangible assets metric system. Retrieved 6 January 2011, from https://livingcapitalmetrics.wordpress.com/2009/11/19/draft-legislation/

Fisher, W. P., Jr. (2009b, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009c). NIST Critical national need idea White Paper: metrological infrastructure for human, social, and natural capital (Tech. Rep. No. http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC:. National Institute for Standards and Technology.

Fisher, W. P., Jr. (2010a, 22 November). Meaningfulness, measurement, value seeking, and the corporate objective function: An introduction to new possibilities., LivingCapitalMetrics.com, Sausalito, California. Retrieved from http://ssrn.com/abstract=1713467

Fisher, W. P., Jr. (2010b). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital, Bridge to Business Postdoctoral Certification, Freeman School of Business, Tulane University (http://ssrn.com/abstract=2340674).

Fisher, W. P., Jr. (2010c). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics: Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2011b). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com. Retrieved 18 January 2011, from Social Science Research Network: http://ssrn.com/abstract=1739386.

Fisher, W. P., Jr. (2012a). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr. (2012b, May/June). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].

Fisher, W. P., Jr. (2015). A Rasch perspective on the law of supply and demand. Rasch Measurement Transactions, in press.

Fisher, W. P., Jr., Harvey, R. F., & Kilgore, K. M. (1995). New developments in functional assessment: Probabilistic models for gold standards. NeuroRehabilitation, 5(1), 3-25.

Fisher, W. P., Jr., Harvey, R. F., Taylor, P., Kilgore, K. M., & Kelly, C. K. (1995, February). Rehabits: A common language of functional assessment. Archives of Physical Medicine and Rehabilitation, 76(2), 113-122.

Fisher, W. P., Jr., & Stenner, A. J. (2011a, January). Metrology for the social, behavioral, and economic sciences (Social, Behavioral, and Economic Sciences White Paper Series). Retrieved 12 January 2014, from National Science Foundation: http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36.

Fisher, W. P., Jr., & Stenner, A. J. (2011b, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium, http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf, Jena, Germany.

Fisher, W. P., Jr., & Stenner, A. J. (2013a). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Fisher, W. P., Jr., & Stenner, A. J. (2013b). Overcoming the invisibility of metrology: A reading measurement network for education and the social sciences. Journal of Physics: Conference Series, 459(012024), http://iopscience.iop.org/1742-6596/459/1/012024.

Fisher, W. P., Jr., & Stenner, A. J. (2015). The role of metrology in mobilizing and mediating the language and culture of scientific facts. Journal of Physics Conference Series, 588(012043).

Fisher, W. P., Jr., & Stenner, A. J. (2015). Theory-based metrological traceability in education: A reading measurement network. Measurement, in review.

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo, in review.

Fisher, W. P., Jr., & Wright, B. D. (1994). Introduction to probabilistic conjoint measurement theory and applications (W. P. Fisher, Jr., & B. D. Wright, Eds.) [Special issue]. International Journal of Educational Research, 21(6), 559-568.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Gleeson-White, J. (2015). Six capitals, or can accountants save the planet? Rethinking capitalism for the 21st century. New York: Norton.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Haraway, D. J. (1996). Modest witness: Feminist diffractions in science studies. In P. Galison & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts, and power (pp. 428-441). Stanford, California: Stanford University Press.

Hawken, P., Lovins, A., & Lovins, H. L. (1999). Natural capitalism: Creating the next industrial revolution. New York: Little, Brown, and Co.

Jensen, M. C. (2001, Fall). Value maximization, stakeholder theory, and the corporate objective function. Journal of Applied Corporate Finance, 14(3), 8-21.

Latour, B. (1990). Postmodern? No, simply amodern: Steps towards an anthropology of science. Studies in History and Philosophy of Science, 21(1), 145-71.

Latour, B. (1993). We have never been modern. Cambridge, Massachusetts: Harvard University Press.

Latour, B. (2005). Reassembling the social: An introduction to Actor-Network-Theory. Clarendon Lectures in Management Studies). Oxford, England: Oxford University Press.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1), http://iopscience.iop.org/1742-6596/459/1/012002/pdf/1742-6596_459_1_012002.pdf.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Pendrill, L. (2014, December). Man as a measurement instrument [Special Feature]. NCSLI Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2013). Quantifying human response: Linking metrological and psychometric characterisations of man as a measurement instrument. Journal of Physics: Conference Series, 459, http://iopscience.iop.org/1742-6596/459/1/012057.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, p. in press. doi: http://dx.doi.org/10.1016/j.measurement.2015.04.010.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.

Rogers, J., & White, A. (2015, April 28). Focusing corporate sustainability ratings on what matters. Huffington Post. Retrieved from http://www.huffingtonpost.com/jean-rogers/focusing-corporate-sustai_b_7156148.html.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459(012025), http://iopscience.iop.org/1742-6596/459/1/012025.

Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2013, August). Causal Rasch models. Frontiers in Psychology: Quantitative Psychology and Measurement, 4(536), 1-14 [doi: 10.3389/fpsyg.2013.00536].

Williamson, O. E. (1981, November). The economics of organization: The transaction cost approach. The American Journal of Sociology, 87(3), 548-577.

Williamson, O. E. (1991). Economic institutions: Spontaneous and intentional governance [Special issue]. Journal of Law, Economics, & Organization: Papers from the Conference on the New Science of Organization, 7, 159-187.

Williamson, O. E. (2005). The economics of governance. American Economic Review, 95(2), 1-18.

Wilson, M. (Ed.). (2004). National Society for the Study of Education Yearbooks. Vol. 103, Part II: Towards coherence between classroom assessment and accountability. Chicago, Illinois: University of Chicago Press.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Wilson, M. R. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.

Wilson, M., Mari, L., Maul, A., & Torres Irribarra, D. (2015). A comparison of measurement concepts across physical science and social science domains: Instrument design, calibration, and measurement. Journal of Physics: Conference Series, 588(012034), http://iopscience.iop.org/1742-6596/588/1/012034.

Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14(2), 97-116 [http://www.rasch.org/memo42.htm].

Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every educator and psychologist should know (pp. 65-104 [http://www.rasch.org/memo64.htm]). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Wright, B. D., & Stone, M. H. (1999). Measurement essentials. Wilmington, DE: Wide Range, Inc. [http://www.rasch.org/measess/me-all.pdf].

Young, J. J., & Williams, P. F. (2010, August). Sorting and comparing: Standard-setting and “ethical” categories. Critical Perspectives on Accounting, 21(6), 509-521.

An Entrepreneurial Investment Model Alternative to Picketty’s Taxation Approach to Eliminating Wealth Disparities

May 14, 2014

Is taxation the only or the best solution to inequality? The way discussions of wealth disparities inevitably focus on variations in how, whom or what to tax, it is easy to assume there are no viable alternatives to taxation. But if the point is to invest in those with the most potential for making significant gains in productivity, so as to maximize the returns we realize, do we not wrongly constrain the domain of possible solutions when we misconceive an entrepreneurial problem in welfare terms?

Why can’t we require minimum levels of investment in social capital stocks and bonds offered by schools, hospitals, NGOs, etc? In human capital instruments offered by individuals? Why should not we expect those investments to be used to create new value? What supposed law of nature says it is impossible to associate new human, social and environmental value with stable and meaningful prices? And if there is such a law (such as Kenneth Arrow (1963) proposed), how can we break it? Why can’t we reconceive human and social capital stocks and flows in new ways?

There is one very good reason why we cannot now make such requirements, and it is the same reason why liberals (including me) had better become accustomed to accepting the failure of their agenda. That reason is this: social and environmental externalities. Inequality is inevitable only as long as we do not change the ways we deal with externalities. They can no longer be measured and managed in the same ways. They must be put on the books, brought into the models, measured scientifically, and traded in efficient markets. We have to invent accountability and accounting systems that harness the energy of the profit motive for the greater good—that actually grow authentic wealth and not mere money—and we have to do this far more effectively than has ever been done before.

It’s a tall order. But there are resources available to us that have not yet been introduced into the larger conversation. There are options to consider that need close study and creative experimentation. Proceeding toward the twin futilities of premature despair or unrealistic taxation will only set up another round of self-fulfilling prophecies inexorably grinding to yet another unforeseen but fully foretold disaster. Conversations about how to shape the roles, rules and institutions that make markets what they are (Miller and O’Leary, 2007) need to take place for human, social, and natural capital (Fisher and Stenner, 2011b). Indeed, those conversations are already well underway, as can be seen in the prior entries in this blog and in the sources listed below.

Arrow, K. J. (1963). Uncertainty and the welfare economics of medical care. American Economic Review, 53, 941-973.

Fisher, W. P., Jr. (2007). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-1093 [http://www.rasch.org/rmt/rmt211.pdf].

Fisher, W. P., Jr. (2009a). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2009b). NIST Critical national need idea White Paper: Metrological infrastructure for human, social, and natural capital (http://www.nist.gov/tip/wp/pswp/upload/202_metrological_infrastructure_for_human_social_natural.pdf). Washington, DC: National Institute for Standards and Technology (11 pages).

Fisher, W. P., Jr. (2010a, 22 November). Meaningfulness, measurement, value seeking, and the corporate objective function: An introduction to new possibilities. Sausalito, California: LivingCapitalMetrics.com (http://ssrn.com/abstract=1713467).

Fisher, W. P. J. (2010b). Measurement, reduced transaction costs, and the ethics of efficient markets for human, social, and natural capital (http://ssrn.com/abstract=2340674). Bridge to Business Postdoctoral Certification, Freeman School of Business: Tulane University.

Fisher, W. P., Jr. (2010c, June 13-16). Rasch, Maxwell’s method of analogy, and the Chicago tradition. In G. Cooper (Ed.), https://conference.cbs.dk/index.php/rasch/Rasch2010/paper/view/824. Probabilistic models for measurement in education, psychology, social science and health: Celebrating 50 years since the publication of Rasch’s Probabilistic Models. FUHU Conference Centre, Copenhagen, Denmark: University of Copenhagen School of Business.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. Journal of Applied Measurement, 12(1), 49-66.

Fisher, W. P., Jr. (2011b, Thursday, September 1). Measurement, metrology and the coordination of sociotechnical networks. In S. Bercea (Ed.), New Education and Training Methods. International Measurement Confederation (IMEKO). Jena, Germany: http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24491/ilm1-2011imeko-017.pdf.

Fisher, W. P., Jr. (2012a). Measure and manage: Intangible assets metric standards for sustainability. In J. Marques, S. Dhiman & S. Holt (Eds.), Business administration education: Changes in management and leadership strategies (pp. 43-63). New York: Palgrave Macmillan.

Fisher, W. P., Jr. (2012b, May/June). What the world needs now: A bold plan for new standards [Third place, 2011 NIST/SES World Standards Day paper competition]. Standards Engineering, 64(3), 1 & 3-5 [http://ssrn.com/abstract=2083975].

Fisher, W. P., Jr., & Stenner, A. J. (2011a, January). Metrology for the social, behavioral, and economic sciences. http://www.nsf.gov/sbe/sbe_2020/submission_detail.cfm?upld_id=36.

Fisher, W. P., Jr., & Stenner, A. J. (2011b, August 31 to September 2). A technology roadmap for intangible assets metrology. In Fundamentals of measurement science. International Measurement Confederation (IMEKO) TC1-TC7-TC13 Joint Symposium. Jena, Germany: http://www.db-thueringen.de/servlets/DerivateServlet/Derivate-24493/ilm1-2011imeko-018.pdf.

Fisher, W. P., Jr., & Stenner, A. J. (2013a). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Fisher, W. P., Jr., & Stenner, A. J. (2013b). Overcoming the invisibility of metrology: A reading measurement network for education and the social sciences. Journal of Physics: Conference Series, 459(012024), http://iopscience.iop.org/1742-6596/459/1/012024.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Creatively Expressing How Love Matters for Justice: Setting the Stage and Tuning the Instruments

April 16, 2014

Nussbaum (2013) argues about the political importance of connecting with our bodies without shame and disgust, and of the relevance musical and poetic public expressions of varieties of love offer to conceptions of justice. Institutions embodying principles of loving justice require media integrating emotional expression with technical calculation, in exactly the same way music does. Being able to dance at the revolution demands instruments tuned to shared scales, no matter if equal temperament, just intonation, meantone tuning, or any of a variety of other well, or irregular, temperaments are chosen.

The physicality of dancing, so often evoking romance and courtship, provides a point of entry to a metaphoric logic of reproduction applicable to the Socratic midwifery of ideas and to the products of social intercourse. Tuning the instruments of the human, social, and environmental arts and sciences to harmonize and choreograph relationships may then enable formulation of nonreductionist approaches to the problem of how to reconcile political emotions with physical or geometrical accounts of the scales of justice.

Historical accounts of (musical, medical, electrical, etc.) metrological standards describe ways in which passionate concern for shared vulnerabilities and common joys have sometimes succeeded in deploying systems realizing higher forms of just relations (Alder, 2002; Berg and Timmermans, 2000;  Isacoff, 2001; Schaffer, 1992). The question of the day is whether we will succeed in creating yet new forms of such relations in the many areas of life where they are needed.

Yes, as Nussbaum (2013, p. 396) admits, the demand for love is a tall order, and unrealistic. But all heuristic fictions, from Pythagorean triangles to the mathematical pendulum, are unrealistic and are never actually observed in practice, as has been pointed out by a number of historians and philosophers (Butterfield 1957, pp. 16-17; Heidegger, 1967, p. 89; Rasch, 1960, pp. 37-38, 1973/2011). These fictions are, however, eminently useful as guides, goals, and as coherent ways of telling our stories, and that is the criterion by which they should be judged.

 

Alder, K. (2002). The measure of all things: The seven-year odyssey and hidden error that transformed the world. New York: The Free Press.

Berg, M., & Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Butterfield, H. (1957). The origins of modern science (revised edition). New York: The Free Press.

Heidegger, M. (1967). What is a thing? (W. B. Barton, Jr. & V. Deutsch, Trans.). South Bend, Indiana: Regnery/Gateway.

Isacoff, S. M. (2001). Temperament: The idea that solved music’s greatest riddle. New York: Alfred A. Knopf.

Nussbaum, M. (2013). Political emotions: Why love matters for justice. Cambridge, MA: The Belknap Press of Harvard University Press.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.)

Rasch, G. (1973/2011, Spring). All statistical models are wrong! Comments on a paper presented by Per Martin-Löf, at the Conference on Foundational Questions in Statistical Inference, Aarhus, Denmark, May 7-12, 1973. Rasch Measurement Transactions, 24(4), 1309 [http://www.rasch.org/rmt/rmt244.pdf].

Schaffer, S. (1992). Late Victorian metrology and its instrumentation: A manufactory of Ohms. In R. Bud & S. E. Cozzens (Eds.), Invisible connections: Instruments, institutions, and science (pp. 23-56). Bellingham, WA: SPIE Optical Engineering Press.

Six Classes of Results Supporting the Measurability of Human Functioning and Capability

April 12, 2014

Another example of high-level analysis that suffers from a lack of input from state of the art measurement arises in Nussbaum (1997, p. 1205), where the author remarks that it is now a matter of course, in development economics, “to recognize distinct domains of human functioning and capability that are not commensurable along a single metric, and with regard to which choice and liberty of agency play a fundamental structuring role.” Though Nussbaum (2011, pp. 58-62) has lately given a more nuanced account of the challenges of measurement relative to human capabilities, appreciation of the power and flexibility of contemporary measurement models, methods, and instruments remains lacking. For a detailed example of the complexities and challenges that must be addressed in the context of global human development, which is Nussbaum’s area of interest, see Fisher (2011).

Though there are indeed domains of human functioning and capability that are not commensurable along a single metric, they are not the ones referred to by Nussbaum or the texts she cites. On the contrary, six different approaches to establishing the measurability of human functioning and capability have been explored and proven as providing, especially in their composite aggregate, a substantial basis for theory and practice (modified from Fisher, 2009, pp. 1279-1281). These six classes of results speak to the abstract, mathematical side of the paradox noted by Ricoeur (see previous post here) concerning the need to simultaneously accept roles for abstract ideal global universals and concrete local historical contexts in strategic planning and thinking. The six classes of results are:

  1. Mathematical proofs of the necessity and sufficiency of test and survey scores for invariant measurement in the context of Rasch’s probabilistic models (Andersen, 1977, 1999; Fischer, 1981; Newby, Conner, Grant, and Bunderson, 2009; van der Linden, 1992).
  2. Reproduction of physical units of measurement (centimeters, grams, etc.) from ordinal observations (Choi, 1997; Moulton, 1993; Pelton and Bunderson, 2003; Stephanou and Fisher, 2013).
  3. The common mathematical form of the laws of nature and Rasch models (Rasch, 1960, pp. 110-115; Fisher, 2010; Fisher and Stenner, 2013).
  4. Multiple independent studies of the same constructs on different (and common) samples using different (and the same) instruments intended to measure the same thing converge on common units, defining the same objects, substantiating theory, and supporting the viability of standardized metrics (Fisher, 1997a, 1997b, 1999, etc.).
  5. Thousands of peer-reviewed publications in hundreds of scientific journals provide a wide-ranging and diverse array of supporting evidence and theory.
  6. Analogous causal attributions and theoretical explanatory power can be created in both natural and social science contexts (Stenner, Fisher, Stone, and Burdick, 2013).

What we have here, in sum, is a combination of Greek axiomatic and Babylonian empirical algorithms, in accord with Toulmin’s (1961, pp. 28-33) sense of the contrasting principled bases for scientific advancement. Feynman (1965, p. 46) called for less of a focus on the Greek chain of reasoning approach, as it is only as strong as its weakest link, whereas the Babylonian algorithms are akin to a platform with enough supporting legs that one or more might fail without compromising its overall stability. The variations in theory and evidence under these six headings provide ample support for the conceptual and practical viability of metrological systems of measurement in education, health care, human resource management, sociology, natural resource management, social services, and many other fields. The philosophical critique of any type of economics will inevitably be wide of the mark if uninformed about these accomplishments in the theory and practice of measurement.

References

Andersen, E. B. (1977). Sufficient statistics and latent trait models. Psychometrika, 42(1), 69-81.

Andersen, E. B. (1999). Sufficient statistics in educational measurement. In G. N. Masters & J. P. Keeves (Eds.), Advances in measurement in educational research and assessment (pp. 122-125). New York: Pergamon.

Choi, S. E. (1997). Rasch invents “ounces.” Rasch Measurement Transactions, 11(2), 557 [http://www.rasch.org/rmt/rmt112.htm#Ounces].

Feynman, R. (1965). The character of physical law. Cambridge, Massachusetts: MIT Press.

Fischer, G. H. (1981). On the existence and uniqueness of maximum-likelihood estimates in the Rasch model. Psychometrika, 46(1), 59-77.

Fisher, W. P., Jr. (1997). Physical disability construct convergence across instruments: Towards a universal metric. Journal of Outcome Measurement, 1(2), 87-113.

Fisher, W. P., Jr. (1997). What scale-free measurement means to health outcomes research. Physical Medicine & Rehabilitation State of the Art Reviews, 11(2), 357-373.

Fisher, W. P., Jr. (1999). Foundations for health status metrology: The stability of MOS SF-36 PF-10 calibrations across samples. Journal of the Louisiana State Medical Society, 151(11), 566-578.

Fisher, W. P., Jr. (2009). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2010). The standard model in the history of the natural sciences, econometrics, and the social sciences. Journal of Physics: Conference Series, 238(1), http://iopscience.iop.org/1742-6596/238/1/012016/pdf/1742-6596_238_1_012016.pdf.

Fisher, W. P., Jr. (2011). Measuring genuine progress by scaling economic indicators to think global & act local: An example from the UN Millennium Development Goals project. LivingCapitalMetrics.com. Retrieved 18 January 2011, from Social Science Research Network: http://ssrn.com/abstract=1739386.

Fisher, W. P., Jr., & Stenner, A. J. (2013). On the potential for improved measurement in the human and social sciences. In Q. Zhang & H. Yang (Eds.), Pacific Rim Objective Measurement Symposium 2012 Conference Proceedings (pp. 1-11). Berlin, Germany: Springer-Verlag.

Moulton, M. (1993). Probabilistic mapping. Rasch Measurement Transactions, 7(1), 268 [http://www.rasch.org/rmt/rmt71b.htm].

Newby, V. A., Conner, G. R., Grant, C. P., & Bunderson, C. V. (2009). The Rasch model and additive conjoint measurement. Journal of Applied Measurement, 107(4), 348-354.

Nussbaum, M. (1997). Flawed foundations: The philosophical critique of (a particular type of) economics. University of Chicago Law Review, 64, 1197-1214.

Nussbaum, M. (2011). Creating capabilities: The human development approach. Cambridge, MA: The Belknap Press.

Pelton, T., & Bunderson, V. (2003). The recovery of the density scale using a stochastic quasi-realization of additive conjoint measurement. Journal of Applied Measurement, 4(3), 269-281.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Reprint, with Foreword and Afterword by B. D. Wright, Chicago: University of Chicago Press, 1980). Copenhagen, Denmark: Danmarks Paedogogiske Institut.

Rasch, G. (1977). On specific objectivity: An attempt at formalizing the request for generality and validity of scientific statements. Danish Yearbook of Philosophy, 14, 58-94.

Stenner, A. J., Fisher, W. P., Jr., Stone, M. H., & Burdick, D. S. (2013). Causal Rasch models. Frontiers in Psychology: Quantitative Psychology and Measurement, 4(536), 1-14.

Stephanou, A., & Fisher, W. P., Jr. (2013). From concrete to abstract in the measurement of length. Journal of Physics Conference Series, 459, http://iopscience.iop.org/1742-6596/459/1/012026.

Toulmin, S. E. (1961). Foresight and understanding: An enquiry into the aims of science. London, England: Hutchinson.

van der Linden, W. J. (1992). Sufficient and necessary statistics. Rasch Measurement Transactions, 6(3), 231 [http://www.rasch.org/rmt/rmt63d.htm].

 

Revisiting the “Glocal” integration of universals and historical context

April 11, 2014

Integrated considerations of the universal and the local, the pure ideal parameters and the messy concrete observations, seem ever more ubiquitous in my reading lately. For instance, Ricoeur (1992, p. 289) takes up the problem of human rights imperfectly realized as a product of Western Europe’s cultural history that has nonetheless been adopted by nearly every country in the world. Ricoeur raises the notion of “universals in context or of potential or inchoate universals” that embody the paradox in which

“on the one hand, one must maintain the universal claim attached to a few values where the universal and the historical intersect, and on the other hand, one must submit this claim to discussion, not on a formal level, but on the level of the convictions incorporated in concrete forms of life.”

I could hardly come up with a better description of Rasch measurement theory and practice myself. Any given Rasch model data analysis provides many times more individual-level qualitative statistics on the concrete, substantive observations than on the global quantitative measures. The whole point of graphical displays of measurement information in kidmaps (Chien, Wang, Wang, & Lin, 2009; Masters, 1994), Wright maps (Wilson, 2011), construct maps and self-scoring forms (Best, 2008; Linacre, 1997), etc. is precisely to integrate concrete events as they happened with the abstract ideal of a shared measurement dimension.

It is such a shame that there are so few people thinking about these issues aware of the practical value of the state of the art in measurement, and who include all of the various implications of multifaceted, multilevel, and multi-uni-dimensional modeling, fit assessment, equating, construct mapping, standard setting, etc. in their critiques.

The problem falls squarely in the domain of recent work on the coproduction of social, scientific, and economic orders (such as Hutchins 2010, 2012; Nersessian, 2012). Systems of standards, from languages to metric units to dollars, prethink the world for us and simplify a lot of complex work. But then we’re stuck at the level of conceptual, social, economic, and scientific complexity implied by those standards, unless we can create new forms of social organization integrating more domains. Those who don’t know anything about the available tools can’t get any analytic traction, those who know about the tools but don’t connect with the practitioners can’t get any applied traction (see Wilson’s Psychometric Society Presidential Address on this; Wilson, 2013), analysts and practitioners who form alliances but fail to include accountants or administrators may lack financial or organizational traction, etc. etc.

There’s a real need to focus on the formation of alliances across domains of practice, building out the implications of Callon’s (1995, p. 58) observation that “”translation networks weave a socionature.” In other words, standards are translated into the languages of different levels and kinds of practice to the extent that people become so thoroughly habituated to them that they succumb to the illusion that the objects of interest are inherently natural in self-evident ways. (My 2014 IOMW talk took this up, though there wasn’t a lot of time for details.)

Those who are studying these networks have come to important insights that set the stage for better measurement and metrology for human, social, and natural capital. For instance, in a study of universalities in medicine, Berg and Timmermans (2000, pp. 55, 56) note:

“In order for a statistical logistics to enhance precise decision making, it has to incorporate imprecision; in order to be universal, it has to carefully select its locales. The parasite cannot be killed off slowly by gradually increasing the scope of the Order. Rather, an Order can thrive only when it nourishes its parasite—so that it can be nourished by it.”

“Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.”

Though Berg and Timmermans show no awareness at all of probabilistic and additive conjoint measurement theory and practice, their description of how a statistical logistics has to work to enhance precise decision making is right on target. This phenomenon of noise-induced order is a kind of social stochastic resonance (Fisher, 1992, 2011b) that provides another direction in which explanations of Rasch measurement’s potential role in establishing new metrological standards (Fisher, 2009, 2011a) have to be taken.

Berg, M., & Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Best, W. R. (2008). A construct map that Ben Wright would relish. Rasch Measurement Transactions, 22(3), 1169-70 [http://www.rasch.org/rmt/rmt223a.htm].

Callon, M. (1995). Four models for the dynamics of science. In S. Jasanoff, G. E. Markle, J. C. Petersen & T. Pinch (Eds.), Handbook of science and technology studies (pp. 29-63). Thousand Oaks, California: Sage Publications.

Chien, T.-W., Wang, W.-C., Wang, H.-Y., & Lin, H.-J. (2009). Online assessment of patients’ views on hospital performances using Rasch model’s KIDMAP diagram. BMC Health Services Research, 9, 135 [10.1186/1472-6963-9-135 or http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2727503/%5D.

Fisher, W. P., Jr. (1992, Spring). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [http://www.rasch.org/rmt/rmt54k.htm].

Fisher, W. P., Jr. (2009, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement, 42(9), 1278-1287.

Fisher, W. P., Jr. (2011a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In N. Brown, B. Duckor, K. Draney & M. Wilson (Eds.), Advances in Rasch Measurement, Vol. 2 (pp. 1-27). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2011b). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science, 2, 705-715.

Hutchins, E. (2012). Concepts in practice as sources of order. Mind, Culture, and Activity, 19, 314-323.

Linacre, J. M. (1997). Instantaneous measurement and diagnosis. Physical Medicine and Rehabilitation State of the Art Reviews, 11(2), 315-324 [http://www.rasch.org/memo60.htm].

Masters, G. N. (1994). KIDMAP – a history. Rasch Measurement Transactions, 8(2), 366 [http://www.rasch.org/rmt/rmt82k.htm].

Nersessian, N. J. (2012). Engineering concepts: The interplay between concept formation and modeling practices in bioengineering sciences. Mind, Culture, and Activity, 19, 222-239.

Wilson, M. R. (2011). Some notes on the term: “Wright Map.” Rasch Measurement Transactions, 25(3), 1331 [http://www.rasch.org/rmt/rmt253.pdf].

Wilson, M. (2013, April). Seeking a balance between the statistical and scientific elements in psychometrics. Psychometrika, 78(2), 211-236.

Comment on Kerrey and Leeds in WSJ

November 20, 2013

Writing in today’s Wall Street Journal, Bob Kerrey and Jeffery T. Leeds note the unintended consequences likely to follow from new higher education regulations proposed by the U.S. Department of Education. Cutting to the chase, Kerrey and Leeds’ key points (emphases added) are that:

  • “Absent innovative, competitive—and, yes, disruptive—pressure to raise quality and lower costs, all the well-intentioned federal regulation in the world will not make college more accessible.”
  • “He [Secretary of Education, Arne Duncan] should insist on real and significant disclosure. Colleges should be required to post their graduation rates, job-placement rates, the average debt of their students upon graduation, their tax status and any and all information that will enable Americans to make informed decisions when choosing a school.”
  • “The department should also work with schools and colleges to address the fundamental causes of rising tuition, and hold schools accountable for student outcomes instead of their debt.”

These are, of course, exactly the themes repeatedly raised in this blog. Measurement quality is unavoidably implicated in holding schools accountable for student outcomes, in enabling consumers to make informed purchasing decisions, and in raising quality and lowering costs.

To meet the challenges we face, measurement quality must be far more than just a matter of precision and rigor. Quality must also speak to relevance, efficiency, and meaningfulness. Recent history has brought home the lesson that annual tests used solely for accountability purposes will not enable rebalanced quality/cost equations, informed consumer decisions, or fair accountability results. But how might these disparate purposes be efficiently and meaningfully realized?

It is essential that, if teachers are to be responsible for student outcomes and for raising the overall quality of education, formative measuring tools must provide the qualitative and quantitative information they need to be able to act responsibly. The irony is, of course, that the way to overcome the problems of a purely summative focus for educational measurement is to measure more! Now, measuring more need not involve devoting more time exclusively to taking tests. Instead, computerized and online assessments are increasingly integrated into instruction so that measures are made in the course of studying (Cheng and Mok, 2007; Wilson, 2004). Measures are thereby continuously updated, and are plotted in growth charts relative to long range outcome goals.

Furthermore, the qualitative information provided by the measurement process is used to inform teachers and students about what comes next in the individualized curriculum, as well as about special strengths and weaknesses. This information has been shown to be unparalleled in its value for advancing learning in the classroom (Black and Wiliam, 1998, 2009; Hattie, 2008).

But formative assessment alone will not be sufficient to the larger tasks of raising quality and lowering costs. For that, systematic quality improvement methods in schools will need to be joined with comparable outcome measures parents and students can use to inform school choice decisions (Fisher, 2013; Lunenberg, 2010).

Kerrey and Leeds rightly seek an infrastructure capable of disruptive effects, of transforming the inflationary economy of education (and health care). To state again a recurring theme in this blog, the command and control hierarchies of regulatory systems can and should be replaced with a metrological infrastructure of common metrics with the scientific, legal, and financial status of common currencies for the exchange of value. Only when such currencies are in place will we be able to set out clear paths for the informed decisions, improved quality, lower costs, and accountability for outcomes that we seek.

References

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74.

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21, 5-31.

Cheng, Y. C., & Mok, M. M. C. (2007). School-based management and paradigm shift in education: An empirical study. International Journal of Educational Management, 21(6), 517-542.

Fisher, W. P., Jr. (2013). Imagining education tailored to assessment as, for, and of learning: Theory, standards, and quality improvement. Assessment and Learning, 2, in press.

Hattie, J. (2008). Visible learning. New York: Routledge.

Lunenberg, F. C. (2010). Total Quality Management applied to schools. Schooling, 1(1), 1-6.

Wilson, M. (Ed.). (2004). Towards coherence between classroom assessment and accountability. (Vol. 103, Part II, National Society for the Study of Education Yearbooks). Chicago, Illinois: University of Chicago Press.

On the IMEKO 2013 Joint Symposium in Genoa, Italy

November 19, 2013

The most recent instance of the IMEKO (International Measurement Confederation) joint symposium (which now also includes TC-13, the technical committee on measurements in biology and health care, along with TC-1 on Measurement Science and TC-7 on Metrology Education) was held in Genoa, Italy, September 4-6, 2013. The papers presented are available in volume 459 of the Journal of Physics Conference Series at http://iopscience.iop.org/1742-6596/459/ .

Mari and Wilson’s keynote providing a “gentle introduction to Rasch measurement models for metrologists” will be of special interest. Additional Rasch-oriented presentations were made by Maul, Torres-Irribarra, and Wilson; Camargo and Henson; Bezruczko; Stenner; Massof; Stephanou, Pendrill; and Fisher. Readers may also be interested in related work presented by Benoit, Crenna, Rossi, Granovskii, Pavese, Ruhm, Thomas, and others.

An exciting new dialogue between the natural and social sciences is underway. Each has much to learn from the other. Metrology has had little need to attend to the individual-level stochastic processes structuring invariant cognitive and behavioral constructs, and measurement practice in psychology and the social sciences has everything to learn about the value of local traceability to globally uniform units. Everyone interested in contributing to or learning from this dialogue is invited to make their voices heard.

A photo of a majority of the Genoa symposium attendees is available at http://spectronet.de/de/vortraege_bilder/vortraege_2013/15th-joint-international-imeko-tc1tc7tc13-symposiu_hlms3467.html.

A number of presentations involving Rasch measurement models, methods, and results were made at previous IMEKO symposia in Annecy (France), Lisbon (Portugal), London (England), and Jena (Germany) in 2008, 2009, 2010, and 2011, respectively (Fisher, 2009, 2010, 2011). A paper based on Mark Wilson’s keynote address at the 2011 meeting has recently been published in the IMEKO journal, Measurement (Wilson, 2013).

This journal also has a forthcoming celebration of the work of the late Ludwik Finkelstein in press (volume 46, number 8, pp. 2885-2992). Finkelstein made a large number of foundational contributions to educational and conceptual issues in measurement science, and had a special interest in exploring the possibility of a unified science of measurement applicable across the natural and social sciences (see, for instance, Finkelstein, 2003, 2009, 2010).

Wilson’s keynote at the Jena IMEKO symposium in 2011 was given at the invitation of Luca Mari, an engineer and philosopher of measurement based at the Universite Cattaneo, in Castellanza, Italy. Wilson reciprocated the invitation by bringing Mari to last summer’s International Meeting of the Psychometric Society in Lincoln, Nebraska. Mari gave a well-attended workshop on metrology, and an invited address.

Mari is also be a visiting scholar in the Graduate School of Education at the University of California, Berkeley, 18-22 November, 2013. In addition to his contributions to the International Electrotechnical Vocabulary (IEV), Mari is intensely involved in the ongoing revisions to the International Vocabulary of Metrology (known as the VIM; Joint Committee on Guides in Metrology, 2008), especially as this involves efforts continuing Finkelstein’s interest in integrating measurement concepts from all fields into a common frame of reference.

References

Finkelstein, L. (2003). Widely, strongly and weakly defined measurement. Measurement, 34(1), 39-48(10).

Finkelstein, L. (2009). Widely-defined measurement—An analysis of challenges. Measurement, 42(9), 1270-1277.

Finkelstein, L. (2010). Measurement and instrumentation science and technology-the educational challenges. Journal of Physics: Conference Series, 238, doi:10.1088/1742-6596/238/1/012001.

Fisher, W. P., Jr. (2008). Notes on IMEKO symposium. Rasch Measurement Transactions, 22(1), 1147 [http://www.rasch.org/rmt/rmt221.pdf].

Fisher, W. P., Jr. (2010). Unifying the language of measurement. Rasch Measurement Transactions, 24(2), 1278-1281  [http://www.rasch.org/rmt/rmt242.pdf].

Fisher, W. P., Jr. (2012). 2011 IMEKO conference proceedings available online. Rasch Measurement Transactions, 25(4), 1349 [http://www.rasch.org/rmt/rmt254.pdf].

Joint Committee for Guides in Metrology (JCGM/WG 2). (2008). International vocabulary of metrology: Basic and general concepts and associated terms, 3rd ed. Sevres, France: International Bureau of Weights and Measures–BIPM. http://www.bipm.org/utils/common/documents/jcgm/JCGM_200_2008.pdf. Accessed 17 October 2013.

Wilson, M. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.