Archive for the ‘standards’ Category

Excerpts and Notes from Goldberg’s “Billions of Drops…”

December 23, 2015

Goldberg, S. H. (2009). Billions of drops in millions of buckets: Why philanthropy doesn’t advance social progress. New York: Wiley.

p. 8:
Transaction costs: “…nonprofit financial markets are highly disorganized, with considerable duplication of effort, resource diversion, and processes that ‘take a fair amount of time to review grant applications and to make funding decisions’ [citing Harvard Business School Case No. 9-391-096, p. 7, Note on Starting a Nonprofit Venture, 11 Sept 1992]. It would be a major understatement to describe the resulting capital market as inefficient.”

A McKinsey study found that nonprofits spend 2.5 to 12 times more raising capital than for-profits do. When administrative costs are factored in, nonprofits spend 5.5 to 21.5 times more.

For-profit and nonprofit funding efforts contrasted on pages 8 and 9.

p. 10:
Balanced scorecard rating criteria

p. 11:
“Even at double-digit annual growth rates, it will take many years for social entrepreneurs and their funders to address even 10% of the populations in need.”

p. 12:
Exhibit 1.5 shows that the percentages of various needs served by leading social enterprises are barely drops in the respective buckets; they range from 0.07% to 3.30%.

pp. 14-16:
Nonprofit funding is not tied to performance. Even when a nonprofit makes the effort to show measured improvement in impact, it does little or nothing to change their funding picture. It appears that there is some kind of funding ceiling implicitly imposed by funders, since nonprofit growth and success seems to persuade capital sources that their work there is done. Mediocre and low performing nonprofits seem to be able to continue drawing funds indefinitely from sympathetic donors who don’t require evidence of effective use of their money.

p. 34:
“…meaningful reductions in poverty, illiteracy, violence, and hopelessness will require a fundamental restructuring of nonprofit capital markets. Such a restructuring would need to make it much easier for philanthropists of all stripes–large and small, public and private, institutional and individual–to fund nonprofit organizations that maximize social impact.”

p. 54:
Exhibit 2.3 is a chart showing that fewer people rose from poverty, and more remained in it or fell deeper into it, in the period of 1988-98 compared with 1969-1979.

pp. 70-71:
Kotter’s (1996) change cycle.

p. 75:
McKinsey’s seven elements of nonprofit capacity and capacity assessment grid.

pp. 94-95:
Exhibits 3.1 and 3.2 contrast the way financial markets reward for-profit performance with the way nonprofit markets reward fund raising efforts.

Financial markets
1. Market aggregates and disseminates standardized data
2. Analysts publish rigorous research reports
3. Investors proactively search for strong performers
4. Investors penalize weak performers
5. Market promotes performance
6. Strong performers grow

Nonprofit markets
1. Social performance is difficult to measure
2. NPOs don’t have resources or expertise to report results
3. Investors can’t get reliable or standardized results data
4. Strong and weak NPOs spend 40 to 60% of time fundraising
5. Market promotes fundraising
6. Investors can’t fund performance; NPOs can’t scale

p. 95:
“…nonprofits can’t possibly raise enough money to achieve transformative social impact within the constraints of the existing fundraising system. I submit that significant social progress cannot be achieved without what I’m going to call ‘third-stage funding,’ that is, funding that doesn’t suffer from disabling fragmentation. The existing nonprofit capital market is not capable of [p. 97] providing third-stage funding. Such funding can arise only when investors are sufficiently well informed to make big bets at understandable and manageable levels of risk. Existing nonprofit capital markets neither provide investors with the kinds of information needed–actionable information about nonprofit performance–nor provide the kinds of intermediation–active oversight by knowledgeable professionals–needed to mitigate risk. Absent third-stage funding, nonprofit capital will remain irreducibly fragmented, preventing the marshaling of resources that nonprofit organizations need to make meaningful and enduring progress against $100 million problems.”

pp. 99-114:
Text and diagrams on innovation, market adoption, transformative impact.

p. 140:
Exhibit 4.2: Capital distribution of nonprofits, highlighting mid-caps

pages 192-3 make the case for the difference between a regular market and the current state of philanthropic, social capital markets.

p. 192:
“So financial markets provide information investors can use to compare alternative investment opportunities based on their performance, and they provide a dynamic mechanism for moving money away from weak performers and toward strong performers. Just as water seeks its own level, markets continuously recalibrate prices until they achieve a roughly optimal equilibrium at which most companies receive the ‘right’ amount of investment. In this way, good companies thrive and bad ones improve or die.
“The social sector should work the same way. .. But philanthropic capital doesn’t flow toward effective nonprofits and away from ineffective nonprofits for a simple reason: contributors can’t tell the difference between the two. That is, philanthropists just don’t [p. 193] know what various nonprofits actually accomplish. Instead, they only know what nonprofits are trying to accomplish, and they only know that based on what the nonprofits themselves tell them.”

p. 193:
“The signs that the lack of social progress is linked to capital market dysfunctions are unmistakable: fundraising remains the number-one [p. 194] challenge of the sector despite the fact that nonprofit leaders divert some 40 to 60% of their time from productive work to chasing after money; donations raised are almost always too small, too short, and too restricted to enhance productive capacity; most mid-caps are ensnared in the ‘social entrepreneur’s trap’ of focusing on today and neglecting tomorrow; and so on. So any meaningful progress we could make in the direction of helping the nonprofit capital market allocate funds as effectively as the private capital market does could translate into tremendous advances in extending social and economic opportunity.
“Indeed, enhancing nonprofit capital allocation is likely to improve people’s lives much more than, say, further increasing the total amount of donations. Why? Because capital allocation has a multiplier effect.”

“If we want to materially improve the performance and increase the impact of the nonprofit sector, we need to understand what’s preventing [p. 195] it from doing a better job of allocating philanthropic capital. And figuring out why nonprofit capital markets don’t work very well requires us to understand why the financial markets do such a better job.”

p. 197:
“When all is said and done, securities prices are nothing more than convenient approximations that market participants accept as a way of simplifying their economic interactions, with a full understanding that market prices are useful even when they are way off the mark, as they so often are. In fact, that’s the whole point of markets: to aggregate the imperfect and incomplete knowledge held by vast numbers of traders about much various securities are worth and still make allocation choices that are better than we could without markets.
“Philanthropists face precisely the same problem: how to make better use of limited information to maximize output, in this case, social impact. Considering the dearth of useful tools available to donors today, the solution doesn’t have to be perfect or even all that good, at least at first. It just needs to improve the status quo and get better over time.
“Much of the solution, I believe, lies in finding useful adaptations of market mechanisms that will mitigate the effects of the same lack of reliable and comprehensive information about social sector performance. I would even go so far as to say that social enterprises can’t hope to realize their ‘one day, all children’ visions without a funding allociation system that acts more like a market.
“We can, and indeed do, make incremental improvements in nonprofit funding without market mechanisms. But without markets, I don’t see how we can fix the fragmentation problem or produce transformative social impact, such as ensuring that every child in America has a good education. The problems we face are too big and have too many moving parts to ignore the self-organizing dynamics of market economics. As Thomas Friedman said about the need to impose a carbon tax at a time of falling oil prices, ‘I’ve wracked my brain trying to think of ways to retool America around clean-power technologies without a price signal–i.e., a tax–and there are no effective ones.”

p. 199:
“Prices enable financial markets to work the way nonprofit capital markets should–by sending informative signals about the most effective organizations so that money will flow to them naturally..”

p. 200:
[Quotes Kurtzman citing De Soto on the mystery of capital. Also see p. 209, below.]
“‘Solve the mystery of capital and you solve many seemingly intractable problems along with it.'”
[That’s from page 69 in Kurtzman, 2002.]

p. 201:
[Goldberg says he’s quoting Daniel Yankelovich here, but the footnote does not appear to have anything to do with this quote:]
“‘The first step is to measure what can easily be measured. The second is to disregard what can’t be measured, or give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily isn’t very important. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.'”

Goldberg gives example here of $10,000 invested witha a 10% increase in value, compared with $10,000 put into a nonprofit. “But if the nonprofit makes good use of the money and, let’s say, brings the reading scores of 10 elementary school students up from below grade level to grade level, we can’t say how much my initial investment is ‘worth’ now. I could make the argument that the value has increased because the students have received a demonstrated educational benefit that is valuable to them. Since that’s the reason I made the donation, the achievement of higher scores must have value to me, as well.”

p. 202:
Goldberg wonders whether donations to nonprofits would be better conceived as purchases than investments.

p. 207:
Goldberg quotes Jon Gertner from the March 9, 2008, issue of the New York Times Magazine devoted to philanthropy:

“‘Why shouldn’t the world’s smartest capitalists be able to figure out more effective ways to give out money now? And why shouldn’t they want to make sure their philanthropy has significant social impact? If they can measure impact, couldn’t they get past the resistance that [Warren] Buffet highlighted and finally separate what works from what doesn’t?'”

p. 208:
“Once we abandon the false notions that financial markets are precision instruments for measuring unambiguous phenomena, and that the business and nonproft sectors are based in mutually exclusive principles of value, we can deconstruct the true nature of the problems we need to address and adapt market-like mechanisms that are suited to the particulars of the social sector.
“All of this is a long way (okay, a very long way) of saying that even ordinal rankings of nonprofit investments can have tremendous value in choosing among competing donation opportunities, especially when the choices are so numerous and varied. If I’m a social investor, I’d really like to know which nonprofits are likely to produce ‘more’ impact and which ones are likely to produce ‘less.'”

“It isn’t necessary to replicate the complex working of the modern stock markets to fashion an intelligent and useful nonprofit capital allocation mechanism. All we’re looking for is some kind of functional indication that would (1) isolate promising nonprofit investments from among the confusing swarm of too many seemingly worthy social-purpose organizations and (2) roughly differentiate among them based on the likelihood of ‘more’ or ‘less’ impact. This is what I meant earlier by increasing [p. 209] signals and decreasing noise.”

p. 209:
Goldberg apparently didn’t read De Soto, as he says that the mystery of capital is posed by Kurtzman and says it is solved via the collective intelligence and wisdom of crowds. This completely misses the point of the crucial value that transparent representations of structural invariance hold in market functionality. Goldberg is apparently offering a loose kind of market for which there is an aggregate index of stocks for nonprofits that are built up from their various ordinal performance measures. I think I find a better way in my work, building more closely from De Soto (Fisher, 2002, 2003, 2005, 2007, 2009a, 2009b).

p. 231:
Goldberg quotes Harvard’s Allen Grossman (1999) on the cost-benefit boundaries of more effective nonprofit capital allocation:

“‘Is there a significant downside risk in restructuring some portion of the philanthropic capital markets to test the effectiveness of performance driven philanthropy? The short answer is, ‘No.’ The current reality is that most broad-based solutions to social problems have eluded the conventional and fragmented approaches to philanthropy. It is hard to imagine that experiments to change the system to a more performance driven and rational market would negatively impact the effectiveness of the current funding flows–and could have dramatic upside potential.'”

p. 232:
Quotes Douglas Hubbard’s How to Measure Anything book that Stenner endorsed, and Linacre and I didn’t.

p. 233:
Cites Stevens on the four levels of measurement and uses it to justify his position concerning ordinal rankings, recognizing that “we can’t add or subtract ordinals.”

pp. 233-5:
Justifies ordinal measures via example of Google’s PageRank algorithm. [I could connect from here using Mary Garner’s (2009) comparison of PageRank with Rasch.]

p. 236:
Goldberg tries to justify the use of ordinal measures by citing their widespread use in social science and health care. He conveniently ignores the fact that virtually all of the same problems and criticisms that apply to philanthropic capital markets also apply in these areas. In not grasping the fundamental value of De Soto’s concept of transferable and transparent representations, and in knowing nothing of Rasch measurement, he was unable to properly evaluate to potential of ordinal data’s role in the formation of philanthropic capital markets. Ordinal measures aren’t just not good enough, they represent a dangerous diversion of resources that will be put into systems that take on lives of their own, creating a new layer of dysfunctional relationships that will be hard to overcome.

p. 261 [Goldberg shows here his complete ignorance about measurement. He is apparently totally unaware of the work that is in fact most relevant to his cause, going back to Thurstone in 1920s, Rasch in the 1950s-1970s, and Wright in the 1960s to 2000. Both of the problems he identifies have long since been solved in theory and in practice in a wide range of domains in education, psychology, health care, etc.]:
“Having first studied performance evaluation some 30 years ago, I feel confident in saying that all the foundational work has been done. There won’t be a ‘eureka!’ breakthrough where someone finally figures out the one true way to guage nonprofit effectiveness.
“Indeed, I would venture to say that we know virtually everything there is to know about measuring the performance of nonprofit organizations with only two exceptions: (1) How can we compare nonprofits with different missions or approaches, and (2) how can we make actionable performance assessments common practice for growth-ready mid-caps and readily available to all prospective donors?”

p. 263:
“Why would a social entrepreneur divert limited resources to impact assessment if there were no prospects it would increase funding? How could an investor who wanted to maximize the impact of her giving possibly put more golden eggs in fewer impact-producing baskets if she had no way to distinguish one basket from another? The result: there’s no performance data to attract growth capital, and there’s no growth capital to induce performance measurement. Until we fix that Catch-22, performance evaluation will not become an integral part of social enterprise.”

pp. 264-5:
Long quotation from Ken Berger at Charity Navigator on their ongoing efforts at developing an outcome measurement system. [wpf, 8 Nov 2009: I read the passage quoted by Goldberg in Berger’s blog when it came out and have been watching and waiting ever since for the new system. wpf, 8 Feb 2012: The new system has been online for some time but still does not include anything on impacts or outcomes. It has expanded from a sole focus on financials to also include accountability and transparency. But it does not yet address Goldberg’s concerns as there still is no way to tell what works from what doesn’t.]

p. 265:
“The failure of the social sector to coordinate independent assets and create a whole that exceeds the sum of its parts results from an absence of.. platform leadership’: ‘the ability of a company to drive innovation around a particular platform technology at the broad industry level.’ The object is to multiply value by working together: ‘the more people who use the platform products, the more incentives there are for complement producers to introduce more complementary products, causing a virtuous cycle.'” [Quotes here from Cusumano & Gawer (2002). The concept of platform leadership speaks directly to the system of issues raised by Miller & O’Leary (2007) that must be addressed to form effective HSN capital markets.]

p. 266:
“…the nonprofit sector has a great deal of both money and innovation, but too little available information about too many organizations. The result is capital fragmentation that squelches growth. None of the stakeholders has enough horsepower on its own to impose order on this chaos, but some kind of realignment could release all of that pent-up potential energy. While command-and-control authority is neither feasible nor desirable, the conditions are ripe for platform leadership.”

“It is doubtful that the IMPEX could amass all of the resources internally needed to build and grow a virtual nonprofit stock market that could connect large numbers of growth-capital investors with large numbers of [p. 267] growth-ready mid-caps. But it might be able to convene a powerful coalition of complementary actors that could achieve a critical mass of support for performance-based philanthropy. The challenge would be to develop an organization focused on filling the gaps rather than encroaching on the turf of established firms whose participation and innovation would be required to build a platform for nurturing growth of social enterprise..”

p. 268-9:
Intermediated nonprofit capital market shifts fundraising burden from grantees to intermediaries.

p. 271:
“The surging growth of national donor-advised funds, which simplify and reduce the transaction costs of methodical giving, exemplifies the kind of financial innovation that is poised to leverage market-based investment guidance.” [President of Schwab Charitable quoted as wanting to make charitable giving information- and results-driven.]

p. 272:
Rating agencies and organizations: Charity Navigator, Guidestar, Wise Giving Alliance.
Online donor rankings: GlobalGiving, GreatNonprofits, SocialMarkets
Evaluation consultants: Mathematica

Google’s mission statement: “to organize the world’s information and make it universally accessible and useful.”

p. 273:
Exhibit 9.4 Impact Index Whole Product
Image of stakeholders circling IMPEX:
Trading engine
Listed nonprofits
Data producers and aggregators
Trading community
Researchers and analysts
Investors and advisors
Government and business supporters

p. 275:
“That’s the starting point for replication [of social innovations that work]: finding and funding; matching money with performance.”

[WPF bottom line: Because Goldberg misses De Soto’s point about transparent representations resolving the mystery of capital, he is unable to see his way toward making the nonprofit capital markets function more like financial capital markets, with the difference being the focus on the growth of human, social, and natural capital. Though Goldberg intuits good points about the wisdom of crowds, he doesn’t know enough about the flaws of ordinal measurement relative to interval measurement, or about the relatively easy access to interval measures that can be had, to do the job.]


Cusumano, M. A., & Gawer, A. (2002, Spring). The elements of platform leadership. MIT Sloan Management Review, 43(3), 58.

De Soto, H. (2000). The mystery of capital: Why capitalism triumphs in the West and fails everywhere else. New York: Basic Books.

Fisher, W. P., Jr. (2002, Spring). “The Mystery of Capital” and the human sciences. Rasch Measurement Transactions, 15(4), 854 [].

Fisher, W. P., Jr. (2003). Measurement and communities of inquiry. Rasch Measurement Transactions, 17(3), 936-8 [].

Fisher, W. P., Jr. (2005). Daredevil barnstorming to the tipping point: New aspirations for the human sciences. Journal of Applied Measurement, 6(3), 173-9 [].

Fisher, W. P., Jr. (2007, Summer). Living capital metrics. Rasch Measurement Transactions, 21(1), 1092-3 [].

Fisher, W. P., Jr. (2009a). Bringing human, social, and natural capital to life: Practical consequences and opportunities. In M. Wilson, K. Draney, N. Brown & B. Duckor (Eds.), Advances in Rasch Measurement, Vol. Two (p. in press []). Maple Grove, MN: JAM Press.

Fisher, W. P., Jr. (2009b, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement (Elsevier), 42(9), 1278-1287.

Garner, M. (2009, Autumn). Google’s PageRank algorithm and the Rasch measurement model. Rasch Measurement Transactions, 23(2), 1201-2 [].

Grossman, A. (1999). Philanthropic social capital markets: Performance driven philanthropy (Social Enterprise Series 12 No. 00-002). Harvard Business School Working Paper.

Kotter, J. (1996). Leading change. Cambridge, Massachusetts: Harvard Business School Press.

Kurtzman, J. (2002). How the markets really work. New York: Crown Business.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-34.

A truly ambitious plan to tackle climate change 

December 3, 2015

A recent story in the NY Times asks just what a truly ambitious plan to tackle climate change would look like. Pledges of emissions cuts being made in Paris this month are projected to fall short of what is needed to solve the problem of climate change. Calls for mass mobilization on the scale of the U.S.’s entry into WWII are met with skepticism at the same time that leaders are signing on for stronger terms in the Paris agreement than their countries have agreed to.

One crucial assumption is made across the full range of the proposals for more stringent standards and innovative technologies. That assumption is that solving the problem of climate change is a matter of marshaling the will to get the job done. On the face of it, of course, it seems inane to consider something as important as will power to be part of the problem. If people don’t want to do something, how could it possibly ever get done?

But as I’ve pointed out in a number of previous posts in this blog, complex problems sometimes cannot be solved from within the conceptual framework that engendered them. We are in this situation in large part because our overall relation to the earth is based on assuming it to be a bottomless well of resources, with the only limitation being the creativity we bring to bear in tapping those resources. Though many of us, perhaps a majority, are seriously committed to reconceiving our relation to the earth in sustainable terms, practical results are nearly impossible to produce within the existing institutional framework. Our economic, legal, accounting, education, etc. systems are all set up to support a consumer ethos that hobbles and undercuts almost all efforts intended to support an alternative sustainability ethos. It is both ironic and counterproductive to formulate solutions to the problem of climate change without first changing the institutional background assumptions informing the rules, roles and responsibilities through which we conceptualize and implement those solutions.

Insight into this problem is provided by recent work on standards for sustainability accounting. It shows that, by definition, efforts targeting change in economic externalities like environmental concerns cannot be scaled up in ways that are needed. This happens simply because balancing mission and margin demands maintenance of the bottom line. Giving away the business in the name of saving the planet might be a noble gesture but it is the opposite of sustainable and more importantly does not provide a viable model for the future.

So how do we model a new kind of bottom line that balances mission and margin in a new way? A way in which institutional rules, roles and responsibilities are themselves configured into the sustainable ecological relations we need? A way in which means and ends are unified? How do we become the change we want to see? How can we mobilize an international mass movement focused on doing what needs to be done to solve the problem of climate change? What possibilities do we have for catalyzing the increasingly saturated solution of global discontent and desire for a new relation to the earth? Can natural social processes of leaderless self organizing systems be seeded and guided to fruition? What would that seeding and guidance look like?

For proposed answers to these questions and more on what a model of a truly ambitious plan to tackle climate change might look like, see other posts in this blog here, here, here, and here.

With Reich in spirit, but with a different sense of the problem and its solution

October 4, 2015

In today’s editorial in the San Francisco Chronicle, Robert Reich seeks some way of defining a solution to the pressing problems of how globalization and technological changes have made American workers less competitive. He rightly says that “reversing the scourge of widening inequality requires reversing the upward distributions [of income] within the rules of the market, and giving average people the bargaining power they need to get a larger share of the gains from growth.”

But Reich then says that the answer to this problem lies in politics, not economics. As I’ve pointed out before in this blog, focusing on marshaling political will is part of the problem, not part of the solution. Historically, politicians do not lead, they follow. As is demonstrated across events as diverse as the Arab Spring and the Preemption Act of 1841, mass movements of people have repeatedly demanded ways of cutting through the Gordian knots of injustice. And just as the political “leadership” across the Middle East and in the early U.S. dragged its feet, obstructed, and violently opposed change until it was already well underway, so, too, will that pattern repeat itself again in the current situation of inequitable income distribution.

The crux of the problem is that no one can give average people anything, not freedom (contra Dylan’s line in Blowin’ in the Wind about “allowing” people to be free) and certainly not a larger share of the gains from growth. As the old saying goes, you can lead a horse to water, but you can’t make it drink. People have to take what’s theirs. They have to want it, they have to struggle for it, and they have to pay for it, or they cannot own it and it will never be worth anything to them.

It is well known that a lack of individual property rights doomed communism and socialism because when everything is owned collectively by everyone, no one takes responsibility for it. The profit motive has the capacity to drive people to change things. The problem is not in profit itself. If birds and bees and trees and grasses did not profit from the sun, soil, and rain, there would be no life. The problem is in finding how to get a functional, self-sustaining economic ecology off the ground, not in unrealistically trying to manipulate and micromanage every detail.

The fundamental relevant characteristic of the profits being made today from intellectual property rights is that our individual rights to our own human and social capital are counter-productively restricted and undeveloped. How can it be that no one has any idea how much literacy or health capital they have, or what it is worth?! We have a metric system that tells us how much real estate and manufactured capital we own, and we can price it. But despite the well-established scientific facts of decades of measurement science research and practice, none of us can say, “I own x number of shares of stock in intellectual, literacy, or community capital, that have a value of x dollars in today’s market.” We desperately need an Intangible Assets Metric System, and the market rules, roles, and responsibilities that will make it impossible to make a profit while destroying human, social, and natural capital.

In this vein, what Reich gets absolutely correct is hidden inside his phrase, “within the rules of the market.” As I’ve so often repeated in this blog, capitalism is not inherently evil; it is, rather, unfinished. The real evil is in prolonging the time it takes to complete it. As was so eloquently stated by Miller and O’Leary (2007, p. 710):

“Markets are not spontaneously generated by the exchange activity of buyers and sellers. Rather, skilled actors produce institutional arrangements, the rules, roles and relationships that make market exchange possible. The institutions define the market, rather than the reverse.”

We have failed to set up the institutional arrangements needed to define human, social, and natural capital markets. The problem is that we cannot properly manage three of the four major forms of capital (human, social, and natural, with the fourth being manufactured/property) because we do not measure them in a common language built into scientifically, economically, legally and financially accountable titles, deeds, and other instruments.

And so, to repeat another one of my ad nauseum broken record nostrums, the problem is the problem. As long as we keep defining problems in the way we always have, as matters of marshalling political will, we will inadvertently find ourselves contributing more to prolonging tragic and needless human suffering, social discontent, and environmental degradation.

Miller, P., & O’Leary, T. (2007, October/November). Mediating instruments and making markets: Capital budgeting, science and the economy. Accounting, Organizations, and Society, 32(7-8), 701-734.

Feminist Diffractions, Stochastic Resonance, and Education, Revisited

May 25, 2015

Lehrer (2015) offers an insightful commentary on Saxe et al’s (2015) recent article in Human Development that prompts some observations.

Two areas for questions and comments come to mind. The first has to do with construing the development and revision of new ways of understanding as contested, which implicitly aligns with Latour’s (1987, pp. 89, 93) sense of the way new constructs are subjected to tests of strength. Haraway (1996) makes an important point in her critique of what she sees as the overly masculinist metaphors of heroic competition and (perhaps not so) sublimated violence in these contests. Her sense of “feminist diffractions” stops short of what I have in mind, but opens the door to an alternative approach to what Lehrer calls the “close coupling of definitions with the development and revision of new concepts and ways of understanding.”

Galison (1997, pp. 843-844), for instance, seeks a metaphor capable of expressing what happens in the conceptual, practical, and argumentative contests between different communities of scientists (instrumentalist technicians, theoreticians, and experimentalists). He wants a metaphor that does justice to the disunified chaos and disorder one finds in the relationships between these different groups, which paradoxically results in such productive and coherent innovations. He recalls Peirce’s and Wittgenstein’s metaphors of cables and threads that take their strength from being intertwined from smaller wires and bits of fiber but finds these images too mechanical for his purposes. He wants something more akin to amorphous semiconductors or laminated materials that can fail microscopically but hold macroscopically better than more structurally homogenous materials.

Berg and Timmermans (2000, pp. 55-56) make a similar observation in their study of the constitution of universalities in medical fields:

“In order for a statistical logistics to enhance precise decision making, it has to incorporate imprecision; in order to be universal, it has to carefully select its locales. … Paradoxically, then, the increased stability and reach of this network was not due to more (precise) instructions: the protocol’s logistics could thrive only by parasitically drawing upon its own disorder.”

The general problem is taken up by Ricoeur (1992, p. 289), who raises the notion of “universals in context or of potential or inchoate universals” that embody the paradox in which

“on the one hand, one must maintain the universal claim attached to a few values where the universal and the historical intersect, and on the other hand, one must submit this claim to discussion, not on a formal level, but on the level of the convictions incorporated in concrete forms of life.”

To repeat another theme that comes up again and again in this blog, this kind of noise-induced order sounds like the phenomenon of stochastic resonance (Fisher, 1992, 2011). The importance of stochastic resonance is that it opens up a way to connect the phenomena of emergent understanding with measurement, both at the local individual and general systemic levels.

This is the crux of some very important issues in the philosophy of science and in philosophy generally. Haraway (1996, pp. 439-440), for instance, points out that “embedded relationality is the prophylaxis for both relativism and transcendence.” And Golinski (2012, p. 35) similarly says, “Practices of translation, replication, and metrology have taken the place of the universality that used to be assumed as an attribute of singular science.”

A start in the direction of embedded relationality, translation, replication, and metrology in education is apparent, for instance, in work that enables teachers to usefully relate individual student performances to general learning progressions, connecting instructional applications with accountability (Fisher & Wilson, 2015; Lehrer, 2013; Lehrer & Jones, 2014; Wilson, 2004). As Lehrer (2015, p. 49) says about the Saxe et al. work, “Recurrent forms of mathematical practice enabled the authors to create compelling trajectories of collective activity and learning over time while preserving the contributions of individual development.”

The second of the two topics I’d like to address comes up here in the closing paragraph of his short commentary, where Lehrer says a “hoped-for future innovation would make it possible to visualize individual and collective trajectories simultaneously.” Though future improvements can certainlty be expected, visualizations of individual and collective trajectories for growth in reading are already being recognized in both educational and metrological contexts (Stenner, Swartz, Hanlon, & Emerson, 2012; Stenner & Fisher, 2013, p. 4) for their potential to serve as the media of an embedded relationality capable of undercutting both the relativism of uncontrolled local variation and the universalist pretensions often built into accountability programs.

With emerging recognition of the potential Rasch’s stochastic approaches to construct mapping (Bond & Fox, 2007; Wilson, 2005) offer in the way of metrological translation networks (Mari & Wilson, 2013; Pendrill, 2014; Pendrill & Fisher, 2015; Fisher & Wilson, 2015; Stenner & Fisher, 2013; Wilson, 2013; Wilson, Mari, Maul, & Torres Irribarra 2015), there are good reasons to expect significant new kinds of progress in fields that rely on assessments and surveys for outcome measurement and management.


Berg, M.,& Timmermans, S. (2000). Order and their others: On the constitution of universalities in medical work. Configurations, 8(1), 31-61.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Fisher, W. P., Jr. (1992). Stochastic resonance and Rasch measurement. Rasch Measurement Transactions, 5(4), 186-187 [].

Fisher, W. P., Jr. (2011). Stochastic and historical resonances of the unit in physics and psychometrics. Measurement: Interdisciplinary Research & Perspectives, 9, 46-50.

Fisher, W. P., Jr., & Stenner, A. J. (2015). The role of metrology in mobilizing and mediating the language and culture of scientific facts. Journal of Physics Conference Series, 588(012043).

Fisher, W. P., Jr., & Wilson, M. (2015). Building a productive trading zone in educational assessment research and practice. Pensamiento Educativo, in review.

Galison, P. (1997). Image and logic: A material culture of microphysics. Chicago: University of Chicago Press.

Golinski, J. (2012). Is it time to forget science? Reflections on singular science and its history. Osiris, 27(1), 19-36.

Haraway, D. J. (1996). Modest witness: Feminist diffractions in science studies. In P. Galison & D. J. Stump (Eds.), The disunity of science: Boundaries, contexts, and power (pp. 428-441). Stanford, California: Stanford University Press.

Latour, B. (1987). Science in action: How to follow scientists and engineers through society. New York: Harvard University Press.

Lehrer, R. (2013, April 29). (Chair). In A learning progression emerges in a trading zone of professional community and identity. American Educational Research Association, Division C on Learning and Instruction, Section 2b on Learning and Motivation in Social and Cultural Contexts, San Francisco, CA.

Lehrer, R., & Jones, S. (2014, 2 April). Construct maps as boundary objects in the trading zone. In W. P. Fisher Jr. (Chair), Session 3-A: Rating Scales and Partial Credit, Theory and Applied. International Objective Measurement Workshop, Philadelphia, PA.

Lehrer, R. (2015). Designing for development: Commentary on Saxe, de Kirby, Kang, Le and Schneider. Human Development, 58(1), 45-49.

Mari, L., & Wilson, M. (2013). A gentle introduction to Rasch measurement models for metrologists. Journal of Physics Conference Series, 459(1),

Pendrill, L. (2014). Man as a measurement instrument [Special Feature]. NCSLi Measure: The Journal of Measurement Science, 9(4), 22-33.

Pendrill, L., & Fisher, W. P., Jr. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46-55.

Ricoeur, P. (1992). Oneself as another. Chicago, Illinois: University of Chicago Press.

Saxe, G. B., de Kirby, K., Kang, B., Le, M., & Schneider, A. (2015). Studying cognition through time in a classroom community: The interplay between “everyday” and “scientific” concepts. Human Development, 58(1), 5-44.

Stenner, A. J., & Fisher, W. P., Jr. (2013). Metrological traceability in the social sciences: A model from reading measurement. Journal of Physics: Conference Series, 459(012025),

Stenner, A. J., Swartz, C., Hanlon, S., & Emerson, C. (2012, February). Personalized learning platforms. Presented at the Pearson Global Research Conference, Fremantle, Western Australia.

Wilson, M. (Ed.). (2004). National Society for the Study of Education Yearbooks. Vol. 103, Part II: Towards coherence between classroom assessment and accountability. Chicago, Illinois: University of Chicago Press.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Wilson, M. R. (2013). Using the concept of a measurement system to characterize measurement models used in psychometrics. Measurement, 46, 3766-3774.