Posts Tagged ‘leadership’

Review of Spitzer’s Transforming Performance Measurement

January 25, 2010

Everyone interested in practical measurement applications needs to read Dean R. Spitzer’s 2007 book, Transforming performance measurement: Rethinking the way we measure and drive organizational success (New York, AMACOM). Spitzer describes how measurement, properly understood and implemented, can transform organizational performance by empowering and motivating individuals. Measurement understood in this way moves beyond quick fixes and fads to sustainable processes based on a measurement infrastructure that coordinates decisions and actions uniformly throughout the organization.

Measurement leadership, Spitzer says, is essential. He advocates, and many organizations have instituted, the C-suite position of Chief Measurement Officer (Chapter 9). This person is responsible for instituting and managing the four keys to transformational performance measurement (Chapters 5-8):

  • Context sets the tone by presenting the purpose of measurement as either negative (to inspect, control, report, manipulate) or positive (to give feedback, learn, improve).
  • Focus concentrates attention on what’s important, aligning measures with the mission, strategy, and with what needs to be managed, relative to the opportunities, capacities, and skills at hand.
  • Integration addresses the flow of measured information throughout the organization so that the covariations of different measures can be observed relative to the overall value created.
  • Interactivity speaks to the inherently social nature of the purposes of measurement, so that it embodies an alignment with the business model, strategy, and operational imperatives.

Spitzer takes a developmental approach to measurement improvement, providing a Measurement Maturity Assessment in Chapter 12, and also speaking to the issues of the “living company” raised by Arie de Geus’ classic book of that title. Plainly, the transformative potential of performance measurement is dependent on the maturational complexity of the context in which it is implemented.

Spitzer clearly outlines the ways in which each of the four keys and measurement leadership play into or hinder transformation and maturation. He also provides practical action plans and detailed guidelines, stresses the essential need for an experimental attitude toward evaluating change, speaks directly to the difficulty of measuring intangible assets like partnership, trust, skills, etc., and shows appreciation for the value of qualitative data.

Transforming Performance Measurement is not an academic treatise, though all sources are documented, with the endnotes and bibliography running to 25 pages. It was written for executives, managers, and entrepreneurs who need practical advice expressed in direct, simple terms. Further, the book does not include any awareness of the technical capacities of measurement as these have been realized in numerous commercial applications in high stakes and licensure/certification testing over the last 50 years (Andrich, 2005; Bezruczko, 2005; Bond & Fox, 2007; Masters, 2007; Wilson, 2005). This can hardly be counted as a major criticism, since no books of this kind have yet to date been able to incorporate the often highly technical and mathematical presentations of advanced psychometrics.

That said, the sophistication of Spitzer’s conceptual framework and recommendations make them remarkably ready to incorporate insights from measurement theory, testing practice, developmental psychology, and the history of science. Doing so will propel the strategies recommended in this book into widespread adoption and will be a catalyst for the emerging re-invention of capitalism. In this coming cultural revolution, intangible forms of capital will be brought to life in common currencies for the exchange of value that perform the same function performed by kilowatts, bushels, barrels, and hours for tangible forms of capital (Fisher, 2009, 2010).

Pretty big claim, you say? Yes, it is. Here’s how it’s going to work.

  • First, measurement leadership within organizations that implements policies and procedures that are context-sensitive, focused, integrated, and interactive (i.e., that have Spitzer’s keys in hand) will benefit from instruments calibrated to facilitate:
    • meaningful mapping of substantive, additive amounts of things measured on number lines;
    • data volume reductions on the order of 80-95% and more, with no loss of information;
    • organizational and individual learning trajectories defined by hierarchies of calibrated items;
    • measures that retain their meaning and values across changes in item content;
    • adapting instruments to people and organizations, instead of vice versa;
    • estimating the consistency, and the leniency or harshness, of ratings assigned by judges evaluating performance quality, with the ability to remove those effects from the performance measures made;
    • adjusting measurement precision to the needs of the task at hand, so that time and resources are not wasted in gathering too much or too little data; and
    • providing the high quality and uniform information needed for networked collective thinking able to keep pace with the demand for innovation.
  • Second, measurement leadership sensitive to the four keys across organizations, both within and across industries, will find value in:
    • establishing industry-wide metrological standards defining common metrics for the expression of the primary human, social, and natural capital constructs of interest;
    • lubricating the flow of human, social, and natural capital in efficient markets broadly defined so as to inform competitive pricing of intangible assets, products, and services; and
    • new opportunities for determining returns on investments in human, community, and environmental resource management.
  • Third, living companies need to be able to mature in a manner akin to human development over the lifespan. Theories of hierarchical complexity and developmental stage transitions that inform the rigorous measurement of cognitive and moral transformations (Dawson & Gabrielian, 2003) will increasingly find highly practical applications in organizational contexts.

Leadership of the kind described by Spitzer is needed not just to make measurement contextualized, focused, integrated, and interactive—and so productive at new levels of effectiveness—but to apply systematically the technical, financial, and social resources needed to realize the rich potentials he describes for the transformation of organizations and empowerment of individuals. Spitzer’s program surpasses the usual focus on centralized statistical analyses and reports to demand the organization-wide dissemination of calibrated instruments that measure in common metrics. The flexibility, convenience, and scientific rigor of instruments calibrated to measure in units that really add up fit the bill exactly. Here’s to putting tools that work in the hands of those who know what to do with them!

References

Andrich, D. (2005). Georg Rasch: Mathematician and statistician. In K. Kempf-Leonard (Ed.), Encyclopedia of Social Measurement (Vol. 3, pp. 299-306). Amsterdam: Academic Press, Inc.

Bezruczko, N. (Ed.). (2005). Rasch measurement in health sciences. Maple Grove, MN: JAM Press.

Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences, 2d edition. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Dawson, T. L., & Gabrielian, S. (2003, June). Developing conceptions of authority and contract across the life-span: Two perspectives. Developmental Review, 23(2), 162-218.

Fisher, W. P., Jr. (2009, November). Invariance and traceability for measures of human, social, and natural capital: Theory and application. Measurement (Elsevier), 42(9), 1278-1287.

Fisher, W. P., Jr. (2010). Bringing human, social, and natural capital to life: Practical consequences and opportunities. Journal of Applied Measurement, 11, in press [Pre-press version available at http://www.livingcapitalmetrics.com/images/BringingHSN_FisherARMII.pdf%5D.

Masters, G. N. (2007). Special issue: Programme for International Student Assessment (PISA). Journal of Applied Measurement, 8(3), 235-335.

Spitzer, D. (2007). Transforming performance measurement: Rethinking the way we measure and drive organizational success. New York: AMACOM.

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.

Advertisements

Information and Leadership: New Opportunities for Advancing Strategy, Engaging Customers, and Motivating Employees

December 9, 2009

Or, What’s a Mathematical Model a Model Of, After All?
Or, How to Build Scale Models of Organizations and Use Them to Learn About Organizational Identity, Purpose, and Mission

William P. Fisher, Jr., Ph.D.

The greatest opportunity and most significant challenge to leadership in every area of life today is the management of information. So says Carol Bartz, CEO of Yahoo! in her entry in The Economist’s annual overview of world events, “The World in 2010.” Information can be both a blessing and a curse. The right information in the right hands at the right time is essential to effectiveness and efficiency. But unorganized and incoherent information can be worse than none at all. Too often leaders and managers are faced with deciding between gut instincts based in unaccountable intuitions and facts that are potentially seriously flawed, or that are merely presented in such overwhelming volumes as to be useless.

This situation is only going to get worse as information volumes continue to increase. The upside is that solutions exist, solutions that not only reduce data volume by factors as high as hundreds to one with no loss of information, but which also distinguish between merely apparent and really reliable information. What we have in these solutions are the means of following through on Carol Bartz’s information leadership warnings and recommendations.

Clearly communicating what matters, for instance, requires leaders to find meaning in new facts and the changing scene. They have to be able to use their vision of the organization, its mission, and its place in the world to tell what’s important and what isn’t, to put each event or opportunity in perspective. And what’s more is that the vision of the organization has to be dynamic. It, too, has to be able to change with the changing circumstances.

And this is where a whole new class of useful information solutions comes to bear. It may seem odd to say so, but leadership is fundamentally mathematical. You can begin to get a sense of what I mean in the ambiguity of the way leaders can be calculating. Making use of people’s skills and talents is a challenge that requires being able to assess facts and potentials in a way that intuitively gauges likelihoods of success. It is possible to lead, of course, without being manipulative; the point is that leadership requires an ability to envision and project an abstract heuristic ideal as a fundamental principle for focusing attention and separating the wheat from the chaff. A leader who dithers and wastes time and resources on irrelevancies is a contradiction in terms. An organization is supposed to have an identity, a purpose, and a mission in life independent of the local particulars of who its actual employees, customers, and suppliers are, and independent of the opportunities and challenges that arise in different times and places.

Of course, every organization is colored and shaped to some extent by every different person that comes into contact with it, and by the times and places it finds itself in. No one wants to feel like an interchangeable part in machine, but neither does anyone want to feel completely out of place, with no role to play. If an organization was entirely dependent on the particulars of who, what, when, and where, it’s status as a coherent organization with an identifiable presence would be compromised. So what we need is to find the right balance between the ideal and the real, the abstract and the concrete, and, as the philosopher Paul Ricoeur put it, between belonging and distanciation.

And indeed, scientists often note that no mathematical model ever holds in every detail in the real world. That isn’t what they’re intended to do, in fact. Mathematical models serve the purpose of being guides to creating meaningful, useful relationships. One of the leading lights of measurement theory, Georg Rasch, said it well over 50 years ago: models aren’t meant to be true, but to be useful.

Rasch accordingly also pointed out that, if we measure mass, force, and acceleration with enough precision, we see that even Newton’s laws of motion are not perfectly true. Measured to the nth decimal place, what we find is that observed amounts of mass, force, and acceleration form probability distributions that do indeed satisfy Newton’s laws. Even in classical physics, then, measurement models are best conceived probabilistically.

Over the last several decades, use of Rasch’s probabilistic measurement models in scaling tests, surveys, and assessments has grown exponentially. As has been explored at length in previous posts in this blog, most applications of Rasch’s models mistakenly treat them as statistical models, as so their real value and importance is missed. But even those actively engaged in using the models appropriately often do not engage with the basic question concerning what the model is a model of, in their particular application of it. The basic assumption seems to be that the model is a mathematical representation of relations between observations recorded in a data set, but this is an extremely narrow and unproductive point of view.

Let’s ask ourselves, instead, how we would model an organization. Why would we want to do that? We would want to do that for the same reasons we model anything, such as creating a safe and efficient way of experimenting with different configurations, and of coming to new understandings of basic principles. If we had a standard model of organizations of a certain type, or of organizations in a particular industry, we could use it to see how different variations on the basic structure and processes cause or are associated with different outcomes. Further, given that such models could be used to calibrate scales meaningfully measuring organizational development, industry-wide standards could be brought to bear in policy, decision making, and education, effecting new degrees of efficiency and effectiveness.

So, we’d previously said that the extent to which an organization finds its identity, realizes its purpose, and advances its mission (i.e., develops) is, within certain limits, a function of its capacity to be independent from local particulars. What we mean by this is that we expect employees to be able to perform their jobs no matter what day of the week it is, no matter who the customer is, no matter which particular instance of a product is involved, etc. Though no amount of skill, training, or experience can prepare someone for every possible contingency, people working in a given job description prepare themselves for a certain set of tasks, and are chosen by the organization for their capacities in that regard.

Similarly, we expect policies, job descriptions, work flows, etc. to function in similar fashions. Though the exact specifics of each employee’s abilities and each situation’s demands cannot be known in advance, enough is known that the defined aims will be achieved with high degrees of success. Of course, this is the point at which the interchangeability of employee ability and task difficulty can become demeaning and alienating. It will be important that we allow room for some creative play, and situate each level of ability along a continuum that allows everyone to see a developmental trajectory personalized to their particular strengths and needs.

So, how do we mathematically model the independence of the organization from its employees, policies, customers, and challenges, and scientifically evaluate that independence?

One way to begin is to posit that organizational development is equal to the differences between the abilities of the people employed, the efficiencies of the policies, alignments, and linkages implemented; and the challenges presented by the market. If we observe the abilities, efficiencies, and challenges in by means of a rating scale, the resulting model could be written as:

ln(Pmoas/(1-Pmoas)) = bm – fo – ca – rs

which hypothesizes that the natural logarithm of the response odds (the response probabilities divided by one minus themselves) is equal to the ability b of employee m minus the efficiency f of policy o minus the challenge c of market a minus the difficulty r of obtaining rating in category s. This model has the form of a multifaceted Rasch model (Linacre, 1989; others), used in academic research, rehabilitative functional assessments, and medical licensure testing.

What does it take for each of these model parameters to be independent of the others in the manner that we take for granted in actual practice? Can we frame our observations of the members of each facet in the model in ways that will clearly show us when we have failed to obtain the desired independence? Can we do that in a way that simultaneously provides us with a means for communicating information about individual employees, policies, and challenges efficiently in a common language?

Can that common language be expressed in words and numbers that capitalize on the independence of the model parameters and so mean the same thing across local particulars? Can we set up a system for checking and maintaining the meaning of the parameters over time? Can we build measures of employee abilities, policy efficiencies, and market challenges into our information systems in useful ways? Can we improve the overall quality, efficiency, and meaningfulness of our industry by collaborating with other firms, schools, non-profits, and government agencies in the development of reference standard metrics?

These questions all have the same answer: Yes, we can. These questions set the stage for understanding how effective leadership depends on effective information management. If, as Yahoo! CEO Bartz says, leadership has become more difficult in the age of blogospherical second-guessing and “opposition research,” why not tap all of that critical energy as a resource and put it to work figuring out what differences make a difference? If critics think they have important questions that need to be answered, the independence and consistency, or lack thereof, of their and others’ responses gives real heft to a “put-up-or-shut-up” criterion for distinguishing signal from noise.

This kind of a BS-detector supports leadership in two ways, by focusing attention on meaningful information, and by highlighting significant divergences from accepted opinion. The latter might turn out to be nothing more than exceptionally loud noise, but it might also signal something very important, a contrary opinion sensitive to special information available only from a particular perspective.

Bartz is right on, then, in saying that the central role of information in leadership has made listening and mentoring more important than ever. Modeling the organization and experimenting with it makes it possible to listen and mentor in completely new ways. Testing data for independent model parameters is akin to tuning the organization like an instrument. When independence is achieved, everything harmonizes. The path forward is clear, since the ratings delineate the range in which organizational performance consistently varies.

Variation in the measures is illustrated by the hierarchy of the policy and market items rated, which take positions in their distributions showing what consistently comes first, and what precedents have to be set for later challenges to be met successfully. By demanding that the model parameters be independent of one another, we have set ourselves up to learn something from the past that can be used to predict the future.

Further and quite importantly, as experience is repeatedly related to these quantitatively-scaled hierarchies, the factors that make policies and challenges take particular positions on the ruler come to be understood, theory is refined, and leadership gains an edge. Now, it is becoming possible to predict where new policies and challenges will fall on the measurement continuum, making it possible for more rapid responses and earlier anticipations of previously unseen opportunities.

It’s a different story, though, when dependencies emerge, as when one or more employees in a particular area unexpectedly disagree with otherwise broadly accepted policy efficiencies or market challenges, or when a particular policy provokes anomalous evaluations relative to some market challenges but not others. There’s a qualitatively different kind of learning that takes place when expectations are refuted. Instead of getting an answer to the question we asked, we got an answer to one we didn’t ask.

It might just be noise or error, but it is imperative to ask and find out what question the unexpected answer responds to. Routine management thrives on learning how to ever more efficiently predict quantitative results; its polar opposite, innovation, lives on the mystery of unexpected anomalies. If someone hadn’t been able to wonder what value hardened rubber left on a stove might have, what might have killed bacteria in a petri dish, or why an experimental effect disappeared when a lead plate was moved, Vulcanized tires, Penicillin, and X-ray devices might never have come about.

We are on the cusp of the information analogues of these ground-breaking innovations. Methods of integrating rigorously scientific quantities with qualitative creative grist clarify information in previously unimagined ways, and in so doing make it more leveragable than ever before for advancing strategy, engaging customers, and motivating employees.

The only thing in Carol Bartz’s article that I might take issue with comes in the first line, with the words “will be.” The truth is that information already is our greatest opportunity.

Creative Commons License
LivingCapitalMetrics Blog by William P. Fisher, Jr., Ph.D. is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at livingcapitalmetrics.wordpress.com.
Permissions beyond the scope of this license may be available at http://www.livingcapitalmetrics.com.