RWJF - President’s Message

Accountability, we believe, requires letting the world know the results of our grantmaking in depth, in ways that can be acted upon. As with any enterprise, foundations need to know if they are making a difference. The question is rarely whether to measure success; it is more often what to measure and how. Foundations lack the usual yardsticks of success used in business, government or academia. No financial bottom line, periodic election returns or U.S. News & World Report rankings exist against which to calibrate our performance.
     Collectively, foundations vary greatly in missions, goals and strategies; the scope, scale and nature of the grants we make; the time frames of our grantmaking; and the degree to which our contributions are even identifiable. Though I have enjoyed reading the occasional reports on the large foundations that are written by professional foundation-watchers (and they have generally been kind to RWJF), they are highly subjective and their methods are not reproducible.
     At RWJF, we have spent a great deal of time and energy developing and pursuing three interrelated approaches to assessing how we are doing: evaluations, performance measurements and public disclosure. Independent, external evaluations of RWJF national programs and some major grants, conducted by some of the health care field’s leading researchers, have long been a hallmark of this organization. In recent years we have developed a variety of internal performance measures—including development and assessment of strategic objectives within our program interest areas and periodic formal and informal assessment of how we are doing as judged by important audiences—with the intent of integrating the results and feeding them back into future grantmaking. We spend an increasingly large proportion of our quarterly Board of Trustees meetings wrestling with how to measure the impact of our proposed and existing programs.
     Some of our pioneering efforts, in my view, lie in the realm of public disclosure—for example, our reports on grant results and the essays in the annual RWJF anthology, To Improve Health and Health Care. Our growing library of 600 Grant Results Reports (available online at www.rwjf.org) looks carefully at what was accomplished by the scores of grants made each year. The anthology—available online and in paperback—attempts to provide a critical, in-depth review of individual Foundation programs, grantmaking approaches and impacts on specific fields. In terms of traditional evaluation, we have committed almost $20 million to new program evaluation in 2001, and we have some $56 million in evaluations ongoing from this and prior years.
     Despite these efforts, our quest for performance measurement remains incomplete. In part this is because it is so difficult to establish causality when we are working on complex social issues, often alongside many others. For example, during the past decade we have invested heavily in programs to reduce the number of Americans who lack health insurance. Despite our efforts, the number of uninsured has resumed its upward climb. Should we accept some blame for that lack of progress? Did our efforts prevent worse outcomes? How can we know?
     We also sometimes make our job harder by not specifying up front exactly what we hope to achieve with a particular grant, program or grantmaking strategy. Sometimes we oversell what we hope to accomplish, because we believe in it and because we want the support of our colleagues. Sometimes we are tempted to tackle trivial problems where we know we can measure our results. I like to think that’s a temptation we usually resist.
     The widely disparate strategies we employ defy ready comparison. For example, seeing the results of our efforts in leadership development takes years, if not decades, as compared to the next-day results we can obtain from a poll on a topical issue. While the latter may help us or others shape a short-term action, design a program or make a policy decision, the former contributes in some way to the development of people who will assume important leadership positions some 20 years hence.
     Despite these difficulties, we in philanthropy owe it to ourselves, our constituencies and the fields in which we work to try as hard as possible to judge the worth of what we do. We must not abandon attempts at assessment because the tools are crude. I have watched RWJF get better and better at evaluation over the past decade, and I know this essential struggle will continue.