Chairman’s Corner: March 2010

‘Social Outcomes’: The Elephants in the Room

by Mario Morino, Co-founder of VPP

In my last column, “‘Social Outcomes’: Missing the Forest for the Trees?,” I wrote about my deep, nagging fear that many efforts to assess outcomes are woefully off track. I pointed to several wonderful beams of shining light in the field, from Youth Villages to the Cleveland Clinic. But my dominant message was that many efforts in our sector are causing more harm than good. That column hit a nerve, triggering a volume and intensity of responses far greater than I had expected.

After struggling though literally 20 drafts, I wasn’t sure whether the piece was even coherent. Yet, it seemingly made some sense to a good number of thought leaders and practitioners I greatly respect—along with many others I don’t know who apparently care deeply about this issue as well.

Of course, not everyone agreed with my analysis. In fact, I got hard pushback on some points, and a few commentators wondered why it had taken me so long to own up to my own limitations in my approach over the years to the topic of outcomes.

The majority, however, agreed with the thesis that we’ve lost sight of the ends we’re trying to advance. In the wise words of David Hunter, Managing Partner of Hunter Consulting and former Director of Assessment for the Edna McConnell Clark Foundation: “It seems to me that the mess you describe indeed is enormous and very destructive—because few people involved in this work have thought deeply about managing towards outcomes and [they] have put the cart before the horse—focusing…on HOW to measure rather than on WHY measure…and WHAT to measure.”

Sins of Commission, Sins of Omission

The feedback confirmed for me that nonprofit executives, staff, and boards; donors; and assessment experts are deeply frustrated with our sector’s work around outcomes.

• I heard a lot of anger at funders who don’t walk (or fund) their talk.
• I heard exasperation with nonprofits that are unwilling to embrace even basic methods of determining whether they’re doing what they say they do.
• I heard disappointment that outcomes assessment has become an exercise focused on cold numbers—the equivalent of Robert McNamara’s simplistic and terribly misleading Vietnam body counts—rather than an effort to help nonprofit leaders achieve lasting impact for those they serve.

We must be intentional about surfacing these roiling frustrations that are rarely getting voiced. If we don’t, we’re going to continue to perpetrate sins of commission and omission that prevent us from making even the slightest dent in the failing status quo that defines education, healthcare, and social services in America.

The most common sin of commission is when we funders, in the name of “measurement” and “accountability,” foist unfunded, often overly simplistic, self-serving mandates on our grantees—rather than genuinely helping them define, create, and use the information they need to be disciplined managers. In the words of Tris Lumley, Head of Strategy for London-based New Philanthropy Capital, “Great organisations…are built around great data. Data that allows them to understand the needs they address, what activities are likely to best address these needs, what actually happens as a result of these activities, and how to allocate resources and tweak what they do for even greater impact. Too often, funders set the agenda with their own requirements [and] cripple the organisations they’re trying to help.”

The sin of omission I often see is when funders and nonprofits run away from outcomes and their measurement altogether—that is, nothing assesses whether nonprofits are delivering on their promises to the families who turn to them for services. As Hunter said, “It is a really, really bad thing for nonprofits to promise to help people improve their lives and prospects…and then, when the matter is looked at closely, it turns out that they aren’t doing that at all!”

It’s About Management, Not Metrics

Based on all the feedback, it is clear to me that our sector needs a major reset on the approach to outcomes—from how we think about them to how we assess them.

If I were a real innovator, I would invent one of those memory-erasing devices that sometimes pop up in science fiction movies and immediately use it to erase every frustrating memory we have built up though our experiences with metrics, measurement, outcomes, and evaluation. With everyone starting with a clean slate and open mind, I would make the case that more than anything else, our sector needs a singular focus on managing to outcomes. And here’s precisely what I mean:

• Gain clarity, through thoughtful introspection, on what change we are trying to create;
• Gain specificity on how we will accomplish that change;
• Determine what information (hard and soft) will be most helpful to understand if we are on course to achieve that change;
• Collect and use this information as a basis for being disciplined within mission—that is, to plan, make important decisions, track, course correct, and improve;
• Combine all of the above with good judgment and keen discernment, which are more important than any metric.

In my experience, some nonprofit leaders inherently think in terms of outcomes or are at least open to doing so. They bring more than intuition and personal agenda; think deeply about the what, how, and why of their services; are evidence-based; and talk naturally and frequently about the change happening in the lives of those they serve. These leaders are genuinely hungry for reliable information to assess their value to those they serve. They want to manage to outcomes. It’s no surprise that these are the leaders we look for in prospective VPP partners.

Leaders who have an innate desire for good information that is aligned with their mission are the ones most likely to develop a true performance culture and make a real difference in the lives of those they serve. And before those readers who rebel against the term “performance culture” get too incensed, please step back from the jargon and debates of the times and ask, “How can anyone with a deep commitment to their mission not want to be rigorous in determining how they can do the most good for those they serve?”  This is what I seek to convey when I use the term.

We have also found that using information to manage to outcomes and having a performance culture are absolutely dependent on an attitude and mindset that must come from within. Trying to impose this orientation on leaders and organizations is as constructive as trying to foist change on your spouse. As my better half will tell you (with a resigned sigh), it ain’t gonna happen.

What Managing to Outcomes Looks Like

Geoff Canada, founder and CEO of Harlem Children’s Zone and one of my heroes, raised a stir with some provocative comments that were published in the New York publication City Limits.

When Canada was asked to define success for HCZ, he said, “The only benchmark of success is college graduation. That’s the only one: How many kids you got in college, how many kids you got out.”

Canada could not have been clearer on the ultimate outcomes HCZ is focused on achieving. It’s not improving reading levels. It’s not getting kids to graduate high school. It’s not helping kids get into college. To HCZ, these are important interim indicators to ensure they are moving in the right direction, but, ultimately, it’s ensuring those young people make it through college that matters.

With that great clarity as a starting point, Canada and his team, with the help of the Edna McConnell Clark Foundation, Bridgespan, and others, have gotten good at identifying the information they need to collect in order to manage to these outcomes. Are all the kids in the HCZ graduating from college? Of course not. But HCZ is on a very promising path.

A Challenge to Us All

Based on what I’ve learned over the years—through my work with VPP; my direct engagement with schools and hospitals in the Cleveland area; my observations of HCZ and other innovators; and my own experience implementing information systems in my business career—I have some thoughts on how we can get from a place of pent-up frustrations to a place of real progress. My next column will go into greater detail, but here is an overview of what’s to come.

If you agree that many efforts in our sector to assess outcomes are causing more harm than good, you can do something right now. Whether a funder, nonprofit executive, staff member, board member, or beneficiary of nonprofit services, you are likely in one of two situations you can influence.

If you’re not focused on outcomes (or doing very little), then please recognize that you—the executives, staff, board, and funders—have an affirmative obligation to engage. It’s mission-critical to know whether you’re on track to deliver what you promise to those you serve. I have great respect for leaders’ intuition, but intuition alone is almost never enough.

If your nonprofit has defined your intended outcomes and maybe even progressed to reporting on them, then please stop, step back, and rigorously question what you’ve done (or plan to do). Rather than think of this as a step backward, realize it may be the first step toward course correcting and getting on a truly productive track.

In either case, please remember the critical, first-order question, “To what end?” Think about Geoff Canada. Are you on a path to gain the clarity he has achieved (after many years of struggle!) on the ends he’s trying to advance for the children and young people he serves? These are difficult and fundamental questions that you owe yourself and those your nonprofit serves to answer honestly and with deep introspection.

I encourage nonprofits to undertake facilitated discussions, perhaps inviting informed voices to brief their boards and staff. For example, I’ve been fortunate to be deeply engaged with The Lawrence School in Northeast Ohio, which serves grades 1-12 students with learning differences and attention-deficit disorders. We have benefited greatly by having a facilitator—a seasoned, skilled professional who understands management and organizations well—lead working groups of board and staff to sort out and define fundamental aspects of what the school does and represents. The facilitator has helped us conduct concerted and lengthy efforts to gain greater clarity of mission and vision and define the school’s guiding principles and underlying values.  Similarly, discussions are well along to clarify and explain more clearly whom the school serves and to define, with specificity, its educational model and how it differentiates itself from other educational approaches. These are difficult matters to take on, but, with a leadership that has embraced this approach and put trust in the process, the gains have been nothing short of transformational.

None of this suggests in any way that summative and formative evaluation are not important, particularly for building information about what works and what doesn’t for the field or a discipline. But if we really want to help organizations deliver quality services most effectively, then our priority must be on identifying the nonprofits with the willingness, propensity, and capacity to manage to outcomes—and then helping them do just that. This will require major commitments of time and money to help these nonprofits build their culture of information-based introspection and the performance-management systems that can deliver the information they need on an ongoing basis, while never losing an ounce of their mission-driven passion.

Thanks to all who took the time to offer comments on my last column—both pushback and push-forward. Surfacing your frustrations helped me refine and crystallize my thinking, so keep ‘em coming