We Must Be Scientists for Change

Steve Andersen is Vice President of Technology and Innovation at Salesforce.com Foundation where he helps nonprofits use Salesforce.com to change the world. He believes we are at a point in time where agents of social change can get access to the same quality technology systems that large corporations can.

"We know very little, and yet it is astonishing that we know so much, and still more astonishing that so little knowledge can give us so much power." - Charles Franklin Kettering (1876-1958) Engineer, inventor, and co-founder of the Sloan-Kettering Institute

We in the social change sector are working to address the most important problems in our world today. Justice, equality, poverty, sustainability...we're not shy in putting our stake in the ground and taking on these daunting issues. I love that about this community -- the willingness to stand in the face of massive adversity, to think creatively, and try new things in the effort to bring about change. We’re innovative, we’re resourceful, and we achieve real results.

And yet as enamored as I am with the chutzpah I see in myself and my colleagues, I am unsettled. I look at my work and at what we've all achieved and I come to the unwelcome conclusion that it's not enough. Change isn’t happening fast enough. I’m not being the change agent I want to be, and the movement isn’t living up to its potential. Why aren’t we farther along our path to change? It’s clearly not from a lack of desire, creativity, hard work, and just plain smarts. Spend one day in the social change arena and you know all of these traits are present in abundance. Why, then, are we all frustrated with the speed of change?

I think part of the answer may be that although we are incredibly creative, we, as a movement, are not scientific enough in our approach. By applying scientific methods more broadly to our work, I think we can accelerate our impact and get closer to our audacious goals.

At heart I'm a scientist -- my first real job was in molecular biology and later I began my tech career at a biotech firm. I developed a deep respect for the scientific method -- how it starts with an idea, poses that idea as a testable question, and then strives to answer that question through experimentation. I am in awe of scientists who see a result and immediately ask the next question. I respect measurement, and I trust in evidence.

In publishing her findings, a scientist is making an argument for how she thinks the world works. At the same time, she publishes her methods -- the techniques and technologies she used to get her results. She shares her methods so that others might replicate her experiments and see for themselves. But even more importantly this shared knowledge can be applied to other problems she hasn’t even considered. The spread of new ideas and technology though scientific publication, this cycle of advancement, has driven the progress of science for centuries.

We must be scientists for change. I think being more scientific is the only way we can make good on the amazing opportunities in front of us. I see four main qualities that we need to foster in our movement to become more effective:

1. A willingness to experiment with bold ideas and strategies in testable ways
2. Excellence in the creative application of technology in service of those experiments
3. Meticulous interpretation and sharing of results
4. Imaginative posing of the logical next question and returning to step 1 as quickly as possible

Testing Bold Ideas

When we come up with a new thought, “I wonder if college students would be more active if they received messages about the issue from their friends,” we should reframe it as a hypothesis. Reframed: “College students will lobby decision makers at a higher rate if they are engaged by their trusted circle in a convenient way.” This is a better way of posing the question, as it can be proven true or false. We’ve removed the ambiguity, and we can design our work to speak to the question.

We can’t test this hypothesis by blasting personalized SMSs to 12,000 students. If we did and we got a low response, what have we learned? There are too many unknowns--maybe they didn’t like getting messages from friends, maybe they were upset by the $0.10 we just added to their phone bill, maybe they were all at the big football game.

Better design can give us more information. We could send SMSs from trusted friends to one group of 3,000 students. Another set could receive a generic action alert via SMS. A third group could receive an email from a trusted friend. The final 3,000 could get an email from our organization. We need to start thinking about controlling our experiments -- isolating the factors we want to test. In the four-part design outlined above, we’re trying to test two main factors -- the method of communication and the messenger. By carefully designing our experiments to isolate what we’re testing, we are guaranteed to learn something from it. In this case we may learn that message and messenger don’t make a lick of difference, but even that would be a valuable lesson.

Excellence in Technical Execution

Our experiments require a certain amount of skill with modern technology, often ones with which we’re not yet comfortable. If designed correctly, our experiments will generate data that will be nearly as important as the outcome of the effort itself. We must become excellent at trying new things with technology and also at building stable systems to support analysis of our data.

Many groups struggle with sending a single mass email and would look at the experiment above and have no idea where to begin making it happen. That needs to change. We need to embrace the use of technology instead of fearing it. We must invest in technical competency within our organizations and partner with proven implementers who can help us with our experiments. Our leaders should be in the vanguard of this effort, evangelizing the importance of technology in service of mission, be visible, competent users of technology, and invest appropriately in systems for their organizations.

Meticulous Measurement and Sharing

The data we generate will hold valuable truths, but only if we build the competency for mining it. Designing our work as testable experiments will cause us to change the way we analyze our efforts. We need more statisticians in our movement, and we’ll seek them out as our experimental results start rolling in.

If we learn from our analysis that college students are 55% more likely to respond to a personal SMS from a trusted friend than to an email, we need to tell that story, and not just in the halls of the next conference we attend. We should develop structures and systems for sharing our experimental design and the results. Our goal should be that others can replicate our work and push it in new directions. This is the best way we can invest in our movement as a whole--by contributing to our shared knowledge, “competitive advantage” be damned.

Relentless, Iterative Experimentation

In our experiment, we learned that students are much more responsive to personal SMSs than to our old way of communicating. We clearly need to start sending text messages in our daily work! But we can’t be satisfied with that single finding. What is our next experiment? What thoughts does this finding spur? We can’t hesitate to integrate what we’ve learned and pose a succeeding hypothesis, thus building our knowledge base.

If we can excel at these four traits -- testing bold ideas, excelling with technology, measuring and sharing, and relentless iteration -- we can accelerate our progress significantly. Of course it's not easy. Being rigorous and scientific in our environment is incredibly challenging. It's risky, requires specialized skills, is more expensive in the short-term, and can produce experimental results that can prove inconvenient at times for us and our organizations.

But we don’t have the luxury of playing it safe. We need to push forward and ask deep questions of ourselves. This kind of searching is going on in many places in our movement, but most notably in the global development sector. Thinkers like William Easterly, Esther Duflo, and David Roodman are laying down the gauntlet and demanding that we stop giving blindly and start running testable experiments. They are pointing out that it is actually possible to know the impact of our efforts, as long as our efforts are thoughtfully designed and meticulously analyzed.

Let’s lead by creating new ways of working for change -- ways to which we will be wedded only as long as they prove to be effective. Let’s treat our movement as a body of knowledge that we all can rely on, contribute to, and debunk parts of as appropriate. Let’s challenge each other and systematically share, all with the aim of building our movement's understanding of how to best bring about change.

Achieving massive impact isn't easy, we all know that. And there is no charted path. We are doing amazing, creative, impactful work. And we have learned so much in the process. Charles Kettering, whose quote opened this post, also said:

"Every honest researcher I know admits he's just a professional amateur. He's doing whatever he's doing for the first time. That makes him an amateur. He has sense enough to know that he's going to have a lot of trouble, so that makes him a professional."

We're attempting to tackle massive, unsolved problems, often with little more than good ideas and an abiding desire to help. We’re inventing strategies and tactics in the moment, as amateurs -- it’s exciting and challenging work. Thinking about our work scientifically can help us be brilliant professionals as well, driving us more quickly and systematically to the impact for which we all strive.

Related reading:
White Man’s Burden, William Easterly
TED Talk: Social Experiments to Fight Poverty, Esther Duflo
Critical Mass, Philip Ball