I’ve been working in the world of program development with a lot of complexity and innovation and co-creation lately and have seen these three terms used sometimes interchangeably to describe a strategic move. As a result, I’ve been adopting a more disciplined approach to these three kinds of activities.
First some definitions.
Taken explicitly from Cynefin, a probe is an activity that teaches you about the context that you are working with. The actual outcome of the probe doesn’t matter much because the point is to create an intervention of some kind and see how your context responds. You learn about the context and that helps you make better bets as you move forward – “more stories like this, less stories like this” to quote Dave Snowden. Probes are small, safe to fail and easily observed. They help to test different and conflicting hypotheses about the context. If 8 out of 10 of your probes are not failing, you aren’t learning much about the limits of your context. Probes are actually methods of developmental evaluation.
A prototype is an activity that is designed to give you an idea of how a concept might work in reality. Prototypes are designs that are implemented for a short time, adjusted through a few iterations and improved upon. The purpose of a prototype is to put something into play and look at its performance. You need to have some success with a prototype in order to know what parts of it are worth building upon. Prototypes straddle the world of “safe to fail” and fail safe. They are both developmental evaluations tools and they also require some level of summative evaluation in order to be fully understood. Prototypes are also probes, and you can learn a lot about the system from how they work.
A pilot is a project designed to prove the worthiness of an approach or a solution. You need it to have an actual positive effect in its outcomes, and it’s less safe to fail. Pilots are often designed to achieve success, which is a good approach if you have studied the context with a set of probes and maybe prototyped an approach or two. Without good intelligence about the context you are working with, pilots are often shown to work by manipulating the results. A pilot project will run for a discrete amount of time and will then be summatively evaluated in order to determine its efficacy. If it shows promise, it may be repeated, although there is always a danger of creating a “best practice” that does not translate across different contexts. If a pilot project is done well and works, it should be integrated with the basic operating procedure of an organization, and tinkered with over time, until it starts showing signs of weakened effectiveness. From then on, it can become a program. And pilots are alos probes, and as you work with them they too will tell you a lot about what is possible in the system.
The distinctions between these three things are quite important. Often change is championed in the non-profit word with the funding of pilot projects, the design of which is based on hunches and guesses about what works, or worse, a set of social science research data that is merely one of many possible hypotheses, privileged only by the intensity of effort that went into the study. We see this all the time with needs assessments, gap analyses and SWOT-type environmental scans.
Rather than thinking of these as gradients on a line though, I have been thinking of them as a nested set of circles:
Each one contains elements of the one within it. Developing one will be better if have based your development on the levels below it. When you are confronted with complexity and several different ideas of how to move forward, run a set of probes to explore those ideas. When you have an informed hunch, start prototyping to see what you can learn about interventions. What you learn from those can be put to use as pilots to eventually become standard programs.
By far, the most important mindshift in this whole area is adopting the right thinking about probes. Because pilot projects and even prototyping is common in the social development world, we tend to rely on these methods as ways of innovating. And we tend to design them from an outcomes basis, looking to game the results towards positive outcomes. I have seen very few pilot projects “fail” even if they have not been renewed or funded. Working with probes turns this approach inside out. We seek to explore failure so we can learn about the tolerances and the landscape of the system we are working in. We “probe” around these fail points to see what we can learn about the context of our work. When we learn something positive we design things to take advantage of this moment. We deliberately do things to test hypotheses and, if you’re really good and you are in a safe-to-fail position, you can even try to create failures to see how they work. That way you can identify weak signals of failure and notice them when you see them so that when you come to design prototypes and pilots, you “know when to hold ‘em and know when to fold ‘em.”
Caitlin and I are hosting a learning process for the Vancouver Foundation which has brought together 11 people from community foundations around BC. We are trying to discover what kinds of new practices community foundations can adopt to roll with the changing nature of philanthropy and community.
It’s a classic complexity problem. The future is unknowable and unpredictable. Data is plentiful but not helpful because context trumps all. There are competing experts with different hypotheses of what should happen. These twelve people are brave. They’re willing to be the innovators in a sector that is by nature fairly conservative when it comes to change.
We are using an architecture combining Theory U and complexity work coming from Cynefin practices. I can maybe write more about our design later, but today I’m struck by a comment one of our participants made when she was reflecting on the past three months of engaging in deep dialogue interviews with people in her community. She talked to a number of people as a way of beginning to understand the context for making change, and noticed that the conversations she was having were taking her away from the rigid roles and responsibilities (and the associated posturing) that comes with trying to do interesting work in a hierarchical, top down and controlling way. Today in our check in she shared this:
“When we are given permission to talk to anyone about anything it’s freeing. We let our roles drop as well our limiting beliefs about what we can and can’t do. We are able to more closely align our actions and our way of being with our intentions.”
A pithy but powerful statement in how changing the way we converse changes the way we are able to act. It’s lovely witnessing the birth of a complexity worker.
One of my favourite concepts from the complexity world is the fallacy of thinking that comes from the truth of retrospective coherence. The mistake is that, because we can look back in time to understand causes of our current condition, we can therefore see forward in time and affect the causes of a future condition. Complex systems are emergent, so we can never be sure what the future holds, regardless of how well we can trace how we got here.
Despite the fact that it is illegal to sell an investment instrument without the warning that “past performance does not guarantee future results” falling for the trap that retrospective coherence gives you a reliable path forward is basically a feature of doing any strategic work at all. It leads to planning that puts out a future preferred state and then backcasts a set of steps that, if we follow them, will take us there or nearly there.
So there are all kinds of issues with this, and the Cynefin framework’s greatest gift is that it helps us create strategy to avoid to pitfall of retrospective coherence.
Today though, a surprise in my morning reading. A lovely article on Robert Frost’s “The Road Not Taken.” We all think we know what that poem is about: about the adventure that will ensue if we just take the less beaten path. But you might be surprised to learn that the poem is actually about retrospective coherence and not adventures strategic planning (emphasis mine):
Most readers consider “The Road Not Taken” to be a paean to triumphant self-assertion (“I took the one less traveled by”), but the literal meaning of the poem’s own lines seems completely at odds with this interpretation. The poem’s speaker tells us he “shall be telling,” at some point in the future, of how he took the road less traveled by, yet he has already admitted that the two paths “equally lay / In leaves” and “the passing there / Had worn them really about the same.” So the road he will later call less traveled is actually the road equally traveled. The two roads are interchangeable.According to this reading, then, the speaker will be claiming “ages and ages hence” that his decision made “all the difference” only because this is the kind of claim we make when we want to comfort or blame ourselves by assuming that our current position is the product of our own choices (as opposed to what was chosen for us or allotted to us by chance). The poem isn’t a salute to can-do individualism; it’s a commentary on the self-deception we practice when constructing the story of our own lives. “The Road Not Taken” may be, as the critic Frank Lentricchia memorably put it, “the best example in all of American poetry of a wolf in sheep’s clothing.” But we could go further: It may be the best example in all of American culture of a wolf in sheep’s clothing.
Brilliant.
A small elevator speech I shared on the OSLIST yesterday:
Self organization works by a combination of attractors and boundaries. Attractors are things that draw components of a system towards themselves (gravity wells, a pile of money left on the ground, an invitation). Boundaries (or constraints) are barriers that constrain the elements in a system (an atmosphere, the edges of an island, the number of syllables in a haiku)
Working together, attractors and boundaries define order where otherwise there is chaos. We can be intentional about some of these, but not all of them. Within complex systems, attractors and constraints create the conditions to enable emergence. What emerges isn’t always desirable and is never predictable, but it has the property of being new and different from any of the individual elements within the system.
Self-organization is where we get new, previously unknown things from.
Dave is working on a theory of change, which I think is a good thing. In this latest post he has a nice summation of the way to move to action in complex situations (like cultures):
So where we are looking at culture change (to take an example), we first map the narrative landscape to see what the current dispositional state is. That allows us to look at where we have the potential to change, and where change would be near impossible to achieve. In those problematic cases we look more to stimulating alternative attractors rather that attempting to deal with the problem directly. Our method is the look at the narrative landscape and then ask the questions What can I (we) do tomorrow to create more stories like these and fewer like those? The question engages people in action without analysis and it allows us to take an approach that measures vectors (speed and direction) rather than outcome. The question also allows widespread engagement in small actions in the present, which reduces the unexpected (and potentially negative) consequences of large scale interventions.
In sum, complexity work is about understanding the context to understand where the potential for evolution might lie. From there you try experiements to see what you can learn, and support what works while removing support for what doesn’t
It’s an old saw, but it’s actually a simple thing. And I keep writing about it because it seems TOO simple for most folks. Shouldn’t strategy be more ordered, laid out and thought through than this.
As always the answer depends, but with complex situations the answer is no. Save your discipline and rigour for understanding things as they evolve rather than trying to get it all right from the start.
via Change through small actions in the present – Cognitive Edge.