Chris Corrigan Chris Corrigan Menu
  • Blog
  • Chaordic design
  • Resources for Facilitators
    • Facilitation Resources
    • Books, Papers, Interviews, and Videos
    • Books in my library
    • Open Space Resources
      • Planning an Open Space Technology Meeting
  • Courses
  • About Me
    • Services
      • What I do
      • How I work with you
    • CV and Client list
    • Music
    • Who I am
  • Contact me
  • Blog
  • Chaordic design
  • Resources for Facilitators
    • Facilitation Resources
    • Books, Papers, Interviews, and Videos
    • Books in my library
    • Open Space Resources
      • Planning an Open Space Technology Meeting
  • Courses
  • About Me
    • Services
      • What I do
      • How I work with you
    • CV and Client list
    • Music
    • Who I am
  • Contact me

Category Archives "Evaluation"

What’s in the central garden?

June 15, 2016 By Chris Corrigan Art of Hosting, Collaboration, Conversation, Evaluation, Facilitation, Leadership, Learning, Organization, Philanthropy 6 Comments

A few years ago, Juanita Brown shared a very powerful image with me. She talked about how those of us that practice dialogue and facilitation in a deep way have access to various gateways that take us into a “central garden.” All of our pathways invite us into this garden where we come to discover and realize something about the role of dialogue, meaning making and collaboration. It is a set of realizations that lies beneath the practice of methods.

On a call today with my friend Mark McKergow, we were discussing this image  There are a bunch of us – although not a large bunch of us – from different practitioner communities who are always interested in transcending our methods and entering into this conversation.   Alongside Juanita, Mark has also been wondering “where is everybody else, and how come we’re not connecting?”

Today we were discussing the failure of dialogue to have enough presence to provide workable and practical alternatives to everything from public policy decisions (such as the EU referendum in Britain, or the polarization of US society) to the everyday challenges of managing and running large organizations, evaluating, strategizing and controlling outcomes, people and money.  

We know that our field of dialogic practice is massive, well researched and well documented.  We know that leadership literature is filled with the importance of relational and sense making work. We know that that mid-career professionals end up coming to our various workshops to take on skills and ideas that are fundamentally transformative to their work and lives and that they go back to places where “it’s difficult to implement” because other mid-career professionals are wedded to globalized management practices that are good enough for what they are trying to do, within the highly constrained performance frameworks within which they are forced to operate. We even know (thanks to people like Jon Husband) that global organizations like Hay Associates have spent the better part of a century ensuring that these management science constraints are widely deployed and understood. They frame everything, not without utility, but to the exclusion of almost every other way of organizing and being together in human endeavour.

So what is the problem? Are we just lousy storytellers? Are we being deliberately marginalized? Is there something fundamentally flawed about the ability of dialogic practice to actually be of value?  And how do we disrupt the standard set of management tools and the narcissism of our own communities of practice in a way that creates some serious openings for change?

What do you think?

Share:

  • Click to share on Mastodon (Opens in new window) Mastodon
  • Click to share on Bluesky (Opens in new window) Bluesky
  • Click to share on LinkedIn (Opens in new window) LinkedIn
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • More
  • Click to share on Reddit (Opens in new window) Reddit
  • Click to share on Tumblr (Opens in new window) Tumblr
  • Click to share on Pinterest (Opens in new window) Pinterest
  • Click to share on Pocket (Opens in new window) Pocket
  • Click to share on Telegram (Opens in new window) Telegram

Like this:

Like Loading...

Exploring future possibilities by mapping “dispositionalities”

April 25, 2016 By Chris Corrigan Art of Harvesting, Art of Hosting, Complexity, Evaluation, Facilitation, Featured, Learning, Philanthropy, Uncategorized, World Cafe 5 Comments

It’s good to have Dave Snowden back from his treks in the Himalayas. He’s been a big influence on my thinking and practice over the past few years and his near daily blog posts are always rich, irreverent and practical. He is in the process of creating an important body of theory and practice that is useful even if the language and the concepts are sometimes a lot of work to grasp. The payoff from wrestling with his ideas is rich.

Today he’s discussing “dispositionality” which simply means that making change in a system is much easier when you have a sense of what the system is pre-disposed to do (and what it is NOT pre-disposed to do…)

Back in the summer Caitlin and I led a learning lab for the board and staff members of various community foundations from around British Columbia.  The five principles that Dave articulated today were very much embedded in our work and they are becoming very much the basis for any change and planning work I do.  Here’s how we made it work, pen and paper style.

1. Map the current state of the system, including its dominant flows, eddy points and whirlpools.

We began with a World Cafe design based on small stories of change. It is always good to ask people about actual decisions or stories that they remember to ground their experience in discovery. If you run a cafe on “What are the big sources of change in our sector?” you get a data set that is divorced from reality, meaning that it is subject to being gamed by the participants. I can just insert the things I want to see in there.  But if I am asked to tell a story about a particular decision I had to made, the data set is richer and we have a good chance to see emerging patterns.

And so our Cafe ran like that: “Tell a story of a time when you knew things needed to change?”

Each person told a story and the other three at the table listened and wrote down what they heard was the impetus for change, with one data point on a post-it note.  We did several rounds of story telling.  At the end of the round, we asked people to give the post-its to the story teller, and we gave the story tellers time to rank each post it note on a scale of 1-3. A one meant that the impetus for change was just known to me (a weak signal),  two meant that a few other people know about this impetus, and a three meant that this change trigger was known by everybody.

We then had the group cluster all the post-its to find major categories, and we sorted post it notes within the categories to produce a map that was rendered by our graphic recorder, Corrina Keeling. You can see that above.

2. Identify the energy gradient associated with existing dominant patterns and what adjacent possible states to any undesirable pattern present themselves.

The resulting map shows the major areas for change making, specific “acupuncture points” and the “energy gradients associated with the dominant patterns.”  Practically what this means is that items marked in yellow were very weak signals and could be candidates for a change initiative that would appear out of left field for the dominant system.  Not a bad thing to do, but it requires a lot of resources and political capital to initiate.  The red items were things that EVERYBODY was talking about, which meant that the space for innovation was quite closed down.  There are a lot of experts, large consulting firms, influential funding pots and politically committed people tackling change at this level because it is perceived to be an influential place to play.  As a result it is generally a zone that is not failure tolerant and so these items are not good candidates for a probe or prototyping approach.

But the orange items were in a kind of Goldilocks zone: there are a few people who know that you can make change here, so you have allies, but the field is not cluttered with competing experts trying to assert their ideological solutions.

The whole map allows you to make choices.

3. Engage in safe-to-fail experiments in parallel either to change the energy gradient or to nudge (or shift) a dominant pattern to a more desirable state ideally through action rather than platitude.

This is of course the best approach to making change within complex systems.  We took time to develop prototypes that were intended to tell us something about the system. A bonus would be that we might might create ideas that would turn into interesting new initiatives, but the primary function of running prototypes is to probe the system to tell us something about what is possible.  Making tentative conclusions from action inspires people to try more, on a path that is a little more blazed.  Just creating platitudes such as “Let’s build networks for knowledge transfer” doesn’t do enough to help change makers poke around and try things that are likely to work.

Each participant in the group created one or two prototypes which they rolled out, seeking to make a bit of change and learn about what helped or hindered change making in a relatively conservative sector of civil society.

4. Monitor the impact in real time and take multiple small actions to reinforce the good and disrupt the bad.

We kept the group together over a few months, having them check in over webinars to share the progress on their prototypes. We deliberately created a space where things were allowed to fail or radically change and we harvested learning all the way along. Where things were working, prototypes evolved in that direction, and we had a little funding to help accelerate them. By simply starting, participants discovered oblique strategies and in some cases entirely new ways to address their basic desire for changing some element of their environment.  Without engaging in a deliberate yet loosely held action-based project, it is very difficult to see the opportunities that lie in the blind spots.

This learning was summarized in a report, but the bigger harvest was the capacity that each participant built to take steps to sense, design and implement change initiatives with a better informed complexity approach.

5. At all costs avoid any announcement of a change initiative or idealistic outcome based targets

I think this goes without saying. Change making in the complex space is essentially learning on overdrive. When we are truly stuck and yet we have a sense that “this might just work” we need good support to explore that instinct.  Being deliberate about it helps.  But announcing that “this is what we are doing and here are the targets we have to meet” will collapse people’s inherent creativity down to narrowing the focus of their work on achieve the pre-determined outcomes.  That is a perfect strategy for destroying the capacity to engage with complexity, and it can result in a myopic approach to change that guarantees “black swan events” and other nasty surprises.

Share:

  • Click to share on Mastodon (Opens in new window) Mastodon
  • Click to share on Bluesky (Opens in new window) Bluesky
  • Click to share on LinkedIn (Opens in new window) LinkedIn
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • More
  • Click to share on Reddit (Opens in new window) Reddit
  • Click to share on Tumblr (Opens in new window) Tumblr
  • Click to share on Pinterest (Opens in new window) Pinterest
  • Click to share on Pocket (Opens in new window) Pocket
  • Click to share on Telegram (Opens in new window) Telegram

Like this:

Like Loading...

PLUME: five principles of harvesting

March 16, 2016 By Chris Corrigan Art of Harvesting, Art of Hosting, Design, Emergence, Evaluation, Facilitation, Featured, Learning, Open Space, World Cafe 3 Comments

 

This morning we began our Harvesting and Collective Sensemaking online course.  Rowan Simonsen, Amy Lenzo and I were really excited to be able to share our first little insights with people, and especially this new mnemonic that we created to capture five key principles of harvesting practice: PLUME. We are excited to introduce this into the world.

Read More

Share:

  • Click to share on Mastodon (Opens in new window) Mastodon
  • Click to share on Bluesky (Opens in new window) Bluesky
  • Click to share on LinkedIn (Opens in new window) LinkedIn
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • More
  • Click to share on Reddit (Opens in new window) Reddit
  • Click to share on Tumblr (Opens in new window) Tumblr
  • Click to share on Pinterest (Opens in new window) Pinterest
  • Click to share on Pocket (Opens in new window) Pocket
  • Click to share on Telegram (Opens in new window) Telegram

Like this:

Like Loading...

Intervening in a complex system: 5 Ps

February 8, 2016 By Chris Corrigan Art of Harvesting, Art of Hosting, Complexity, Conversation, Design, Emergence, Evaluation, Featured, Leadership, Stories

When I was up in Whitehorse last week I got to spend time with folks from the Public Service Commission discussing a project that would see us looking at discriminations in the workplace from a complexity angle.  Using Cynefin and SenseMaker(tm), we hope to understand the ways in which the landscape of discrimination shifts and changes over time so that the PSC can make wiser decisions about the kinds of initiatives it sculpts.  One of the problems with diversity initiatives in the public service (in any large public organization really) is the feeling that they need to be broad based and rolled out to everyone.  This usually results in a single initiative that spreads across the whole organization, but except for a little awareness raising, does little to address specific instances of discrimination.  Everything from awareness raising “cultural competency training” to zero tolerance accountability measures have limited effect because a) discriminatory behaviour is highly context and situation dependant and b) the public service has a permeable boundary to the outside world, meaning ideas, behaviours and people move between the two contexts all the time.  The larger your organization, the more like the real world you have to be.

At any rate, I took a bit of time to do a mini-Cynefin teaching to explain how strategy works in the complex domain.  and my friend Pawa Haiyupis and I added two Ps to my concentric circles of intervention in a complex system.  So to review:

  • Patterns: Study the patterns in a complex setting using narrative capture and sense-making.  This can be done with the SenseMaker(tm) software, and it can also be done with dialogic interventions.  The key thing is to let the people themselves tag their stories or at the very least have a group of people reviewing data and finding patterns together.  For example, you might notice a correlation between stressful times in an organization and an increase in feelings of discriminatory behaviour
  • Probe: Once you have identified some patterns, you can make some hypotheses about what might work and it’s time to develop some safe to fail probes.  These aren’t meant to be successful: they are meant to tell you whether or not the patterns you are sensing have developmental potential.  Failure is entirely welcome. What if we offered stress reduction activities during high stress times to help release pent up feelings? We want to be okay with te possibility that that might not work.
  • Prototype: If a probe shows some promise, you might develop a prototype to develop a concept. Prototypes are designed to have tolerance for failure, in that failure helps you to iterate and improve the concept.  The goal is to develop something that is working.
  • Pilot: A pilot project is usually a limited time proof of concept.  Roll it out over a year and see what you learn.  In Pilot projects you can begin to use some summative evaluation methods to see what has changed over time.  Because of their intensive resource commitment, pilot projects are hardly ever allowed to fail, making them very poor ways of learning and innovating, but very good ways to see how stable we need to make an approach.
  • Project/Program/Policy: Whatever the highest level and most stable form of an initiative is, you will get to there if your pilot shows promise, and the results are clear. Work at this level will last over time, but needs regular monitoring so that an organization knows when it’s time to tinker and when it’s time to change it.

Cynefin practitioners will recognize that what I’m writing about here is the flow between the complicated and the complex domains, (captured by Dave Snowden’s Blue dynamic in this post.)  My intention is to give this some language and context in service organizations, where design thinking has replaced the (in some ways more useful) intuitive planning and innovation used in non-profits and the public service.

Since October, when I first starting sketching out these ideas, I’ve learned a few things which might be helpful as you move through these circles.

  1. Dialogue is helpful at every scale.  When you are working in a complex system, dialogue ensures that you are getting dissent, contrary views and outlying ideas into the process.  Complex problems cannot be addressed well with a top-down roll out of a change initiative or highly controlled implementations of a single person’s brilliant idea.  If at any point people are working on any stage of this alone, you are in danger territory and you need another pair of eyes on it at the very least.
  2. Evaluation is your friend and your enemy. At every stage you need to be making meaning and evaluating what is going on, but it is critically important to use the right evaluation tools.  Developmental evaluation tools – with their emphasis on collective sense making, rapid feedback loops and visible organizational and personal learning – are critical in any complexity project, and they are essential in the first three stages of this process.  As you move to more and more stable projects, you can use more traditional summative evaluation methods, but you must always be careful not to manage to towards targets.  Such an error results in data like “We had a 62% participation rate in our diversity training” which tells you nothing about how you changed things, but can shift the project focus to trying to acheive a 75% participation rate next cycle.  This is an especially pervasive metric in engagement processes. And so you must…
  3. Monitor, monitor, monitor. Intervening in a complex system always means acting without the certainty that what you are doing is helpful.  You need data and you need it on a short term and regular basis.  This can be accomplished by formal and informal ongoing conversations and story captures about what is happening in the system (are we hearing more stories like the ones we want?) or through a SenseMaker(tm) monitoring project that allows employees to end their data with a little data capture.
  4. These practices are nested, not linear. An always to remember that this is not a five step process to intervening in a complex system.  In a large organization, you can expect all of these things to be going on all the time.  Building the capacity for that is a kind of holy grail and would constitute a 21st century version of the Learning Organization in my books.

Share:

  • Click to share on Mastodon (Opens in new window) Mastodon
  • Click to share on Bluesky (Opens in new window) Bluesky
  • Click to share on LinkedIn (Opens in new window) LinkedIn
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • More
  • Click to share on Reddit (Opens in new window) Reddit
  • Click to share on Tumblr (Opens in new window) Tumblr
  • Click to share on Pinterest (Opens in new window) Pinterest
  • Click to share on Pocket (Opens in new window) Pocket
  • Click to share on Telegram (Opens in new window) Telegram

Like this:

Like Loading...

Probes, Prototypes and Pilot projects

October 5, 2015 By Chris Corrigan Complexity, Evaluation, Featured One Comment

I’ve been working in the world of program development with a lot of complexity and innovation and co-creation lately and have seen these three terms used sometimes interchangeably to describe a strategic move. As a result, I’ve been adopting a more disciplined approach to these three kinds of activities.

First some definitions.

Taken explicitly from Cynefin, a probe is an activity that teaches you about the context that you are working with. The actual outcome of the probe doesn’t matter much because the point is to create an intervention of some kind and see how your context responds. You learn about the context and that helps you make better bets as you move forward – “more stories like this, less stories like this” to quote Dave Snowden. Probes are small, safe to fail and easily observed. They help to test different and conflicting hypotheses about the context. If 8 out of 10 of your probes are not failing, you aren’t learning much about the limits of your context. Probes are actually methods of developmental evaluation.

A prototype is an activity that is designed to give you an idea of how a concept might work in reality. Prototypes are designs that are implemented for a short time, adjusted through a few iterations and improved upon. The purpose of a prototype is to put something into play and look at its performance. You need to have some success with a prototype in order to know what parts of it are worth building upon. Prototypes straddle the world of “safe to fail” and fail safe. They are both developmental evaluations tools and they also require some level of summative evaluation in order to be fully understood. Prototypes are also probes, and you can learn a lot about the system from how they work.

A pilot is a project designed to prove the worthiness of an approach or a solution. You need it to have an actual positive effect in its outcomes, and it’s less safe to fail. Pilots are often designed to achieve success, which is a good approach if you have studied the context with a set of probes and maybe prototyped an approach or two. Without good intelligence about the context you are working with, pilots are often shown to work by manipulating the results. A pilot project will run for a discrete amount of time and will then be summatively evaluated in order to determine its efficacy. If it shows promise, it may be repeated, although there is always a danger of creating a “best practice” that does not translate across different contexts. If a pilot project is done well and works, it should be integrated with the basic operating procedure of an organization, and tinkered with over time, until it starts showing signs of weakened effectiveness. From then on, it can become a program. And pilots are alos probes, and as you work with them they too will tell you a lot about what is possible in the system.

The distinctions between these three things are quite important. Often change is championed in the non-profit word with the funding of pilot projects, the design of which is based on hunches and guesses about what works, or worse, a set of social science research data that is merely one of many possible hypotheses, privileged only by the intensity of effort that went into the study. We see this all the time with needs assessments, gap analyses and SWOT-type environmental scans.

Rather than thinking of these as gradients on a line though, I have been thinking of them as a nested set of circles:

PPPsEach one contains elements of the one within it. Developing one will be better if have based your development on the levels below it. When you are confronted with complexity and several different ideas of how to move forward, run a set of probes to explore those ideas. When you have an informed hunch, start prototyping to see what you can learn about interventions. What you learn from those can be put to use as pilots to eventually become standard programs.

By far, the most important mindshift in this whole area is adopting the right thinking about probes. Because pilot projects and even prototyping is common in the social development world, we tend to rely on these methods as ways of innovating. And we tend to design them from an outcomes basis, looking to game the results towards positive outcomes. I have seen very few pilot projects “fail” even if they have not been renewed or funded. Working with probes turns this approach inside out. We seek to explore failure so we can learn about the tolerances and the landscape of the system we are working in. We “probe” around these fail points to see what we can learn about the context of our work. When we learn something positive we design things to take advantage of this moment. We deliberately do things to test hypotheses and, if you’re really good and you are in a safe-to-fail position, you can even try to create failures to see how they work. That way you can identify weak signals of failure and notice them when you see them so that when you come to design prototypes and pilots, you “know when to hold ‘em and know when to fold ‘em.”

Share:

  • Click to share on Mastodon (Opens in new window) Mastodon
  • Click to share on Bluesky (Opens in new window) Bluesky
  • Click to share on LinkedIn (Opens in new window) LinkedIn
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
  • More
  • Click to share on Reddit (Opens in new window) Reddit
  • Click to share on Tumblr (Opens in new window) Tumblr
  • Click to share on Pinterest (Opens in new window) Pinterest
  • Click to share on Pocket (Opens in new window) Pocket
  • Click to share on Telegram (Opens in new window) Telegram

Like this:

Like Loading...

1 … 7 8 9 10 11

Find Interesting Things
Events
  • Art of Hosting November 12-14, 2025, with Caitlin Frost, Kelly Poirier and Kris Archie Vancouver, Canada
  • The Art of Hosting and Reimagining Education, October 16-19, Elgin Ontario Canada, with Jenn Williams, Cédric Jamet and Troy Maracle
Resources
  • A list of books in my library
  • Facilitation Resources
  • Open Space Resources
  • Planning an Open Space Technology meeting
SIGN UP

Enter your email address to subscribe to this blog and receive notifications of new posts by email.
  

Find Interesting Things

© 2015 Chris Corrigan. All rights reserved. | Site by Square Wave Studio

%d