This morning we began our Harvesting and Collective Sensemaking online course. Rowan Simonsen, Amy Lenzo and I were really excited to be able to share our first little insights with people, and especially this new mnemonic that we created to capture five key principles of harvesting practice: PLUME. We are excited to introduce this into the world.
Just about to leave Montreal this morning for Toronto and north to Thornbury, Ontario to visit family. I was here for the conference of the Canadian Evaluation Society, where I participated on a panel on innovative dialogue methods (and yes I noted the irony in my remarks) and later led a World Cafe where I presented some of the sense-making processes I’ve been working on. I was here on the recommendation of Junita Brown who has been in some good conversations with evaluators around the use of the World Cafe for evaluation purposes. Originally Amy Lenzo and I were scheduled to host a cafe here that was much more ambitious: a plenary cafe with the participants to explore the learning field of the conference. Through various machinations that was cut back to a panel presentation and a very small world cafe at the end of the day with 16 people. The conference was one of those highly scripted and tightly controlled affairs that I hardly ever go to.
The session before us was a case competition where student teams were responding to a mock RFP from Canada World Youth to evaluate an Aboriginal Youth leadership Program. Not a single team had an Aboriginal person on it, and every single presentation was basically the same: full of fundamental flaws about what constitutes success (“Did the youth return to their communities”) or what constitutes a cultural lens (“We are using a medicine wheel to understand various parts of the program). One group of fresh faced non-Aboriginal students even had the temerity to suggest that they were applying a decolonizing strategy. Their major exposure to indigenous communities was through a single book on decolonizing methodology and some internet searches about medicine wheels. It was shocking actually, because these were the students that made the finals of this competition. They looked like fresh versions of the kinds of evaluation firms that show up in First Nations certain they know what’s going on.
To make matters worse, the case competition organizer had a time mix up with the conference planner meaning that our panel started 30 minutes late which gave me very little time to present. As I as doing a a cafe directly afterwards I ceded most of my time to my panel colleagues Christine Loignon, Karoline Truchon who did a very interesting presentation on their use of PhotoVoice. It was clear to me at the conference that the practitioners among us had a better grasp of complexity theory, power and non-linear sense-making than any of the professional evaluators I met.
I presented most of the work that I have been documenting here over the last few months, and later led a small group through a cafe where we engaged in the creation of a sensemaking framework and used a pen and paper signification framework.
By far the better experience for me was hanging out with friends and colleagues. On the first night I arrived I had dinner and drinks with my friends from Percolab: Paul Messer, Samatha Slade and Elizabeth Hunt. We ate fish and chips, drank beer and whisky and caught up. On Sunday I met Jon Husband for lunch on the grass at McGill with his delightful godson and then joined the Percolab folks for a visit to the new co-operative ECTO co-working space on Mount Royal in the Plateau, followed by a barbeque with family and friends.
And Last night, after my presentations a great evening with Juan Carlos Londono and Lisa Gravel. We had dinner at Lola Rosa and spent hours going over the new French translation of the GroupWorks Pattern Language Deck. This was a brilliant time. I learned a bunch of new French words and most fun of all we discussed deeper etymology, nuance and the limitations and benefits of our respective languages in trying to convey some of the more esoteric practices of hosting groups. The new deck has some beautiful reframing and some names for patterns that need some work. But it’s exciting to see this translation and I always love diving into the language.
I really do like Montreal a lot and in the past number of years come to love it more as I have lost my inhibition about speaking French. the more French I speak, the more French I learn and the more the heart of the city opens up. Many English Canadians have the idea that Montreal is a cold hearted city to English speakers, but I find that isn’t true at all. Just offer what you can in French and people open up. And if you’re lucky enough to sit down with lovers of words like the friends I have, your learning explodes.
Off for a couple of days to visit family and then home to Bowen Island for a series of small local facilitation gigs, all of which will tell me something deeper about my home place.
I was back at St. Aidan’s United Church in Victoria yesterday, hosting another conversation in their continued evolution into their next shape. Last December we worked together to explore four possible scenarios that were being proposed for the congregation. In the past few months they have been working on implementing one of these scenarios – the one which featured a plan to develop a Spiritual Learning Centre. Yesterday was a short strategic conversation called to explore the shape of what that Centre could be and how it will change life at the church.
Two weeks ago in our Leadership 2020 program I experimented with using a signification framework to harvest a World Cafe. We are beginning another cohort this week and so I had a chance to further refine the process and gather much more information.
We began the evening the same way, with a World Cafe aimed at exploring the shared context for the work that these folks are in. Our cohort is made up of about 2/3rds staff from community social services agencies and 1/3 staff from the Ministry of Children and Family Development. This time I used prepared post it notes for the sense making exercise, which you can see here:
Our process went like this:
- At Cafe tables of five for 20 minutes, discuss the question “What is a story of the future you are anticipating for this sector?”
- Second round, new tables, same question, 20 minutes
- About ten minutes of hearing some random insights from the group, and checking to see how those resonate.
- 2 minutes of silent reflection on the question of ‘What do you need to learn here that will help us all move forward?”
- Each participants took a pink and blue post it note. On the blue post it they wrote what they needed to learn that would be immediately applicable and on the red ones, learning that is needed to prepare for the future.
- Participants filled out the post-its and then were instructed on how to signify the data on a triangle framework that helped them signify whether what they needed to learn would help them “in their personal life,” “do their jobs” and/or “make change.”
- Participants also indicated on the post-its whether the worked for the Ministry or worked for a community organization.
At the conclusion of the exercise we had a tremendous amount of information to draw from. Our immediate use was to take a small group and use affinity grouping to identify the themes that the whole has around their learning and curiosity. We have used these themes to structure a collective story harvest exercise this morning.
But there is some much more richness that can come from this model. Here are some of the ways people are playing with the date:
- Removing all the pink post-its to see what the immediate learning needs are and vice versa.
- Looking at and comparing the learning needs between the two sectors to see where the overlaps and differences are
- examining the clusters at the extremes to see what ot tells us about personal needs, and professional needs.
- Uncovering a theory of change by looking at the post its clustered around the “Making change” point and also seeing if these theories of change are different between the community and the government.
And of course because the data has been signified on each post it, we can recreate the framework easily. The next level for me will be using this data to create a Cynefin framework using the four-points contextualization exercise. Probably won’t happen in this cohort.
Big learning is the rich amount of data that proceeds from collecting finely-grained objects, allowing for disintermediated sense-making, and seeing all these multiple ways in which signified data can be used to address complex challenges obliquely, which allows you to get out of the pattern entrainment that blinds you to the weak signals and emergent patterns that are needed to develop emergent practice. This pen and paper version is powerful on its own. You can imagine how working with SenseMaker across multiple signification frameworks can produce patterns and results that are many magnitudes richer.
Hmmm…maybe I should have let the group do this…
When I popped off to London last week to take a deep dive into Cognitive Edge’s work with complexity, one of the questions I held was about working with evaluation in the complex domain.
The context for this question stems from a couple of realities. First, evaluation of social programs, social innovation and other interventions in the human services is a huge industry and it holds great sway. And it is dominated by a world view of linear rationalism that says that we can learn something by determining whether or not you achieved the goals that you set out to achieve. Second, evaluation is an incredibly privileged part of many projects and initiatives and itself becomes a strange attractor for project planning and funding approval. In order for funders to show others that their funding is making a difference, they need a “merit and worth” evaluation of their funds. The only way to do that is to gauge progress against expected results. And no non-profit in its right mind will say “we failed to achieve the goals we set out to address” even though everyone knows that “creating safe communities” for example is an aspiration out of the control of any social institution and is subject to global economic trends as much as it is subject to discrete interventions undertaken by specific projects. The fact that folks working in human services are working in a complex domain means that we can all engage in a conspiracy of false causality in order to keep the money flowing (an observation Van Jones inspired in me a while ago.) Lots of folks are making change, because they know intuitively how to do this, but they way we learn about that change is so tied to an inappropriate knowledge system, that I’m not convinced we have much of an idea what works and what doesn’t. And I’m not talking about articulating “best practices.”
The evaluation methods that are used are great in the complicated domain, where causes and effects are easy to determine and where understanding critical pathways to solutions can have a positive influence on process. in other words, where you have replicable results, linear, summative evaluation works great. Where you have a system that is complex, where there are many dynamics working at many different scales to produce the problems you are facing, an entirely different way of knowing is needed. As Dave Snowden says, there is an intimate connection between ontology, epistemology and phenomenology. In plain terms, the kind of system we are in is connected to the ways of knowing about it and the ways of interpreting that knowledge.
I’m going to make this overly simplistic: If you are working with a machine, or a mechanistic process, that unfolds along a linear trajectory, than mechanistic knowledge (problems solving) and interpretive stratgies are fantastic. For complex systems, we need knowledge that is produced FROM the system and interpreted within the system. Evaluation that is done by people “outside” of the system and that reports finding filtered through “expert” or “disinterested” lenses is not useful for a system to understand itself.
Going into the Cynefin course I was interested to learn about how developmental evaluation fit into the complex domain. What I learned was the term “disintermediated sensemaking” which is actually the radical shift I was looking for. Here is an example of what it looks like in leadership practice.
Most evaluation uses processes employing a specialized evaluator undertaking the work. The problem with this is that it places a person between the data and experience and the use of the knowledge. And it also increases the time between an experience and the meaning making of that experience, which can be a fatal lag with strategy in emergent systems. The answer to this problem is to let people in the system have direct experience of the data, and make sense of it themselves.
There are many many ways to do this, depending on what you are doing. For example:
- When clustering ideas, have the group do it. When only a few people come forward, let them start and then break them up and let others continue. Avoid premature convergence.
- When people are creating data, let them tag what it means, for example, in the decision making process we used last weekend, participants tagged their thoughts with numbers, and tagged their numbers with thoughts, which meant that they ordered their own data.
- Produce knowledge at a scale you can do something about. A system needs to be able to produce knowledge at a scale that is usable, and only the system can determine this scale. I see many strategic plans for organizations that state things like “In order to create safe communities for children we must create a system of safe and nurturing foster homes.” The job of creating safe foster homes falls into the scope of the plan, but tying that to any bigger dynamics gets us into the problem of trying to focus our work on making an impact we have no ability to influence.
- Be really clear about the data you want people to produce and have a strategy for how they will make sense of it. World Cafe processes for example, often produce scads of data on table cloths at the centre of the table, but there is often so little context for this information that it is hard to make use of. My practice these days is to invite people to use the table cloths as scratch pads, and to collect important data on post it notes or forms that the group can work with. AND to do that in a way that allows people to be tagging and coding the data themselves, so that we don’t have to have someone else figure out what they meant.
- Have leaders and teams pour over the raw data and the signification frameworks that people have used and translate it into strategy.
These just begin to scratch the surface of this inquiry in practice. Over the next little while I’m going to be giving this approach a lot of thought and try it out in practice as often as I can, and where the context warrants it.
If you would like to try an exercise to see why this matters try this. the next time you are facilitating a brainstorm session, have the group record dozens of insights on post its and place them randomly on a wall. Take a break and look over the post its. Without touching the post its, start categorizing them and record your categorization scheme. Then invite the group to have a go at it. Make sure everyone gets a chance to participate. Compare your two categorization schemes and discuss the differences. Discuss what might happen if the group were to follow the strategy implicit in your scheme vs. the strategy implicit in their scheme.