A couple of good blog posts in my feed this morning that provoked some thinking. These quotes reminded me how much evaluation and planning is directed towards goals, targets and patterns that cause us to look for data that supports what we want to see rather than learning what the data is telling us about what’s really going on. These helped me to reflect on a conversation I had with a client yesterday, where we designed a process for dealing with this.
Coming back from Campbell River tonight. I was working with a group of Churches who are currently trying to understand their future. The United Church of Canada is in a period of massive restructuring due to all kinds of causes. We are very clear, retrospectively, what these causes are…everything from demographics shifts to the overall decline of Christendom. Most folks I have been working with over the past few years actually welcome these dynamics and realities, even though it means that the Church has plunged into a period of deep uncertainty. For people that are both spiritual AND religious in the liberal tradition of the United Church, this is actually a good thing and an opportunity for practice. In short, an uncertain world is where a spiritual practice comes into it’s own.
And of course from a complexity perspective, this is where the tools of complexity thinking comes into play as well, although that is largely a harder sell. In the United Church, leadership has emerged over several generations firmly in the managerial model. In fact some Churches I have been in don’t even start their board meetings with a prayer, claiming that “this is business.” Which is shocking, actually.
Ironically complexity thinking tools are perhaps in short supply but the scriptural texts contain a fantastic set of heuristics (experienced based principles) with which to understand and live with complexity and change. The New Testament, for example contains letters from various apostles to the new Churches scattered throughout the Levant, Turkey, Greece and Rome. These letters contain guideline after guideline for living together in community true to the message of Jesus, which was essentially that love is the number one heuristic. Practice that and your community thrives. Forget that and things founder. More stories lie that, less like this.
At any rate, despite it being right there, in plain view, I find myself over and over having to reintroduce and reinforce the need to think differently when confronted with the complexity of what is happening to the Church. And make no mistake, this is a dress rehearsal for the inevitable collapse of many social institutions that we take for granted, so doing this work has been illuminating. I have been trying to simplify the Cynefin framework’s distinctions between ordered and unordered systems and, inspired a bit by Dave Snowden’s recent post, have started teaching from a list that invites new strategies for planning. In it I contrast complexity strategy with strategy used to solve technical problems and knowable situations. In Cynefin terms, this is complex vs. complicated.
Here are a few of the contrasts that I talked about today, complexity first and its correlation second:
The basic difference between complex problems and complicated problems comes down to whether a problem is solvable or not. Is there a stable outcome? Is there an end state? Can research and expertise provide us with answers? Is the situation predictable? Answer yes to these questions and you have a complicated problem. Answer no and you have a complex one. It comes down to the difference between building a community and building a building.
- Complex problems aren’t solvable; complicated ones are.
- Address complexity by sense patterns and weak signals and amplifying them; solve complicated problems by analysing data and problem solving.
- In complexity, pay attention to what works and ask why?; for complicated problems, keep your eyes on the prize and study gaps (ask why not?)
- Be informed in your strategy by stories, myths and parables that translate across many contexts; for complicated problems, adopt “best” practices and rule based solutions.
- Employ collaborative leadership to address complexity; employ experts to solve complicated problems.
- In complexity, truth is found in stories; for complicated situations, truth is found in facts.
- Complex planning requires anticipatory awareness, meaning that you have to constantly scan for meaning through the system; a vision won;t help you. In complicated situations a vision is useful and the end state can be achieved with logical, well planned steps.
- In complexity, the future is already here, but it is quiet and hidden in the noise of the culture. in complicated systems the future is not here and it is well understood what it will take to get there from here.
- In complex systems, the solutions will come at you obliquely, out of the blue and in surprising ways, so you need to cultivate processes that allow that to happen. In complicated systems, problems are tackled head on from a position of knowing as much as you can about how to proceed and then choosing the best course of action.
The point of this list is to make a crude distinction in order to have people understand that they need new strategy tools to address the situation they are in. In the Church, leaders have had to confront a situation of such fraught complexity in many generations, and so the leadership that has brought them to this point no longer answers all the questions. This can be a profoundly traumatic experience for people who are used to being able to understand what is going on and influence the situation. So there is lots in this work, and a gentle, clear and fierce introduction to complexity thinking is really needed now. That’s what I’m after here.
Regular readers will know that I’ve been thinking a lot about evaluation for many years now. I am not an evaluator, but almost every project I am involved in contains some element of evaluation. Sometimes this evaluation is well done, well thought through and effective and other times (the worst of times, more often than you think) the well thought through evaluation plan crumbles in the face of the HIPPO – the Highest Paid Person’s Opinion. So how do we really know what is going on?
When I stumbled across Michael Quinn Patton’s work in Developmental Evaluation, a whole bunch of new doors opened up to me. I was able to see the crude boundaries of traditional evaluation methods very clearly and was able to see that most of the work I do in the world – facilitating strategic conversations – was actually a core practice of developmental evaluation. Crudely put, traditional “merit and worth” evaluation methods work well when you have a knowable and ordered system where the actual execution can be evaluated against a set of ideal causes that lead to an ideal state. Did we build the bridge? Does it work according to the specifications of the project? Was it a good use of money? All of that can be evaluated summatively.
In the unordered systems where complexity and emergence is at play, summative evaluation cannot work at all. The problem with complex systems is that you cannot know what set of actions will lead to the result you need to get to, so evaluating efforts against an ideal state is impossible. Well, it’s POSSIBLE, but what happens is that the evaluator brings her judgements to the situation. Complex problems (or more precisely, emergent problems generated from complex systems) cannot be solved, per se. While it is possible to build a bridge, it is not possible to create a violence free society. Violent societies are emergent.
So that’s the back story. Last December I went to London to do a deep dive into how the Cynefin framework and Cognitive Edge’s work in general can inform a more sophisticated practice of developmental evaluation. After a few months of thinking about it and being in conversation with several Cognitive Edge practitioners including Ray MacNeil in Nova Scotia, I think that my problem is that that term “evaluation” can’t actually make the jump to understanding action in complex systems. Ray and I agreed that Quinn Patton’s work on Developmental Evaluation is a great departure point to inviting people to leave behind what they usually think of as evaluation and to enter into the capacities that are needed in complexity. These capacities include addressing problems obliquely rather than head on, making small safe to fail experiments, undertaking action to better understand the system rather than to effect a change, practicing true adaptive leadership which means practicing anticipatory awareness and not predictive planning, working with patterns and sense-making as you go rather than rules and accountabilities, and so on.
Last night a little twitter exchange between myself, Viv McWaters and Dave Snowden based on Dave’s recent post compelled me to explore this a bit further. What grabbed me was especially this line: “The minute we evaluate, assess, judge, interpret or whatever we start to reduce what we scan. The more we can hold open a description the more we scan, the more possibility of seeing novel solutions or interesting features.”
What is needed in this practice is monitoring. You need to monitor the system in all kinds of different ways and monitor yourself, because in a complex system you are part of it. Monitoring is a fine art, and requires us to pay attention to story, patterns, finely grained events and simple numbers that are used to measure things rather than to be targets. Monitoring temperatures helps us to understand climate change, but we don’t use temperatures as targets. Nor should we equate large scale climate change with fine grained indicators like temperature.
Action in complex systems is a never ending art of responding to the changing context. This requires us to be adopting more sophisticated monitoring tools and using individual and distributed cognition to make enough sense of things to move, all the while watching what happens when you do move. It is possible to understand retrospectively what you have done, and that is fine as long as you don’t confuse what you learn by doing that with the urge to turn it into a strategic plan going forward.
What role can “evaluation” have when your learning about the past cannot be applied to the future?
For technical problems in ordered systems, evaluation is of course important and correct. Expert judgement is required to build safe bridges, to fix broken water mains, to do the books, audit banks and get food to those who need it. But in complex systems – economies, families, communities and democracies, I’m beginning to think that we need to stop using the word evaluation and really start adopting new language like monitoring and sense-making.
Two weeks ago in our Leadership 2020 program I experimented with using a signification framework to harvest a World Cafe. We are beginning another cohort this week and so I had a chance to further refine the process and gather much more information.
We began the evening the same way, with a World Cafe aimed at exploring the shared context for the work that these folks are in. Our cohort is made up of about 2/3rds staff from community social services agencies and 1/3 staff from the Ministry of Children and Family Development. This time I used prepared post it notes for the sense making exercise, which you can see here:
Our process went like this:
- At Cafe tables of five for 20 minutes, discuss the question “What is a story of the future you are anticipating for this sector?”
- Second round, new tables, same question, 20 minutes
- About ten minutes of hearing some random insights from the group, and checking to see how those resonate.
- 2 minutes of silent reflection on the question of ‘What do you need to learn here that will help us all move forward?”
- Each participants took a pink and blue post it note. On the blue post it they wrote what they needed to learn that would be immediately applicable and on the red ones, learning that is needed to prepare for the future.
- Participants filled out the post-its and then were instructed on how to signify the data on a triangle framework that helped them signify whether what they needed to learn would help them “in their personal life,” “do their jobs” and/or “make change.”
- Participants also indicated on the post-its whether the worked for the Ministry or worked for a community organization.
At the conclusion of the exercise we had a tremendous amount of information to draw from. Our immediate use was to take a small group and use affinity grouping to identify the themes that the whole has around their learning and curiosity. We have used these themes to structure a collective story harvest exercise this morning.
But there is some much more richness that can come from this model. Here are some of the ways people are playing with the date:
- Removing all the pink post-its to see what the immediate learning needs are and vice versa.
- Looking at and comparing the learning needs between the two sectors to see where the overlaps and differences are
- examining the clusters at the extremes to see what ot tells us about personal needs, and professional needs.
- Uncovering a theory of change by looking at the post its clustered around the “Making change” point and also seeing if these theories of change are different between the community and the government.
And of course because the data has been signified on each post it, we can recreate the framework easily. The next level for me will be using this data to create a Cynefin framework using the four-points contextualization exercise. Probably won’t happen in this cohort.
Big learning is the rich amount of data that proceeds from collecting finely-grained objects, allowing for disintermediated sense-making, and seeing all these multiple ways in which signified data can be used to address complex challenges obliquely, which allows you to get out of the pattern entrainment that blinds you to the weak signals and emergent patterns that are needed to develop emergent practice. This pen and paper version is powerful on its own. You can imagine how working with SenseMaker across multiple signification frameworks can produce patterns and results that are many magnitudes richer.
How’s this for a conference centre?
Last week, we hosted a group of 35 emerging and legacy leaders in the human services sector on Bowen Island to kick off our sixth Leadership 2020 cohort. Hosting the group on Bowen Island is a powerful way to begin and end this ten month program, and there is tremendous value offered by hosting it on Bowen Island.
We are a small island with a working village and we have evolved an inventive way of hosting gatherings. We call it “Village as a Venue” a name coined by my friend Tim Merry to describe the way he hosts gatherings in Mahone Bay, Nova Scotia. This is a way to reimagine the local economy of small villages who can compete in unorthodox ways with larger venues in nearby cities for conference and meeting business.
On Bowen Island, our village as a venue model starts with one of the retreat centres on island We use the Bowen Island Lodge mostly for our work (and sometimes we host at Rivendell and Xenia as well). The Lodge is ideal because it is set up to host groups (as opposed to acting like a hotel), it is right on the water, and is only a five minute walk from the ferry dock and the village, meaning that people can actually arrive using public transit from anywhere in Vancouver. It is located in a neighbourhood so we keep a careful eye on our noise levels at night, but if people want to socialize in a rowdy way, there are pubs nearby. The Lodge is also perfect in that it is not a high end retreat facility, and it provides an incredibly affordable and accessible venue to accommodate and host people. It has shared rooms and shared washrooms, but the beds are comfortable and when we are there we have the whole space to work in. Overflow registrants are housed at the Lodge at the Old Dorm and other local B&Bs.
The Bowen Island Lodge is a dry rental, meaning that they don’t have their own catering staff. This means that we get to hire local friends to provide us with food. Usually we have our events catered by The Snug which is a little cafe that has always punched above it’s weight in terms of quality. Over the years, both The Snug and the Sam Trethewy, the manager at the lodge have come to appreciate to people we bring to Bowen, who are often social workers and others on the front lines of human services. They treat them well, with good food and sensitive hosting which makes for a superior experience for people.
Spreading the joy further, we always schedule a night out at Rustique, where our friend Thierry Morbach cooks us up a rural French feast. We book the whole restaurant for this, and it becomes a raucous and memorable dinner. On other nights we will head up to the pub for drinks (this past week a group of 15 or so invaded on a Tuesday night, which is no small boost to Glen’s business on a January night). On the Thursday night we usually have a celebration at the Lodge which necessitates folks walking up to the Beer and Wine Store for supplies.
During the day, we give people a couple of hours at lunch to be hosted on the island. Many folks end up going to the village to walk around, buy chocolate and meet folks. They get to see our village for what it is, a friendly working commercial centre. It is not set up to attract tourist dollars, and my friend Edward Wachtman and his partner Sheree Johnson has just completed a study that shows that tourists are looking for something other than that tourist experiences that are sold in many other small towns on the coast. What they find on Bowen is authentic community. They notice the way we look after each other, the way people talk and discuss issues. They often head out for early morning walks or runs on the nearby trails and stop in at The Snug and get to see a community as it is. I hear story after story of these encounters and we often talk about the friendliness of the village and what it says about leadership and community. What happens on Bowen becomes a living teaching for how it is possible to live and work together, and visitors SEE that.
And finally, we use the island itself to host. Bowen is a beautiful place and to get there you need to cross three miles of water. this is an almost archetypal journey, and it marks a thresh hold to a different experience. When you arrive you are received in Snug Cove, and when you leave again, it is as if you are birthed back out into the world. While on the island, we often take people out on the land, to experience the serene calm of the place and to spend time in reflection about their lives. There are so few places in the modern world, especially in the social services sector, where people can just slow down and reflect and pause, surrounded by forest and water and ravens and deer. It becomes transformative, which is the point. Edward’s survey revealed that this is a primary reason why people come to Bowen Island.
We are in a loose conversation with friends in Mahone Bay and in Ballyvaughn, Co. Clare in Ireland about this concept. In Ballyvaughn a group called The Burren Call has set up to host gatherings at the Burren College of Art and on the land around it as well. This pattern is repeating and it takes these places of beauty and transformative potential and leverages what we already have to provide experiences for vistors that also benefit us locals, both financially (and especially in the off-season) as well as psychologically. There is nothing nquite like having your place seen through the eyes of visitors and reflected back.
For Bowen that reflection is that we have a special place, a beautiful natural setting, a friendly and welcoming community and an authentic working village. Locals are always curious about what our visitors are up to and Piers at The Snug or Paul Ricketts at the Beer and Wine Store are always curious and, its fair to say, appreciative of the folks who are “in that workshop with Chris and Caitlin.”
Village as a Venue holds a lot of promise for villages like ours. Having run more than 30 events on Bowen like this, I think we have hit a stride in bringing people over for 3, 4 and 5 days. It is the unique and quirky local character of our community and the beauty of the land and seas that makes this possible. These are strong assets and contribute to the visitor experience of renewal, restoration and serenity.