We were working with a local government client last week in a meeting that had a very contentious subject matter focused on the return of land and uses of that land, to First Nations owners. There was an important conversation as a part of this work that involved removing a structure that had some historical significance to the community but was seen as a mark of an oppressive history by the First Nations owners who could not contemplate it remaining on their land. It is a wickedly complicated issue right at the heart of what reconciliation really means: returning land, transferring ownership and working with history.
Our client did an incredible job of preparing multiple stakeholders to participate in this discussion, by meeting each group personally and hearing their thoughts on the situation. All the stakeholders, twenty in total, agreed to come to a two hour dialogue to discuss the issues at hand. Our client put together a beautiful 8 page booklet with much of the technical information in it about proposals and process and sharing some of the things they had heard in the pre-meetings. The format of the day included a presentation from the First Nations about what they were proposing and why, with most of the meeting involving a World Cafe for dialogue.
It went well. We received a couple of really powerful pieces of positive feedback.
These kinds of conversations are the sharp edge of the reconciliation wedge. It is one thing to conduct a brief territorial acknowledgement at the beginning of a meeting or event, it is entirely another for people to sit down and discuss the issues around the return of land.
In debriefing with our client this week, she made the following observations about what contributed to the usefulness of the container for this conversation:
- Very small groups – no more than four at a table – meant that there was no need for people to “take their best shot” as they would have in a larger plenary format. Groups smaller than five reduce the performative nature of conversation and allow dialogue to fully unfold. This enabled people who needed to invest a lot of emotional energy and attention in their speaking and listening, to operate in a more relaxed way.
- The questions for the dialogue were very broad. Sometimes the most powerful question is “what are you thinking and feeling about what you just heard?” This question kicked off 45 minutes of intense learning, listening and story telling at the tables.
- The invitation process is everything. We helped our client design an invitation process but she took the lead in going to each group separately and talking to them about why they were needed in the conversation.
- There were no observers. Everyone in the room was at a table except for me and our graphic recorder. Everyone at a table had a question they needed answered or a curiosity about the outcome. there was no certainty in the room, no positionality, and yet, each person spoke about their own experience and their own perspective and listened carefully to what others said. Also, everyone in the room had to stretch their perspectives to participate. This was not comfortable for anyone, because this work isn’t comfortable for anyone. It is literally unsettling.
- The First Nations leadership pulled no punches in explaining their reasons for their proposal and why it was important that the structure be removed from their lands. This can be a very tricky thing because while it is important for non-indigenous stakeholders to hear First Nations perspectives, there is a tremendous amount of emotional labour involved in talking about traumatizing history. We had one of our own team prepared to talk about the history and emotional legacy of the structure. She had interviewed people from her community and was well positioned to share the rationale but on the day we didn’t need to her to tell the story as the leadership were willing to tell that story themselves. Enabling this to happen well is important.
Reconciliation is nothing without the return of the lands or the influence over the lands which we acknowledge as “unceded territory.” What stops people from going much further than territorial acknowledgement is the fear of being unsettled in the conversation. But we can’t do this work without holding containers that allow for people to be unsettled. Only that way to we share perspectives and find possibilities and to do so in small, deep conversations where stories can be shared, perspectives understood and . Or sometimes not. But the path to reconciliation requires us to try, and these few notes and observations might help in that.
Where I live, on a small island off the west coast of Canada, the traditional Celtic season markers make more sense for our community rhythms and the cycles of our landscape than the solar seasonal calendar, and I’m not as versed in the Skwxwu7mesh seasons well enough to relate to those.
Today is Lughnasa, the traditional commencement of the harvest season. The province of British Columbia is burning in many places, and today the winds have brought us smoke from the interior to colour the sun pink in a grey and orange sky.
The produce in our local markets is showing tremendous diversity, as the brassicas and squashes and fruits that were planted in the spring join the early harvest of greens and peas. On the land and sea, salmon are returning, the deer have dropped their fawns, and already there are signs up for shares in pigs and turkey’s and sheep for the winter.
It’s also a time of harvest for me from a year that has seen much in the way of professional and personal growth. I am moving from a deep study of theory to a deeper informed approach to practice, wanting now to focus my professional craft on simplicity while beginning to think about how to share everything I’ve been learning over the past 8 years or so for the benefit of other practitioners, especially those who are starting out. I am also looking deeply into my own life and where I am on this journey that has delivered 49 years of living and still confounds me.
There are some new learning offerings being planned for this year, including a session on using complexity for social change that I’m doing with Bronagh Gallagher here in Vancouver and over in Glasgow. I’m also preparing an online course with my friends at Beehive Productions on Chaordic Design. Add to that two Art of Hosting workshops in November: our 14th annual offering here on Bowen Island and one in Amsterdam with old friends. These are all harvests for me.
We are embarking on a innovative approach to a social problem and we need a framework to guide the evaluation process. As it is a complex challenge, we’re beginning with a developmental evaluation framework. To begin creating that,I was at work for most of the morning putting together a meta-framework, consisting of questions our core team needs to answer. In Art of Hosting terms, we might call this a harvesting plan.
For me, when working in the space of developmental evaluation, Michael Quinn Patton is the guy whose work guides mine. This morning I used his eight principles to fashion some questions and conversation invitations for our core team. The eight principles are:
- Developmental purpose
- Evaluation rigor
- Utilization focus
- Innovation niche
- Complexity perspective
- Systems thinking
- Co-creation
- Timely feedback
The first four of these are critical and the second four are kind of corollaries to the first and the first two are essential.
I think in the Art of Hosting and Art of Harvesting communities we get the first principle quite well, that participatory initiatives are, by their nature, developmental. They evolve and change and engage emergence. What I don’t see a lot of however is good rigour around the harvesting and evaluation.
All conversations produce data. Hosts and harvesters make decisions and choices about the kind of data to take away from hosted conversations. Worse, we sometimes DON’T make those decisions and then we end up with a mess, and nothing useful or reliable as a result of our work.
I was remembering a poorly facilitated session I once saw where the facilitator asked for brainstormed approaches to a problem. He wrote them in a list on a flip chart. When there were no more ideas, he started at the top and asked people to develop a plan for each one.
The problems with this approach are obvious. Not al ideas are equal, not all are practical. “Solve homlessness” is not on the same scale as “provide clothing bundles.” No one would seriously believe that this is an effective way to make a plan or address an issue.
You have to ask why things matter. When you are collecting data, why are you collecting that data and how are you collecting it? What is it being used for? Is it a reliable data source? What is your theoretical basis for choosing to work with this data versus other kinds of data?
I find that we do not do that enough in the art of hosting community. Harvesting is given very little thought other than “what am I going to do with all these flipcharts?” at which point it is too late. Evaluation (and harvesting) rigour is a design consideration. If you are not rigourous in your data collection and your harvesting methods, others can quite rightly challenge your conclusions. If you cannot show that the data you have collected is coherent with a strategic approach to the problem you are addressing, you shouldn’t be surprised if your initiative sputters.
In my meta-framework the simple questions I am using are:
- What are our data collection methods?
- What is the theoretical basis and coherence for them?
That is enough to begin the conversation. Answering these has a major impact on what we are hosting.
I high recommend Quinn Patton et. al.’s book Developmental Evaluation Exemplars for a grounded set of principles and some cases. Get rigourous.
My friend Avril Orloff shared this beautiful quote on her facebook page.
“Nobody tells this to people who are beginners. I wish someone told me. All of us who do creative work, we get into it because we have good taste. But there is this gap. For the first couple of years you make stuff, it’s just not that good. It’s trying to be good, it has potential, but it’s not. But your taste, the thing that got you into the game, is still killer. And your taste is why your work disappoints you.
A lot of people never get past this phase – they quit. Most people I know who do interesting, creative work went through years of this. We know our work doesn’t have this special thing that we want it to have. We all go through this. And if you are just starting out or you are still in this phase, you gotta know it’s normal and the most important thing you can do is do a lot of work…
It is only by going through a volume of work that you will close that gap, and your work will be as good as your ambitions. And I took longer to figure out how to do this than anyone I’ve ever met. It’s gonna take awhile. It’s normal to take awhile. You’ve just gotta fight your way through.”
That is from Ira Glass.
Evaluation is such an influential constraint in organizational and community life. When resources and attention are tied to evaluation results, a kind of tautology gets set up. One begins managing projects towards the evaluation outcomes, in order to give the best chance of an initiative surviving and continuing to attract resources. One of the things I appreciate about developmental evaluation is its deliberate engagement with emergence. Making sense of emergence however can be a really time consuming affair, and so I’m thinking about how we can use good use of time to use dialogue and collective meaning making to help make sense of data and direction.
Developmental evaluation is for the complex domain. That means that we are not working with evaluating actions against desired end states, but instead noticing and paying attention to vectors and directions – intentions and hypotheses that help shape emerging strategy. Developmental evaluation is the process of gathering information about our work to give us some intelligence about what we are doing.
Think of the information needs of two different kinds of athletes. A golf player relies on solid objective data (how many yards to the hole, where the wind is coming from, the nature of the lie of the ball and so on) and interprets that data through her own self-knowledge (I hit a five iron 160 yards. Adjusting for wind and lie and the target topography, I should hit a 4 iron with backspin…) Of course the better a golfer one is, the easier it is to execute a plan and understand exactly where one succeeded or failed.
By contrast soccer players work in a dynamic environment. The information available to them only becomes apparent as they begin to play the match. They may know something about the other team, but they learn rapidly in the first ten minutes or so how the game is going to go. A team will discover where the opposition’s weakness is, or what its attacking strategy is, or where the open spots are on the pitch. Making good use of this information requires excellent communication in real time to share what is being learned. It requires players to play with potentials and patterns rather than certainties. Every move provides yet more information. The better a team works together, the faster they can adjust their strategy to take advantage of potentials.
When we are evaluating work there is a mix of these two types of approaches at play. Summative evaluation will look at the gap between expected outcomes and what actually happened and suggest how to adjust for next time. Budget planning and auditing is a good example of this technical kind of results based evaluation. Count the money and compare against projections. Look for causes. Some of these causes will be technical and some will be down to culture.
Developmental evaluation requires a different strategic approach, and simply put, it might fall into these four things (I’m trying for simplicity here, to try to be able to describe this in an easy way):
- Data points that give us the ability to capture information about a current state of an evolving system. This can render a series of pictures that will allow us to see patterns and trends. You need multiple snapshots over time to make sense of what is happening. One photo of a soccer game in progress tells you nothing. You need to monitor indicators not manage end points. Soccer is much more than just putting the ball in the net, even though that is the desired end result.
- Feedback loops from data to human sensemaking so that data can be used in real time to develop strategy and adjustments to the directionality of work.
- A facilitated sensemaking process to bring together multiple perspectives to interpret what is happening. In a complex system the data won’t give you answers. It will provide information to form hypotheses about the patterns that are emerging, and that information can give you guidance for action.
- A way of acting that doesn’t over commit resources to emerging potential strategies, but which gives enough momentum to see if we can shift things in a desired way. Snowden calls this “safe-to-fail.” This is tricky and calls for good context dependant leadership, but it is the essence of good decision making.
There are all kinds of ways of implementing these strategies. You can use surveys to discover what people are accessing on your website and you can use interviews or sensemaking tools to find out HOW they are using that information. You can use a strategic group to interpret these results and see how they are either coherent with our intentions, or at odds with them. You can then create new initiatives that support what is emerging or figure out ways to abandon what is not working. There are thousands of dialogue methods and processes to use to ask questions about and develop action around the data that is emerging.
Importantly, developmental evaluation needs to be a part of the way you work strategically. It needs a rhythm and a cadence to it, so that you know you are coming back on a regular basis to the emerging picture of what is happening. You need outsiders occasionally to come in and disrupt your point of view and offer alternative views of the patterns, and you need to choose a longer rhythm to continue to develop and refine your evaluation strategy as a whole.
I want this to be simple as a process to use. Strategy without information is just a wild guess. But if we tie our decisions too closely to the data emerging from dynamic systems we can get equally stuck making decisions that try to game the system towards desired results, with sometimes disastrous results for clients, customers and ultimately, organizational integrity. It’s a balance and a practice. How can we make this easy?