Online Edition - January 2000
How Is It
We spend a lot of effort in meetings, conferences and training sessions designed to initiate or sustain changes.
--This is the final of three columns intended to shift our thinking about what is essential in the conduct of these gatherings. Most of our traditional planning is about the actions of the presenters, what we want to present and how we want to present it.
--We think power is the point. We seem to ignore that it is people's active experience in the room that will affect their emotional response to change, and ultimately the quality of action.
of experience that we undervalue are
--Our business here is evaluation. Nowhere is Mechanical Age more deeply honored than in our love of evaluation, and this is not an argument against it. The argument is about how we think about evaluation. What we evaluate and when we evaluate it. As an example, every conference I have ever attended has an evaluation form and they are all alike. They each imply a relationship between how well the material was presented and the value received by participants.
--One set of questions is about the presenter; were they dynamic, well organized, did they present what was promised, did they know their stuff, how good were the audio-visuals, were the handouts useful?
--The second set of questions is about value received. What did you learn, was it useful, what will you do differently tomorrow? These measures are based on the belief that cause is in the front of the room and effect is in the audience. If the front of the room is clear and on target, the meeting will be a success.
--It is our elitist, leadership compulsion reinforced by an entertainment and spectator culture that leads us in this direction. If we seek accountable employee action, we need to move the spotlight from the stage and into the audience.
--O'Henry begins a book of his short stories with the statement: "Not very long ago some one invented the assertion that there were only 'Four Hundred' people in New York City who were really worth noticing. But a wiser man has arisen-the census taker-and his larger estimate of human interest has been preferred in marking out the field of these little stories of the 'Four Million'."
--He understood that the character of a city was better defined by the story of its citizens than simply the story of its leaders. Same with our meetings, training events, conferences and change efforts; the character of the event is better defined by the story of the participants than the excellence of the presenter.
Why It Matters
High Accountability Evaluation
--As a start, we might design our evaluation around the following questions:
-- -What was your purpose in attending this event. What are you doing here? Even if you were sent, you decided to come and brought expectations that were solely yours? In asking this, we start with the belief that each of us is creating our own experience, even if we did not ask to be born.
-- -How well are you achieving what you came for? Raises the question of whose conference, or change process is this. Are we active in seeking our outcomes or are we waiting for room service?
-- -Where in the room did you place yourself? Where we sit is a physical fact and an expression of the initial nature of our engagement, commitment, willingness to be present. We each rationalize our seating by ease of entry, need to make a call, not wanting to be called on, that was the only open seat. All rationalizations, so why not use the evaluation form to push back on our passivity.
-- -When during the day, have you been bored or disappointed and how are you dealing with this? Confronts the question of energy and relevance. Whose job is to maintain interest, be relevant. When energy drains, where does it go. Why is it the leader's job to keep the power plant running. This does not let the leader go free, but it reduces them to one player, not the only player.
-- -What change on your part, in this event, might give you different results? The idea that we are constantly co-creating our world is implied in this question as in the others. This takes the questions we ask ourselves as leaders and trainers and places them equally on the shoulders of the audience.
-- The particular questions are not really the point, they will change with the purpose of the event. What is key is that evaluation can be a means of real time reflection, done for the purpose of creating accountable communities and used to carry the message that self-evaluation is at the essence of change and learning.
Now is the Hour
-- Well, in fact there won't be a next time. Even if we hold another event, the same people will not be in the room, the purpose will be changed, the reality of the world will have shifted somewhat. Evaluating an event at the end has us care only about the past, and misses the opportunity to confront how we are doing while there is still time to do something about it.
-- To complete an evaluation as we leave the room also makes it too easy for people to express their frustration as a parting shot, avoiding any engagement or responsibility. We also abandon the leader/trainer when we evaluate at the end. In my own case, I usually get participant reactions after the event. Some thought I exaggerated, that my humor and cynicism was distracting, and that I consistently finished other people's sentences for them.
--Well, where were these people when I needed them? When did I lose them? Why did they believe the situation was unsalvageable. There is truth in what they said, but now the opportunity to respond is forever lost. If evaluation is intended to improve our performance and not just judge it (big if!), it has to occur in the middle of things. Kathie Dannemiller, goddess of large group methodology, understands this and takes a "pulse" reading at key points along the way, and then makes it public information. Smart strategy.
--We shouldn't wait until the end of our life to evaluate it, same with our meetings, trainings, and conferences. Place the evaluation early, in the middle, and often.
--One more point: Evaluation is not about ratings, it is about learning. It should be a conversation among participants. Get it in writing first, but then make it the beginning of a conversation, make it public, invite people to own their response to the event. It gives everyone information they can act on, engages the people who are drifting away and thus reinforces the belief that we can shape our destiny not just observe and remember it.
--And, by the way, Happy New Year.