ASQ - Team and Workplace Excellence Forum

Online Edition - January 2000
---

---Issue Highlight
---Peter Block on Meetings & Evaluations

--
"Evaluation is not about ratings, it is about
learning. It should be a conversation among
participants."

In This Issue...
Ken Blanchard
----Interview

Year 2000 Recruitment
----Tactics

Teamwork at NASA
Team Effectiveness
----In Health Care

The Downside of
----Upsizing


Features...
Peter Block Column
Views for a Change
Pageturners
Briefcases

 

 

How Is It Going?
by Peter Block

We spend a lot of effort in meetings, conferences and training sessions designed to initiate or sustain changes.

--This is the final of three columns intended to shift our thinking about what is essential in the conduct of these gatherings. Most of our traditional planning is about the actions of the presenters, what we want to present and how we want to present it.

--We think power is the point. We seem to ignore that it is people's active experience in the room that will affect their emotional response to change, and ultimately the quality of action.

--Three elements of experience that we undervalue are
-- (1) the questions we ask of participants,
-- (2) the design and structure of the room and
-- (3) the evaluation of the event.

--Our business here is evaluation. Nowhere is Mechanical Age more deeply honored than in our love of evaluation, and this is not an argument against it. The argument is about how we think about evaluation. What we evaluate and when we evaluate it. As an example, every conference I have ever attended has an evaluation form and they are all alike. They each imply a relationship between how well the material was presented and the value received by participants.

--One set of questions is about the presenter; were they dynamic, well organized, did they present what was promised, did they know their stuff, how good were the audio-visuals, were the handouts useful?

--The second set of questions is about value received. What did you learn, was it useful, what will you do differently tomorrow? These measures are based on the belief that cause is in the front of the room and effect is in the audience. If the front of the room is clear and on target, the meeting will be a success.

--It is our elitist, leadership compulsion reinforced by an entertainment and spectator culture that leads us in this direction. If we seek accountable employee action, we need to move the spotlight from the stage and into the audience.

--O'Henry begins a book of his short stories with the statement: "Not very long ago some one invented the assertion that there were only 'Four Hundred' people in New York City who were really worth noticing. But a wiser man has arisen-the census taker-and his larger estimate of human interest has been preferred in marking out the field of these little stories of the 'Four Million'."

--He understood that the character of a city was better defined by the story of its citizens than simply the story of its leaders. Same with our meetings, training events, conferences and change efforts; the character of the event is better defined by the story of the participants than the excellence of the presenter.

Why It Matters
How we evaluate has a power that we need to appreciate:
-- 1. Evaluation is not a benign event. It carries as clear a message of our intentions as anything else we do. It says more by the way questions are composed than by the answers it provides.
-- 2. The message the evaluation carries takes a stance on who is responsible for learning and outcomes, even for culture. Is it the leader/trainer, or is it the citizen/participant? If a meeting or training event does not go well the evaluation questions imply that the leaders may have to do it differently next time. Our questions rarely indicate that next time the participants will have to change the nature of their participation.
-- 3. Evaluation also takes a stance on how engineerable or predictable the world can be. It most often re-enforces the illusion that change is predictable and knowable in advance, and the ones who know are the leaders and trainers. If we find out that the event was characterized by surprise, chaos, or confusion, we consider that a failure of leadership.
-- Maybe surprise and confusion are the essential ingredients to learning and change. Maybe if everything goes as planned, the meeting failed.
-- 4. Evaluation is not a neutral, "objective" inquiry. It is an active element in learning and changes what it touches.
-- Tim Gallwey, author of the Inner Game series of books, commented recently that we are always ready to evaluate the effects of training, but we never measure the effects of the evaluation.

High Accountability Evaluation
Here are some ways to treat evaluation as powerful, invasive, means to confront participant accountability.

--As a start, we might design our evaluation around the following questions:

-- -What was your purpose in attending this event. What are you doing here? Even if you were sent, you decided to come and brought expectations that were solely yours? In asking this, we start with the belief that each of us is creating our own experience, even if we did not ask to be born.

-- -How well are you achieving what you came for? Raises the question of whose conference, or change process is this. Are we active in seeking our outcomes or are we waiting for room service?

-- -Where in the room did you place yourself? Where we sit is a physical fact and an expression of the initial nature of our engagement, commitment, willingness to be present. We each rationalize our seating by ease of entry, need to make a call, not wanting to be called on, that was the only open seat. All rationalizations, so why not use the evaluation form to push back on our passivity.

-- -When during the day, have you been bored or disappointed and how are you dealing with this? Confronts the question of energy and relevance. Whose job is to maintain interest, be relevant. When energy drains, where does it go. Why is it the leader's job to keep the power plant running. This does not let the leader go free, but it reduces them to one player, not the only player.

-- -What change on your part, in this event, might give you different results? The idea that we are constantly co-creating our world is implied in this question as in the others. This takes the questions we ask ourselves as leaders and trainers and places them equally on the shoulders of the audience.

-- The particular questions are not really the point, they will change with the purpose of the event. What is key is that evaluation can be a means of real time reflection, done for the purpose of creating accountable communities and used to carry the message that self-evaluation is at the essence of change and learning.

Now is the Hour
In addition to what we evaluate, the timing of our evaluation also is critical. We typically evaluate an event after it is over, filling out the forms on the way out of the room. Or we send in evaluations a month after the event. Implied is that the feedback will help us do better next time.

-- Well, in fact there won't be a next time. Even if we hold another event, the same people will not be in the room, the purpose will be changed, the reality of the world will have shifted somewhat. Evaluating an event at the end has us care only about the past, and misses the opportunity to confront how we are doing while there is still time to do something about it.

-- To complete an evaluation as we leave the room also makes it too easy for people to express their frustration as a parting shot, avoiding any engagement or responsibility. We also abandon the leader/trainer when we evaluate at the end. In my own case, I usually get participant reactions after the event. Some thought I exaggerated, that my humor and cynicism was distracting, and that I consistently finished other people's sentences for them.

--Well, where were these people when I needed them? When did I lose them? Why did they believe the situation was unsalvageable. There is truth in what they said, but now the opportunity to respond is forever lost. If evaluation is intended to improve our performance and not just judge it (big if!), it has to occur in the middle of things. Kathie Dannemiller, goddess of large group methodology, understands this and takes a "pulse" reading at key points along the way, and then makes it public information. Smart strategy.

--We shouldn't wait until the end of our life to evaluate it, same with our meetings, trainings, and conferences. Place the evaluation early, in the middle, and often.

--One more point: Evaluation is not about ratings, it is about learning. It should be a conversation among participants. Get it in writing first, but then make it the beginning of a conversation, make it public, invite people to own their response to the event. It gives everyone information they can act on, engages the people who are drifting away and thus reinforces the belief that we can shape our destiny not just observe and remember it.

--And, by the way, Happy New Year.

Return to January 2000 News for a Change Homepage

 
  • Print this page
  • Save this page

Average Rating

Rating

Out of 0 Ratings
Rate this item

View comments
Add comments
Comments FAQ

ASQ News