measuring outcomes – finding solutions

10May12

In regards to measuring the outcomes, I have been worried about how time-consuming it was to get the women to respond to 30 multiple choice questions. One of my initial ideas was to have someone read off each of the questions, and then have the women check off the box (ie. strongly agree – strongly disagree which would be symbolized by smiley faces) themselves. Therefore, it wouldn’t take any reading or writing.

I know researchers often have trouble administering these types of questionnaires to particular groups, and I could clearly see why. I thought the questions were straightforward and easy to understand, and given the responses of the initial group when answering orally, the problem didn’t seem to be that the questions were too difficult or confusing. I thought I had by-passed the issue of literacy by having someone read out the questions to them, but as I saw the women glance at each other uncertainly, I knew I had mis-read the situation. During the deliberative dialogues, although they often seemed unsure of the format and hesitated before speaking up, when they finally decided to participate, they spoke with a lot of clarity and confidence. When they were asked to fill out the questionnaires, it was the first time I saw doubt and discomfort in their demeanor. In the end, they were just looking at each other’s sheets to see what the next person had checked off in order to figure out what the right answer was. I almost started laughing because the answers were obviously useless. Everyone had written down the same thing. We ended up using volunteers to sit down with each of the participants to fill out the sheets individually, but that method was simply not practical or sustainable.

However, an interesting thing has happened during this long, labor-intensive process. As the women have been able to have their questions fully answered – “Why do we need to check off these boxes, why don’t you just ask us?” “You have asked us if we would recommend these dialogues to our friends, but all the dialogues are filled up so no one else can join. So how could we recommend it?”  –  I’ve realized these kind of questionnaires are completely foreign to them, and it’s something they’ve never had to do. For example, they signed their names on the front of the sheet – I imagine for them, anything they have to sign is usually a signal of something serious, like a legal contract or government-related, so they were not sure what they were getting themselves into. But as the volunteers have patiently sat down with each of them, explained the questionnaire, explained how to answer in a multiple choice format, and answered all of their questions, we came to realize that the first time is the hardest. After they understand the context, they will be able to fill out the forms for themselves in the future. Once they sat down with the volunteers and were told what each of the multiple choice answers actually meant, almost all of them said, “That isn’t what I think, I want to change my answer.” The next time, they won’t need anyone to explain the purpose of the questionnaires or to help them to fill it out.

It was the entire concept, the idea and the format of the questionnaires, that was unfamiliar to them, not simply the literacy aspect of it. When they were able to ask all their questions fully, from what the 5-point scale meant  to how the questionnaires would be used to improve the program, our biggest problem solved itself. Although further clarification of the  scaling system may still be necessary and adjustments will still have to be made, the women have a much better understanding of the entire questionnaire process and we should be able to move forward without needing volunteers for each individual participant.

Advertisements


No Responses Yet to “measuring outcomes – finding solutions”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: