conducting questionnaires

Conducting Questionnaires | Jonathan Clifton | November 12th, 2000

Hi to everybody,

I am coming to the end of my MAP assignment and have just done a classroom questionnaire for my students. In the past I have always researched adult classes and have always received lots of useful feedback either informally or as a response to qualitative questionnaire. This time I was working with a group of young adults. I used a quantitative questionnaire e.g. how do you rate these activities /materials on a scale of one to five and a space at the bottom of the questionnaire for another comments. The questionnaire came back with various ratings but the space for any comments was very disappointing- only 4 comments from 10 questionnaires. One of which said "you are very nice" and another just said, "OK". I suppose it was a mistake not guiding the students more. Also, after the lesson I asked for any comments and again drew a blank. Has anybody else "out there" had similar experiences of students either being unwilling or unable to discuss the teaching/learning situation, especially young adults who are maybe unused to the teacher actually asking for their opinion????

Best wishes to all

Jonathan Clifton

Conducting Questionnaires | Pinkie | November 13th, 2000

Hi Jonathan:

Thanks for the lead on authentic texts. As regards your interesting point about questionnaires, I certainly experience similar problems: you give a questionnaire asking what students think and most of them say "OK", which may be because that's what they think or may just be because they're polite, and in any case getting useful criticism out of them is like getting water out of a stone.

So you try to structure things a bit more (for example, you give examples of the sort of thing you're looking for (e.g. "More vocab activities, please"), and sure enough 95% of students reply "More vocab activities, please".

I wonder whether one useful approach would be to get them thinking about things a few days before they actually complete the questionnaire, so they have time to develop their ideas? Has anyone out there looked into this sort of thing in depth?

Pinkie

Spain

Conducting Questionnaires | Dominic Marini | November 14th, 2000

Hi Pinkie,

(Caveat: I've just jumped into the list after an absence of a few weeks so I may be repeating something already said).

There is a lot of research in polling research (I took political polling and stats in my former student existence) that predicts exactly what happened to you. When you give poll respondents a model answer they will tend to copy it in their response, because respondents may feel that that type of answer is somehow "better". So, if you are going to give example answers it is suggested that you give a balanced set of contradictory examples, to communicate to respondents that a wide range of opinions are acceptable.

For example, "Some students say they want more vocabulary work but other students say vocabulary work is not interesting. What is your opinion on vocabulary work?"

Alternatively, if you are concerned that students are being merely polite in their answers, and you want them to critique your teaching, you could signal to them that you welcome critical responses. You could give them some example answers at the beginning of your survey to skew (on purpose!) students' responses towards criticism. For example, "Some students think the teacher does not know how to teach vocabulary. What do you think? What should the teacher do to improve?"

On the other hand, I've moved towards ongoing student evaluation of the course and my teaching, in the hope that this would open a space where students could give critical opinions about the course. The students I've done this with are intermediate-advanced. Each week I give them an open-ended question about the course and ask them to write (anonymously) whatever they think. Sometimes I ask more focused questions.

Then, I respond to the class with my own letter, reviewing the comments students gave me and me giving them my response. Students have responded favorably to this two way evaluation process.

Since the process is ongoing during the class I can change aspects of the course that students complain about and even implement student suggestions. As a matter of fact, if many students make the same criticism and suggestion for improvement I am (morally) bound to implement their suggestion. Otherwise I risk undermining the process.

This creates a "virtuous circle" where student feedback and suggestions becomes implemented in the class that in turn motivates students to express themselves more. Eventually though students become tired of writing weekly comments, or have less to say, and these comments become a trickle and eventually peter out.

For my MET I've tried videotaping a class discussion (after small group discussion) on one problem: why students don't ask questions. The next week the class watched the video of themselves and we used that as a springboard (again, after small group discussion) for further class discussion. I haven't finished transcribing that class discussion yet but it seems that the students' turns were longer during the second class discussion and they made more concrete suggestions for improving the class.

It takes a while, but continuously asking students for their "help" to improve the course gets through to them.

Good luck,

Dominic Marini
Japan

Conducting Questionnaires | James Hobbs | November 14th, 2000

Dear All,

I've done many questionnaires with university classes and have often had the same experience as Jonathan, ie no comments or useless "very nice" - type comments. Through a process of trial and error I've developed my own 'rules' for giving questionnaires, as follows.

1 Make questionnaires a regular thing in class. Have a few "practice runs" before you throw in the really important questions. I find that the feedback gets progressively better as the students become more accustomed to doing questionnaires.

2 Teacher feedback - show the students that their opinions count. When I recently did a survey in the first session with a new group, I found some strong opinions about evaluation - ongoing in-class evaluation, or end-of-term speech to the whole class? There were some interesting opinions on both sides, so I made "how to decide grades" the theme for the whole of the next lesson. We finished up with a paper vote, which went narrowly in favour of ongoing evaluation (fortunately nobody requested a recount or complained about the design of the ballot paper), but the whole class was clearly energized by the realization that I really was open to suggestions for change. If the only questionnaire comes at the end of a course, maybe students don't realize that their comments really matter?

3 Give the option of answering in the students' native language. I find that if I insist on English most students interpret the activity as a test and look for "correct" answers. Moreover, comments tend to seem like a reflection of what students CAN say, rather than of what they really WANT to say.

4 Anonymous questionnaires. However many reassurances you give, I suspect most students still won't really believe that their grade isn't going to be affected by saying "This class sucks". But if it's anonymous they tend to be a bit more adventurous (but nobody has told me that my class sucks ........ not yet, anyway).

Perhaps a lot depends on local context (I'm in Japan). What 'rules' have people in other countries come up with?

James Hobbs

Conducting Questionnaires | Andy Boon | November 14th, 2000

This survey is very long but if anyone is interested in the topic and would like to complete it I would be very grateful.

TEACHER SURVEY

EFFECTIVE PROCEDURAL INSTRUCTIONS:

Can you help me?

I am currently researching an aspect of my teaching for the Masters Course in Teaching English to Speakers of Other Languages. I am examining the use of procedural instructions in the classroom. Procedural instructions are instructions given by the teacher to inform students of what to do at each stage of the lesson. I would be extremely grateful if you could complete the survey. This will help me complete my research and to find more effective methods of giving procedural instructions in the lessons.

PLEASE BE AS HONEST AS POSSIBLE WHEN ANSWERING THE QUESTIONS!

Thank you for your co-operation and time in completing the survey.

1. How do you usually initiate Procedural instructions in your lessons?

(e.g. Verbal - Written - Modeling - Others)

GIVE EXAMPLES:

2. To what extent do you explain, as opposed to demonstrate what you want students to do? (If you explain, generally how long are your explanations?).

3. What do you find to be the most effective way to give Procedural instructions in your classes? Why?

GIVE REASONS:

4. Are there instances of procedural instruction that fail to set up a particular task or create confusion in your class?

YES/NO

If YES - How does it fail?

Why do you think it fails?

GIVE EXAMPLES:

5. Are tasks always done as you expected or are there instances of tasks being done differently than you expected by students?

If differently How? Why?

GIVE EXAMPLES:

6. To what extent (if at all) do you use Japanese in Procedural instructions?

7. How simple/complicated is your use of the target language in relation to the students' linguistic ability? (Consider a) sentence length b) idiom usage c) vocabulary d) speed of speech e) other factors).

8. Do you repeat or modify Procedural instructions? YES/NO.

9. Do you have instances of student silence after you have given Procedural instructions? YES/NO.

If YES: How do you distinguish between student thinking time and genuine confusion by students of what they are supposed to do?

10. How long does it usually take for students to begin a task after you have given Procedural instructions?

11. Do you use any means to check that students have understood the Procedural instructions? YES/NO.

IF YES- GIVE EXAMPLES:

12. Do students ask clarification questions if they are unsure about the Procedural instructions? YES/NO.

13. Do you usually mark the change in your lessons from the end of one task to the next Procedural instruction?- (For example: Okay, we are going to move on to .....) YES/NO.

IF YES, HOW? GIVE EXAMPLES:

14. Think back to a lesson you have taught recently. How did you give the Procedural instructions?

15. You are organizing a role-play in your lesson. You want your students to demonstrate the use of household appliances using sequence markers (first, next, then, after that, finally).

Student A will ask the question How do I use (this microwave oven)?

Student B will demonstrate how to use the appliance using sequence markers.

How would you give the Procedural Instructions to set this task up in your class?

Please answer on a separate piece of paper and GIVE REASONS for your chosen procedure.

THANK YOU

Andy

Conducting Questionnaires | Andy Boon | November 20th, 2000

I'd just like to say thanks to all teachers who completed my survey. Your comments were very useful. Thank you for taking the time to reply.

Andy Boon

 

Archive Categories