Materials Trialing
Materials Trialing | Richard Goldie | October 1st, 2004
Hi,
I've been involved at various stages of the process, from data collection, syllabus design, materials writing, proofing, layout and page setting, and, recently, trialing. Oh, and from time to time, I am drafted back to the classroom to plug a gap. The work is often frustrating: with such a large and rigid organization as the Air Force, there are so many factors (tests, resources, styles of teaching) outside our control. Despite this, much of it is enjoyable and I count myself lucky to be part of such a large-scale materials writing set-up.
I've written a brief account of a recent trial we have done: if anyone is interested in reading it, I'll post it. Also, if has been involved in trialing (or materials writing), I'd be interested in hearing from you!
Cheers,
Richard Goldie
Re: Richard's recent trailing | Jerry Talandis Jr. | October 3rd, 2004
Hi Richard,
Nice to have you on board!
I'm not currently doing MAP, but I would be interested in reading your account on this discussion list. I think if you posted it, we could all read and comment about it.
Jerry Talandis Jr.
Toyama, Japan
Re: trialing | Richard Goldie | October 4th, 2004
Hi,
Thanks for the welcome. Hope I didn't step on Nuna's toes in my first contribution!
Jerry suggested I give an account of a recent trial I have just done of new materials produced in my place of work: so here it is. Feel free to ignore / respond as you wish. Perhaps what I'll do today is just give an outline of what happened and leave discussion of the broader issues till later.
We have just finished trialing the first module of a course for aircraft technicians. It is targeted at a range of specializations and so must appeal to all, while not being specialized to any one (rather difficult to achieve). It is mainly a reading skills course, with language focus on vocab and grammar that a needs analysis found to be relevant to all specialties. It was written by a team of writers (including myself) in one location but will be taught around Saudi Arabia in other bases. It therefore has to appeal to a wide range of "customers".
Trialing can obviously throw light on a whole range of issues in the material; the overriding issue in ours was (perceived) "level of difficulty". Our immediate bosses thought that the course was FAR too easy (indeed, "shamefully" easy!) and that it did not reflect the level that technicians of that category (should) possess. Initial feedback from other bases, however, indicated that the course was FAR too demanding and should not be implemented! The issue is obviously contentious. The trial, which lasted two weeks and which was difficult to set up, comprised a colleague teaching most of it, me observing key lessons and teaching some too.
As to the nub question of level of difficulty, our preliminary findings have been interesting. The course has, in fact, assumes too great a level of general "air force" knowledge. It also assumes a bit too much linguistic knowledge. But the students were, surprisingly, willing and able to tackle reading skills work with texts quite a bit beyond their level - because the tasks were achievable. Also, the choice of vocab and grammar was not in question - what they needed more of was vocabulary preparation before reading a text and consolidation afterwards (we sometimes gave them far too little).
I guess that the general issues I am interested in with regards to trialing are:
· What is "level of difficulty", how can it be measured? Obviously we need to distinguish between the difficulty of the text (linguistic / conceptual / structural), the task (depth of comprehension required) and language (are some words "more difficult to learn" than others?
· how to collect data / info on a trial
· the difference between teacher feedback, based on what they read (predictive) and feedback based on actually having taught it (retrospective)
· to what extent the teacher trialing the material stamps his/her own style on the material (e.g. in order to teach a certain text, I felt it necessary to deal with some vocab first (eliciting and pre-teaching); others would not have done this.
· Different impressions two observers have of exactly the same event!
If anyone would like to pick up on any of these, great!
Richard
Re: Trialing | Mike McDonald | October 4th, 2004
Richard,
I'm afraid I have no experience of the kind of trialing you describe. However, I can recommend Hutchinson and Waters, "English for Specific Purposes", especially chapters 7-12, for a discussion of the various considerations in designing and evaluating ESP courses. Sorry if this is old news to you.
Mike McDonald
with 
and 
