Data & action research

Data | Robert Haines | May 20th, 2004

At a recent conference I heard two interesting statements about data that I think apply to action research.

The first came from a university professor who was presenting at the conference when she commented on a paper she'd read. The paper impressed her because it had "a lot of data".

The second comment came from Henry Widdowson during his participation in a colloquium on Applied Linguistics. Widdowson remarked, "Data is not experience".

What are the implications for action research and other forms of scientific inquiry when it comes to better understanding our learners and ourselves as teachers?

Rob

Re: Data | Jerry Talandis Jr. | May 20th, 2004

Hello Robert,

I'll take a stab at your question.

I don't know the context in which the quotes were made, but the first one   ("a lot of data") gives me the impression that the professor felt that "more is more." In other words, she may have felt that a large quantity of accumulated data was a good thing, adding to the value of the overall work. Quantity = quality. If this is an accurate reading of her meaning (which I have no idea if it is), then I'd have to disagree. More is not always "more." You can have a little data and do a lot with it, as I've been reading in the IIC module. As far as implications for AR, I'd think the message here is that you don't have to go killing yourself to accumulate tons and tons of data in order to do your subject justice.

"Data is not experience" tells me that no matter how much you try to "measure" life, in the end, you just can't capture all of it. For example, from my house I can see the Tateyama mountain range clearly (a main reason why we chose our location). No matter how often I look at those magnificent peaks, I never get tired of them. Of course I've tried taking their picture on numerous occasions, trying in vain to capture their majestic quality. I've never yet managed to do so. While I can never tire of looking at   the mountains directly, I easily tire of looking at my flat, lifeless, two-dimensional pictures of them. I sense there is an element of my direct experience with life that I'll never be able to capture and preserve. So it goes with AR- as we go about measuring, recording, transcribing, interviewing, we need to keep in mind that what we are trying to capture (life experience) can never be fully captured by any method we now use. This should not keep us from trying, but instead keep us humble as we go about searching for our truths. Remembering, "data is not experience" can keep our egos in check as we go about our work. I think that's a good thing!

Any other reactions to these quotes?

Jerry
Japan

Re: Data | Jake Kimball | May 20th, 2004

Like Jerry, I'd also like to hear more about the context of the talks.    But I'll take a stab at it.

"She was impressed by the paper because it had "a lot of data".

My take on it is that by "lots of data" means the writer included statistical analysis or even triangulated data.    Regardless, the result is that the amount and variety of data provide validity and reliability.

"Data is not experience." 

There's substitute for rolling up one's sleeves and getting one's hands dirty.    I'm thinking of students who passively sit in L1 lectures to learn languages.    I imagine there are also many researchers who do studies but no longer teach.

One reason ar isn't held as high as other research methods is that it is hard to reproduce.    ARs answers to questions are for practicing classroom teachers who need unique solutions to unique problems.    The solutions may not lead to what researchers desire, namely validity and reliability across classrooms or contexts. 

Regarding research, I think that research is subject to fads.   Has anyone else noticed that quantitative studies have been becoming far and few between in TESOL Quarterly.    The past year or two has been mostly qualitative, with only one or two articles in total even including a chart or more of statistical data.

Jake

Tateyama Mountains | Steve Mann | May 20th, 2004

Hi all

I can't see the Tateyama mountain range at all from my house in Staffordshire. But, in investigating the importance in peoples' lives of mountain views (or views of water for that matter), I might use your snapshots (even if they are flat, lifeless, two-dimensional pictures) in conjunction with your comments to construct such an account.

Data represented in an account or description is not direct experience but a version of experience. Pieces of data create dialogue and new experience.

Yesterday I went for a walk with 10 Chinese students. I thought an alternative to B'ham city centre was a good idea. We walked to grass meadows in flatlands by the River Trent. They spent most of their time taking pictures in pairs and groups. The ran screaming from dragonflies and kept asking me if I was sure there were no snakes in the long grass.

A couple of them asked me if there was anything wrong because they thought I was 'quiet' and 'emotional'. One asked me if I was okay. But,  I think my experience of a May Day in the sunshine in England was very different for their experience. We all experienced the same views in different ways.

My account of an 'experience' yesterday is in many ways a partial one. It might be used as data in a number of ways. It is my perspective. I may be wrong about their experience. It is data but the extent to which it is a full record of experience is certainly open to debate.

Today some of the Chinese students have promised to send some of these digital photographs. Will they help me recall details of my experience yesterday? Or will I construct new versions of my experience?

Jerry, you send me a picture of the Tateyama mountains and I'll send you a picture of 10 students on a May day in England by the River Trent.

Best

Steve

Re: Tateyama Mountains | Robert Haines | May 20th, 2004

This may be one of the most coherent and reasonable messages I've ever received from a tutor.

Thank you, Steve.

Rob

Re: Data | Martin Lovatt |   May 20th, 2004

Hi Rob,

It's a trick question, right? I mean, asking people to comment on quotes without any contextual background is like taking a dab of green paint from a painting and asking someone who has never seen the picture what green depicts in it. (Although, while we're on the topic of paintings, I should say that my MET assignment is beginning to look more like an abstract than a landscape...)

Anyway, I thank you for the test, Rob.

And who said Steve could play? I thought this list was for us bottom-dwellers? Tateyama mountains... and the Trent river. I'm from Staffordshire, a quaint little spot called Smoke-on-Stench (aka Stoke-on-Trent) and I don't recommend going out of your way to see the Trent river... but I like the sound of the Tateyama mountains.

Keep 'em coming, Rob.

Best regards to all,

Martin

Fukuoka, Japan (And I can see 'aburayama' from here which means 'greasy mountain'... I think I might paint it green.)

Continuous assesment | Tisa | May 22nd, 2004

Better leave it here.   I'd love to contribute more -that thread on data etc was cool - but I just can't keep up with you Pinkies.

Take care,

Tisa, who despite having a magnificent view on the Alps and the Jura from her apartment in Lausanne, Switzerland, still misses the Pacific Ocean and Coastal Mountains of her native Vancouver, Canada!)

"Data" discussion question | Jerry Talandis Jr. |   May 23rd, 2004

Hello. Regarding the recent discussion about "data", I have a question, out of curiosity:

Robert, when you made your original post, you asked us to comment on a couple of quotes that you had heard at a conference:

The first came from a university professor who was presenting at the conference when she commented on a paper she'd read. She was impressed by the paper because it had "a lot of data".

The second comment came from Henry Widdowson during his participation in a colloquium on Applied Linguistics. Widdowson remarked, "Data is not experience".

But, you presented these quotes without giving much context in which they were given, as three of us picked up in our responses:

Jerry: I don't know the context in which the quotes were made, but the first...

Jake: Like Jerry, I'd also like to hear more about the context of the talks. But I'll take a stab at it.

Martin: It's a trick question, right? I mean, asking people to comment on quotes without any contextual background is like taking a dab of green paint from a painting and asking someone who has never seen the picture what green depicts in it... Anyway, I thank you for the test, Rob.

So, I was wondering: was the lack of context intentional or not? Like I said, I'm just curious because the resulting discussion turned up some interesting perspectives. Take a look at mine, for example:

("a lot of data") gives me the impression that the professor felt that "more is more." In other words, she may have felt that a large quantity of accumulated data was a good thing, adding to the value of the overall work. Quantity = quality.

I immediately felt that the researcher in question didn't have a clue- she thought that just because there was a lot of accumulated data, then the research was automatically good. I think this reflects my attitude towards quantitative research- I get turned off or intimidated when I see lots of numbers and formulas, so I responded with a sort of negative reaction to the quote. I felt like the researcher had "lost touch" with reality in a way, getting lost in volume and numbers. Compare this with Jake's response:

"She was impressed by the paper because it had "a lot of data".

My take on it is that by "lots of data" means the writer included statistical analysis or even triangulated data.     Regardless, the result is that the amount and variety of data provide validity and reliablility.

To my eyes, Jake had a completely different reaction. He seemed comfortable with quantitative research and automatically assumed that the positive reaction to the paper was because the writer had done a good job utilizing the data. Jake even went so far as to fill-in imaginary information (statistical analysis & triangulation) that would justify the positive reaction.

It seems that we filled in the vacuum in our own unique ways, based on our personal outlooks and work experience. If you had given clear context in which those quotes were given, we may all well have given you the same (or similar) answers. As a result, our responses were varied, thus making for a livelier discussion. So, was the way you presented your question intentional or did you just get lucky? Inquiring minds want to know! :-)

Jerry

Re: "Data" discussion question | Jerry Talandis Jr. | May 24th, 2004

Robert Haines wrote:

I did not consciously leave out the context. My perspective of it is that Widdowson was commenting during his keynote address called " Applied Linguistics, interdisciplinarity, and disparate realities". The colloquium was entitled "Directions in applied linguistics: Colloquium in honor of Robert B. Kaplan".

The professor who made the comment about data was sitting with me at lunch, referring to a paper that I can't remember much about. I had the impression she meant that more is more when it comes to data.

Thanks Robert, for your response. I was just curious. This thread turned out to be kind of like a "Rorschach Test": put some image out there, ask for people to comment on it. The vacuum created by the lack of context made for some interesting discussion. Thanks. Also, it gave me an idea for a communication activity with my students. I could, for example, show ss a picture and ask them to comment about it, whatever comes to mind. If they are beginners, I could ask them questions about people in the picture, such as "who is he/she?" "What is he/she doing? Why" "What is her favorite color?" Etc. This could be an interesting way to liberate their imaginations and get them to think creatively, using language they already had. Different personalities will fill in details in their own unique ways, which would make the activity interesting.

Jerry
Toyama, Japan

Lost in numbers | Jake Kimball | June 10th, 2004

I'd really like to get to the topic of research and data and how we feel about the different methods and approaches to research available to us. Jerry's really got me thinking the past week or so.

I think this reflects my attitude towards quantitative research- I get turned off or intimidated when I see lots of numbers and formulas, so I responded with a sort of negative reaction to the quote. I felt like the researcher had "lost touch" with reality in a way, getting lost in volume and numbers. 

I suspect that many of us (practicing teacher-researchers) take one side or the other.    But the qualitative/quantitative paradigm isn't as clear-cut as it seems.    I don't want to identify myself with one or the other.     There are actually many different types of each.    I would like to say that there is a middle ground.  I see no reason why an approach to research cannot be determined by the experiment being set up.    Let the research questions determine how to plan and carry out a study.    Secondly, I think that individuals with an advanced degree should be able to recognize the benefits and appropriacy of both kinds of research and everything in between.

From my reading, qualitative studies use more subjective sources of data such as questionnaires, interviews, observations, diary journals, etc.    I do feel that qualitative studies are more descriptive and context-based and therefore have more intuitive appeal to practicing teachers.    The narratives that we write are perhaps recognizable.

Quantitative studies, if designed properly, may be more objective and help us to explain cause and affect relationships and to test a hypothesis.  The value in statistics is that we can control or isolate variables.  

So what is research?    Interestingly, I went back to to Brown and Rogers Doing Second Language Research (Oxford).    This was the topic of the very first chapter in which Brown described a TESOL Task Force question "How would you define research?"    13 definitions came out of those questions.  I won't bore you with more on that.    The main point being that our brief discussion mirrors that of other professionals.

For those of you still reading I would like to mention some quality reserach books that got me over the quantitative hump.    I still don't know much but I'm not put off by stats any more and actually enjoy reading literature with stats--mostly because I now have a basic understanding of them.    They all have something different to offer.

Research Methods For English Language Teachers (McDonough& McDonough)

Doing Second Language Research (Brown & Rogers)

Understanding Reserach in second Lnaguage Learning (Brown)

Research Methods in Language Learning (Nunan)

I enjoyed McDonough the most because it has the most extensive reference list.    They've put together a lot of 'research' on research.    It's mostly about research design, not research.

The Brown book is 100% stats but the other 3 books cover both qualitative and quantitative projects.  

I'm not the smartest apple in the tree so more than one reading of each book will be needed for me to really grasp the statistics.

Jake

Re: Lost in numbers | Jerry Talandis Jr. | June 10th, 2005

Hi Jake, (and anyone else interested in this topic)

Thanks for responding to this topic of qualitative vs. quantitative research. I had thought that previous "data" thread dead, but no- it lives! Just goes to show that because traffic on the list appears light, that doesn't mean nothing is happening. Cool.

In my previous post I tried to express why I wasn't attracted to qualitative research, saying, in effect, that I was afraid of it. Jake, your response has helped me see that there is nothing to be afraid of! In fact, I've been reading that Brown book as well- it's quite excellent (very hands-on & interactive). I'd recommend it to any CP. It should be added to the FND required reading list, actually. And thanks for all those references, Jake. It's good info everyone should have knowledge about.

My point is that when we learn about stuff that makes us uncomfortable, we then naturally lose much of any fear. Education is light! :-) I'm a naturally intuitive sort of chap, so I'm drawn to the quantitative side. But, the more I learn about quantitative research, the more confident and "well-rounded" I feel. There's a lot of advantage in that- kind of like a switch hitter in baseball, being able to hit from the left or the right side of the plate (or, for all you Brits, being able to kick the ball with either foot equally well).

Any other comments out there about the qualitative/quantitative research paradigm? Which side are you drawn to, if any?

Jerry

Re: Lost in numbers | Sarah Haas | June 10th, 2005

Hi Jerry,

Just to chuck in my two cent's worth:   I think it's good, when practical, to have a balance of qualitative and quantitative data and analyses.   I think having both can add depth to 'multi-perspectives.'

Here's a (approximate--I don't have the book in front of me) quote from Eisner (pg. 186-187)  

...at the outset I feel compelled to say that in qualitative inquiry, numbers are OK. I want to say that again: numbers are OK... in fact, the use of numbers where appropriate in qualitative research often gives readers the sense that researchers were reflective about their methods...

Eisner, E. 1998. The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice.   Prentice Hall. New Jersey.                                             

Sarah
Japan  

Re: Lost in numbers | Alex | June 10th, 2005

Hi,

Reflecting on what I've read about qualitative and quantitative data -mostly in the list, thanks to all of you who contributed- I tried to figure out what kind of data led me to choose this topic. I think that keeping a class diary and recording lessons which are essential qualitative research tools helped a lot to come up with some quantitative data which in this case was that my students take turns in pair or group work activities mostly when they are directly selected or asked by the speaker. So, looking over the qualitative data enabled me to make a remark but I had to take into account quantification in order to avoid wrong generalizations.

Therefore, I agree with Sarah that keeping a balance is essential.

Best wishes,

Alex

Re: Lost in numbers | Jerry Talandis Jr. | June 11th, 2005

Hi Sarah, Alex, and everyone else out there,

Cool messages. Sarah, that quote you gave was very helpful. Next time I read some hard-core quantitative research article, I'll close my eyes, take some deep breaths and repeat over and over, "numbers are ok... numbers are ok... numbers are ok..." until I calm down! :-)

Alex, reading your testimony about how this discussion on qualitative vs. quantitative research helped you reflect on your work in a new way was also very helpful. Seems like we've all come to the conclusion that "balance is best," but to me "balance" is one of those amorphous words that sound great rolling off the tongue, get me nodding my head, but then leave me with a rather unclear picture of how that actually works out in reality. Your example shows us one way how it can.

Any other examples out there of "balance in action?"

Jerry

Re: Balance in action | Tisa | June 11 th , 2005

Dear Jerry and anyone else interested in an example of 'balance in action',

Some time back I read an interesting little treatise on writing up qualitative research   (Wolcott, H. 1990. Writing Up Qualitative Research. Sage Publications Inc.). One of the points he really hammered home was that "a critical task in qualitative research is not to accumulate all the data that I can, but to get rid of most of it".   As a result, the qualitative researcher's job largely becomes one of what he refers to as "winnowing...to reveal essences".

I've taken this deceptively simple piece of advice to heart since it really forces me to 'balance' what is and is not important to my main argument.   I have a tendency to want to say and include everything that, even if I could write a book, wouldn't be possible. I also suffer occasional pangs of guilt that basing an argument on only 4 questionnaires is not 'balanced' enough.   Yet if those 4 questionnaires are from the four students on which I've framed my case study, then why do I necessarily need more, especially if I'm specifically interested in what's going on within that unique context?   Wolcott's call to discard data is, I think, an argument in favour of small-scale qualitative studies that judiciously use small amounts of data (based on specific contexts) to create illuminating case studies.   In other words, it's what you do with what you've got that makes for quality qualitative research.   That's one way I attempt to achieve 'balance in action' anyway.

Better get back to IIC!   Like you, Jerry, I'm shooting for the July 1 st deadline to get my assignment into Keith.

Have a great weekend!

Tisa
Switzerland

Re: Balance in action | Jerry Talandis Jr. | June 12th, 2005

Hi Tisa,

Thanks, Tisa, for your quote and comments. I also have trouble like you do, putting into practice the "less is more" ideal. Recently for my TD assignment I had reams of transcripts I wanted to include, feeling that all of it was crucial for making my case the way I wanted to make it. But, due to the 2,000 word limit, there was no way I could fit all of it in. It was brutal, but I cut out entire sections and rewrote. Of course it turned out much better, and I ended up getting a very good grade. I could appreciate why they impose word limits. That "short & little" assignment was one of the most difficult I've had to do yet. Finding balance is not easy to achieve, I think, but well worth any effort you can make.

Jerry

 

Archive Categories