Following the conference several participants continued the
on-line discussion. One issue which particularly interested
me was our discussion about "validity" and the significance
of this construct within an action research context. The prompt
which triggered the discussion had been posted a month earlier;
it was my response to Shankar's message which set the conversation
in motion.
I happened across this conversation while browsing my saved
email. I was surprised at the coherence of the contributions
and the specific interplay between the other correspondents
and myself. The various responses to my writing prompted me
to flesh out my theoretical views. I had never before written
about "validity" -- it was in the course of this
conversation that I was able to make explicit for myself what
I considered to be some of the critical issues for action
researchers to consider.
Without further ado, let me share the written conversation
which took place in just a single week.
Dear all,
I am about to put finishing touches to the final chapter of
my action research thesis and am trying to defend it for rigour
in terms of Validity of my findings.
I have tried to do this using criteria suggested by both
from the qualitative research literature and action research/
action inquiry literature....
Shankar
Date: Tue, 09 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com>
Shankar,
The issue of "validity" is, I think, a "red
herring" in our line of work. The point of a piece of
action research isn't to "prove" anything -- the
most any research account can really do (no matter what the
flavour) is to allow the reader to take a fresh look at his
or her own work. More traditional forms of research wave a
"validity" flag because rarely do readers see any
real connection to their work so the researcher attempts to
persuade us readers that this work matters with a discussion
of validity. But if "truth" isn't what I care about,
if what I care about is whether I can see the problematic
in my own work with new eyes, if your inquiry raises questions
that I think I might find it useful to ask of myself, then
that's what I think this enterprise is about. How do I find
out whether my discussion resonates for my readers? Well,
I have to give it to some folks to discover what their response
is.
Judith
Date: Wed, 10 Jun 1998
From: Brian Murphy <bmurphy@iol.ie>
Judith,
-
You wrote:
> the most any research account can really do (no matter
what the flavour)
> is to allow the reader to take a fresh look at his
or her own work.
Do you mean by "his or her own work" the work of
the reader, as opposed to the work of the individual reported
in the research account?
If the former, then it seems to be suggesting that the only
purpose for reading research is to understand better what
we are doing ourselves -- we are not really interested in
the "account in itself" only in the leverage it
gives us. This seems unduly self-centred.
Sure, when we read research accounts we try to relate them
to our own work and interests but we also sometimes try to
"judge" them or come to some opinion about them.
In particular, where we have doubts about the plausibility
and credibility of the research we need to be convinced by
the nature of the evidence presented. (Though I accept that
some people will accept an account on the basis of a researcher's
claims to "intensive, personal involvement"). So,
to me, this implies that validity is an important consideration
and should be taken seriously.
Regards,
Brian
Date: Wed, 10 Jun 1998
From: Jack Whitehead <edsajw@bath.ac.uk>
I've enjoyed so much of Judith's work and agreed with her
ideas. So, this morning I was startled by her points about
validity. I'm smiling as I write because I know they challenge
a fundamental commitment of mine to validity! As a university
researcher and teacher I have a commitment to help my Ph.D.
students to make original contributions to educational knowledge.
All my students are educational action researchers who are
asking questions of the kind: How do I improve what I am doing?
As they describe and explain their own professional learning
(create their own living educational theories) I help to establish
validation groups in which groups of critical friends subject
their explanations to questions of the sort: Is the explanation
comprehensible? Are the assertions sufficiently supported
by evidence? Are the values clarified and justified in the
course of their emergence in practice? Does the explanation
live in the sense of containing an evaluation of past practice
and an intention to create something better in the future?
I'm still holding to the idea of validity as crucially important
to action researchers who are seeking to make significant
and original contributions to knowledge.
Warm regards
Jack
Date: Wed, 10 Jun 1998
From: Bob Dick <bdick@scu.edu.au>
Dear Judith
Before I get too far into the current discussion I think
it might be useful if I define my understanding of the terms.
I have in mind "validity" and "truth,"
both of which I find problematic.
By "validity" I mean something like "agreement
with reality" or "more than fiction or fantasy.
"
By "truth" I mean something similar. In both instances
I recognise that my views of the world are constructions of
mine. At the same time, I hope in developing these constructions
I am influenced by the data. I hope, further, that these data
(whatever form they take) are influenced by what happens in
the "real world. "
(Yes, I do assume there is a world out there, and that it
is to some extent knowable.)
When I do action research I have some interest in changing
my behaviour or in helping others decide if they wish to change
their behaviour. To do this, I think, requires that I or they
develop models of the world that somehow match that world.
I think of this as "validity. "
Defined in this way, it is a concern of mine. From what you
say, it may or may not be a concern of yours.
In the interests of my own education, I invite you to say
more about your position.
Warm regards
Bob
Date: Wed, 10 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com>
I guess I hit some kind of nerve with my thoughts on "validity."
This will be a long reply because I'm going to respond to
Bob, Jack and Brian by dialoguing with their words.
Judith
-
Bob Dick wrote:
> Before I get too far into the current discussion
I think it might
> be useful if I define my understanding of the terms.
I have in
> mind "validity" and "truth,"
both of which I find problematic.
>
> By "validity" I mean something like "agreement
with reality" or
> "more than fiction or fantasy. "
>
> By "truth" I mean something similar. In
both instances I
> recognise that my views of the world are constructions
of mine.
> At the same time, I hope in developing these constructions
I
> am influenced by the data. I hope, further, that
these data
> (whatever form they take) are influenced by what
happens in the
> "real world. "
>
> (Yes, I do assume there is a world out there, and
that it is to
> some extent knowable.)
I think I've become more of a deconstructionist than Bob
is -- that is, while I accept there is a "real"
world "out there" I'm not convinced it's knowable
in any "objective" sense. I think we're always dealing
as blind men with the elephant -- and it's precisely our different
views of "reality" that are important to understand
so that from time to time we can step out of our own limited
perception and entertain alternate interpretations of that
world.
-
> When I do action research I have some interest in
changing my
> behaviour or in helping others decide if they wish
to change their
> behaviour. To do this, I think, requires that I or
they develop
> models of the world that somehow match that world.
I think of this
> as "validity. "
I also have that same interest in changing my behaviour and
in helping others decide if they wish to change theirs. But
I don't think it's possible for anyone to develop models of
the world that "somehow match that world" because
any "knowing" is necessarily "interpretation."
So while I accept there is some direct link between a "real"
world out there and my sensory responses -- my "knowing"
what those responses mean is an interpretive act and it's
at that point that any direct connection to any "reality"
is severed. I attempt to "validate" my interpretations
by touching base with other's interpretations (it's like creating
intersecting sets in a Venn diagram) -- and the extent to
which my interpretations have something in common with somebody
else's reflects a communal interpretation -- a "knowing"
that is more than just my own personal construction.
-
> Defined in this way, it is a concern of mine. From
what you say,
> it may or may not be a concern of yours.
>
> In the interests of my own education, I invite you
to say more
> about your position.
I am concerned with tabulating my personal interpretations
with those of a wider interpretive community. I came to my
current set of beliefs about meaning and interpretation many
years ago when I taught a course on "Reader Response"
-- one of the most interesting texts with which the teachers
and I grappled was Stanley Fish's "Is there a text in
this class?" from his book of the same name in which
he contends:
a system of intelligibility cannot be reduced to a list
of the things that it renders intelligible. What Abrams
and those who agree with him do not realize is that communication
occurs only "within' such a system (or context, or
situation, or interpretive community) and that the understanding
achieved by two or more persons is specific to that system
and determinate only within its confines. Nor do they realize
that such an understanding is enough and that the more perfect
understanding they desire -- an understanding that operates
above or across situations -- would have no place in the
world even if it were available, because it is only in situations
-- with their interested specifications as to what counts
as fact, what it is possible to say, what will be heard
as an argument -- that one is called on to understand (p.
305). 2
What I understand Fish to be arguing is that we make personal
sense by constructing meaning within an interpretive community
and that the sense made within any particular interpretive
community must, in turn, be explored in terms of other communities'
interpretations but that there is no "match" with
an objective world other than a collective interpretation
of that world.
-
Jack Whitehead wrote:
> As a university researcher and teacher I have a commitment
to
> help my Ph.D. students to make original contributions
to educational
> knowledge. All my students are educational action
researchers who are
> asking questions of the kind: How do I improve what
I am doing? As they
> describe and explain their own professional learning
(create they own
> living educational theories) I help to establish
validation groups in
> which groups of critical friends subject their explanations
to questions
> of the sort: Is the explanation comprehensible? Are
the assertions
> sufficiently supported by evidence? Are the values
clarified and justified
> in the course of their emergence in practice? Does
the explanation live in
> the sense of containing an evaluation of past practice
and an intention to
> create something better in the future?
I am asking the same questions, Jack -- How do I improve
what I am doing? What counts as evidence? How do my interpretations
of a situation connect with others' understanding? I'd contend
that what's happening in the "validation groups"
is that personal understanding/interpretations are being tested
within an interpretive community, that is, the "data,"
the ideas are responded to so that the "researcher"
can see in what ways his/her perceptions are similar to and
different from those of others. It's important, however, to
keep in mind that the interpretive community creates its own
sense of "reality" which is still an interpretation,
albeit a more widely shared one than one's personal construction.
I do believe I need to situate my ideas/hypotheses within
an interpretive community but I'm hesitant to call that "validity"
because of the baggage that this word carries -- it implies
some kind of more direction connection with "reality"
than I believe exists.
-
> I'm still holding to the idea of validity as crucially
important to action
> researchers who are seeking to make significant and
original contributions
> to knowledge.
I think I've abandoned a concern with "validity"
and replaced it with a need to find/create an interpretive
community within which data, ideas, arguments resonate. I
am concerned about making "significant and original contributions"
not to knowledge but to the understanding of the interpretive
community.
-
Brian Murphy wrote:
> Judith,
> You wrote:
> the most any research account can really do (no matter
what the flavour) is to
> allow the reader to take a fresh look at his or her
own work.
>
> Do you mean by "his or her own work" the
work of the reader, as opposed to
> the work of the individual reported in the research
account?
When I read a research account, whether it's technical/rational
(as Schon3
would call it) or an account of reflection-in-action, I'm
using it to test my ideas, beliefs, experiences against those
of the interpretive community. I don't read any account as
standing on its own, but rather as situated in a discourse
tradition, and offering interpretations of experience against
which I think about my own interpretations. As Margaret Meek
4 puts
it: "the text lets me read myself."
-
> If the former, then it seems to be suggesting that
the only purpose for
> reading research is to understand better what we
are doing ourselves -- we
> are not really interested in the "account in
itself" only in the leverage
> it gives us. This seems unduly self-centred.
It's not a matter of self-centred -- that is what text does,
that's how it operates -- the ink marks on the page (or coloured
pixels on a screen) carry no meaning in and of themselves
-- what sense we make of the impression they make on our retinas
is the result of interpretive processes -- we make sense based
on what we bring to the text in terms of our personal experiences,
our beliefs, the collective meaning we carry around with us.
I can "read" an account from what Louise Rosenblatt
5 calls
an "efferent" stance (what facts does this poem
teach us?) but ultimately any reading is done against the
backdrop of "my theory of the world in my head."
-
> Sure, when we read research accounts we try to relate
them to our own work
> and interests but we also sometimes try to "judge"
them or come to some
> opinion about them. In particular, where we have
doubts about the
> plausibility and credibility of the research we need
to be convinced by the
> nature of the evidence presented. (Though I accept
that some people will
> accept an account on the basis of a researchers claims
to "intensive,
> personal involvement'). So this to me implies that
validity is an important
> consideration and should be taken seriously.
But I would argue that "judging" them or "coming
to some opinion about them" is part and parcel of relating
them to our own work and interests -- since our work and interests
are the backdrop against which we are reading. It's not possible
to come to a text as a clean slate -- we bring who we are
to each and every reading, and that's what makes reading so
interesting because who I am is changed by every reading I
do, consequently rereading a text is a new experience since
it's a new "I" who is doing the reading.
Judith
Date: Thu, 11 Jun 1998
From: Brian Murphy <bmurphy@iol.ie>
Hi Judith,
Thanks for the dialogue. After reading it a few times I couldn't
but help feel that I agreed with just about everything you
say, and yet I still cannot abandon "validity" as
you do. The source of this agreement -- yet ultimate disagreement
-- puzzled me somewhat. Where in all these paragraphs did
we actually part company?
I eventually isolated two paragraphs:
-
> I do believe I need to situate my ideas/hypotheses
within an interpretive
> community but I'm hesitant to call that "validity"
because of the baggage
> that that word carries -- it implies some kind of
more direction connection
> with "reality" than I believe exists.
>
> I think I've abandoned a concern with "validity"
and replaced it with a
> need to find/create an interpretive community within
which data, ideas,
> arguments resonate. I am concerned about making "significant
and original
> contributions" not to knowledge but to the understanding
of the interpretive
> community.
The first paragraph above suggests that a lot of our debate
is due to the "baggage" that the word "validity"
carries, mostly coming from a positivist, scientific community.
I do not believe that the word needs to carry that baggage
and it need not imply direct connection with "reality."
I view readers of research accounts as being part of an interpretive
community who apply community interpretive rules or norms
in response to the words on the page. These norms differ greatly
between communities but usually require (among other things)
a connection between the claims made in the account and the
evidence presented. It is this notion of validity that I would
not like to jettison, as otherwise it may be difficult to
discriminate the bogus, fictional, anecdotal or concocted.
Your second paragraph says you have replaced "validity"
with the notion of "resonance." Can you say a bit
more about this? I do not understand it, so I cannot say whether
I personally could substitute it for validity.
So a lot of the debate could be due to terminology? Maybe
definitions of validity and resonance in qualitative research
generally, and action research in particular, would help.
But I'm a novice here and so hesitate to put my toe in the
water.
Regards,
Brian
Date: Thu, 11 Jun 1998
From: Bob Dick <bdick@scu.edu.au>
Hello all
I'm enjoying the discussion, and hope it will help me understand
some aspects of constructivism that I don't presently understand
very well.
In taking the discussion further, I have no intention of
trying to persuade anyone to my point of view. I presume each
of us have belief systems that work for us. My interest is
in increasing my understanding.
I'd like to follow Brian's post by trying to define what
Judith and I agree about. If we can establish common ground,
the differences are more likely to be educational, I think.
So, I think Judith and I are agreed ...
- that there is a real world
- that we can know it only indirectly
- that for this reason there are no objective grounds on
which we can say that this interpretation is certainly better
than that interpretation [I would add, except perhaps in
the extremes]
and we may be agreed ...
- that we wish to impact in some way on that world
I also suspect the issues about "baggage" are about
labels, not realities. I don't expect them to be important.
"Validity" works well for me, for reasons I'll
try to explain below. But that doesn't give me any reason
to expect others to find it useful.
I use a non-reductionist methodology within a faculty that
is reductionist. So I am often regarded as somewhat marginal.
No doubt this has had some influence on my language and practice.
Speaking personally, in such a situation I find it useful
to be able to justify my choice of methodology in terms that
make sense to my colleagues. Supervising theses in this situation,
I find it helpful to be able to assist thesis candidates to
defend their thesis according to the rules of the dominant
paradigm.
For these purposes, "validity" is a useful term.
My colleagues don't have any reason to learn my dialect. It
therefore makes sense to talk to them in their own.
However, I don't have any ideological commitment to "validity"
as a word. "Resonance" will do fine if we can agree
on what it means, and if it serves the purposes I require
it to serve.
There does seem to be a difference in the conclusions we
draw about the knowability of the world. This is what I'd
like to explore.
Judith, you seem to be saying either that you can't know
the world, or that it doesn't matter. This puzzles me, not
just here, but when I come across it in some of the literature.
What I don't understand is how I and others can act effectively
on the world if our perceptions of it bear no correspondence
to that reality.
Warm regards
Bob
Date: Thu, 11 Jun 1998
From: Jack Whitehead <edsajw@bath.ac.uk>
-
Judith writes:
> I think I've abandoned a concern with "validity"
and replaced
> it with a need to find/create an interpretive community
within
> which data, ideas, arguments resonate. I am concerned
about
> making "significant and original contributions"
not to knowledge
> but to the understanding of the interpretive community.
I agree with all the points Judith has made about interpretive
communities. I also agree with Bob's views on truth and validity.
It may be that Judith can abandon a concern with "validity"
because she is focusing on "understanding" rather
than "knowledge." Like Bob I'm also interested in
the meaning of "resonate." One of the standards
of judgement examiners of a Ph.D. Thesis are required to use
is that it should make an original contribution to "knowledge."
Because of this it still seems important to stress the importance
of "validity" in the sense of testing the validity
of a claim to knowledge. I'm happy to embrace both the concepts
associated with hermeneutics and epistemology in the sense
of valuing "understanding, "ways of knowing"
and "knowledge." I don't yet see them as conflicting.
Warm regards
Jack
Date: Thu, 11 Jun 1998
From: Pam Swepson <p.swepson@uq.net.au>
Dear Judith and Bob and all
I too am enjoying the discussion on validity and resonance.
I have read a few brief words about what Gareth Morgan has
said about resonance and I have some difficulty with the concept.
Perhaps someone can help me here.
My concern is that resonance sounds like either rationalisation
or "group think. " To explain further, if as some
people think, human beings have a tendency to form an opinion
on very little data and then spend the rest of our time finding
data to justify it, either individually or as a group, is
this what resonance is? Feeling some justification for what
we want to believe is true? Where am I wrong here, please?
If I am right, then it seems to me that to check our own
"validity" (if you want to use that word) then you
have to work very hard at finding very different opinions
to compare your thinking with and hanging in long enough to
make an honest comparison. Tough stuff, I think.
Any thoughts?
Pam
Date: Thu, 11 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com>
Brian and Bob, another longish musing.
What I find fascinating is how if we unpack our thoughts
more fully we find we have similar values -- that the language
we are using serves as the barrier. I guess I'm particularly
sensitive to the language I choose to use. Let me relate an
incident which occurred twenty years ago which had a significant
impact on how I engage with the academic world.
I had recently finished my PhD. I was at a conference having
just attended a session given by my major mentor Frank Smith
(a literacy researcher) and we were talking about some reading/writing
research I was then engaged in. I was describing what was
going on in the classrooms, the "reading" and "writing"
activities we were attempting with the kids when Smith turned
to me and asked "How can you distinguish them?"
I paused for a moment and replied "I guess I can't."
And he responded "Well, why do you talk about it that
way?"
It was the "Why do you talk about it that way?"
that got me -- I couldn't know if a child was really learning
about reading from the "writing" activities instead
of the "reading" activities -- Smith's point, well
taken, is that there is no way of tracing exactly how a learner
constructs his or her understanding of how text operates.
I found myself, after that, thinking a lot about how I "talked"
about ideas.
Now this notion of "validity" is one of those ideas
that I've thought about quite a bit -- why do I talk about
it that way? I realized I couldn't. Like you, Bob, I wasn't
coming from the same place as folks who use the term -- it
comes from a technical/rationalist tradition. The idea was
invoked to regulate research activity and keep wide ranging
inquiry in check. The problem is nevertheless an interpretive
one -- who gets to decide what is "valid" and what
isn't? It's still readers of a research account (whether that
reader is a thesis advisor, or journal reviewers, or colleagues
in granting agencies) who serve as the gate-keepers -- who
make the interpretive judgement about whether a piece of research
seems plausible or not. For when it comes down to it a judgement
about "validity" is a judgement about "plausibility."
Really, there is no objective test of "validity"
-- the technical/rational research community is forced to
judge "validity" in an interpretive way. And the
myth is that if there is "validity" and "reliability"
then there will be "replicability," right? But when
you look at reflective practice6 (I'm using Schon's dichotomy for convenience
sake) it's clear that replicability isn't a goal here. As
Schon made abundantly clear in his work, the situations on
which we are reporting are unique from moment to moment and
the characteristics of the situation in which I work are likely
to be quite different from those in which you work and so
the kind of test which makes some sort of sense isn't this
notion of "validity" but of something else -- I've
opted for an openly subjective idea like "resonance"
-- my test when I read a piece of research includes the following:
- does it offer enough thick description for me to "live"
in the situation, to see it in some depth (it's never possible
to offer more than a slice of an interpretation of what's
going on, and what's written about is selected from a huge
store of evidence so that the writer can focus on what he
or she decides are critical ideas)
- does the account situate itself in the research conversation,
does the writer make clear the debates which have influenced
his or her thinking?
- are the writer's assumptions made explicit and do they
reflect on them?
- does the account offer some thoughts on "So What?"
-- how has this work affected the way in which the writer
engages in his or her professional work?
The stance I take as a reader is an aesthetic one as Louise
Rosenblatt7
calls it -- I read research accounts as I do novels -- I attempt
to "live through" the experience. As with a good
novel, if the account lets me in, allows me to live in that
fictional world and I can see connections to my world, it's
likely to permit me to see my own work in new ways. I'm not
looking for correspondence -- I know my work situation is
unique, as was the research situation being described -- rather
I'm looking for what I have come to call "resonance"
-- does the account seem believable, does it help me think
about the problematic of my working situation, does it help
me name or reframe the tensions in my work so that I might
do something about them?
Sondra Perl,8
a writing researcher invokes the notion of "felt sense"
in writing -- you can tell when some writing is going well
because it "feels" right -- she even goes so far
as to explore how the body identifies what this "feeling"
is. I think it's the same with research accounts -- particularly
action research accounts which have a strong narrative flavour
-- they "feel" right. I go away from my reading
wondering in new ways.
Now, Bob, you're concerned about living with colleagues --
that's certainly a concern. I guess I discovered I was what
Doris Lessing9
referred to as a "zone three person in a zone four world
(The Marriages Between Zone Three, Four, and Five) -- that
novel helped me understand the dissonance I was feeling living
in an academic setting, it let me "read myself."
It allowed me to recognize that I was always going to be at
odds with colleagues over questions about teaching, about
assessment, about learning above all. So I made the decision
that I could free myself from talking about things in the
way they did -- and having made that decision I've been able
to get on with the kind of inquiry I think is useful to me
personally and which seems to resonate for teachers -- they
respond to what I write in a reflective way which seems to
impact on how they think about their own work. Since teachers
are my audience, not my academic colleagues, it's their judgement
about the utility of my research and writing for them that
counts for me.
Judith
Date: Thu, 11 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com>
-
Bob Dick wrote:
> Judith, you seem to be saying either that you can't
know the world, or
> that it doesn't matter. This puzzles me, not just
here, but when I
> come across it in some of the literature. What I
don't understand is
> how I and others can act effectively on the world
if our perceptions
> of it bear no correspondence to that reality.
Bob, clearly there has got to be some correspondence between
our perceptions and reality "out there." I just
don't think we can "know" it in any objective way.
My interpretation of what's happening in my middle/inner ear
as it's receiving speech depends on a whole host of judgements
I'm making about whose speaking, what they're likely to be
speaking about, what they know, what my expectations are,
and on and on. There's a field of study -- Signal Detection
Theory10 -- dealing with radar operators -- the
"reading" of a radar screen is very complex -- the
reader has to make a judgement about whether a blip represents
an enemy aircraft or not. They end up with "hits', "misses"
and "false alarms" -- the gist of the arguments
is that it is not possible to make perfect identifications
because you can't completely eliminate the "noise"
from the system. Even reading a radar screen is an interpretive
process!
Judith
Date: Sat, 13 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com >
Jack, I don't have your message in front of me -- I seem
to have lost it from my hard drive. But what I remember of
the gist of it is that you're uncomfortable with my abandonment
of the notion of validity, that you feel my rejection of the
notion somehow implies that you must rethink it, too.
I have entered this discussion, not to change anyone's mind
about ideas such as this, but to share my journey, to let
you see the path I've taken and why.
I simply stopped playing the academic game about 15 years
ago. I decided that the kind of work involving teachers that
I felt I needed to do needed a different character -- one
that the academy didn't understand or support. I remember
a major funding agency rejecting my research proposal as a
"promising reject" because I couldn't lay out the
inquiry in the expected fashion. I decided, then, that rather
than rewrite and resubmit the proposal, I would conduct what
research I wished to do unfunded. That has been a wise decision
for me. It has let me work with teachers, and the teachers
with their students, exploring questions of interest to us
all. We have been successful publishing some of this work
in periodicals which do support teacher action research, and
as collected articles in books, now on the internet. We've
been able to get our ideas into the research conversation
and to grow from other's responses to what we've had to say.
It's that public response that has been important for shaping
our thinking.
But as for attempting to change how you think about things
-- that hasn't been my intention. I've simply wanted to share
how I've come to think about things, and if those thoughts
resonate for anyone, that's fine, but it's not necessary that
they do.
Judith
Date: Sat, 13 Jun 1998
From: John J. O'Brien <jobrien@access.victoria.bc.ca>
Interesting thread. To add to the melange -- Pam queries
whether resonance is about "feeling some justification
for what we wish to believe is true. " I understand and
concur with the danger inherent in seeking confirmation for
preconceptions. However, I do not think that resonance is
about that. Things resonant when we disagree with them, too.
Perhaps resonance is a matter of confirmation of a felt sense
of truth in some regard -- but I don't think it implies agreement.
It is, perhaps, another facet for observation and learning.
Thoughts?
John
Date: Sat, 13 Jun 1998
From: Judith Newman < jnewman@hfx.andara.com>
John, you've put that nicely. I'd say that "resonance"
is that sense that this connects/ extends my sense of the
world, this doesn't feel right -- the sort of response that's
happening with this very discussion -- for some, the notion
of "resonance" seems to fit and for others it's
proving uncomfortable. I'd say that we're testing the "validity"
of the argument/data against what we believe or have experienced
about the world -- that's all any of us can do -- I think.
Judith
Date: Sat, 13 Jun 1998
From: Dina Boogaard <DJBoogaard@aol.com>
Judith,
I just wanted to say that I deeply appreciate where you are
coming from regarding the concept of validity and the limiting
aspects of the Academy. And I think you have explained it
eloquently and clearly. I also very much appreciate the contributions
of everyone inquiring into this arena. It is all contributing
to my own understanding.
I'm just going to add a few paragraphs from some thoughts
about qualitative research that I wrote down a few months
ago, in my quest to better understand what I'm dealing with
as I plunge into my dissertation (this is an excerpt):
Many scientists, particularly social scientists, have already
come to realize that a perfectly objective, measurable,
predictable world is not really an option. That means that
all research is a mixture of subjective, caring involvement
with the people being studied and a honing of the researcher's
capacity to be appropriately detached, analytical, and reflexive.
The feminist perspective, a form of critical theory, contributes
a particular and important voice to qualitative research.
It emphasizes the point that all research is subjective,
that we socially construct our worlds, and this does not
indicate a lack of rigor or epistemological validity to
the work. What has come to the fore because of feminist
inquiry is that most knowledge has been generated and defined
by white males and that, although they espouse an important
perspective it is not the only one, and perhaps not always
appropriate. "Feminists have challenged not only the
view of the way in which knowledge is produced but also
whose view of the world it represents." As Chisholm
(1990)11
puts it, "the collision between theory and praxis [abstracted
reflection on practice] is as emotionally significant as
it is intellectually interesting" -- there is a need
to look coolly and passionately" (Robson, 1993, p.
65). 12
Reliability and Validity
Although qualitative research must contend with the issues
of reliability and validity, there are strong voices suggesting
that these criteria are neither directly applicable nor
congruent to qualitative research, that they are positivist
notions that are not transferable to qualitative, postmodern
methodologies. As I mentioned earlier, Guba and Lincoln
13 have suggested that alternate terms,
"Terms such as credibility, transferability, dependability
and conformability replace the usual positivist criteria
of internal and external validity, reliability and objectivity."
(p. 14)
It seems that in qualitative research we go to the participants
themselves and our own souls to verify, rather than validate,
whether or not we captured what was important and useful.
It is a chorus of voices that determine the standards and
verification required for a study to be authentic and done
with the "right" intention and processes. Guba
and Lincoln 14 bring some compelling thoughts to the
discourse regarding quality issues based on their long and
extensive research into the area of emerging criteria in
qualitative research: "establishing the criteria of
"fairness'(a balance of stakeholder views), sharing
knowledge, and fostering social action." The new emerging
approach to quality is based on three new commitments: to
emergent relations with respondents, to a set of stances,
and to a vision of research that enables and promotes justice.
(Creswell, 1998) 15
Dina Boogaard
Date: Sun, 14 Jun 1998
From: Pam Swepson <p.swepson@uq.net.au>
Dear Judith, Dina and John
I certainly have some sympathies for the idea of "resonance."
So, for those who have thought about it more than I, may I
ask, how can you distinguish between the ahha! of insight
from plain old kidding ourselves to maintain old prejudices?
Pam
Date: Sat, 13 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com>
Pam,
I guess what I attempt to do in my own work is focus on the
problematic -- the stuff that pops up which I don't understand.
And via the relating of what I call "critical
incidents" I attempt to identify the gap between
what I think I believe and what my actions are likely conveying
in these situations.
The process as it has evolved for me, and the teachers I
have been working with, is focused on exposing those "old
prejudices" and trying to reframe the problematic of
our working lives in new ways. In other words, the process
is aimed specifically at naming prejudices and trying to gain
a new perspective on what might be going on.
For example, a year or so ago I was working with a group
of teachers, trying to help them deal with "change"
initiatives they were facing. I have long thought that if
I could enable folks to talk about what was problematic in
their work they were more likely to find new ways of working.
What I discovered when I was reading their reflections was
that it wasn't a lack of knowledge about current educational
issues that was holding folks back from changing how they
worked in their classrooms -- they named a whole host of situational
factors which create barriers of which I tacitly aware, but
the teachers' writing made me stand back and rethink how I
was engaging with them because I could no longer ignore the
political realities of their lives and working situations.
[See: "We
Can't Get There From Here."]
So, for me, action research sorts of inquiry are most suited
to uncovering our prejudices and framing new ways of understanding.
Judith
Date: Sun, 14 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com >
Dina, I like what you've written here. I think it's important
to understand that we don't have to play in a pre-defined
court -- we can and must create the game as we go along. I
don't believe we will have much luck convincing died-in-the-wool
technical / rationalists anyway. Seems to me our efforts are
better used for building this alternative way of viewing the
world.
This discussion we've been having reminds me a lot of James
Burke's TV series "The Day the Universe Changed"
16 -- in each episode
he explored an instance where our perception of the "universe"
changed. He traces the small shifts of thought which led to
a radical reformulation of how we (western/European thought)
understood how the heavens worked, how we reconceptualized
medical practice, etc. His underlying message is that while
the physical reality remained unchanged, how we thought about
it was forever altered. I think we're embroiled in that kind
of shift with regard to research as an activity. Interesting
times to be living in.
Judith
Date: Mon, 15 Jun 1998
From: Pam Swepson <p.swepson@uq.net.au>
Dear Judith,
-
You wrote, in part:
> So, for me, action research sorts of inquiry are
most suited to uncovering
> our prejudices and framing new ways of understanding.
Thanks for your story and explanation. Could you further
explain, please, how you think action research does help uncover
prejudices. It would seem to me that processes for involving
divergent opinions (both of my own and of my "group")
is the critical factor here. It also seems to me that the
literature on AR is deficient in methodologies for this sort
of critical participation.
Look forward to your comments
Pam
Date: Mon, 15 Jun 1998
From: Jack Whitehead <edsajw@bath.ac.uk>
Hi Judith -- I do like the idea that you are not trying to
change anyone's mind but sharing a journey you have taken.
That's the spirit in which I respond to your work.
-
You also said,
> I simply stopped playing the academic game about
15 years ago.
> I decided that the kind of work involving teachers
that I felt I
> needed to do teachers needed a different character
-- one that the academy
> didn't understand or support.
My own journey has been focused on contributing to the transformation
of what counts as educational knowledge/theory in the academy
so that teachers could gain accreditation for their professional
knowledge in a way which doesn't distort their own educational
theories.
One of the questions I have is that if action researchers
abandon concepts such as "validity" will this be
contributing to the process of "balkanisation" described
by Donmoyer in Educational Researcher? I am wondering if you
see what you are doing in terms of "paradigms" or
whether this concept isn't significant in your enquiry.
Warm regards
Jack
Date: Mon, 15 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com >
-
Pam Swepson wrote:
> Thanks for your story and explanation. Could you
further explain,
> please, how you think action research does help uncover
prejudices. It
> would seem to me that processes for involving divergent
opinions (both
> of my own and of my -group-) is the critical factor
here. It also seems
> to me that the literature on AR is deficient in methodologies
for this
> sort of critical participation.
Pam, I think the way I've come to sort out this business
of uncovering prejudices (I talk about assumptions) seems
to be via critical incidents which serve as the evidence on
which the research is based. In my experience, the bulk of
the incidents seem to focus on situations in which my assumptions
are brought into focus, are highlighted in some way or other.
A poem as example:
-
Mirror 17
Chaos confronted me as I entered the hall:
kids running around pulling stuff from one another's lockers.
I was standing there hands on hips
when twelve year-old Sally
sidled up leaned close and said,
"Pretty rowdy, aren't they?"
"Yes, I concurred, "like wild Indians."
She leaned closer --
"I'm part Indian, you know," she confided.
A friendly overture
reflecting an image I was shocked to see.
The moments I seem to capture all have this kind of flavour
-- as a rule, they make what's transparent visible in ways
that allow me to clearly see contradictions between belief
and action, hidden assumptions, unexpected prejudices.
Judith
Date: Tue, 16 Jun 1998
From: Bob Dick <bdick@scu.edu.au>
Pam said, in part, about surfacing prejudices ...
-
> It also seems to me that the literature on AR is
deficient in
> methodologies for this sort of critical participation.
For me, part of the appeal of Chris Argyris' 18 work is that he provides models and processes
for doing so.
Bob
Date: Tue, 16 Jun 1998
From: Pam Swepson <p.swepson@uq.net.au>
Judith
Nice story, nice poem. I am still feeling unsatisfied but
can't spell out my concerns any further at the moment. I think
it is something about "groups" having the same experience
that you have as an individual. I will try to keep thinking
to clarify my worry.
Pam
Date: Tue, 16 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com >
Pam, somehow I don't think "groups" have "experiences"
-- individuals do, and my hunch is that even when in a "group"
the experience for each individual is unique, dependent on
his / her history, values, beliefs, etc. And I guess what
I look for from others is some element of common experience,
common values which allow me to sense community and at the
same time offers me difference in perspective which keeps
me refining my interpretations of what's going on. -- Like
what's happening with this ACTlist conversation.
Judith
Date: Tue, 16 Jun 1998
From: Jack Whitehead <edsajw@bath.ac.uk>
-
> Twenty years ago I abandoned positivist ways of
viewing the
> world. Working with very young children making sense
of language,
> oral and written, quickly moved me into a different
theoretical
> place. I can't see going back.
Judith -- I can recall a similar experience in 1971 when
I abandoned my positivist approach to educational research
because it did not permit me to understand my educative influence
with my pupils. When you say you moved to a different theoretical
place, I am trying to understand your view of educational
theory and educational theorising and to inquire if your "theoretical
place" can exist without the concepts of validity, knowledge
and explanation.
Warm regards
Jack
For some reason, I don't now recall, I didn't reply to Jack.
The conversation on ACTlist moved on and people began discussing
other concerns. But Jack's question affords me the opening
to put this written discussion into some kind of perspective
for myself and for anyone else interested in such arcane issues
as "validity" in action research.
Jack asked me whether my "theoretical place" can
exist without the concepts of validity, knowledge and explanation.
It's not that I'm not concerned with what connections my perception
of the world of schools and classrooms has with some "real"
world situations -- I try to offer as detailed descriptions
of the situations and the participants as I can so that readers
may judge the "reality" for themselves -- but I
am also aware that whatever I write is an interpretation,
my interpretation, of what I've experienced or what the others,
whose stories I include in my writing, have experienced. Even
if I were to have used quantitative data, the written account
would still my interpretation of what those data might mean.
As I see it, there is simply no way to step outside of interpretation.
So a notion, like "validity" doesn't seem to me
to contribute in any useful way to my work as an action researcher.
I am careful to situate events, moments, critical incidents
as fully as I am able but in the end I realize that, no matter
what I write, it is, in some sense, a fiction. Just as a more
traditional research account is a fiction -- there are no
"average" students, no "typical" teacher,
no "representative" classroom; students, teachers
and classrooms are unique. The most any research account can
afford is a partial snapshot that may or may not offer insight
about the daily concerns which students and teachers face.
As for the construct "knowledge" and the role of
"explanation" -- we come back to a researchers'
view of the world. As a "constructivist" I accept,
as I wrote Bob Dick, that there is a physical reality "out
there" but my senses filter all incoming data and interpret
them based on my prior experience, on my theories of the world
that I've built up over a lifetime of living. How, for example
do I "see" 13 -- in the context 13all I
interpret letters, in 713 4 I perceive numerals. There are
marks on the page, or lit pixels on a screen, but what sense
I make of sensory data depends on context and my interpretation
of it.
I believe I have a responsibility to provide a well-documented
account of the particular experiences I choose to share. However,
I am aware that my "knowledge" is a personal construction,
constantly being shaped by both personal first-hand experience
and by the conversations I have with others. I am influenced
by how others interpret the world and my understanding of
situations is certainly affected by accounts that I read and
by any written or face-to-face discussions in which I participate.
As an action researcher I struggle to maintain an "open"
mind -- to allow myself to be surprised by situations and
to use that surprise to examine my unexamined assumptions.
For me, it's that continuously making visible of the taken-for-granted
that provides the rational for engaging in inquiry into my
professional work.
Needless to say the conversation doesn't end here. The other
day I received email from Pat D'Arcy (Jack Whitehead had passed
on to her an early version of this written conversation I'd
sent him) in which she added her voice.
Date: Mon, 26 Oct 1998
From: Pat D'Arcy <edppmda@bath.ac.uk>
Dear Judith,
I've been sitting here on a wild, wet and windy autumn afternoon
reading the batch of e-mails to you and from you that arrived
in a package from Jack [Whitehead] this morning and they have
set so many thoughts racing through my head that I just wanted
to join in the conversation -- even if somewhat belatedly!
So let me just plunge in! Validity... to me, if action research
has any validity then it is to show that what the researcher
set out to change is indeed changeable by providing demonstrable
evidence that changes in actual practice have indeed occurred.
To take my research as the example (not surprisingly) that
comes most readily to mind. I set out to offer guidelines
for myself and for the teachers I was working with based on
Rosenblatt's aesthetically transactional approach 19 to reading stories which would enable us
to respond to pupils' stories in a way that was both personally
meaningful to ourselves and subsequently to the children who
had written the stories in the first place.
I was able to compare the responses that we made (in writing)
to their stories with the skill-based forms of response currently
in use in our National Tests to show how much those assessments
were missing in terms of the pupils' achievements through
their highly creative constructs as meaning-making individuals.
To me, my research was valid because it provided the evidence
for these differences of what we 'saw' depended on how we
looked, which could then lead me to make a stronger case for
more interpretive forms of assessment which were both valid
(the term that Louise Rosenblatt 20
also uses) and warrantable (the term that Fran Claggett 21 uses in her book "A Measure of Success")....
Pat
Date: Wed, 28 Oct 1998
From: Judith Newman <jnewman@hfx.andara.com>
Pat,
Pat, thanks for jumping into this conversation. Your two
cents at this point are useful.
-
> ... to me, if action research has any validity then
it is to show that what
> the researcher set out to change is indeed changeable
by providing
> demonstrable evidence that changes in actual practice
have indeed
> occurred....
The sort of inquiry I find myself engaging in doesn't necessarily
set out with change goals worked out. I don't begin with anything
specific that I want to change -- as a teacher/teacher educator,
I do have a broad intention of helping teachers examine their
assumptions and practices in order to understand better the
connection and the impact of their instructional decisions
on students. I do want them to examine prevailing paradigms
in order to understand the beliefs underlying them. But as
for specific changes, I can't say in advance what those might
be. Nor can the teachers I work with. For example, one teacher
who is currently writing a Master of Education thesis thought
she was exploring how her grade one students learned to read,
only to discover that her piece really is about how her instructional
decisions are affected by outside pressures. She found that
out from an analysis of her daily writing about what was happening
in her classroom. She didn't know to what extent parents'
beliefs, administrative directives, school district policy,
provincial government control affected her moment-to-moment
classroom decisions until she began the actual writing of
the thesis. So the particular criterion you set out for deciding
whether the work has "validity" doesn't apply in
this situation. I'm not sure what sorts of criteria would,
other than the internal consistency and richness of the narratives,
the thoroughness of Claire's analysis, her connection with
the research literature. Ultimately, anyone reading her work
will have to make some judgement about whether her discussion
raises any questions that are relevant or interesting for
him or her.
-
> To me, my research was valid because it provided
the evidence for these
> differences of what we 'saw' depending on how we
looked...
I can see how, in a situation where you have a particular
goal in mind as you did, that demonstrating a difference between
one way of engaging in instruction or assessment and another
(and examining the paradigmatic beliefs which shape those
different implementations) would lead to an argument in support
of one set of instructional or assessment practices rather
than another. However, I'm not sure why you need to invoke
"validity" as a construct to be able to say there's
something interesting and perhaps useful in that comparison.
My work seems to shape itself differently -- I find myself
beginning in the middle of whatever is going on and suddenly
encountering something problematic. I know before I begin
a new course that I'll stumble across something that I haven't
thought about before but I have no idea before I begin what
that might be. So I find myself writing about issues of power
and control, or problems I face as a "change agent"
in helping teachers actually do something different in their
classrooms, or I come face to face with the pressures that
are affecting them and limiting what they're willing to try.
But I only discover that when I stand back from the critical
incidents I've been collecting and look for issues which they
allow me now to see.
It seems to me you have a responsibility as researcher /
writer to provide me, your reader, with as much evidence as
you can (descriptive, narrative, quantitative, whatever) so
that I can understand your biases and then be able to make
a judgement about whether your arguments offer me useful things
to think about myself. I'm not interested in "what tips
does this research offer me" as many of the teachers
I work with would be. I want your research account to show
me what impact your research had on you own theoretical development.
I want to know how the questions you were asking changed,
how you now understand better the political pressures which
impact on your decision making, and so on. The more clearly
the "journey" you've taken is portrayed, the more
useful is your account for allowing me to think about my own
situation and the problematic I encounter.
Judith
Date: Thu, 29 Oct 1998
From: Jack Whitehead <edsajw@bath.ac.uk>
I do feel that I'm coming to understand your views more fully.
We do appear to be understanding similar meanings using different
words. I wonder if you would expand a bit on the meanings
you give to 'theory' and 'theoretical development' when you
say at the end of the paper:
> "I want your research account to show me what
impact your research
> had on your own theoretical development".
What I'm really interested in is what, for you, characterises
'theory' or 'theoretical development' in the narratives/interpretations,
which are constructed by educational action researchers.
Warm regards
Jack.
Date: Tue, 03 Nov 1998
From: Judith Newman <jnewman@hfx.andara.com>
Jack,
My first encounter with "ethnography" in education
was through Marjorie Siegel's PhD thesis. 22 Margie was struggling with the situation
in the classroom where she was observing/participating and
with her role in the research. One of her advisors, Bill Corsaro,
said to her " What you see as problems, I see as data."
Margie's thesis became an account of her changing assumptions
as she worked through problems.
Her experience shaped my subsequent work because I realized
that my responsibility as researcher was to make what was
happening to me as a result of my experience open for others
to see. I stopped being concerned about whether my experience
could be generalized, (Schon 23
would argue it was unique, anyway) and worked more at shaping
the narrative and argument so that others might use it to
examine their own work more critically.
To respond to your question, then, Jack. The "theory"
in my work can be represented by the intersection set of "my
experience", "what I've read that allows me to take
an interpretative stance on that experience" and "why
I think it's significant for my work; how I see my work differently
as a result of having explore some new contradictions."
I've drawn a Venn diagram for you to show how I see the interplay
among these elements.
So, theory, for me is that constantly changing "space"
where experience, interpretation, and research discourse interact
/ intersect. "Theoretical development" occurs during
the construction of the narratives -- the struggle to decide
which critical incidents contribute to some bigger picture,
how to cast them, what issues do I now understand that I didn't
before, my attempt to relate how I see this inquiry affecting
my work, etc. all are part of my ongoing theoretical development.
Does any of this help?
Judith
And there, for now, I end my musings. At the end of this
written conversation I understand in a new way the relationship
between my views about the construction of knowledge and the
research methodologies I (and the teachers I've worked with)
have been evolving over the past fifteen years. During that
time we've learned how to record our experiences and the insights
they promote, to concatenate events so that a story emerges
and the issues which surface are discussed explicitly. Our
objective is to write in such a way that others may use our
journeys to help them "read themselves."
1. The conference description and the papers
which served as the basis for the month-long on-line discussion
can be found at http://users.andara.com/~jnewman/Papers.html
Return 1
NOTE : Email addresses may not be accurate;
several have changed. I've left them in the excerpts to reflect
the international character of the conversation. Return
2. Fish, Stanldy 1980 "Is there a text
in this class?" In: Is There a Text in this Class? Cambridge
MA: Harvard University Press: 303-321. Return
2
3. Schön, Donald 1983 The Reflective
Practitioner: How Professionals Think in Action. New York:
Basic Books, Inc., Publishers. Return
3
4. Spencer Margaret (Meek), 1987 Text in
Hand: Explorations in the Networking of Literacy and Literature
or New Literacies, New Texts, Old Teachers. Paper presented
at the 5th Invitation Riverina Literacy Centre Conference,
Wagga Wagga, NSW 20-22 August, 1987. Return
4
5. Rosenblatt, Louise 1978 The Reader, the
Text, the Poem: The Transactional Theory of the Literary Work.
Carbondale, IL: Southern Illinous University Press. Return
5
6. Schön, Donald 1983 op cit. Return
6
7. Rosenblatt, Louise 1978 op cit. Return
7
8. Perl, Sondra 1983 Understanding Composing.
In: J.N. Hayes et al (Eds) The Writer's Mind: Writing as Mode
of Thinking. Ubrana, IL: NCTE: 43-51. Return
8
9. Lessing, Doris 1980 The Marriages Between
Zones Three, Four, and Five (as narrated by the Chroniclers
of Zone Three). London: Granada. Return
9
10 . Swets, J.A., W. P. Tanner, jr., &
T. G. Birdsall 1961 Decision processes in perception. Psychological
Review, 68: 301-320. Smith, Frank 1971 Understanding Reading:
A Psycholinguistic Analysis of Reading and Learning to Read.
New York: Holt, Rinehard & Winston: 23-26. Return
10
11. Cited in Robson, Colin 1993 below. Return
11
12. Robson, Colin 1993 Real World Research:
A Resource for Social Scientists and Practitioner-Researchers.
United Kingdon: Blackwell Publishers, Ltd. Return
12
13. Guba, Egon G. & Yvonna S. Lincoln
1994 Competing Paradigms in Qualitative Research. In: Denzin,
Norman & Yvonna Lincoln (Eds) Handbook of Qualitative
Research. Thousand Oaks, CA: Sage Publications, Inc. Return
13
14. Guba, Egon G. & Yvonna Lincoln 1994
op cit. Return 14
15. Creswell, John W. 1998 Qualitative Inquiry
and Research Design: Choosing Among Five Traditions. Thousand
Oaks, CA: Sage Publications, Inc. Return
15
16. Burke, James 1985 The Day the Universe
Changed. London: British Broadcasting Corporation. Return
16
17. Newman, Judith M. 1998 Sabbatical In:
Tensions of Teaching. New York: Teachers' College Press, pp.
176-177. Return 17
18. Argyris, Chris 1976 Increasing Leadership
Effectiveness. New York: Wiley & Sons. Return
18
19. Rosenblatt, Louise 1985 The Transactional
Theory of the Literary Work. In: Charles Cooper (Ed) Researching
Response to Literature and the Teaching of English. Norwood
NJ: Ablex Pub. Corp. Return
19
20. Rosenblatt, Louise 1985 op cit. Return
20
21. Claggett, Fran 1995 A Measure of Success.
Portsmouth, NH: Boynton/Cook Publishers. Return
21
22. Siegel, Marjorie 1982 Reading as Signification.
Bloomington Indiana: University of Indiana. Unpublished Doctoral
Thesis. Return 22
23. Schön, Donald op cit. Return
23
* March, 1999
This article was originally published at: http://casino.cchs.usyd.edu.au/arow//reader/newman.htm.
The formatting has never been quite right, hence this electronic
version.
The article should be cited as:
Newman J (1999) Validity and Action Research: An Online Conversation
in I. Hughes (ed) Action Research Electronic Reader online
at http://www.lupinworks.com/article/validity.html
[ Top ] [Return
to Articles]
