From: "Brad Rose" <Bradrose1-AT-comcast.net>
Subject: BHA: Fw: Experimental Design as the "Gold Standard" of Science: 20 points extra for a randomized experimental design
Date: Tue, 5 Aug 2003 18:41:51 -0400


Dear List Members

I am forwarding to the Bhaskar list a snippet (see below) from a recent
discussion that has occurred on the American Evaluation Association
listserv.  (The discussion was prompted by an observation that applications
for major Federal grants are now awarded an extra 20 points by reviewers, if
they include experimental design--regardless of the appropriateness of that
design for the object of study)  I forward this comment because it may be of
interest to some on this list, as it critiques the Bush administration's
apparent apotheosis of experimental
design in the social sciences as THE exclusive method for obtaining
scientific truth.

My concern, which some on this list may share, is that increasingly
"science" is very narrowly
and overtly politically defined, as is both the knowledge that science
produces and
the truth's that it reveals. The comment below, from Michael Quinn Pattonl
an evaluator in the US, briefly touches upon the ideological and political
valence of the administrations preference for the "gold standard," in social
science (in this case evaluation of social programs.)  Although I do not
agree with what I take to be Patton's implicit ontological pluralism
(relativism), I do agree with his epistemological pluralism, and his
perceptive identification of the Bush administration's ideological
definition of science.

I would welcome comments from Bhaskar list members about the narrowing of
science to a largely positivist construal of what counts as scientific
method.

Cheers,

Brad Rose

----- Original Message -----
From: "Michael Q. Patton" <MQPatton-AT-PRODIGY.NET>
To: <EVALTALK-AT-BAMA.UA.EDU>
Sent: Wednesday, August 06, 2003 12:22 AM
Subject: Re: 20 points extra for a randomized experimental design



>
> As you well know, crowning experiments as the holy grail of evaluation is
an
> ideological assertion not a scientific one.  It is asserted by those in
> power as a canon with all the rule-based fundamentalist fervor of religous
> zealots.  Those who assert experiments as superior to all other methods
> share with religious zealots a certainty in the rightness of their own
> rules, the absolute truth of their inherited and bounded wisdom, and
> evangelical intolerance for thinking, the ambiguities of the real world,
and
> the relativistic assertion that there is more than one right way to do
> something.  This latter they hate more than anything -- and I use the word
> hate advisedly, having discussed these matters with some of them at
length.
> Adulation of experiments as the only real source of truth is brought to us
> by the same people who are purging governement websites of any reseach
> findings, including experiment-based findings, that do not support their
> preconceived ideologies, so it's not as if these people are actually
> interested in empirical findings.
>
> This is not, by the way, a replay of the old paradigms debate.  That
debate
> was about the strengths and weaknesses of various approaches and led, as
you
> nicely summarized in your email, to "A basic principle of meaningful
> evaluation -- start with the evaluation purpose and question first, and
only
> then decide what method - or most likely methods - are most appropriate.
> Every method has its strengths and limitations; no single method is ever
> appropriate in all possible situations."  The problem with this approach
is
> that it requires thought and understanding -- and is philosophically
> relativistic.
>
> The current U.S. administration is not a government that tolerates debate.
> (Those who have and do oppose the Iraq war, or even want to debate its
> merits, are instantly branded traitors.)  Ideology and power are
intolerant
> of debate. This administration has attracted and appointed senior people
> intolerant of debate.   So the rule-makers simply assemble and assert
their
> narrow biases as policy.  Of course, all governments do this to some
extent,
> even to a great extent.  The difference at the moment is a matter of
degree.
>
> Evaluators, reflecting the panorama of human inclinations, count among our
> number a small but prestigious and influential group who share this belief
> in the holiness of experiments and have found common ground with the
> methodological zealots of the current administration.
>
> Hunker down in France, Burt.  It's not a pretty picture on this side of
the
> pond.
>
> Michael.
>
> Michael Q. Patton
> Union Institute & University
> Utilization-Focused Evaluation
> 3228  46th Ave. South
> Minneapolis, MN 55406
>
> MQPatton-AT-Prodigy.net
>
>



     --- from list bhaskar-AT-lists.village.virginia.edu ---