|
Of course no single test, or in the case of the GCP data and
analysis, no single event can prove or discount a proposed
hypothesis. But it came to me that the Glenn Beck Rally might be
an opportunity to test the "Experimenter Effect," in a
half-joking way. (Note: this is an exploration, not a formal
event.) I wouldn't
have thought more about the infamous rally Beck set up for
the steps of Lincoln Memorial on the anniversary of Martin
Luther King's "I have a dream" speech. But Mike Breland
brought it up, with a tone of distaste in his email voice.
And I responded that I guess we could take a look, if I
could find the event timing, gritting my teeth, or, rather,
suppressing my gorge. It was all too easy to find the times
-- 11 to 1 pm on Saturday the 28th of August.
So, here we
are. The prediction, based on the notion of an Experimenter
Effect, would naturally be, ta ta ... No effect. I would
personally hate to see the GCP data rocketing off into
significance with persistent increased variance, so of
course any experimenter effect I am
conscious of would suppress that tendency. And what do we
get?
Well, it is either ambiguous or evidence for a
moderately successful suppression. As a person
who's basically disgusted by Beck, the most pleasing or
satisfying result would be a flat trend, no sign of an
effect. But the result isn't exactly that. For the first
hour there is a persistent decrease of variance, of
sufficient magnitude that it is unlikely at about the 5%
level, or would be if our prediction was for reduced rather
than increased variance, and if the event were just one hour
instead of three. (The observant will recognize these "ifs"
as the reason event specifications must be made a priori.) If that trend were to continue for
the full 3 hours it would be impressive indeed, and might be
interpreted as an effect on the network. But it did not
continue, and in the end, the event has a non-significant
outcome of about Z = -0.777. The result is neither here nor
there, and doesn't amount to much as a test of the
Experimenter Effect. It also doesn't amount to much as an
indication of interest in Beck on the part of Global
Consciousness. That's no surprise to me, because the only
interest I can muster is the sort that compels people to
look at accident scenes. It's not something I can do
for very long.
It is important to keep in mind that we have only a tiny
statistical
effect, so that it is always hard to distinguish signal from
noise. This means that every "success" might be largely
driven by chance, and every "null" might include a real
signal overwhelmed by noise. In the long run, a real effect
can
be identified only by patiently accumulating replications of
similar analyses.
|