New humanist has an article by Wiseman, i guess as a promo for the book

Things that go bump in the night
Why do people think they can see ghosts, ghoulies and gods? Richard Wiseman explains

http://newhumanist.org.uk/2498/things-t ... -the-night

jerome wrote:

ersby wrote:

jerome wrote:Talking of Chris French, the superb APRU invited speaker podcasts which combine scepticism with the highest standards of research can be found here http://itunes.apple.com/gb/podcast/anom ... d394249539

Recommended!
j x

Interesting. I'll try and come along.

I'm planning to ask Matthew Smith of various ganzfeld experiments and jaytee fame over for an afternoon soon, or maybe just to give a talk. Perhaps we could all meet up?

j x

Definitely. Let's get him drunk and try to convince him he's psychic!

VK-machine wrote:

VK-machine wrote:Do I still need to dig up my notes and the paper? There's not much more to be said, I think, but maybe I'm misrecalling something.

Well?

Yes, that's sort of Betty Marwick's issue with it. Give me a few days to get back to that, as I need to get a copy of a JSPR from 2008, and write to her her and the authors to ask a few questions (plus I have rather a lot on.) Don't worry, I'll reply VK - your critique is spot on in terms of what has been suggested. If the authors did not get so upset and defensive we might have resolved whether it holds weight or more, but they are nice people so I'll chase it up

j x

jerome wrote:

VK-machine wrote:

VK-machine wrote:Do I still need to dig up my notes and the paper? There's not much more to be said, I think, but maybe I'm misrecalling something.

Well?

Yes, that's sort of Betty Marwick's issue with it. Give me a few days to get back to that, as I need to get a copy of a JSPR from 2008, and write to her her and the authors to ask a few questions (plus I have rather a lot on.) Don't worry, I'll reply VK - your critique is spot on in terms of what has been suggested. If the authors did not get so upset and defensive we might have resolved whether it holds weight or more, but they are nice people so I'll chase it up

j x

Lol. Whatever happened to "no explanation"?

The omission of the results as well as the way this fact is obscured suggests an intentional deception. Authors getting upset and defensive is quite consistent with that.

jerome wrote:As a book debunking the paranormal it is a failure

What, in your opinion, would count as 'debunking the paranormal'? What would the book have to achieve?

Over the ... er ... millennia there have been countless claims that paranormal events have happened. Clearly, no book could ever debunk more than a tiny subset of such claims - and, as I understand it, the book which you were reviewing does this: debunks a subset of paranormal claims.

Aren't you criticising the book for failing to achieve something which is, even in principle, unachievable? Or have you some clear idea of how the paranormal might be debunked? In order to satisfy you, what would the author of such a book have to show?

On Roy & Robertson:

There have been a few recent attempts to apply statistical tests to determine whether the accuracy of statements made by mental mediums exceeds that which would be expected by chance. Robertson and Roy (2004) report the results of eight experiments designed to eliminate cues such as body language of the part of sitters and expectancy effects. However, McCue (2004) has raised the point that Robertson and Roy’s statistical analysis used the individual statement as the unit of analysis, whereas statements cannot be treated as independent events (such as statements that the target person has an injured leg and that he uses a cane or crutch). Also, Robertson and Roy’s evidence could be interpreted as evidence of psi powers on the part of the medium rather than as evidence for a survival discarnate personality.

(http://www.newdualism.org/papers/D.Stok ... d-book.htm)

Maybe the McCue paper is of interest?

There also have been some discussions on it on the mind-energy-/Skeptiko-forum:

http://forum.mind-energy.net/skeptiko-p ... n#post6664
http://forum.mind-energy.net/skeptiko-p ... n#post9201
http://forum.mind-energy.net/scientific ... n#post9481

Dunno what to make of it, but maybe it is useful to you, Jerome!

Will S wrote:

jerome wrote:As a book debunking the paranormal it is a failure

What, in your opinion, would count as 'debunking the paranormal'? What would the book have to achieve?

Over the ... er ... millennia there have been countless claims that paranormal events have happened. Clearly, no book could ever debunk more than a tiny subset of such claims - and, as I understand it, the book which you were reviewing does this: debunks a subset of paranormal claims.

Aren't you criticising the book for failing to achieve something which is, even in principle, unachievable? Or have you some clear idea of how the paranormal might be debunked? In order to satisfy you, what would the author of such a book have to show?

I think Jerome's main criticism is not that the book fails to debunk every single paranormal event that has ever been claimed to occur, but rather than it fails to discuss the more convincing evidence in favour of its existence. So perhaps instead of discussing a talking weasel that only 3-4 people witnessed, his time might have been better spent discussing the psi experiments with, supposedly, a rigorously designed experimental protocol that still managed to achieve statistical significance.

Mr.Samsa wrote:I think Jerome's main criticism is not that the book fails to debunk every single paranormal event that has ever been claimed to occur, but rather than it fails to discuss the more convincing evidence in favour of its existence.

In that case, I hope that Jerome will tell us more. Does he think that there are studies which definitely prove the reality of the paranormal? Which are these? To understand them, do we need to read the published papers, or is it realistic to summarise their findings in a forum like this?

Another important consideration is: how recent are the studies? If he could point to work published, say 20 years ago, which has never been seriously challenged, or (better still!) has been replicated several times by later researchers, then that would be very impressive.

Will S wrote:

Mr.Samsa wrote:I think Jerome's main criticism is not that the book fails to debunk every single paranormal event that has ever been claimed to occur, but rather than it fails to discuss the more convincing evidence in favour of its existence.

In that case, I hope that Jerome will tell us more. Does he think that there are studies which definitely prove the reality of the paranormal? Which are these? To understand them, do we need to read the published papers, or is it realistic to summarise their findings in a forum like this?

Look at it from a PR perspective. His review suggests to the reader that there is some real good evidence out there that Wiseman ignores.
In truth the papers he mentions are a mixture of massive incompetence and borderline fraud. If one were to mention these in a book then they would damn parapsychology. Surely something that Jerome would not like either.

The argument is pretty common. Believers know that the evidence is out there. If a skeptic shows up something as not convincing or even stupid then, of course, she just hasn't looked in the right place.
In Jerome's case the argument seems more like intentional PR than the honest misunderstanding (or delusion) of the average believer. He has demonstrated in this thread that he cannot be trusted to give the whole truth and not expected to notice obvious errors.

Another important consideration is: how recent are the studies? If he could point to work published, say 20 years ago, which has never been seriously challenged, or (better still!) has been replicated several times by later researchers, then that would be very impressive.

Crappy research often goes unchallenged. Rebuttals take time and effort which is a waste when the wrongness is obvious.
In parapsychology you'll find a lot of miracle stories that have gone completely unchallenged. Stories, for example, of people who won the lottery or the casino...

VK-machine wrote:Crappy research often goes unchallenged. Rebuttals take time and effort which is a waste when the wrongness is obvious.
In parapsychology you'll find a lot of miracle stories that have gone completely unchallenged. Stories, for example, of people who won the lottery or the casino...

Fair comment, I think.

I'm increasingly interested in the question of what would count as 'debunking the paranormal'? What would the debunker have to achieve?

Some time ago, I OP'd another topic on this subject, http://www.rationalskepticism.org/paran ... ml#p398564 using a very good article in 'The Oxford Companion to the Mind' as the starting point. I think it's fair to say that the author's opinion on the subject wasn't refuted, or even seriously challenged: paranormal phenomena are characterised in such a way that you can't debunk them. The sceptic is fighting a Hydra, and every time he chops off a head, two more grow.

That's why I'd like to hear more from Jerome concerning his criticism of Wiseman's book:

Jerome wrote:As a book debunking the paranormal it is a failure

What does he think Wiseman should have done?

VK-machine wrote:
In truth the papers he mentions are a mixture of massive incompetence and borderline fraud.

In Jerome's case the argument seems more like intentional PR than the honest misunderstanding (or delusion) of the average believer.

These are very bold comments and I suggest that you better back them up with good evidence (esp. the claim that Jerome conducts intentional PR and isn't honest). There may be flaws in the study and Jerome may have missed something, but I don't think your comments are fair!

sir_gannayev wrote:

VK-machine wrote:
In truth the papers he mentions are a mixture of massive incompetence and borderline fraud.

In Jerome's case the argument seems more like intentional PR than the honest misunderstanding (or delusion) of the average believer.

These are very bold comments and I suggest that you better back them up with good evidence (esp. the claim that Jerome conducts intentional PR and isn't honest). There may be flaws in the study and Jerome may have missed something, but I don't think your comments are fair!

I said it seems more like intentional PR. I don't know if it is intentional. The point is simply that if someone were doing it intentionally it would look quite similar or even identical.
Why don't you quote the next sentence, too, where I point to his behavior in this thread? First he talked up the Robertson/Roy papers, telling us how he has searched for an answer for years. How he has solicited the help of others, real statisticians even without any hint of an explanation.
And suddenly, whoom, it's a 180. As soon as he realizes that some real criticism is coming his way, he knew it all along.


As far as borderline fraud in parapsychological papers is concerned, may I ask what your idea of proper scientific conduct is?

sir_gannayev wrote:On Roy & Robertson:

There have been a few recent attempts to apply statistical tests to determine whether the accuracy of statements made by mental mediums exceeds that which would be expected by chance. Robertson and Roy (2004) report the results of eight experiments designed to eliminate cues such as body language of the part of sitters and expectancy effects. However, McCue (2004) has raised the point that Robertson and Roy’s statistical analysis used the individual statement as the unit of analysis, whereas statements cannot be treated as independent events (such as statements that the target person has an injured leg and that he uses a cane or crutch). Also, Robertson and Roy’s evidence could be interpreted as evidence of psi powers on the part of the medium rather than as evidence for a survival discarnate personality.

(http://www.newdualism.org/papers/D.Stok ... d-book.htm)

Maybe the McCue paper is of interest?

Absolutely. I have a McCue paper on ghosts from about then (he cites one of my papers so I was pleased with it, lol!) but I don't know the paper referenced here. Bizarrely it's not in the references for the article by Stokes. Normally I'd fire up LEXSCIEN but my membership has temporarily run out, so until I can pay to resubscribe I'm not sure which McCue paper it is. I'll ask Sunchime to look it up for me, but thanks Sir Gannyev, that is extremely useful!

sir_gannayev wrote:
There also have been some discussions on it on the mind-energy-/Skeptiko-forum:

http://forum.mind-energy.net/skeptiko-p ... n#post6664
http://forum.mind-energy.net/skeptiko-p ... n#post9201
http://forum.mind-energy.net/scientific ... n#post9481

Dunno what to make of it, but maybe it is useful to you, Jerome!

Once again I am in your debt. A friend has pointed out to me Betty Markwick's critique, which I need to re-read, but I have not so far been able to access the JSPR for the same reason. That's why I have not been replying for a couple of days (well that and a rather annoying and unpleasant illness, and being busy with the Parnormality review etc).

I'll play catch up this week Thanks for all your help Sir Gannyev!

j x

VK-machine wrote:
The omission of the results as well as the way this fact is obscured suggests an intentional deception. Authors getting upset and defensive is quite consistent with that.

Absolutely. No argument there. It can however have other explanations i expect, but generally I find it the case. However until I can access LEXSCIEN I'm a bit buggered to discuss it properly. If you have access turn to the jSPR for 2008 and have a look through the letters, where the correspondence occurred.

As i recall, and memory is extremely dodgy as we all know, there was a discussion about the suitability of the distribution used. No statistician sees to think it should make any difference to the outcomes - but it was an odd choice according to some, though another psychologist replied as I recall and and said it was a textbook example, as in the textbooks say it is perfectly appropraite use of the binomial distribution. This however is all from memory. Still it does not effect on the outcomes. As i noted, LEXSCIEN cuts off in 2008 (it may have been expanded since I last has access as more journals are scanned) so I'm missing the last parts of the discussion, if any further discussion occurred.

When I asked on the JREF I received no replies, but I did speak to the ever sensible fls over there, who raised a point about the results. I think this is that as you say there are a lot of different things tested in the two year set of trials. This was not a case of the results being subjected to post-hoc evaluation for positives, as so often in bad research; it was built in to the design, but it certainly makes it complex to assess what was found. Irwin & Watt (2007) cover it very briefly in http://www.amazon.co.uk/Introduction-Pa ... 0786430591 but they merely note the "complexity" of the results as a negative in mdesigning mediumship protocols, nothing more, nothing less. (They incidentally do mention O'Keeffe, C. & Wiseman, R. (2005). Testing alleged mediumship: Methods and results. The British Journal of Psychology, 96(2), 165-179. -- http://www.richardwiseman.com/resources/MediumBJP.pdf -- an excellent design which gave negative results, well worth reading).

I'll happily discuss the various ways the data was tested, and the trials; but I want to check the Skeptico threads and try to find what McCue wrote. So far I still have not found anything much, Sir Gannyev is doing far better than I am - so if you can find a published refutation with a full description of the methodological errors, please publish it here, or write one.

j x

jerome wrote:

VK-machine wrote:I'll happily discuss the various ways the data was tested, and the trials; but I want to check the Skeptico threads and try to find what McCue wrote. So far I still have not found anything much, Sir Gannyev is doing far better than I am - so if you can find a published refutation with a full description of the methodological errors, please publish it here, or write one.

I'll write one but note that besides the omission of the results the other issues pale into insignificance.

Don't expect too much from the skeptico forum. The owner keeps skeptics on a short leash which does nothing for the quality of the criticisms.

A little googling finds a short summary of the Marwick critique:
Another study (Robertson and Roy, 2004) also involved a
double-blind method (Roy and Robertson, 2001) in which the target
sitters were chosen randomly from a group and the medium gave the
reading from another room, isolated from the group. There were 10
mediums and 300 participants. Unfortunately, instead of conducting
a single study or series of studies using the same procedure, the
investigators conducted 13 different experimental sessions in which
several different experimental designs were implemented. Moreover,
the results were evaluated by a complicated statistical analysis
of probability values given to individual statements. Although the
authors reported highly significant results, the statistical methods
have been strongly criticized (Markwick, 2007). Perhaps more
disturbingly, however, because the investigators did not report
separately the results of different experiments or experimental conditions,
it is impossible to determine the actual results and compare
those trials with more stringent conditions and those with less
stringent or otherwise different conditions.
in
http://www.newdualism.org/nde-papers/Ke ... -11-17.pdf

Will S wrote:

jerome wrote:As a book debunking the paranormal it is a failure

What, in your opinion, would count as 'debunking the paranormal'? What would the book have to achieve?

You don't need to debunk the paranormal. I don't think the notion of "paranormal" is coherent anyway (see endless previous discussions, I'll link some if you like.) My issue is not that Wiseman does not manage to address every paranormal claim ever made - though he certainly gives the impression he has throughout the book. What surprised and disappointed me was that while he offers some research, he largely ignores all data that is inconsistent with his hypotheses, and ore importantly all the experimental evidence. As I said, read his partners (with Irwin) Caroline Watt's book http://www.amazon.co.uk/Introduction-Pa ... skept01-21 - it's well worth the money, and see how much of the academic literature is addressed? Wiseman does deal with a huge range of targets in a popular accessible style, but he presents an astonishingly one sided view of an area he knows well. I am an academic by nature: I strive for balance; I find something as "evangelical" as this distasteful, not least because I know full well Richard is actually extremely aware of the literature he is ignoring.

I'm going to make a bold claim, I first raised on the JREF: "The strangest point was made by my ex-wife, who notes with interest that in the UK there is much more money to be made by sceptical books (think Dawkins, Wiseman, and the excellent Shermer & especially Ben Goldacre) than by the woo stuff by the likes of Derek Acorah and the psychics, and very little to be made on the academic parapsychology books, that sell so few copies as to be marginal at best. I think this is abundantly true, as Paranormality has been in the Amazon top ten much of the week, and Wiseman received a five figure fee for it. Good for him! This however may be in marked contrast to the USA - but Sceptics in the Pub in my home town is getting at least ten times the number of people who attend the local psychic research group meetings, and the market sector seems prosperous.Hey, I may have to write a really woo book denouncing sceptics fleecing unbelievers for their cash, lol! Problem is it wouldn't sell!"

Will S wrote:
Over the ... er ... millennia there have been countless claims that paranormal events have happened. Clearly, no book could ever debunk more than a tiny subset of such claims - and, as I understand it, the book which you were reviewing does this: debunks a subset of paranormal claims.

Yes it does; but it claims to show the whole paranormal is bunk,and down to Hyper sensitive agency detection, and a handful of other theories from Anomalistic Psychology. As I said in my Amazon review " a great book; but not what it says on the tin". Paranormality is a blistering popular science attack on superstition - but in very few of the things he targets does he actually address the contrary academic literature - read his book and his partners, and you will immediately see my frustration. And often his conclusions run far beyond what he offers: I think for example he is right to dismiss recording theories of ghosts, but I would disagree with each and every one of the grounds he does so on. (So as so often, we agree in conclusion, disagree on how that conclusion is reached.) A while ago we saw headlines saying "Psychic tweeters theory is strictly for the birds" etc etc -- http://www.scotsman.com/news/Psychic-tw ... 6705893.jp - and numerous discussions of a Wiseman Watt experiment that was hailed as "disproving ESP" by some -- yet I have just read the experiment in the latest EJP, a proof of principle test, and in fact while the results were negative that in itself does not justify these strong claims made, as Wiseman and Watt are very clear about in their paper.

Will S wrote:
Aren't you criticising the book for failing to achieve something which is, even in principle, unachievable? Or have you some clear idea of how the paranormal might be debunked? In order to satisfy you, what would the author of such a book have to show?

[/quote][/quote]

Simply addressed those academic peer reviewed experiments that appear to stand contrary to the main point of the book, and the "paranormal can now safely be put aside" argument thereof. Wiseman the parapsychologist appears a very different fellow to Wiseman the pop author: and I think the answer lies in nothing more sinister than market forces, his agent, and his publishers wish to capitalise on the UK sceptics market, now a multi-million pound one, as opposed to the rather less exciting prospects for a decent academic text like his partners?

j x

VK-machine wrote:


A little googling finds a short summary of the Marwick critique:
Another study (Robertson and Roy, 2004) also involved a
double-blind method (Roy and Robertson, 2001) in which the target
sitters were chosen randomly from a group and the medium gave the
reading from another room, isolated from the group. There were 10
mediums and 300 participants. Unfortunately, instead of conducting
a single study or series of studies using the same procedure, the
investigators conducted 13 different experimental sessions in which
several different experimental designs were implemented. Moreover,
the results were evaluated by a complicated statistical analysis
of probability values given to individual statements. Although the
authors reported highly significant results, the statistical methods
have been strongly criticized (Markwick, 2007). Perhaps more
disturbingly, however, because the investigators did not report
separately the results of different experiments or experimental conditions,
it is impossible to determine the actual results and compare
those trials with more stringent conditions and those with less
stringent or otherwise different conditions.
in
http://www.newdualism.org/nde-papers/Ke ... -11-17.pdf

That is excellent. So what we need is the raw data? I don't think Betty Markwick's criticism was that strong: but the failure to report the individual results would be bizarre, and highly suggestive something is being concealed. Cheers VK, I'm on it. I'm going to have a very detailed look again at the way the results are reported. As usual it may take a day or two, but like toothache I will am persistent, and will be back. My hackles were raised not by the statistical claims, but by the authors defensive attitudes in the discussions -- so I am going to look very closely to see how they present their results now. If you find more, do share! Is Ersby about? has he had a look at this?

j x

VK-machine wrote:
Look at it from a PR perspective. His review suggests to the reader that there is some real good evidence out there that Wiseman ignores.

Yes, as I keep saying, you can look at Dr Watt's book for a rapid overview?

VK-machine wrote:
In truth the papers he mentions are a mixture of massive incompetence and borderline fraud. If one were to mention these in a book then they would damn parapsychology. Surely something that Jerome would not like either.

Strangely that is not the opinion of anyone I have met who has actually read the research; you have raised an interesting point about the presentation of results in one paper; yet as even noted sceptics have admitted, parapsychology does have a cumulative database. You seem terribly opposed to the whole endeavour: but what for example do you make of Ersby's dedicated work on the Ganzfeld meta analysis?

VK-machine wrote:
The argument is pretty common. Believers know that the evidence is out there. If a skeptic shows up something as not convincing or even stupid then, of course, she just hasn't looked in the right place.
In Jerome's case the argument seems more like intentional PR than the honest misunderstanding (or delusion) of the average believer. He has demonstrated in this thread that he cannot be trusted to give the whole truth and not expected to notice obvious errors.

No one has ever shown that Robertson/Roy is as fatally flawed as you propose ; the critique is that it is overly complex as a design as i understand it. Until I have re-read the paper I won't comment further, simply because i don';t know, I may be wrong and you may be right - I'm often wrong, and freely admit it. I don't know that the evidence is out there: I know the papers that present the evidence. There is evidence for and against the psi hypothesis, as with most things. (I don't personally hold the psi hypothesis as it happens, but that is another story.) What I do know ois there ais a tremendous body of well researched material you appear to be intent on dismissing as "massive incompetence and borderline fraud", something that would certainly cause you a row with even arch sceptic Ray Hyman, let alone Richard Wiseman, who both have considerable respect for the methodological diligence of the parapsychological community?

VK-machine wrote:

Another important consideration is: how recent are the studies? If he could point to work published, say 20 years ago, which has never been seriously challenged, or (better still!) has been replicated several times by later researchers, then that would be very impressive.

Crappy research often goes unchallenged. Rebuttals take time and effort which is a waste when the wrongness is obvious.
In parapsychology you'll find a lot of miracle stories that have gone completely unchallenged. Stories, for example, of people who won the lottery or the casino...

[/quote]

The ganzfeld?

j x

VK-machine wrote:
And suddenly, whoom, it's a 180. As soon as he realizes that some real criticism is coming his way, he knew it all along.

Er hardly; I mentioned the JSPR critiques by Markwick, that I thought they were weak, and my efforts to find a decent discussion of this anywhere, and am happy to discuss it, but I have not yet changed my views strongly -- I'm just accepting you may have made a fir critique, so being an open minded sort of chap I'm going to check it out! I admitted "Don't worry, I'll reply VK - your critique is spot on in terms of what has been suggested. " - meaning you raise the same points; that hardly constitutes a 180!

j x

Robertson and Roy wrote 3 papers on mediumship. 2 were published in 2001 and 1 in 2004, all in JSPR.

#1

A PRELIMINARY STUDY OF THE ACCEPTANCE BY NON-
RECIPIENTS OF MEDIUMS' STATEMENTS TO RECIPIENTS
T. J. ROBERTSON and ARCHIE E. ROY
ABSTRACT
A test was made of the sceptical hypothesis that the statements made by mediums to recipients are so
general that they could as readily be accepted by non-recipients. A two-year study involving 10
mediums, 44 recipients and 407 non-recipients ostensibly falsified that hypothesis. The average
fraction of the set of statements accepted by the recipient was significantly larger than the average
fraction of the same set of statements accepted by non-recipients, the probability of the results
being due to chance being 5.37 χ 10-". Details are given of the procedure of data collection and
analysis, and an objective method of weighting the statements is described. A number of non-paranormal
factors are listed and assessed as possible reasons for the seeming falsification of the hypothesis.

There are a number of books on "cold reading", which is the art of faking knowledge when you have
none. There are also a number of accounts of how mediums obtain information about their clients to
perform so called "hot readings".

The most important factor in such "cold readings" is confirmation bias. This is the human tendency to
look for information that is in accordance with a view, rather than information that contradicts it.
One way in which this shows itself is as the Forer effect which is in skeptical circles often called
Barnum effect. This is shown by giving a number of people each the same text and then telling them
that it is a description of their personality. If the text is suitable most of the subject will
endorse it as an accurate description of themselves without realizing how widely it is endorsed.

This is probably how Robertson and Roy arrived at their "sceptical hypothesis" but it must be
remembered that this is only one factor. Most importantly, a real life medium should be expected to
"hot read".

The way that the 2 set about to test this hypothesis was by gathering a number of subjects and having
a medium perform a reading on 1 of them. This reading was then broken down into individual statements
of fact.
The subjects were then asked if each of these individual items was true for them or not.

Unsurprisingly the intended recipients endorsed more items than the other subjects. Of course, they
knew that they were supposed to endorse many items, while the others knew they were not to.
The results of this experiment tell us nothing that we didn't already know.
In particular we are unable to draw any conclusions about the validity of Robertson and Roy's "skeptical hypothesis".
In short, the paper is a waste of tree.


#2

A Double-Blind Procedure for Assessing a Medium's Statements
by ARCHIE E. ROY and T. J. ROBERTSON
ABSTRACT
In a previous study, it was shown that a significantly higher percentage of a set of statements given
by a medium to a recipient was accepted by the recipient compared with the percentage of the same set
accepted by non-recipients. A number of non-paranormal factors were identified that might diminish the
large gap between the acceptability levels of recipients and non-recipients.
In the present paper a hard protocol is described that may be used to assess the effect of each factor
separately. The protocol's single, double and triple blind nature in testing each factor is discussed.

Aparently the failure of the first paper prompted Robertson and Roy to try and make amends.
There is no point in critiquing the proposed protocol as they eventually implemented an improved
version but one must certainly commend their effort in correcting their earlier errors.


#3

RESULTS OF THE APPLICATION OF THE ROBERTSON-ROY
PROTOCOL TO A SERIES OF EXPERIMENTS
WITH MEDIUMS AND PARTICIPANTS
by T. J. Robertson and A. E. Roy
ABSTRACT
This paper is the third in a series of papers by Robertson and Roy that together describe and test a
method of assessing claims of mediumistic communication. In this paper we describe the results
obtained by applying the Robertson-Roy Protocol (RRP) in a designed suite of experiments that enables
in each experiment (a) the categories (such as a recipient who believes he or she is a non-recipient)
of all participants present to be unambiguously determined, (b) the operation of a variety of normal
factors (such as body language and verbal response to a medium) to be controlled. The RRP was tested
over two and a half years in a study involving 13 sessions held in a number of locations in England
and Scotland, with some 300 participants from a wide variety of cultural backgrounds. Ten mediums
delivered 73 sets of statements during these sessions. The study demonstrated that the RRP, although
time-consuming both in application and reduction of acquired data, is a practical, repeatable and
useful procedure in assessing the ability of mediums to transmit relevant information to recipients.
The results of the study provided a reliable, and objective, quantitative measure of the significance
to be placed in the higher fraction accepted by the recipients of the number of statements in the sets
delivered to the recipients than those accepted by non-recipients in those sessions. Due to the design
of the experiments the results cannot be due to normal factors such as body language and verbal
response. The probability that the results are due to chance is one in a million. The evaluation by
the Robertson-Roy weighting procedure of the statements delivered by the mediums is also shown to
support the negation of a sceptical hypothesis.

Unfortunately, the abstract is highly misleading, even outright deceptive.

Rather than just reporting the results of the "rigorously blinded" protocol the paper also contains a
number of experiment that vary the protocol.
What they all have in common is that one has a room full of subjects who all rate a reading. Ideally, this would be done under certain strict controls. That is what Robertson and Roy promised in their 2nd paper and it is what they did in some experiments.

But perhaps it is best to mention a high point first. In one variation, the experimenter misleads the
subjects about for who the reading is intended.
As one would expect, this subject then endorses the reading overall as fitting while the others don't.
While this is the expected result, it is nice to have confirmation like that, coming from people who
are biased against such an effect. Effectively this completely debunks their first paper, although they don't point that
out. In fact, they mention this result only in passing e and only only to talk it down, yet we should be thankful they don't cover it up.

Basically, whenever the subjects know which of them is the recipient, he or she will tend to endorse
the reading as fitting while the others will reject it.
Now take a number of such experiments. Then take a number of experiments where everyone has endorsed
the reading equally. Pool them and ask if the intended recipients, on average, endorsed the reading
more than the others.
The answer will be yes and that's the explanation for the "results" given in the abstract.

I could critique each experimental design in the paper seperately but that won't be necessary. Some
are open to the previously mentioned effect, others not. Some of those have their own problems but
that's not important now.

One would hope that the paper contains a table with the results of each individual type of experiment
but no. One would hope that it at least contains the results of the much hyped double-blind
experiments but no again.
We are given results for only a few types. We are also given some pooled results where disparate
experiments were thrown together. The analysis of such pooled results takes up much of the paper and
is completely pointless. It would be tempting to rant on how nonsensical it all is but I would only
bore you, dear reader, without ever managing to convey the full scale of it.