Would love to see some discussions on these topics - I recently rejoined the SPR (stuff on my blog here - http://jerome23.wordpress.com/2010/01/2 ... -research/ ) and have been involved in the field for twenty years now. We had a lively debate going on the old forum about evidence for ESP, and in particular the ganzfeld experiments - if anyone wants ot pick it up, I'd be very happy to participate! (See i'm not just a religious woo...)

j x

What are your opinions on the validity of the Ganzfeld research versus the autoGanzfeld research?

I had read about it years ago, but I was refreshing my memory last week when I was going to reply to something you had said on RDF and with my increase of knowledge of science methodology since my last reading of the research I was pretty appalled at the design of the Ganzfeld tests. I came to the conclusion that they could only, at best, be used as indicators of something for further research but should never be used as evidence for or against a claim.

Is this in line with your thinking?

(I also have my reservations about the autoGanzfeld methodology, but that's for later).

I have never actually seen a pre-Ganzfeld set up run: obviously I have seen a few auto-Ganzfeld sessions. As far as i know the only difference is in the automation of the targets yes? Berger (1983?) The randomization issue was at the heart of the Blackmore/Sargent row: I have spoken to people on both sides and am no clearer to be honest. After the Hyman & Honorton joint communique ( I posted part or all of it on the old forum as I recall) in 1986 autoGanzfeld was standard, but the PRL trials were fully automated and double blind before, as were a number of others. The crucial aspect is that when you use measure of experimental quality, it makes no difference to the results: this is the mystery really. The decline effect would be explicable if it cut in after the introduction of automation: it didn't. The mystery persists to this day, simply because no single cause has yet been found that would explain the results in terms of methodological errors or file draw issues - which is LONG LONG way from saying one does not exist, and why I encourage everyone ot take a look at the actual methodology and stats and look for an explanation. I'll ask Ersby over - he really si the best expert (and a hard sceptic by nature) on this issue.

"Hard skeptic"? Hmm, another label I don't really understand.

But as for the ganzfeld versus autoganzfeld, I'm not sure what question is being asked. There was no sudden drop in results when the new protocol was adopted (the autoganzfeld only became widespread after 1990, when the PRL results were all published). But I think there was a drop when the autoganzfeld experiments went from storing targets digitally rather than on tapes - I'd have to check, though. And there are still a few experiments being carried out which, strictly speaking, aren't autoganzfeld - the ones completed in South America - and these tend to be very successful.

But as I said - I'm not sure what the question is.

You bother to check the facts, unlike most sceptics, and you discuss this on the JREF - so you are hard!

j x

Oh, right. I thought you meant "hard" as in "difficult" or "unyeilding".

Not in the slightest. Your 7 part file is the best summary I have yet seen of the state of play of the ganzfeld.

j x

Well, this appears to be the most methodologically sound Ganzfeld experiment to date:

http://www.anomalistik.de/images/stories/pdf/sdm/ejpv22-1_puetz%20et%20al.pdf

Still getting the same approximate ~30% hit rate.

Darn, that looks like it takes care of the randomization problem, sensory leakage, and the VHS tape wear problems. Dog-gone-it, what could be possibly causing those pesky consistent, above-chance hit rates?

It can't possibly be the experimental hypothesis they are testing for. Psi is impossible. It MUST be something else.

Well, first off, it's not the most methodoligically sound ganzfeld experiment to date. I'd say that since the introduction of computers to store/select/display targets (around the year 2000) there's hardly been a hair's breadth between the different labs in Europe and the US in terms of rigour.

The problem with that paper is that it goes against the debate over the ganzfeld that played out during the 1990s, ie, the concept of "standardness". It is a covert psi experiment, and back in the 1980s there was a similar experiment by Raburn whose negative results were explained away because the subject didn't know they were trying to do. If Raburn was dismissed by parapsychologists on these grounds, then shouldn't this one be dismissed too?

There's also a minor problem of statistical significance. Its p number is 0.03, but the sum of ranks was insignificant. If there are two measures (direct hit and sum of ranks), shouldn't the level of significance be adjusted from p=0.05 to something lower?

But apart from that, it is kind of suggestive of psi, I suppose.

Ah, I get what you are saying about standardness. I don't think ANY data should ever be dismissed; unless it is proven fraud.

The difference in results between the Raburn experiment and the Wackermann experiment could mean many different things. It could support the null hypothesis. It could also mean that there were any number of different variables between the two studies; you are dealing with two entirely different groups of people. The Ganzfeld experiments do not have 100% of the variables nailed down, and this is in no way unique to parapsychology. Good ol' regular psychology has to deal with this problem all the time. But this is why parapsychology is a true science: it has a body of knowledge that is continually expanding, and we're honing in on what works and what doesn't work with a Ganzfeld experiment. If there were no psi effect, I think it is much less likely that we would have found a pretty stable relationship with the following variables over time:

1. Warm, comfortable, inviting experimental atmosphere (as opposed to clinical)
2. People that are open to the belief in psi
3. Practicing meditation
4. Mental quiescence
5. Creativity (some form of it, what is interesting is that new research has isolated artistic talent and controlled for all else {you can't do this with a pseudoscience} and found that that specific variable does not correlate with psi skill)
6. Strong, emotional attachment between sender/receiver pairs
7. Dynamic visual targets with emotional component

As for the statistical analysis, I know exactly what you're talking about, I don't think it's a problem, because it's technically two different hypothesis that are being tested, the first is the basic Ganzfeld correct target hit effect, the overall hit rate of the primary target is going to be higher than expected by chance. The second (I think) is looking to see if any other form of information is being transmitted. I think this avoids familywise error inflation, one could argue that they should have used ANOVA, but I think they have a valid case for looking at it with a hierarchical approach, with the simple total hit rate being the primary factor.

Interesting study, but I have a couple of questions (hopefully someone knows the answers or can help me find them).

1) How were the target videos selected? No mention is made of the pseudo-random generator used in the experiment.

2) Why was subject A required to give a verbal report to EA before the rating session? If the videos weren't randomly selected, or if there was some communication between the two experimenters when signalling the end of phases, then the experimenter could give clues as to whether they were on the right track or not.

3) Why was subject A taken the same room as where subject B viewed the target video? Surely this just introduces unnecessary potential contamination?

4) Have any experiments been done where subject B does not view the target video but one is still selected from the set of 4? This seems to me to be an easy way of separating statistical anomalies from 'true' psi, since even though the theoretical probability of a subject selecting a given video out of 4 is .25, the actual probability will differ according to certain biases (i.e. some subjects picking the top option more than the other 3 when rating them, etc.).

I did like the conclusion at the end though where it suggested that the research could be used as evidence against the sheep-goat effect.

ersby wrote:
There's also a minor problem of statistical significance. Its p number is 0.03, but the sum of ranks was insignificant. If there are two measures (direct hit and sum of ranks), shouldn't the level of significance be adjusted from p=0.05 to something lower?

Yeah it should.

The problem with creating a list of psi-conducive protocols is that you can only work this out by looking at the methods of the papers and finding what was done during successful ones. This creates a bit of circular logic as people then claim these protocols are associated with high hit rates. Of course they are. That’s why they were chosen in the first place.

Nucleic Acid’s list of seven is pretty familiar (except “mental quiescence” which I’m not sure I understand) but how many of those need to be fulfilled? If one or two aren’t followed, how much difference would that make? How many need to be in place for an experiment to count as “standard”?

1) Strangely, they don't mention it. They were really thorough about everything else, so they either forgot (comparatively a minor gaffe), or decided it wasn't a hugely important factor. Either way, given the number of trials, it is not going to affect the results, especially considering the measures they took to make sure there were no selection biases by the subjects.

2) That's how the Ganzfeld procedure works. The subject A talks out loud what they are visualizing. The issue with communication and sensory leakage was already addressed, along with basically every Ganzfeld debate since the 80's. Ea is totally blind to the target.

3) It sounds like it was just equipment and space limitation. I don't know exactly why they chose to do it like this, but I'm guessing they only had one computer with the ganzfeld playback and selection software on it. I agree that it is sloppy, but if done right, it shouldn't incur any sensory leakage.

4) Yes, there have been several attempts to differentiate telepathy (person to person) from clairvoyance (person just receives), and I don't have any of those handy. But the bias issue you mentioned is well known and typically controlled for as a default.

As for the sheep-goat effect, yes and no. If you read more carefully you'd see it's not evidence against the sheep-goat effect. The whole point of the study is they are trying to tease apart the variables of whether belief is a necessary factor, or whether the subject believes they can perform a psi task. We already know that if you take someone who believes psi is bunk, and give them a psi task, they will not do as well. This experiments tests whether that is an intrinsic part of them (e.g. CAN skeptics do psi at all?), or whether the ACT of disbelieving causes the effect. Psi has long been hypothesized to be a subconscious process, therefore we would expect to see it, so long as the subject isn't consciously surpressing it.

As for the statistical significance, yes and no.

Of main interest was the performance of the participants in terms of target identification. All other reported statistics were post-hoc analyses.

I am in my stats class right now, and I just asked about this. Basically, if you wanted to look at ALL the independent variables (hit rate, ranking, etc) you would use ANOVA, which automatically corrects for familywise error (which does not affect the p value at all, it changes degrees of freedom). However, the experimenters are not using the other independent variables for their hypothesis. If the overall hit rate were insignificant but the order effects were significant, it would not support the main hypothesis, but it would be something interesting to look at further.

The problem with creating a list of psi-conducive protocols is that you can only work this out by looking at the methods of the papers and finding what was done during successful ones. This creates a bit of circular logic as people then claim these protocols are associated with high hit rates. Of course they are. That’s why they were chosen in the first place.

I've heard a lot of very bright scientists make this statement, which just goes to show how pervasive myths are in the field. The protocols are based on directional hypotheses that were formulated way before the Ganzfeld was created, based on a basic understanding of psi theory. These were explored in small scale studies, and then later replicated. This is NO different than any other science. The whole reason the Ganzfeld was created was because psi researchers already figured out that dreamlike states are conducive to psi. They said, "Well, we have found that dream telepathy works really well. I wonder if the Ganzfeld state would work just as well for telepathy. Let's test and find out."

It is not retrospective "looking at what works." (which is fine for determining a new way to go, so long as you run NEW tests to test for that specifically)

Mental quiescence is having a "quiet mind," e.g. not in a distracted, stressed, or worrying mind state.

Independent variables are not something to be "fulfilled." They are interaction effects, and they all have (at least theoretically) some effect on the dependent variable. The more you include, the more of the variance in the dependent variable you can explain. As far as what constitutes an experiment being "standard," well obviously you need the Ganzfeld state, which almost universally encompasses a comfortable atmosphere, pleasant sensory homogeneity, and the encouragement by the experimenters to relax and empty the mind of trivial things.

Belief in psi (or rather the negative effect of disbelief), meditation practice, creativity, dynamic/static, and emotional connection are all other independent variables that you can test for within the general Ganzfeld architecture. Or ANY psi task, for that matter. If psi in all these different types of experiments is all the same phenomena, then you would expect to see continuity of most of these variables across experimental conditions. Which you do. Oh, look, science.

NucleicAcid wrote:1) Strangely, they don't mention it.

The link's blocked at work, but as I recall they reference another paper for a detailed methodology. They used the computer's RNG to chose targets. I'd need to check, though.

I've heard a lot of very bright scientists make this statement, which just goes to show how pervasive myths are in the field. The protocols are based on directional hypotheses that were formulated way before the Ganzfeld was created, based on a basic understanding of psi theory.

These protocols are from the dream ESP studies? The Maimonides ones? I'm not very well read on that. Do you have a reference?

My understanding was that what was/is considered "standard" in ganzfeld experiments was based on the Joint Communique in 1985 and the annotated debated in 1999.

Sort of. You're totally right, the modern Ganzfeld standard procedure was crystallized in the Joint Communique, which is built on decades of other experiments and research (the paper I showed is obviously a deviation from the standard, but hey, scientists get bored too )

The IDEA of using the Ganzfeld state was developed because dream telepathy (the Maimonides ones and others, I think) worked well, and the Ganzfeld-induced state is similar to dreaming. It's mentioned in Entangled Minds by Dean Radin.

The protocols of separate rooms, noise isolation, CCTV, blind experimenters, etc is all from the evolution of parapsychology, from a guy holding up a Zener card, to where it is at today. The notion of testing for certain other variables, like creativity and past psi experiences, is information that was picked up on the way.

But there's a difference between Standardness and Methodological rigour.

NucleicAcid wrote:The protocols of separate rooms, noise isolation, CCTV, blind experimenters, etc is all from the evolution of parapsychology, from a guy holding up a Zener card, to where it is at today.

These are all to do with the method. To make sure that ESP doesn't stand for "Error Some Place". By removing all normal means of exchanging information, the hope is that "psi" is the only explanation left.

The notion of testing for certain other variables, like creativity and past psi experiences, is information that was picked up on the way.

To my mind, this is Standardness. And Standardness is a more nebulous concept: "It's a good idea to use creative subjects/ red light/ one target per day/ half an hour". A good idea, but not essential. Go back to your list of seven, Willin's 1994 experiment used people who were friends or musical students or had paranormal experiences and so fulfilled at least three of those seven. But was deemed very non-standard due to the nature of the targets: music. The worry is, was it considered non-standard only after the results were known?

Yes, we want to make sure that all participants are "open" to psi. That will maximize the chances of a successful confirmation bias.

You believers tend to forget: reality doesn't care whether you believe or not, or are "open" or not.

NucleicAcid wrote:1) Strangely, they don't mention it. They were really thorough about everything else, so they either forgot (comparatively a minor gaffe), or decided it wasn't a hugely important factor.

Ersby thinks that they used a RNG which would be good, but I can't find a link to it (although it could be Goulding et al. 2004, which doesn't exist online apparently). However, I don't understand why you aren't concerned about this unless you are confident that they used an RNG. Still, the omitting of this information (assuming that it wasn't linked to in another paper) is at best a serious error which makes this experiment impossible to replicate, making the results fairly useless, or at worst left out because the experimenters (or even the subjects!) picked the targets, obviously making the experiment a joke.

Since they used the computer to display the videos though, I think we could potentially assume that they used an RNG.

NucleicAcid wrote:Either way, given the number of trials, it is not going to affect the results, especially considering the measures they took to make sure there were no selection biases by the subjects.

Well the number of trials is what worried me - there are only 120 right? Or have I misread it? In any case, they only needed 3 or 4 hits "above chance" (a misleading term, as chance results are often above chance) in order to get a significant result. When dealing with such a tiny statistical effect, they obviously need to do many, many more trials.

And how did they make sure there were no selection biases by subjects? I don't think that's possible.

NucleicAcid wrote:2) That's how the Ganzfeld procedure works. The subject A talks out loud what they are visualizing. The issue with communication and sensory leakage was already addressed, along with basically every Ganzfeld debate since the 80's. Ea is totally blind to the target.

Yeah I understand that's how the Ganzfeld procedure is done but my question is why? My point was simply that if they didn't use a RNG, then the experimenter wouldn't be blind to the target - making it impossible to eliminate sensory leakage.

NucleicAcid wrote:3) It sounds like it was just equipment and space limitation. I don't know exactly why they chose to do it like this, but I'm guessing they only had one computer with the ganzfeld playback and selection software on it. I agree that it is sloppy, but if done right, it shouldn't incur any sensory leakage.

If done right, sure, but taking risks like that when we're looking for such a small statistical effect is going to be troublesome..

NucleicAcid wrote:4) Yes, there have been several attempts to differentiate telepathy (person to person) from clairvoyance (person just receives), and I don't have any of those handy. But the bias issue you mentioned is well known and typically controlled for as a default.

Really? How do they control for it?

And my question wasn't about differentiating telepathy from clairvoyance, I'm talking about differentiating telepathy from real world statistics, that is, do people still score 30% when there is no "sender", which would indicate that this is just a result of the experiment and not psi.

I am not troubled by the lack of mention of the RNG because they (I would assume) used a PRNG Random function call in the Automated Digital Ganzfeld software, which would have been build directly into the code (and most likely addressed in the research that originally used the software). I can't find the specs on that program, but a similar program makes mention of it:

...using a random function to select the target (from a pool of 100 digital movies, each lasting 60 seconds, stored on the computers' hard drive)

From "Experimenter Effects and Psi Performance Using A Digital Autoganzfeld System, Smith & Savva 2004 [url]http://www.parapsych.org/papers/54.pdf
[/url]

I imagine this random function is analogous to the Mersenne twister (I code in Python mostly, and I know the random.random() function uses the twister), which is more than adequate for the purpose used.

Yes, you are correct in not so many words. There is an approximately 3% chance of 39 hits occurring out of 120 trials by chance. Clearly, this trial isn't trying to "prove psi." It is exploratory.

I got a bit thrown off by all the matrix math - they were controlling to make sure that there was maximum contrast between the 4 pictures in a set. I mistook this for meaning that they made sure that there weren't any pictures that people would have a greater tendency to pick.

Experimenter A is in Room A at the time the video is selected in Room B. As mentioned, the video is selected by the program; the Experimenter A has no way of knowing what the target it.

Again, I don't think the point of this study is to prove anything to anyone. It is to explore the possibility of using covert psi tasks. I chose this example because, frankly, I was reading it at the time I was perusing this forum, and it was super convenient. I'm sure if I dug I could find an even better example of a tightly controlled experiment. My statement that 'it is the most methodologically sound to date' was hyperbole. I was feeling more snarky and less scientific.

Controlling for clairvoyance vs telepathy is simple. All you do is take the sender out of the room and have an empty room with the video playing on the screen to no-one.

The only real way to differentiate ESP from real world statistics would be to basically make the experiment as psi-hostile as possible. I call this thought experiment the Evil Ganzfeld for obvious reasons; sounds of babies screaming, flashing lights, rigid metal chairs, apathetic rude greeters, etc. However, this discussion has basically already been had many times, and the consensus is that as long as the subject's selection is (other than the psi) completely random, and the target presentation is randomized to even out placement effects, then it should always average to the MCE after enough runs. As long as the setup is built well (which was mostly determined in the Joint Communique, and improved upon since then with computers), there should be no way in which anyone can get an 'edge'. Then it's just a matter of deciding how far you need to go in terms of accrued statistical significance in order to say there is something there.

Personally, I'm not a big fan of this approach, and I think most parapsychologists today also agree. If the Updated Ganzfeld Database didn't settle that problem, then it is unlikely any massing of experiments is going to prove psi to the skeptics. I think the advancements will be made in understanding more about how psi works, and which variables reliably correlate with improved psi performance (again, if psi doesn't exist, then all target selection is random, therefore ANY correlation with psi performance should asymptote to zero as number of trials increases)

NucleicAcid wrote:I am not troubled by the lack of mention of the RNG because they (I would assume) used a PRNG Random function call in the Automated Digital Ganzfeld software, which would have been build directly into the code (and most likely addressed in the research that originally used the software). I can't find the specs on that program, but a similar program makes mention of it:

...using a random function to select the target (from a pool of 100 digital movies, each lasting 60 seconds, stored on the computers' hard drive)

From "Experimenter Effects and Psi Performance Using A Digital Autoganzfeld System, Smith & Savva 2004 [url]http://www.parapsych.org/papers/54.pdf
[/url]

I imagine this random function is analogous to the Mersenne twister (I code in Python mostly, and I know the random.random() function uses the twister), which is more than adequate for the purpose used.

Ah yeah, I saw them mention the software package. Thanks for that.

NucleicAcid wrote:Yes, you are correct in not so many words. There is an approximately 3% chance of 39 hits occurring out of 120 trials by chance. Clearly, this trial isn't trying to "prove psi." It is exploratory.

Of course. And this is scientific methodology, so you can't "prove" anything anyway.

NucleicAcid wrote:I got a bit thrown off by all the matrix math - they were controlling to make sure that there was maximum contrast between the 4 pictures in a set. I mistook this for meaning that they made sure that there weren't any pictures that people would have a greater tendency to pick.

Yeah that part of the paper took me a second to figure out what the hell they were trying to demonstrate. It made sense in the end, it just seemed overly convoluted..

NucleicAcid wrote:
Again, I don't think the point of this study is to prove anything to anyone. It is to explore the possibility of using covert psi tasks.

Sure, but to explore the possibility of covert psi abilities, you need to be able to separate it from non-psi effects.

NucleicAcid wrote:I chose this example because, frankly, I was reading it at the time I was perusing this forum, and it was super convenient. I'm sure if I dug I could find an even better example of a tightly controlled experiment. My statement that 'it is the most methodologically sound to date' was hyperbole. I was feeling more snarky and less scientific.

Fair enough, I understand that.

NucleicAcid wrote:Controlling for clairvoyance vs telepathy is simple. All you do is take the sender out of the room and have an empty room with the video playing on the screen to no-one.

That's what confuses me, because I'd see that as a set up to compare real psi effects from statistical anomalies..

NucleicAcid wrote:The only real way to differentiate ESP from real world statistics would be to basically make the experiment as psi-hostile as possible. I call this thought experiment the Evil Ganzfeld for obvious reasons; sounds of babies screaming, flashing lights, rigid metal chairs, apathetic rude greeters, etc. However, this discussion has basically already been had many times, and the consensus is that as long as the subject's selection is (other than the psi) completely random, and the target presentation is randomized to even out placement effects, then it should always average to the MCE after enough runs. As long as the setup is built well (which was mostly determined in the Joint Communique, and improved upon since then with computers), there should be no way in which anyone can get an 'edge'. Then it's just a matter of deciding how far you need to go in terms of accrued statistical significance in order to say there is something there.

Personally, I'm not a big fan of this approach, and I think most parapsychologists today also agree. If the Updated Ganzfeld Database didn't settle that problem, then it is unlikely any massing of experiments is going to prove psi to the skeptics. I think the advancements will be made in understanding more about how psi works, and which variables reliably correlate with improved psi performance (again, if psi doesn't exist, then all target selection is random, therefore ANY correlation with psi performance should asymptote to zero as number of trials increases)

Hmm.. well no, even if we agreed that all the Ganzfeld studies were sound and showed a true effect, then the strongest claim we can make is that some factor is giving us a result above chance. We can claim it's psi if we like, but to say that we've established it already and need to now work out how it works is a bit premature.

First, we'd need to ask why it is that this effect only occurs under the conditions you listed earlier - you can call them "psi conducive elements" if you like, but I think most scientists would call them potential confounds. We particularly need to look at the "sheep-goat" effect which, if proposed in any other field of science, would be seen as a way of trying to dismiss negative findings.

The problem I have with set ups like this is that it supposes equal responding across the four alternatives in the null hypothesis, and this will never happen. The best you can hope for is minimal bias. However, since most people have similar biases (e.g. "pick the option at the top of the rank selection screen") then we're going to get more responses made to those videos, meaning that they will inevitably score more hits there. Only a slight bias would be necessary to explain the 3-4 hits that make the result significant. I suppose it's similar to how we view the odds of a coin toss - we assume that under fair conditions, no tampering and a perfectly made coin, that we should achieve a 50:50 chance of heads or tails. And this simply isn't the case in the real world, as the odds are 51:49 (and can even reach as high as 80:20 I think depending on the type of toss used). Now, claiming I can alter the probabilities of a coin toss with my mind is the same as claiming the results found here, if accepted as valid, are evidence of psi. They are simply evidence against our initial assumptions - in other words, we have evidence against the null hypothesis but we don't have evidence for the alternative hypothesis since psi abilities is not the logical negation of assumed probabilistic chance.