## Nice Article on Bayes and the Scientific Method

Astronomy, Chemistry, Earth Sciences, Mathematics & Physics.

### Nice Article on Bayes and the Scientific Method

http://www.theguardian.com/science/life-and-physics/2014/sep/28/belief-bias-and-bayes

A prior assumption of zero probability can never be changed. Thus, for example, if you absolutely believe that the Earth is 5000 years old or so, no amount of evidence can change your mind. If your unshakeable faith tells you there are only red stones, then the fact that I appear to have a blue one is simply god, or possibly satan, making a red stone look blue to test your faith. Just like he did with the fake fossils and the cosmic microwave background. No evidence will modify your prior belief. Your faith makes you impervious.

I guess Bayesian statistics provides a mathematical definition of a closed mind. Anyone with a prior of zero about something can never learn from any amount of evidence, because anything multiplied by zero is still zero.
A soul in tension that's learning to fly
Condition grounded but determined to try
Can't keep my eyes from the circling skies
Tongue-tied and twisted just an earthbound misfit, I

twistor59
RS Donator

Posts: 4966

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

Nice article indeed. In the wake of the fact that the '7σ detection' of primordial polarization from BICEP2 may now turn out to be galactic dust, I read this article about the pitfalls of p-values:
http://telescoper.wordpress.com/2013/11/12/the-curse-of-p-values/

The p-value merely specifies the probability that you would reject the null-hypothesis if it were correct. This is what you would call making a Type I error. It says nothing at all about the probability that the null hypothesis is actually a correct description of the data. To make that sort of statement you would need to specify an alternative distribution, calculate the distribution based on it, and hence determine the statistical power of the test, i.e. the probability that you would actually reject the null hypothesis when it is correct. To fail to reject the null hypothesis when it’s actually incorrect is to make a Type II error.

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed. Unquestioning application of off-the-shelf statistical methods in inappropriate situations is a serious problem in many disciplines, but is particularly prevalent in the social sciences when samples are typically rather small.

Statistics is a bitch.
"The longer I live the more I see that I am never wrong about anything, and that all the pains that I have so humbly taken to verify my notions have only wasted my time." - George Bernard Shaw

Pulsar

Posts: 4618
Age: 43

Country: Belgium
Print view this post

### Re: Nice Article on Bayes and the Scientific Method

"there's lies, damn lies, then statistics"
"You can fuck the fuck off, you fucking fucker" - L. Salander

colubridae

Posts: 312
Age: 69

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

The Central Limit Theorem has a lot to answer for.
Signature temporarily on hold until I can find a reliable image host ...

Calilasseia
RS Donator

Posts: 22042
Age: 58

Country: England
Print view this post

### Re: Nice Article on Bayes and the Scientific Method

Pulsar wrote:Nice article indeed. In the wake of the fact that the '7σ detection' of primordial polarization from BICEP2 may now turn out to be galactic dust, I read this article about the pitfalls of p-values:
http://telescoper.wordpress.com/2013/11/12/the-curse-of-p-values/

The p-value merely specifies the probability that you would reject the null-hypothesis if it were correct. This is what you would call making a Type I error. It says nothing at all about the probability that the null hypothesis is actually a correct description of the data. To make that sort of statement you would need to specify an alternative distribution, calculate the distribution based on it, and hence determine the statistical power of the test, i.e. the probability that you would actually reject the null hypothesis when it is correct. To fail to reject the null hypothesis when it’s actually incorrect is to make a Type II error.

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed. Unquestioning application of off-the-shelf statistical methods in inappropriate situations is a serious problem in many disciplines, but is particularly prevalent in the social sciences when samples are typically rather small.

Statistics is a bitch.

Yes I remember a while back reading some stuff on the badscience forum about the way p values were applied in medical science - drug testing etc, and being mildly concerned. Can't remember the details unfortunately.
A soul in tension that's learning to fly
Condition grounded but determined to try
Can't keep my eyes from the circling skies
Tongue-tied and twisted just an earthbound misfit, I

twistor59
RS Donator

Posts: 4966

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

Calilasseia wrote:The Central Limit Theorem has a lot to answer for.

Yeah, at school with maths A level, the thickies were taught the statistics option and the swots were taught mechanics. Perhaps it should have been the other way round!
A soul in tension that's learning to fly
Condition grounded but determined to try
Can't keep my eyes from the circling skies
Tongue-tied and twisted just an earthbound misfit, I

twistor59
RS Donator

Posts: 4966

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

"The longer I live the more I see that I am never wrong about anything, and that all the pains that I have so humbly taken to verify my notions have only wasted my time." - George Bernard Shaw

Pulsar

Posts: 4618
Age: 43

Country: Belgium
Print view this post

### Re: Nice Article on Bayes and the Scientific Method

Pulsar wrote:Another article on statistics... what are the odds, eh?

Pfff...
Half-Life 3 - I want to believe

Rumraket

Posts: 13211
Age: 39

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

Pulsar wrote:Nice article indeed. In the wake of the fact that the '7σ detection' of primordial polarization from BICEP2 may now turn out to be galactic dust, I read this article about the pitfalls of p-values:
http://telescoper.wordpress.com/2013/11/12/the-curse-of-p-values/

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed.

Statistics is a bitch.

There's also publication bias, which leads to a greater number of false positives. But on the "falsely assuming normal distributions" front, I have a paper in press which makes that point for a particular type of question. It has been known for ages that the variates are not normally distributed and still people used statistical tests that use the assumption of normalcy. In this particular case a key issue arising from that assumption is that the probability of equality for two RVs with normal distributions is 0. In the paper we worked out the real (discrete) probabilities and could give probabilities for the equality of two variates - in the worst case for the normal distribution that was a whopping 0.91.
susu
susu.exp

Posts: 1690

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

What was/were the variates you were observing? Gene related?
Lowpro

Posts: 70

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

In this case it's some ecological data I had from my diploma thesis. Basically we are counting the number of damage types on fossil leaves and to compare different sites, subsamples of equal size are drawn from the leaves and you look at how many damage types are represented (clearly an RV taking only non-negative integer values) . There's some literature dealing with the maths involved, but it only treats the special case where each leaf has a maximum of 1 damage type and it only gives the mean and the variance - which obviously is enough to fuel a normal approximation but little else. We reconstructed the complete distribution in the general case where any leaf can have arbitrarily many damage types present.

I've got my molecular programs running, requiring little attention right now, so that stuff is getting turned into an R package to make it easy to use.
susu
susu.exp

Posts: 1690

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

I am entirely out of my depth in this, so please pardon if I am wasting pixels with my ignorance. This does sound fascinating though.

My question, which I hope actually applies:

how does the observer calculate the likelihood that they are observing from a point of view which is itself, the result of us being within an anomalous, rather than a "mainstream" section of existence possibilities? And thus, that what calculations are made, however carefully and however accurately applied, will also be even less applicable to reality?

Again, please pardon the very crude language.

igorfrankensteen

Name: michael e munson
Posts: 2114
Age: 67

Country: United States
Print view this post

### Re: Nice Article on Bayes and the Scientific Method

@susu:

Are you trying to compare multiple leaf damage PCA to the general cases and manage the 1-damage types because they're inflating the general variance?
Lowpro

Posts: 70

Print view this post

### Re: Nice Article on Bayes and the Scientific Method

Nothing that complicated. It's really just taking a subsample of the leaves and looking at how many types of damage are represented. Generally the leaves with multiple types of damage increase variance - a very simple illustration is a subsample of 1 - if you only had leaves with at most one damage type you would always get either 0 or 1, but if there are leaves with more damage types you could also have larger numbers.
Basically it's just a way to remove effects from unequal sampling - the localities to be compared have yielded vastly different numbers of leaves and rarer forms of insect damage will be more likely to show up in an extensive collection than in a small one.
susu
susu.exp

Posts: 1690

Print view this post