Nice Article on Bayes and the Scientific Method

Astronomy, Chemistry, Earth Sciences, Mathematics & Physics.

Moderators: Calilasseia, ADParker

Nice Article on Bayes and the Scientific Method

#1  Postby twistor59 » Sep 29, 2014 8:00 am

http://www.theguardian.com/science/life-and-physics/2014/sep/28/belief-bias-and-bayes

A prior assumption of zero probability can never be changed. Thus, for example, if you absolutely believe that the Earth is 5000 years old or so, no amount of evidence can change your mind. If your unshakeable faith tells you there are only red stones, then the fact that I appear to have a blue one is simply god, or possibly satan, making a red stone look blue to test your faith. Just like he did with the fake fossils and the cosmic microwave background. No evidence will modify your prior belief. Your faith makes you impervious.

I guess Bayesian statistics provides a mathematical definition of a closed mind. Anyone with a prior of zero about something can never learn from any amount of evidence, because anything multiplied by zero is still zero.
A soul in tension that's learning to fly
Condition grounded but determined to try
Can't keep my eyes from the circling skies
Tongue-tied and twisted just an earthbound misfit, I
User avatar
twistor59
RS Donator
THREAD STARTER
 
Posts: 4966
Male

United Kingdom (uk)
Print view this post

Ads by Google


Re: Nice Article on Bayes and the Scientific Method

#2  Postby Pulsar » Sep 29, 2014 2:57 pm

Nice article indeed. In the wake of the fact that the '7σ detection' of primordial polarization from BICEP2 may now turn out to be galactic dust, I read this article about the pitfalls of p-values:
http://telescoper.wordpress.com/2013/11/12/the-curse-of-p-values/

The p-value merely specifies the probability that you would reject the null-hypothesis if it were correct. This is what you would call making a Type I error. It says nothing at all about the probability that the null hypothesis is actually a correct description of the data. To make that sort of statement you would need to specify an alternative distribution, calculate the distribution based on it, and hence determine the statistical power of the test, i.e. the probability that you would actually reject the null hypothesis when it is correct. To fail to reject the null hypothesis when it’s actually incorrect is to make a Type II error.

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed. Unquestioning application of off-the-shelf statistical methods in inappropriate situations is a serious problem in many disciplines, but is particularly prevalent in the social sciences when samples are typically rather small.

Statistics is a bitch.
"The longer I live the more I see that I am never wrong about anything, and that all the pains that I have so humbly taken to verify my notions have only wasted my time." - George Bernard Shaw
User avatar
Pulsar
 
Posts: 4618
Age: 42
Male

Country: Belgium
Belgium (be)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#3  Postby colubridae » Sep 29, 2014 3:03 pm

"there's lies, damn lies, then statistics"
"You can fuck the fuck off, you fucking fucker" - L. Salander
User avatar
colubridae
 
Posts: 312
Age: 69

Print view this post

Re: Nice Article on Bayes and the Scientific Method

#4  Postby Calilasseia » Sep 30, 2014 12:05 am

The Central Limit Theorem has a lot to answer for. :)
Signature temporarily on hold until I can find a reliable image host ...
User avatar
Calilasseia
RS Donator
 
Posts: 22019
Age: 58
Male

Country: England
United Kingdom (uk)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#5  Postby twistor59 » Sep 30, 2014 6:41 am

Pulsar wrote:Nice article indeed. In the wake of the fact that the '7σ detection' of primordial polarization from BICEP2 may now turn out to be galactic dust, I read this article about the pitfalls of p-values:
http://telescoper.wordpress.com/2013/11/12/the-curse-of-p-values/

The p-value merely specifies the probability that you would reject the null-hypothesis if it were correct. This is what you would call making a Type I error. It says nothing at all about the probability that the null hypothesis is actually a correct description of the data. To make that sort of statement you would need to specify an alternative distribution, calculate the distribution based on it, and hence determine the statistical power of the test, i.e. the probability that you would actually reject the null hypothesis when it is correct. To fail to reject the null hypothesis when it’s actually incorrect is to make a Type II error.

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed. Unquestioning application of off-the-shelf statistical methods in inappropriate situations is a serious problem in many disciplines, but is particularly prevalent in the social sciences when samples are typically rather small.

Statistics is a bitch.


Yes I remember a while back reading some stuff on the badscience forum about the way p values were applied in medical science - drug testing etc, and being mildly concerned. Can't remember the details unfortunately.
A soul in tension that's learning to fly
Condition grounded but determined to try
Can't keep my eyes from the circling skies
Tongue-tied and twisted just an earthbound misfit, I
User avatar
twistor59
RS Donator
THREAD STARTER
 
Posts: 4966
Male

United Kingdom (uk)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#6  Postby twistor59 » Sep 30, 2014 6:43 am

Calilasseia wrote:The Central Limit Theorem has a lot to answer for. :)


Yeah, at school with maths A level, the thickies were taught the statistics option and the swots were taught mechanics. Perhaps it should have been the other way round!
A soul in tension that's learning to fly
Condition grounded but determined to try
Can't keep my eyes from the circling skies
Tongue-tied and twisted just an earthbound misfit, I
User avatar
twistor59
RS Donator
THREAD STARTER
 
Posts: 4966
Male

United Kingdom (uk)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#7  Postby Pulsar » Oct 01, 2014 11:51 am

"The longer I live the more I see that I am never wrong about anything, and that all the pains that I have so humbly taken to verify my notions have only wasted my time." - George Bernard Shaw
User avatar
Pulsar
 
Posts: 4618
Age: 42
Male

Country: Belgium
Belgium (be)
Print view this post

Ads by Google


Re: Nice Article on Bayes and the Scientific Method

#8  Postby Rumraket » Oct 01, 2014 12:38 pm

Pulsar wrote:Another article on statistics... what are the odds, eh?

Pfff... :smug:
Half-Life 3 - I want to believe
User avatar
Rumraket
 
Posts: 13184
Age: 39

Print view this post

Re: Nice Article on Bayes and the Scientific Method

#9  Postby susu.exp » Oct 01, 2014 12:47 pm

Pulsar wrote:Nice article indeed. In the wake of the fact that the '7σ detection' of primordial polarization from BICEP2 may now turn out to be galactic dust, I read this article about the pitfalls of p-values:
http://telescoper.wordpress.com/2013/11/12/the-curse-of-p-values/

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed.

Statistics is a bitch.


There's also publication bias, which leads to a greater number of false positives. But on the "falsely assuming normal distributions" front, I have a paper in press which makes that point for a particular type of question. It has been known for ages that the variates are not normally distributed and still people used statistical tests that use the assumption of normalcy. In this particular case a key issue arising from that assumption is that the probability of equality for two RVs with normal distributions is 0. In the paper we worked out the real (discrete) probabilities and could give probabilities for the equality of two variates - in the worst case for the normal distribution that was a whopping 0.91.
susu
susu.exp
 
Posts: 1690

Print view this post

Re: Nice Article on Bayes and the Scientific Method

#10  Postby Lowpro » Oct 01, 2014 5:23 pm

What was/were the variates you were observing? Gene related?
Lowpro
 
Posts: 70

United States (us)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#11  Postby susu.exp » Oct 01, 2014 10:58 pm

In this case it's some ecological data I had from my diploma thesis. Basically we are counting the number of damage types on fossil leaves and to compare different sites, subsamples of equal size are drawn from the leaves and you look at how many damage types are represented (clearly an RV taking only non-negative integer values) . There's some literature dealing with the maths involved, but it only treats the special case where each leaf has a maximum of 1 damage type and it only gives the mean and the variance - which obviously is enough to fuel a normal approximation but little else. We reconstructed the complete distribution in the general case where any leaf can have arbitrarily many damage types present.

I've got my molecular programs running, requiring little attention right now, so that stuff is getting turned into an R package to make it easy to use.
susu
susu.exp
 
Posts: 1690

Print view this post

Re: Nice Article on Bayes and the Scientific Method

#12  Postby igorfrankensteen » Oct 02, 2014 1:58 am

I am entirely out of my depth in this, so please pardon if I am wasting pixels with my ignorance. This does sound fascinating though.

My question, which I hope actually applies:

how does the observer calculate the likelihood that they are observing from a point of view which is itself, the result of us being within an anomalous, rather than a "mainstream" section of existence possibilities? And thus, that what calculations are made, however carefully and however accurately applied, will also be even less applicable to reality?

Again, please pardon the very crude language.
User avatar
igorfrankensteen
 
Name: michael e munson
Posts: 2114
Age: 66
Male

Country: United States
United States (us)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#13  Postby Lowpro » Oct 02, 2014 2:56 am

@susu:

Are you trying to compare multiple leaf damage PCA to the general cases and manage the 1-damage types because they're inflating the general variance?
Lowpro
 
Posts: 70

United States (us)
Print view this post

Re: Nice Article on Bayes and the Scientific Method

#14  Postby susu.exp » Oct 03, 2014 2:44 pm

Nothing that complicated. It's really just taking a subsample of the leaves and looking at how many types of damage are represented. Generally the leaves with multiple types of damage increase variance - a very simple illustration is a subsample of 1 - if you only had leaves with at most one damage type you would always get either 0 or 1, but if there are leaves with more damage types you could also have larger numbers.
Basically it's just a way to remove effects from unequal sampling - the localities to be compared have yielded vastly different numbers of leaves and rarer forms of insect damage will be more likely to show up in an extensive collection than in a small one.
susu
susu.exp
 
Posts: 1690

Print view this post


Return to Physical Sciences & Mathematics

Who is online

Users viewing this topic: No registered users and 1 guest

cron