Help Needed - New Scientist Article

Tracking down papers ...

Discuss the language of the universe.

Moderators: Calilasseia, ADParker

Help Needed - New Scientist Article

#1  Postby Calilasseia » Apr 24, 2020 6:44 pm

Since I don't have a subscription to New Scientist, I can't read [url=https://www.newscientist.com/article/mg24632790-700-correlation-or-causation-mathematics-can-finally-give-us-an-answer/]this article in full[/url[, and when you click on the link, it should be pretty obvious why I'm interested in the contents thereof, including any references (ideally with DOI numbers) for any scientific papers cited therein.

If anyone can provide me with the DOI numbers for all papers cited in this article, this will be extremely welcome. :)

The article claims that a solution to the "correlation versus causation" problem has been found, and I want to check if the content does indeed match the billing. Which, if true, is going to be a major step forward for obvious reasons.
Signature temporarily on hold until I can find a reliable image host ...
User avatar
Calilasseia
RS Donator
THREAD STARTER
 
Posts: 22082
Age: 59
Male

Country: England
United Kingdom (uk)
Print view this post

Ads by Google


Re: Help Needed - New Scientist Article

#2  Postby newolder » Apr 24, 2020 6:59 pm

A similar article from 2014 at quartz makes reference to this arXiv paper: Distinguishing cause from effect using observational data: methods and benchmarks by Joris M. Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, Bernhard Schölkopf and last updated in 2015. 152 citations from later work might help too. Journal ref: Journal of Machine Learning Research 17(32):1-102, 2016


Probably related to your search but I've no NS sub. either so :dunno: for sure.
Last edited by newolder on Apr 24, 2020 7:29 pm, edited 1 time in total.
I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops. - Stephen J. Gould
User avatar
newolder
 
Name: Albert Ross
Posts: 7310
Age: 1
Male

Country: Feudal Estate number 9
Print view this post

Re: Help Needed - New Scientist Article

#3  Postby Ironclad » Apr 24, 2020 7:11 pm

I do. Give me a date. Do you want the magazine posted, if there's one to accompany?
For Van Youngman - see you amongst the stardust, old buddy

"If there was no such thing as science, you'd be right " - Sean Lock

"God ....an inventive destroyer" - Broks
User avatar
Ironclad
RS Donator
 
Name: Nudge-Nudge
Posts: 23879
Age: 51
Male

Country: Wink-Wink
Indonesia (id)
Print view this post

Re: Help Needed - New Scientist Article

#4  Postby Ironclad » Apr 24, 2020 7:12 pm

Your URL is borked, that's why I ask. I can't follow it
For Van Youngman - see you amongst the stardust, old buddy

"If there was no such thing as science, you'd be right " - Sean Lock

"God ....an inventive destroyer" - Broks
User avatar
Ironclad
RS Donator
 
Name: Nudge-Nudge
Posts: 23879
Age: 51
Male

Country: Wink-Wink
Indonesia (id)
Print view this post

Re: Help Needed - New Scientist Article

#5  Postby chairman bill » Apr 24, 2020 7:13 pm

If you don't mind waiting a few days I'll be able to get you a pdf of the article
“There is a rumour going around that I have found God. I think this is unlikely because I have enough difficulty finding my keys, and there is empirical evidence that they exist.” Terry Pratchett
User avatar
chairman bill
RS Donator
 
Posts: 28305
Male

Country: UK: fucked since 2010
United Kingdom (uk)
Print view this post

Re: Help Needed - New Scientist Article

#6  Postby felltoearth » Apr 24, 2020 7:22 pm

PM sent
"Walla Walla Bonga!" — Witticism
User avatar
felltoearth
 
Posts: 14008
Age: 53

Canada (ca)
Print view this post

Re: Help Needed - New Scientist Article

#7  Postby Ironclad » Apr 24, 2020 7:26 pm

:lol:
That's some teamwork!
For Van Youngman - see you amongst the stardust, old buddy

"If there was no such thing as science, you'd be right " - Sean Lock

"God ....an inventive destroyer" - Broks
User avatar
Ironclad
RS Donator
 
Name: Nudge-Nudge
Posts: 23879
Age: 51
Male

Country: Wink-Wink
Indonesia (id)
Print view this post

Ads by Google


Re: Help Needed - New Scientist Article

#8  Postby zoon » Apr 24, 2020 10:12 pm

I googled "correlation causation 2020", and one of the top links here is an article "Artificial intelligence can spot when correlation means causation" on the University College London website, which is a non-specialist piece about a February 2020 paper (the paper itself, "Integrating Overlapping Dasets Using Bivariate Causal Discovery" is here). I'm only guessing this is the article mentioned in the New Scientist April 2020 issue, though the author of the New Scientist article (here, mostly behind a paywall) has the same name as one of the authors of the paper, which would be a remarkable correlation if uncaused.

It seems to be something to do with causes being earlier, and so less complex, than effects, but I don't understand it. Quoting much of the non-specialist piece:
A new artificial intelligence (AI) has allowed researchers at UCL and Babylon Health, for the first time, to demonstrate a useful and reliable way of sifting through masses of correlating data to spot when correlation means causation.

By fusing old, overlapping and incomplete datasets this new method, inspired by quantum cryptography, paves the way for researchers to glean the results of medical trials that would otherwise be too expensive, difficult or unethical to run. The research is being published at the prestigious and peer-reviewed Association for Advancement of Artificial Intelligence (AAAI) conference in New York.
..................
Dr Ciarán Lee, Senior Research Scientist at Babylon and Honorary Senior Research Associate at UCL Physics & Astronomy, explained: "Scientists have it hammered into them that correlation does not mean causation; ice-cream sales don't cause sunburn despite rates of both shooting up during the summer. To find the exact cause of sunburn we whittle down or control as many variables as possible. Then when our datasets show that a change in sun exposure matches a change in sunburn, we can be confident the sun exposure was the causative variable. The problem is the real world is rarely neat and tidy and it can be really hard to control all the variables and work out which is causative."

Scientists started looking for other ways to help spot causative variables. A theory born from physics suggests that everything becomes more disordered and complicated with time, so a cause should be less disordered and complex than its effect. Dr Lee said "If you take your dataset and give each of the variables a complexity rating you can work backwards and spot which one is the cause. But that just helps for that one dataset - we wanted to see if there was a way of combining datasets, ones with gaps or where researchers were asking different questions to what they're interested in now. That could be a game-changer."

Dr Lee was inspired by quantum cryptography. The strange laws of quantum physics mean that two users can send a message and then use a mathematical formula to prove whether someone else is eavesdropping on their conversation. Dr Lee realised that datasets could work in a similar way, but thinking of a potential causative variable from another dataset as the eavesdropper. "If one dataset shows us that obesity causes heart disease, and another shows vitamin D causes obesity we can use a mathematical formula to prove whether vitamin D causes obesity or not. This is what our AI is doing."
User avatar
zoon
 
Posts: 3230

Print view this post

Re: Help Needed - New Scientist Article

#9  Postby Calilasseia » Apr 27, 2020 5:27 pm

Thanks to FE above, who has earned the bottle of champagne, so to speak!

Apparently, the ideas in question centre upon the work of one Judea Pearl, a computer science researcher who has apparently derived some seriously important results in the field of Bayesian Networks. One seminal paper from the researches thereof is this one:

Fusion, Propagation And Structuring in Belief Networks by Judea Pearl, Artificial Intelligence, 29: 241-288 (1986)

and was apparently one of the papers that led to him being granted the Turing Award in 2011. The abstract thereof reads as follows:

Pearl, 1986 wrote:ABSTRACT

Belief networks are directed acyclic graphs in which the nodes represent propositions (or variables), the arcs signify direct dependencies between the linked propositions, and the strengths of these dependencies are quantified by conditional probabilities. A network of this sort can be used to represent the generic knowledge of a domain expert, and it turns into a computational architecture if the links are used not merely for storing factual knowledge but also for directing and activating the data flow in the computations which manipulate this knowledge.

The first part of the paper deals with the task of fusing and propagating the impacts of new information through the networks in such a way that, when equilibrium is reached, each proposition will be assigned a measure of belief consistent with the axioms of probability theory. It is shown that if the network is singly connected (e.g. tree-structured), then probabilities can be updated by local propagation in an isomorphic network of parallel and autonomous processors and that the impact of new information can be imparted to all propositions in time proportional to the longest path in the network.

The second part of the paper deals with the problem of finding a tree-structured representation for a collection of probabilistically coupled propositions using auxiliary (dummy) variables, colloquially called "hidden causes." It is shown that if such a tree-structured representation exists, then it is possible to uniquely uncover the topology of the tree by observing pairwise dependencies among the available propositions (i.e., the leaves of the tree). The entire tree structure, including the strengths of all internal relationships, can be reconstructed in time proportional to n log n, where n is the number of leaves.



I'm no expert in this field by any means, but I know enough to recognise this as an important result. Now all I have to do is track down some of his other papers, and see what else turns up in the mix ...
Signature temporarily on hold until I can find a reliable image host ...
User avatar
Calilasseia
RS Donator
THREAD STARTER
 
Posts: 22082
Age: 59
Male

Country: England
United Kingdom (uk)
Print view this post

Re: Help Needed - New Scientist Article

#10  Postby newolder » Apr 27, 2020 6:35 pm

There's a reference to a J. Pearl in the Mooji et al paper mentioned above: J. Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000.

It's not a complete reference but might be a relevant summary of their work. :dunno:
I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops. - Stephen J. Gould
User avatar
newolder
 
Name: Albert Ross
Posts: 7310
Age: 1
Male

Country: Feudal Estate number 9
Print view this post


Return to Mathematics

Who is online

Users viewing this topic: No registered users and 1 guest