Video Reconstruction From Human Brain

Sovereign · #1 by **Sovereign** » Sep 23, 2011 4:01 pm

I can't wait to see what this becomes in 10 years!

BlackBart · #2 by **BlackBart** » Sep 23, 2011 4:38 pm

Interesting that human faces produce a more recognisable image.

NineBerry · #3 by **NineBerry** » Sep 24, 2011 7:39 am

At 0:21, that's clearly Jesus at the right side!

Ironclad · #4 by **Ironclad** » Sep 24, 2011 7:50 am

NineBerry wrote:At 0:21, that's clearly Jesus at the right side!

Beat me, dagnabbitt :grin:

LucidFlight · #5 by **LucidFlight** » Sep 24, 2011 7:56 am

Impressive and a little spooky.

Sovereign wrote:I can't wait to see what this becomes in 10 years!

DreamTube™

byofrcs · #6 by **byofrcs** » Sep 24, 2011 8:17 am

Hey at last we can get to see what's on the Cartesian theater.

JoeB · #7 by **JoeB** » Sep 24, 2011 2:41 pm

The youtube description says this:

The left clip is a segment of the movie that the subject viewed while in the magnet. The right clip shows the reconstruction of this movie from brain activity measured using fMRI. The reconstruction was obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video. (In brief, the algorithm processes each of the 18 million clips through the brain model, and identifies the clips that would have produced brain activity as similar to the measured brain activity as possible. The clips used to fit the model, those used to test the model and those used to reconstruct the stimulus were entirely separate.) Brain activity was sampled every one second, and each one-second section of the viewed movie was reconstructed separately.
For a related video see: http://www.youtube.com/watch?v=KMA23JJ1M1o
For more information about this work, please check our lab web site: http://gallantlab.org

The right side video isn't directly from the brain, it's a composition from video material which causes similar brain activity. Still pretty impressive though!

John P. M. · #8 by **John P. M.** » Sep 24, 2011 2:53 pm

Yeah, I was about to say; it may seem a bit misleading (depending on what one expects, I suppose).

But at the same time, it does show that we can now 'tap' brain activity and make the output reflect what the brain is actually doing, which is both impressive and interesting. I mean - for the algorithm to be able to pick out the most fitting image for a brain state, it must "know" what that brain state represents; compare it. And then when we see it played back like this, we see that it works.

DavidMcC · #9 by **DavidMcC** » Feb 04, 2012 3:12 pm

Late comment:

What seems to be missing in this work is a "control run", using a completely different set of images to fit to the measured data. If, and only if, the result is similar with a different set of initial images, will it be conclusive, IMO.

DavidMcC · **#10** by **DavidMcC** » Feb 04, 2012 3:15 pm

... The problem may be that the images do not form a set of "orthogonal eigenfunctions", to use an analogy from QM. Therefore, even the best fit is not usually a good one.