Posted: Apr 29, 2017 11:51 am
VazScep wrote:It's not beyond the realms of reason. We know we can't just increase speed/connections/memory for free. We've already hit thermal limits with speed. Eventually, you have so many parallel CPUs, spread over so much space, that the amount of time they are waiting on inter-CPU communication means you might as well have just used a single CPU. Since the 90s, the stupid facts about the speed-of-light and memory access times have put stupid constraints on how we program, even as Moore's law was claiming to march us all forward.
There are major physical limits to scaling computation which we've known for a long time. I doubt the human brain has found them all, but they'll be there.
I'm not arguing for a limitless increase in computation, only that the author seems to me unable to see how there's a rather huge gap between better than what we have now but not that much better, and limitless increases.
VazScep wrote:I think it's a common trope in AI speculation to wonder about this. Have you heard of the "wirehead" problem? It's the reinforcement feedback AI which figures out how its reward function is embedded in the real world, takes control over it, and then stimulates itself indefinitely by having it send maximum reward forever. It's the same situation as someone who figures out that they'd be pretty happy dosed up forever with morphine. I haven't actually read any standard solutions to this problem, but I assume they are the same ones that would be used for a machine that can reprogram itself.
Yeah, I've read a lot of science fiction. Wasn't that in the William Hurt movie, Altered States, either his character or someone else was sitting permanently in a chair, twitching as he was orgasming every few seconds and they couldn't turn it off? Something like that, god it's been really long ago since I saw that movie, which is now 37 years old. [That just made me really depressed]. But, yeah, such a trope is common in other versions, like in Discworld, when Granny Weatherwax goes 'borrowing', living in other animals minds, there is always the risk they'll get stuck there, losing their humanity if they stay too long and not being able to find their way back. 'It's not nice to fool mother nature' Shit, that just popped into my head and I know it's a TV commercial but can't for the life of me remember what for.