Posted: Mar 07, 2017 9:54 pm
Our go-to scenario is --- perhaps naive in itself --- throwing AI at tax-code systems.
My pessimistic mate paints a much bleaker scenario. He thinks that a general AI will always be some sort of reinforcement learner, and that as soon as it gets smart enough, it's first job will be to take control of its reward mechanism, which he thinks inevitably means deploying Machiavellian plots against its human controllers and ultimately getting us out of the picture. I think that idea is hopelessly naive.
My pessimistic mate paints a much bleaker scenario. He thinks that a general AI will always be some sort of reinforcement learner, and that as soon as it gets smart enough, it's first job will be to take control of its reward mechanism, which he thinks inevitably means deploying Machiavellian plots against its human controllers and ultimately getting us out of the picture. I think that idea is hopelessly naive.