AI Succession
Rich Sutton0 HN recommenders · 2023–2025
13K views · talk · evergreen · Uploaded 2023-09-09
A short pre-recorded talk by Rich Sutton for the World AI Conference in Shanghai arguing that succession from humanity to AI is inevitable. He lays out his view of that transition.
Cited critically, as evidence of the unsettling perspective of a prominent AI researcher who seems to accept humanity being replaced.
15:01What Happened to Horses Is Happening to Us
CGP Grey
10:36Why Would AI Want to do Bad Things? Instrumental Convergence
Robert Miles AI Safety
1:15:11Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller Explains
Perimeter Institute for Theoretical Physics
13:03Intelligence and Stupidity: The Orthogonality Thesis
Robert Miles AI Safety
22:56We're building a dystopia just to make people click on ads | Zeynep Tufekci
TED
18:05Intro to AI Safety, Remastered
Robert Miles AI Safety
48:32Sparks of AGI: early experiments with GPT-4
Sebastien Bubeck
15:27Why Not Just: Think of AGI Like a Corporation?
Robert Miles AI Safety
4 comments from Hacker News
Clearly Rich Sutton is a giant in AI for his contributions in RL, but his recent brief talk "AI Succession" ( [VIDEO] ) made me worry a bit about the sort of perspective he has on what the "good" outcome here looks like. I say this as someone who is generally optimistic about the promise of AGI. I have no love for the machines as a "species", and by Rich's definition here, yeah, I am "specist" in favor of humans. I think we should use technology for our own benefit, and that it's not inevitable that machines "replace" us. I also think his framing of the counterarguments is not charitable. The serious AI-risk arguments do not argue that a super-intelligent AI will necessarily be evil. They only argue that its motivations will be unaligned with ours, that it will be more competent in achieving goals than us, and that this will be bad for humans as a side effect. I think a good comparison is humans building a highway that incidentally crushes an ant colony. They didn't set out on an evil mission to destroy ants because they hate them, it just happened as a side effect of something the humans wanted. No evil required.
Richard Sutton has done fascinating RL research in the past. Lately, though, he has been advocating for ending humanity, which is a bit more chilling[1]. [1]: [VIDEO]
He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. His risk calculations derive primarily from the AI itself, not weaponization of AI by others. Rich Sutton seems to agree with this take and embraces our extinction: [VIDEO]
Good time to remind everyone that Sutton is a human successionist and doesn't care if humans all die. He is not to be trusted nor celebrated: [VIDEO]