Superintelligence
Paths, Dangers, Strategies
by Nick Bostrom · first published 2014
52% recommend40% neutral8% critical
Why HN recommends it
Thorough, accessible case for why a superintelligent AI could be dangerous and hard to control, including goal misspecification and arms-race risks; suits readers new to AI safety who want the core arguments.
Common criticism: Some find the prose abstruse, repetitive, and poorly organized
Summarised by Claude from recommending comments
The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species then would come to depend on the actions of the machine superintelligence. But we have one advant
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2009–2025Global catastrophic risks
Nick Bostrom
Recommended by 3 · mentioned by 53 recommend · 5 mention3 recommend · 53 recommend
2020–2026Human Compatible
Stuart J. Russell
Recommended by 9 · mentioned by 189 recommend · 18 mention9 recommend · 189 recommend
2014–2026Two Faces of Tomorrow
James P. Hogan
Recommended by 7 · mentioned by 97 recommend · 9 mention7 recommend · 97 recommend
2017–2026Life 3.0
Max Tegmark
Recommended by 25 · mentioned by 4525 recommend · 45 mention25 recommend · 4525 recommend
2010–2026Mind children
Hans Moravec
Recommended by 1 · mentioned by 91 recommend · 9 mention1 recommend · 91 recommend
2015–2023This Is Not My Hat
Jon Klassen
Recommended by 4 · mentioned by 44 recommend · 4 mention4 recommend · 44 recommend
Deep Utopia
Nick Bostrom
Recommended by 2 · mentioned by 82 recommend · 8 mention2 recommend · 82 recommend
…have to re-articulate their raison d'etre in the face of superhuman machine intelligence. I would expect an LSD guy like Leary to have a better grasp on the purpose of being human than that. See also Superintelligence by Nick Bostrom Life Extension: One of the key features of being human is being mortal. It's not just an inconvenience to be done away with. (That's a personal view; disagreement is welcome.) I sort…
on “Silicon Valley’s Technolibertarian Dream Was Invented By Timothy Leary” ↗…you read the arguments for caring you realise that they are actually well thought-out. For AI risk, check out http://waitbutwhy.com/2015/01/artificial-intelligence-revolu... and https://www.amazon.co.uk/Superintelligence-Dangers-Strategie.... For an argument for tackling death, checkout http://www.nickbostrom.com/fable/dragon.html. Also, there's a clear answer to 'If after decades we can't improve quality of life in…
on “The Moral Economy of Tech” ↗…inevitable. It's very likely to happen in our lifetime. And it's very likely to kill us all. And explaining every reason why I believe those things could fill an entire book, so I seriously recommend reading Superintelligence by Nick Bostrom.
on “The Myth of AI – Jaron Lanier” ↗I highly recommend the book referenced in the article: Nick Bostrom's Superintelligence. https://www.amazon.com/Superintelligence-Dangers-Strategies-... It has helped me make informed, realistic judgments about the path AI research needs to take. It and related works should be in the vocabulary…
on “Tool AIs want to be agent AIs” ↗Here are a few more excellent resources on the potential and the dangers of machine intelligence. Even if you don't expect to read Nick Bostrom's Superintelligence - a deep, provocative, and thoughtful book, but also very verbose - the above links will give you an excellent primer on humanity's prospects if and when we develop a true general AI: [Wait But Why]…
on “Machine intelligence, part 1” ↗The purpose of the profile isn't to argue a risk exists. We largely defer to the people we take to be experts on the issue, especially Nick Bostrom. We think he presents compelling arguments in Superintelligence, and although it's hard to say anything decisive in this area, if you think there's even modest uncertainty about whether AGI will be good or bad, it's worth doing more research into the risks. If you…
on “Artificial intelligence risk research” ↗…"meaningful" sense. Neither of these objections hold much water IMO, which is why I take the intelligence explosion scenario described by Oxford philosopher Nick Bostrom seriously: http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
on “AlphaGo Can't Beat Me, Says Chinese Go Grandmaster Ke Jie” ↗…technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it. If you want to delve into the best version of the AI risk argument, I recommend Superintelligence by Nick Bostrom.
on “Experts who say we shouldn't worry about superintelligent AI are wrong?” ↗If you're actually interested in understanding the arguments for worrying about AI safety, consider reading "Superintelligence" by Bostrom. http://www.amazon.com/Superintelligence-Dangers-Strategies-N... It's the closest approximation to a consensus statement / catalog of arguments by folks who take this position (although…
on “Elon Musk AMA” ↗…when powerful new technologies are introduced. (Reposting my earlier comment from a few weeks ago:) If you are interested in understanding the arguments for worrying about AI safety, consider reading "Superintelligence" by Bostrom. http://www.amazon.com/Superintelligence-Dangers-Strategies-N... It's the closest approximation to a consensus statement / catalog of arguments by folks who take this position (although…
on “Microsoft's Bill Gates insists AI is a threat” ↗I really recommend that (thoughtful) people interested in this subject read ‘Superintelligence: Paths, Dangers, Strategies’ by Nick Bostrom. http://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dange...
on “New York judge grants writ of habeus corpus to two chimpanzees” ↗…DEMONS". If you want to laugh at people with crazy beliefs, then go ahead; but if not the best popular account of why Elon Musk believes that superintelligent AI is a problem comes from Nick Bostrom's SuperIntelligence: http://smile.amazon.com/Superintelligence-Dangers-Strategies... (Note I haven't read it, although I am familiar with the arguments and some acquaintances tend to rate it highly)
on “Web Design: The First 100 Years (2014)” ↗Superintelligence - Nick Bostrom Infinite Jest - David Foster Wallace 100 Years of Solitude - Gabriel Garcia Marquez
on “Ask HN: Best books you've read in 2015” ↗I think you haven't done even cursory research into A.I super-intelligence before dismissing it. Try reading Nick Bostrom's "Superintelligence: Paths, Dangers, Strategies"[1] and then I think you will change your mind. [1]http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
on “The Scariest, Most Boring Armageddon” ↗…recommend a couple of books about the present and possible futures of human progress as well: E.O. Wilson. Consilience. https://www.amazon.com/Consilience-Knowledge-Edward-Osborne-... Nick Bostrom. Superintelligence: Paths, Dangers, Strategies. https://www.amazon.com/Superintelligence-Dangers-Strategies-...
on “Saudi Arabia's Oil Wealth Is About to Get a Reality Check” ↗