Parallel distributed processing
by David E. Rumelhart · first published 1986
28% recommend67% neutral6% critical
Why HN recommends it
Foundational connectionist text bridging classical AI and modern neural networks; valued for its argument on how learning systems should be structured and its theory of backprop as error minimization. For AI readers.
Summarised by Claude from recommending comments
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2018–2025The organization of behavior
D. O. Hebb
Recommended by 1 · mentioned by 31 recommend · 3 mention1 recommend · 31 recommend
2016–2025Talking nets
James A. Anderson, Edward Rosenfeld
Recommended by 2 · mentioned by 32 recommend · 3 mention2 recommend · 32 recommend
2021–2026Genius Makers
Cade Metz
Recommended by 4 · mentioned by 74 recommend · 7 mention4 recommend · 74 recommend
2008–2026Perceptrons
Marvin Minsky, Seymour Papert, Léon Bottou
Recommended by 4 · mentioned by 394 recommend · 39 mention4 recommend · 394 recommend
2007–2026Artificial Intelligence
Stuart J. Russell, Peter Norvig
Recommended by 60 · mentioned by 13660 recommend · 136 mention60 recommend · 13660 recommend
2010–2026Computing machinery and intelligence
Alan Mathison Turing, John von Neumann
Recommended by 10 · mentioned by 5410 recommend · 54 mention10 recommend · 5410 recommend
…(imho ofcourse). in my opinion, using mathematica (or some such) would have been a far saner choice, to get to the essence of the subject. also, a dosage of classical (pioneering ?) work as described in (parallel-distributed-processing vol-1&2) http://www.amazon.com/Parallel-Distributed-Processing-Vol-Fo..., would be just great.
on “Neural Networks Virtual Study Group” ↗may i also humbly suggest the two vol. series called "parallel distributed processing" (rumelhart et al) which provides an excellent overview of early NN research.
on “A visual proof that neural nets can compute any function” ↗Coming from cognitive neuroscience surprised that Explorations in Parallel Distributed Processing by McClelland and Rumelhart doesn’t get more attention as a classic in bridging old school AI approaches with the modern paradigm. https://psycnet.apa.org/record/1988-97441-000
on “Ask HN: What are the foundational texts for learning about AI/ML/NN?” ↗Highly recommend the exercises in Rumelhart and McClelland - Parallel Distributed Processing: Explorations in the Microstructure of Cognition from 1986-1987 (two volumes) https://direct.mit.edu/books/book/4424/Parallel-Distributed-...
on “Neural Networks in the 1990s” ↗Rumelhart et al wrote "Parallel Distributed Processing"; there's a chapter where he proves that the backprop algorithm maximizes "harmony", which is simply a different formulation of error minimization. I remember reading this book enthusiastically back…
on “Who Invented Backpropagation?” ↗…interesting and relevant strand of philosophy of mind is the connectionism of Rumelhart, McClelland and the PDP Research Group (including "godfather of deep learning" Geoff Hinton), exemplified in the tome "Parallel Distributed Processing" released in 1986 It even has an accompanying manual on how to implement artificial neural networks. http://www.cnbc.cmu.edu/~plaut/IntroPDP/index.html More along this line is found in David Marr's…
on “The Best Books on the Philosophy of Mind” ↗I was just reading 1986 book Parallel Distributed Processing (https://mitpress.mit.edu/books/parallel-distributed-processi...) and was quite surprised by the relevance of their arguments how learning systems should be built and structured, which map quite accurately…
on “Frank Rosenblatt's perceptron paved the way for AI 60 years too soon (2019)” ↗Rumelhart is the co-author of a multi volume work called "Parallel Distributed Processing". PDP was the old term in use when Rumelhart and McClelland repopularized artificial neural network techniques in the 80s. Interestingly enough, rumelhart and McClelland were cognitive scientists by training,…
on “The Basic Ideas in Neural Networks (1994) [pdf]” ↗Have you read Parallel Distributed Processing by McClelland and Rumelhart? I haven’t finished reading it, but they make the case that if humans are conscious, then PDP is the way Strong AI will be figured out. https://mitpress.mit.edu/books/parallel-distributed-processi...
on “Deep learning is hitting a wall?” ↗…2. See here for a nice explanation of the backprop algorithm, http://neuralnetworksanddeeplearning.com/chap2.html or if you want the OG (original Geoff) paper on the topic, it's in the proceedings for Parallel Distributed Processing 1986 -- "Learning representations by back-propagating errors", Rumelhart, Hinton, and Williams.
on “Show HN: Improved Seam Carving Using Forward Energy” ↗As all the other responses point out, NNs were red hot back then. The interest in NNs was ignited (in part) by this double volume collection of essays called "Parallel Distributed Processing" edited by Rumelhart and McClelland. Dean even cites them. And, if you read the contributors, it contains many (though not all) of the heavy hitters. Reading back on it, it will sound very familiar.…
on “Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]” ↗If the article is talking about the neural network in McClelland and Rumelhart’s Parallel Distributed Processing, there’s actually a paper by Steven Pinker and some other linguists drilling into it and finding that it doesn’t model children’s language acquisition nearly as closely or as well as M&R think it does.
on “Is my toddler a stochastic parrot?” ↗It's a chapter from here: Rumelhart, D. E., McClelland, J. L., & the PDP research group. (1986). Parallel distributed processing: Explorations in the microstructure of cognition. Volume I. Cambridge, MA: MIT Press.
on “Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]” ↗sorry, but i think neural-networks came way before 2012, notably the works of rumelhart, mccleland etc. see the 2 volume "parallel distributed processing" to read almost all about it. the book(s): https://direct.mit.edu/books/monograph/4424/Parallel-Distrib... a-talk: https://www.youtube.com/watch?v=yQbJNEhgYUw
on “Ilya Sutskever NeurIPS talk [video]” ↗…Winter", I remember it as a time when it became clear that logic programming over a collection of facts was never going to get us to AGI. Same time, Rumelhart and McClelland published their books on "Parallel Distributed Processing", training neural networks via back-propagation; McClelland has continued this work and has a current version at https://web.stanford.edu/group/pdplab/pdphandbook/handbook.p... But Hameroff - an anesthesiologist…
on “Dr. Stuart Hameroff: Consciousness Is More Than Computation [video]” ↗