Sparse distributed memory
by Pentti Kanerva · first published 1988
38% recommend63% neutral
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2008–2026Fluid Concepts and Creative Analogies
Douglas R. Hofstadter, Fluid Analogies Research Group
Recommended by 17 · mentioned by 3117 recommend · 31 mention17 recommend · 3117 recommend
2009–2026Le Ton beau de Marot
Douglas R. Hofstadter
Recommended by 27 · mentioned by 5227 recommend · 52 mention27 recommend · 5227 recommend
2009–2026Metaphors We Live By
George Lakoff, Mark Johnson, Lakoff
Recommended by 56 · mentioned by 8156 recommend · 81 mention56 recommend · 8156 recommend
2007–2026I Am a Strange Loop
Douglas R. Hofstadter
Recommended by 101 · mentioned by 204101 recommend · 204 mention101 recommend · 204101 recommend
2007–2026Gödel, Escher, Bach
Douglas R. Hofstadter
Recommended by 505 · mentioned by 1,055505 recommend · 1,055 mention505 recommend · 1,055505 recommend
Kanerva is also the author of the (vastly underrated IMHO) book Sparse Distributed Memory from the late 80s that set the stage for a lot of this and predates Numenta by a couple of decades...
on “Maryland research could improve the AI task of sensorimotor representation” ↗Kanerva's Sparse Distributed Memory is quite remarkable: https://en.wikipedia.org/wiki/Sparse_distributed_memory it's a book though: https://www.amazon.com/Sparse-Distributed-Memory-MIT-Press/d...
on “Ask HN: What is your favorite CS paper?” ↗"Sparse Distributed Memory" is a lovely book by Pentti Kanerva in which he hypothesizes that high dimensional binary vectors (hypervectors) is key to understanding human memory and cognition. To a large part Kanerva is concerned…
on “Counterintuitive Properties of High Dimensional Space” ↗I don't keep around many items from my collection of my mid-90s AI books, but for some reason Kanerva's Sparse Distributed Memory book always managed to make the cut whenever I go in to cull things. There is something about the idea that seems appealing and I keep waiting for some excuse to see if any of the ideas can be melded…
on “Sparse Distributed Memory” ↗Vector Symbolic Architecture (VSA) is the European name. Base is Pentti Kanerva's 1988 book, "Sparse Distributed Memory". Now referred to as "HDC" or "HD". Search Pentti Kanerva now Staff Scientist at UC Berkeley Redwood Center for Theoretical Neuroscience or Mohsen Imani, now UC Irvine former at UC San Diego. Introductory…
on “Hyperdimensional Computing” ↗…some structures (e.g. the thalamus, that is a kind of "hub of information") connections going backward (toward the input) exceed the connections going forward by almost a factor of ten. * Brain uses Sparse Distributed Memory (SDM). See SDM by Pentti Kanerva (NASA researcher). * Neuron models have many more variable/parameters (that can be used to transfer or process information) than usual nodes/links from artificial neural…
on “The future of deep learning” ↗If you're interested in this you may want to take a look at SDM content-addressable memory, which uses neural networks as address encoders/decoders: https://en.wikipedia.org/wiki/Sparse_distributed_memory it was developed by Pentti Kanerva at NASA in the 80s For related memory-augmented models see: https://en.wikipedia.org/wiki/Deep_learning#Networks_with_se... There was also "Reasoning, Attention,…
on “One-shot Learning with Memory-Augmented Neural Networks” ↗…field, Jeff Hawkins for example, are collaborating, just indirectly. The "Pentti Kanerva" described in the article as an "old friend of Hofstadter" and his first wife, Carol, was the originator of the "sparse distributed memory" idea that appears to be what NuPIC is based on and, I believe, either co-founded Redwood Neuroscience / Numenta with Hawkins or soon joined it. (I'm sorry, I'm fuzzy on the details.) Researchers such…
on “The Man Who Would Teach Machines to Think” ↗…how thoughts and memories are encoded and stored. I'm curious as to whether Hofstadter will address this in this book - I would imagine so as he was long ago excited by earlier similar ML approaches (Sparse Distributed Memory).
on “Douglas Hofstadter has a new book” ↗The whole idea that memory is stored in a distributed, lossy, and redundant fashion is hardly a new one; I read about the concept of Sparse Distributed Memory in Science News as a youngster more than 3 decades ago, and it in turn was informed by earlier ideas of sparse and spatial-coded memory (e.g. holographic metaphors of recollection). LLMs provide evidence…
on “The brain is not a computer (2016)” ↗