An introduction to information theory
by John Robinson Pierce · first published 1980
88% recommend12% neutral
Why HN recommends it
Accessible yet technically credible introduction to Shannon's information theory, building intuition for entropy, randomness, language and noise; suits beginners and curious readers wanting a first, readable grounding.
Summarised by Claude from recommending comments
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2014–2025Symbols, signals, and noise
John R. Pierce
Recommended by 12 · mentioned by 1312 recommend · 13 mention12 recommend · 1312 recommend
2010–2021Grammatical man
Jeremy Campbell
Recommended by 7 · mentioned by 87 recommend · 8 mention7 recommend · 87 recommend
2020–2026A Short Stay in Hell
Steven L. Peck
Recommended by 13 · mentioned by 1513 recommend · 15 mention13 recommend · 1513 recommend
2008–2026Fortune's Formula
William Poundstone
Recommended by 32 · mentioned by 3732 recommend · 37 mention32 recommend · 3732 recommend
2017–2026A mind at play
Jimmy Soni, Rob Goodman
Recommended by 21 · mentioned by 3021 recommend · 30 mention21 recommend · 3021 recommend
A Book of Abstract Algebra
Charles C. Pinter
Recommended by 25 · mentioned by 2725 recommend · 27 mention25 recommend · 2725 recommend
2009–2026The mathematical theory of communication
Claude Elwood Shannon
Recommended by 47 · mentioned by 10347 recommend · 103 mention47 recommend · 10347 recommend
If you want an actual introduction to Information Theory, I'd recommend "An Introduction to Information Theory: Symbols, Signals and Noise". It deals with complicated information theory topics similar to Kolmogorov Complexity (albeit by a different name) in…
on “Kolmogorov Complexity - it's a bit silly” ↗The information theory (mathematical) definition of randomness is explained nicely in Chapter V of An Introduction to Information Theory: http://www.amazon.com/Introduction-Information-Theory-Symbol... In the particular case of a series of n equally-probably, independent events, the entropy is given as H = - log 1/n, measured in bits.…
on “Show HN: WordPass – password generator giving over 90 bits of entropy” ↗For those intrigued, the Dover edition of An Introduction to Information Theory: Symbols, Signals and Noise by Pierce is a very good book. Accessible, but with a technically credible author.
on “Celebrating Claude Shannon” ↗Dover's excellent 'An Introduction to Information Theory' by John Pierce is dedicated to Claude and Betty Shannon.
on “The Man Who Invented Information Theory (2017)” ↗…fundamental to a CS education (I've more often drawn on discrete math/symbolic logic) but 3Blue1Brown has great series on introductory linear algebra and calculus. And Symbols, Signals, and Noise: An Introduction to Information Theory by John Pierce is also good. My own view echoes this (https://meaningness.com/metablog/how-to-think), which is that it's often useful to know a little about a lot of different kinds of math. That way…
on “Computer Science from the Bottom Up (2016)” ↗_Introduction to Information Theory: Symbols, Signals and Noise (Dover Books on Mathematics)_ by Pierce is another such book. It is of course only an introduction to the field, but it had an immense impact on how I saw information (hence,…
on “Ask HN: What OR Are there some multiple perspective books in mathematics?” ↗My introduction to Shannon and information theory through this book [1], which talks briefly about language entropy, is what set me on the path to my career. [1]. An Introduction to Information Theory, Symbols, Signals & Noise By John Robinson Pierce · 1980 https://www.google.com/books/edition/An_Introduction_to_Info...
on “Prediction and Entropy of Printed English – Claude Shannon (1950) [pdf]” ↗…introduction to differential privacy. 2. Tales of the Weird: An Uncanny Introduction (2023) A collection of short stories from the British Library to introduce the many flavors of weird fiction. 3. An Introduction to Information Theory: Symbols, Signals and Noise (1980) by John R. Pierce A fun survey of information theory with great anecdotes about Bell Labs in mid-century. See my full list here: https://bcmullins.github.io/interesting-books-2025/
on “Ask HN: End of Year Book Recommendations” ↗…you can take away from this book. For example, he talks about his wife's death, and ties that into his theory of mind and explains the unstated purposes of why we have funerals/wakes for people. - An Introduction to Information Theory by John R. Pierce. For someone like me who doesn't really have a very strong math background, this was a very clear intro to the ideas behind information theory, and why they're important historically.…
on “Ask HN: Best books you read in the past decade?” ↗…Incompleteness Paperback" by Hal Prince (Author) "Gödel, Escher, Bach: An Eternal Golden Braid" by Douglas R Hofstadter "Information Theory: A Tutorial Introduction (2nd Edition)" by James V Stone "An Introduction to Information Theory: Symbols, Signals and Noise" Subsequent Edition by John R. Pierce "Coding the Matrix: Linear Algebra through Applications to Computer Science" by Philip N. Klein "Land of Lisp: Learn to Program in Lisp,…
on “Ask HN: I'm thinking of writing a book – Why would you want a book?” ↗…textbook. Nahin - Inside interesting Integrals. Lots of fun integrations! Nahin - He also wrote some books about euler and imaginary numbers which ive partially read, they seemed good. Pierce - An introduction to information theory; symbols signals and noise. sounds and looks like a dover textbook but is a non-fake pop science book. --------------- Boyer - The History of Calculus and its Conceptual Development. A bit old, and…
on “Classic Mathematics Books for Lifelong Learners” ↗…notable exception of Smith's Wealth of Nations which is disturbingly un- and mis-read: 1. Grammatical Man, by Jeremy Campbell (1982) https://www.worldcat.org/title/grammatical-man-information-e... My introduction to information theory and its diverse set of interrelated applications and phenomena. 2. Ecology and the Politics of Scarcity, by William Ophuls (1977) https://www.worldcat.org/title/ecology-and-the-politics-of-s... Distills…
on “Ask HN: What are the best unknown books you have read?” ↗Hello folks, Recently, I developed a genuine interest in information theory and communication systems, I gave "An Introduction to Information Theory: Symbols, Signals and Noise" a try (https://www.amazon.com/dp/0486240614) and I really enjoyed it. I'd like to dig deeper into these topics but I don't know what a good introductory resource (book, course,…
on “Ask HN: Resources to Self-Study Communication Systems” ↗…a big if), it seems the same problem applies to gambling. The energy it takes to communicate whether a gamble has paid off is at least as big as what can be recovered from the gamble. The book is "An Introduction to Information Theory" by John R Pierce, and it's the most I've ever learned from a $10 paperback.
on “The Gambler, Maxwell’s New Demon” ↗Beginners might want to start with the following works; 1) An Introduction to Information Theory, Symbols, Signals and Noise by John R. Pierce - A classic general text to understand the concepts and build intuition. Other books by the same author are also excellent. 2) Information Theory: A Tutorial…
on “Essential Coding Theory [pdf]” ↗