Probability Theory
The Logic of Science
by E. T. Jaynes · first published 2003
56% recommend44% neutral
Why HN recommends it
Builds probability from the ground up as extended logic for reasoning under uncertainty, with a strong Bayesian and maximum entropy perspective. Suits readers with solid math who want depth beyond practical recipes.
Summarised by Claude from recommending comments
Publisher Description: The standard rules of probability can be interpreted as uniquely valid principles in logic. In this book, E. T. Jaynes dispels the imaginary distinction between "probability theory" and "statistical inference", leaving a logical unity and simplicity, which provides greater technical power and flexibility in applications. This book goes beyond the conventional mathematics of probability theory, viewing the subject in a wider context. New results are discussed, along with applications of probability theory to a wide variety of problems in physics, mathematics, economi
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2009–2026Bayesian data analysis
Andrew Gelman, John B. Carlin, Hal S. Stern, Donald B. Rubin
Recommended by 27 · mentioned by 4927 recommend · 49 mention27 recommend · 4927 recommend
2016–2026Statistical Rethinking
Richard McElreath
Recommended by 81 · mentioned by 9281 recommend · 92 mention81 recommend · 9281 recommend
2010–2022Probability with martingales
Williams, David
Recommended by 4 · mentioned by 54 recommend · 5 mention4 recommend · 54 recommend
2009–2025Mathematical foundations of the calculus of probability
J. Neveu
Recommended by 3 · mentioned by 63 recommend · 6 mention3 recommend · 63 recommend
2011–2024Real Analysis
H. L. Royden, Patrick Fitzpatrick, Halsey Royden, Royden
Recommended by 4 · mentioned by 64 recommend · 6 mention4 recommend · 64 recommend
A user's guide to measure theoretic probability
Pollard, David
Recommended by 2 · mentioned by 22 recommend · 2 mention2 recommend · 22 recommend
Basic Algebra
Anthony Knapp
Recommended by 2 · mentioned by 22 recommend · 2 mention2 recommend · 22 recommend
I have a favourite. Probability Theory: The Logic of Science by E.T. Jaynes. I just love it for the writing and the way it teaches probability. I haven't encountered any book in probability that is like this book. Every other book I have…
on “Ask HN: Math books you recommend just for the way they are written?” ↗You absolutely need a solid grounding in multi-variable calculus, linear algebra, probability theory and information theory. It will also be helpful to be well versed in graph theory. In my opinion one of the best starting points is "Information Theory, Inference and Learning Algorithms" by David MacKaye.…
on “Ask HN: What maths are critical to pursuing ML/AI?” ↗For me the best so far book on Bayesian probability was "Probability Theory: The Logic of Science: Principles and Elementary Applications" by E. T. Jaynes. The book starts from the deduction of Bayesian theorem from the first principles of logic and shows its applications to…
on “Think Bayes: Bayesian Statistics Made Simple (2012)” ↗…will help you build a good foundation for mathematical thinking. Concrete Mathematics by Graham, Knuth, Patashnik Spivak’s Calculus How to Prove It by Velleman Polya’s How to Solve It E.T. Jaynes’ Probability Theory Conceptual Mathematics by Lawvere and Schanuel
on “Book: Mathematics for Machine Learning” ↗Before all that, you need to choose your statistician. My advice is: resuscitate E.T. Jaynes. Failing that, find one of his disciples. Failing that, find a Bayesian. Failing that, read Probability Theory: the Logic of Science, and just do the analysis yourself. Failing that, maybe a Frequentist statistician will do. Maybe.
on “How to share data with a statistician” ↗If you are interested in a deeper dive into the topics covered in this paper I would suggest "Probability Theory: The Logic of Science" by E. T. Jaynes available athttp://www-biba.inrialpes.fr/Jaynes/cpreambl.pdf It is probably (ha!) the most enlightening math book I have ever read.
on “In Defense of Probability (1985) [pdf]” ↗…"Think of a number which you are allowed to change after each number I suggest. If for any number I suggest you change it for a bigger one, then what you think of is infinity." Chapter 15: Paradoxes of probability theory in Jaynes's "Probability Theory: Logic of Science" is a great reading on the topic (you can find a pdf easily on google). It starts with a quote from Gauss: "I protest against the use of infinite magnitude…
on “How to explain infinity to kids” ↗I think there is something to be said for when one first discovers Bayes; not least because its often presented like this article as a sort of enlightenment. Perhaps the finest work on the topic is probability theory by Jaynes; if the underpinning of the Bayesian sect was to have a bible that would be it and BDA3 would be its practical counterpart. I say this both owning and having read both and having invested a…
on “Bayes Theorem: A Framework for Critical Thinking” ↗E.T.Jaynes: "Probability Theory, The Logic Of Science"
on “Ask HN: Book which comprehensively covers probability theory?” ↗…escape-able problem). If you want a book that makes many of the same philosophical arguments about probability but also backs it up with math and descriptions on how to do Bayesian inference, Edwin Jaynes's Probability Theory: The Logic of Science is for you. It's quite literally a modern cult classic. People that read this book tend to go out and evangelize for it. My final comment is that there is a place for frequentist…
on “Eugenics Shaped Statistics” ↗See also: Probability, The Logic Of Science, by E.T. Jaynes. http://www.amazon.com/Probability-Theory-Logic-Science-Vol/d... Available here: http://bayes.wustl.edu/etj/prob/book.pdf The introduction/first chapter has a nice example about a policeman concluding a crime is being committed that's very…
on “Terence Tao on the relationship between classical and Bayesian reasoning” ↗Probability Theory: The Logic of Science by E. T. Jaynes http://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp... http://bayes.wustl.edu/etj/prob/book.pdf http://omega.albany.edu:8008/JaynesBook.html
on “Suggest good sites/books on probability” ↗E.T. Jaynes book, "Probability Theory: the Logic of Science" may come close to what you want. It emphasize that there are rules of thought, which lead to Bayesian statistics. As such, Bayesian statistics aren't just a recipe, but the law.…
on “Think Bayes - Bayesian Statistics Made Simple” ↗Oh, I didn't realize that. Well, the book is Probability Theory: The Logic of Science ET Jaynes 2003 I highly suggest buying it or finding it from your library!
on “Unusual applications of Bayesian reasoning [pdf]” ↗> For what he thought about "ill posed", etc., problems, I couldn't find. Jaynes' book has a full chapter on "Paradoxes of Probability Theory", including Borel's paradox (or Borel-Kolomogorov paradox). See also http://www.cims.nyu.edu/~csplash/data/notes/2010-14.pdf
on “Frequentism and Bayesianism: A Practical Introduction” ↗