Understanding Machine Learning
From Theory to Algorithms
by Shai Shalev-Shwartz, Shai Ben-David · first published 2014
75% recommend25% neutral
Why HN recommends it
Rigorous theory-first treatment of machine learning balancing statistical and computational learning theory with algorithms; suits readers with strong math backgrounds who want the foundations, not a practical toolkit.
Summarised by Claude from recommending comments
"Machine learning is one of the fastest growing areas of computer science, with far-reaching applications. The aim of this textbook is to introduce machine learning, and the algorithmic paradigms it offers, in a principled way. The book provides an extensive theoretical account of the fundamental ideas underlying machine learning and the mathematical derivations that transform these principles into practical algorithms. Following a presentation of the basics of the field, the book covers a wide array of central topics that have not been addressed by previous textbooks. These include a discussi
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2014–2025Foundations of machine learning
Mehryar Mohri
Recommended by 4 · mentioned by 84 recommend · 8 mention4 recommend · 84 recommend
2011–2025Learning From Data
Yaser S. Abu-Mostafa
Recommended by 31 · mentioned by 4131 recommend · 41 mention31 recommend · 4131 recommend
2014–2026Counterexamples in analysis
Bernard R. Gelbaum
Recommended by 11 · mentioned by 1411 recommend · 14 mention11 recommend · 1411 recommend
2007–2026The Elements of Statistical Learning
Trevor Hastie, Robert Tibshirani, Jerome Friedman
Recommended by 144 · mentioned by 227144 recommend · 227 mention144 recommend · 227144 recommend
2014–2026Introduction to Statistical Learning
Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani
Recommended by 85 · mentioned by 10885 recommend · 108 mention85 recommend · 10885 recommend
2007–2026Pattern Recognition and Machine Learning
Christopher M. Bishop
Recommended by 80 · mentioned by 10480 recommend · 104 mention80 recommend · 10480 recommend
2009–2026Convex optimization
Stephen P. Boyd
Recommended by 40 · mentioned by 5740 recommend · 57 mention40 recommend · 5740 recommend
2008–2026Speech and language processing
Dan Jurafsky, Daniel Jurafsky, James H. Martin
Recommended by 41 · mentioned by 4541 recommend · 45 mention41 recommend · 4541 recommend
I really enjoyed Ben-David (and Shalev-Schwartz)'s book "Understanding machine learning". It's essentially about theory though, not "learn all about machine learning in torchsorflow in 10 days".
on “Mathematician Solves Sensitivity Conjecture in Two Pages (2019)” ↗I skimmed over Mohri's book and I think the topics it covers are quite narrow. For mathematical foundations of ML, I would recommend the book "Understanding Machine Learning: From Theory to Algorithms" by Shai Shalev-Shwartz. A brief version of the book is available to download on the author's website: http://www.cs.huji.ac.il/~shais/Handouts.pdf
on “Level-Up Your Machine Learning” ↗I recommend Understanding Machine Learning for a textbook, as I rather enjoyed a class using it. http://www.cambridge.org/al/academic/subjects/computer-scien...
on “Ask HN: How do I go about learning machine learning?” ↗…Theory, Convex Optimization Algorithms. Nesterov - Lecture Notes (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.693...) Statistical/Theoretical: Shai Shalev-Schwartz & Shai Ben-David's Understanding Machine Learning (http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning...) Mohri's Foundations of Machine Learning (https://cs.nyu.edu/~mohri/mlbook/) The two above courses could share SSS's Online Learning…
on “Confession as an AI researcher; seeking advice” ↗…typically recommend User Story Mapping by Jeff Patton and Making Things Happen by Scott Berkun, which are both excellent how-tos with great examples to build on. If you're strong in math, I recommend Understanding Machine Learning from Theory to Algorithm by Shalev-Shwartz and Ben-David, as it has all the mathematics for ML and actually has some pseudocode for implementation which helps bridge the gap into actual software development…
on “The Hundred-Page Machine Learning Book” ↗…a (mostly discrete) set of random variables and draw inferences from them. So this is about smart and efficient ways to marginalize and condition and compress. Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning From Theory to Algorithms, 2014 This is also a great book. You will see a lot of KTC here but mostly within the framework of convexity and duality.
on “Kenneth Arrow and the golden age of economic theory” ↗I recommend Shai Shalev-Schwartz and Shai Ben-David's Understanding Machine Learning: From Theory to Algorithms [0]. I've also used and found Mohri's Foundations of Machine Learning quite insightful [1]. Usually, between the two books, at least one's proof is easy to follow. Get both…
on “Book: Mathematics for Machine Learning” ↗…areas of mathematics to do theory in ML (and to do ML more generally!) -probability/concentration/hoeffding bounds [the PAC model] [Key] -linear algebra [key] -optimization [key] for books -understanding machine learning by shai ben david This book is nice since it really balances theory with a more practical understanding. -An Introduction to Computational Learning Theory by kearns is a classic [low priority]. this…
on “Dear Academia, I loved you, but I’m leaving you (2014)” ↗If you are looking to have a mathematical understanding of the field then I really enjoyed "Understanding Machine Learning: From Theory to Algorithms" by Shalev-Shwartz and Ben-David.
on “The Mathematics of Artificial Intelligence (2022)” ↗I have not seen mentioned so far in this thread the following book, which I can't recommend more highly: Understanding Machine Learning: From Theory To Algorithms – Shai Shalev-Shwartz
on “Ask HN: What are the foundational texts for learning about AI/ML/NN?” ↗…that other ML approaches are computational statistics. The goalpost for AI keeps moving. If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gentle intro would be Learning From Data [2]. If someone recommends a book/paper it would be awesome, I'm always on the look. [1] https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearni... [2]…
on “The Mathematics of Artificial Intelligence (2022)” ↗Having read significant chunks of both ESL and Understanding Machine Learning (albeit UML much more recently) I would argue that for many readers UML is superior. ESL pays short shrift to the computational complexity of learning whereas UML explicitly handles both statistical…
on “Understanding Machine Learning: From Theory to Algorithms (2014)” ↗Yes, they use what's called the VC-dimension. Chapter 20 of the book Understanding Machine Learning (available for free online http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning...) has a good introduction to the basics of this theory. The short story is that VC dimension describes "worst…
on “Neural Nets Can Learn Function Type Signatures from Binaries [pdf]” ↗…Glivenko Cantelli. In Kevin P. Murphy, Machine Learning: A Probabilistic Perspective, ISBN 978-0-262-01802-9, MIT Press, 2012. I saw no mention of Vapnik. In Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning From Theory to Algorithms, 2014. I see Vapnik, V. (1992), Principles of risk minimization for learning theory, in J. E. Moody, S. J. Hanson & R. P. Lippmann, eds, `Advances in Neural Information Processing Systems…
on “Kenneth Arrow and the golden age of economic theory” ↗…Bitcoin Standard: The Decentralized Alternative to Central Banking by Saifedean Ammous Financial Shenanigans by Howard M. Schilit Up next: Life and Fate by Vasily Grossman These Truths by Jill Lepore Understanding Machine Learning: From Theory to Algorithms by Shalev Shwartz and Ben David
on “Ask HN: Which book do you want to read next?” ↗