Algorithms for Optimization
by Mykel J. Kochenderfer, Tim A. Wheeler · first published 2019
45% recommend55% neutral
Why HN recommends it
Broad, accessible survey of optimization techniques that shows which classes of methods suit which problems; includes code and exercises, making it a good fit for developers moving into ML or scientific computing.
Summarised by Claude from recommending comments
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
Probabilistic Machine Learning
Kevin P. Murphy
Recommended by 27 · mentioned by 3527 recommend · 35 mention27 recommend · 3527 recommend
2010–2026Numerical linear algebra
Lloyd N. Trefethen
Recommended by 26 · mentioned by 2926 recommend · 29 mention26 recommend · 2926 recommend
2016–2026Statistical Rethinking
Richard McElreath
Recommended by 81 · mentioned by 9281 recommend · 92 mention81 recommend · 9281 recommend
2007–2026The Elements of Statistical Learning
Trevor Hastie, Robert Tibshirani, Jerome Friedman
Recommended by 144 · mentioned by 227144 recommend · 227 mention144 recommend · 227144 recommend
2014–2026Introduction to Statistical Learning
Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani
Recommended by 85 · mentioned by 10885 recommend · 108 mention85 recommend · 10885 recommend
2007–2026Pattern Recognition and Machine Learning
Christopher M. Bishop
Recommended by 80 · mentioned by 10480 recommend · 104 mention80 recommend · 10480 recommend
2008–2026Linear Algebra Done Right
Sheldon Jay Axler
Recommended by 130 · mentioned by 179130 recommend · 179 mention130 recommend · 179130 recommend
…Python code available. Steve Brunton's lectures on YouTube are pretty damn good and compliment the book well: https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/fea... Another really cool book is Algorithms for Optimization by Mykel Kochenderfer and Tim Wheeler: https://mitpress.mit.edu/books/algorithms-optimization. Julia code used in book.
on “A 2020 Vision of Linear Algebra” ↗…what is going on though, the path I am having some early success with (as a long time developer / data pipeline guy, but newly into the standard python / ML practice) is to run through Kochenderfer's "Algorithms for optimization" from 2019 (MIT press), including implementing the exercises, as optimization is the cornerstone of the majority of ML methods. Some of the most fun I've had in a long time. Freely available here: https://algorithmsbook.com/optimization/…
on “Ask HN: In 2022, what is the proper way to get into machine/deep learning?” ↗…in good condition. The book covers a lot more than linear programming. A solution manual for both editions can be found on the internet. A good "free-pdf" optimization book, to support the above is, Algorithms for Optimization by Kochenderfer & Wheeler ( https://algorithmsbook.com/optimization/ ). It has a chapter on constrained linear optimization with Julia code and is a good secondary resource. Kochenderfer, Wheeler, and…
on “Scaling up linear programming with PDLP” ↗The first author (MK) also wrote "Algorithms for Optimization" (MIT Press), which is also implemented in (and is a nice guide to) the Julia language, which is itself a pretty sweet language.
on “Algorithms for Decision Making” ↗For those who want to learn more, I really like the book "Algorithms for Optimization": https://algorithmsbook.com/optimization/#outline It's a great intro because it covers a large breadth and gives you a sense of what class of techniques are useful for what (rather than just going super…
on “Gradient Free Optimizers: A collection of modern optimization methods in Python” ↗…series. [1] It's great that they are making the labs free and putting them on GitHub. Even so, the fact that the textbooks are not free and open source make this much less interesting. In contrast, "ALGORITHMS FOR OPTIMIZATION" is available online [2] as well as code from the book, and two more books by the same authors [3]. Seems like university departments should start funding fully open, downloadable books by their faculty…
on “Python Applied Mathematics Labs” ↗I've been wanting to read books like Algorithms for Optimization (https://www.amazon.com/gp/product/0262039427/) and Probabilistic Machine Learning (https://www.amazon.com/gp/product/0262046822/) but when I open them up I immediately think "wow I really need to refresh…
on “How to choose a textbook that is optimal for oneself?” ↗Preview of the second edition of Algorithms for Optimization by Kochenderfer and Wheeler. Three new chapters and various other changes, see the preface for details.
on “Algorithms for Optimization (second edition preview) [pdf]” ↗…anybody getting into this field? Is there any classical literature for optimization related to ML? Thanks! My current list includes: 1. Numerical Optimization by Jorge Nocedal Stephen J. Wright 2. Algorithms for Optimization - introduction to optimization with a focus on practical algorithms 3. Algorithms for Decision Making - a broad introduction to algorithms for decision making under uncertainty [2] https://algorithmsbook.com/optimization/…
on “Ask HN: Literature for mathematical optimization?” ↗The Tufte package also allows for typesetting a book with executable Julia code, only in LaTeX: https://github.com/sisl/tufte_algorithms_book It was used to make Algorithms for Optimization and Algorithms for Decision Making.
on “Books.jl: Create Books with Julia” ↗Looks like a different book, "Algorithms for Optimization", by two of the three authors of "Algorithms for Decision Making".
on “Algorithms for Decision Making” ↗Algorithms for Decision Making was recently released as open access. The authors were able to get open access rights for their previous book, Algorithms for Optimization. It is a general reference book for optimization algorithms, with concrete implementations in the Julia programming language.
on “Algorithms for Optimization Now Open Access” ↗…only written in Standard ML (and now includes Haskell as well I believe). The code there is fairly non-trivial and it makes sense to use a language that has become the standard for FP (Haskell). Take Algorithms for Optimization, a book written entirely in Julia, using more complex code than Introduction to Statistical.... I don't even really know Julia and mentally translating the code to Python while reading requires very little…
on “An Introduction to Statistical Learning with Applications in Python” ↗