Optimization by Vector Space Methods
by David G. Luenberger · first published 1968
88% recommend13% neutral
Why HN recommends it
Uses functional analysis and vector space geometry to build optimization theory, giving a strong sense of least-squares and infinite-dimensional methods; suited to readers wanting mathematical depth.
Summarised by Claude from recommending comments
Unifies the field of optimization with a few geometric principles The number of books that can legitimately be called classics in their fields is small indeed, but David Luenberger's OPtimization by Vector Space Methods certainly qualifies. Not only does Luenberger clearly demonstrate that a large segment of the field of optimization can be effectively unified by a few geometric principles of linear vector space theory, but his methods have found applications quite removed from the engineering problems to which they were first applied. Nearly 30 years after its initial publication, athis bo
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
Mentions per year
What Hacker News says
Related books
2011–2025Foundations of optimal control theory
E. B. Lee
Recommended by 1 · mentioned by 21 recommend · 2 mention1 recommend · 21 recommend
Mathematical Theory of Optimal Processes
Pontryagin Ls
0 recommend · 3 mentioned0 recommend · 3 mention0 recommend · 30 recommend
2012–2021Advanced calculus for applications
Francis Begnaud Hildebrand
Recommended by 1 · mentioned by 21 recommend · 2 mention1 recommend · 21 recommend
2009–2025Real and complex analysis
Walter Rudin
Recommended by 25 · mentioned by 4025 recommend · 40 mention25 recommend · 4025 recommend
2010–2018Probability
Leo Breiman
Recommended by 5 · mentioned by 75 recommend · 7 mention5 recommend · 75 recommend
2011–2025Deterministic and Stochastic Optimal Control
Wendell H. Fleming, Raymond W. Rishel
Recommended by 1 · mentioned by 31 recommend · 3 mention1 recommend · 31 recommend
2010–2025The art and theory of dynamic programming
Stuart E. Dreyfus
Recommended by 1 · mentioned by 41 recommend · 4 mention1 recommend · 41 recommend
2010–2025Controlled Markov processes
E. B. Dynkin
Recommended by 1 · mentioned by 41 recommend · 4 mention1 recommend · 41 recommend
…that answer anyway! For the rest, the Hilbert and Banach spaces won't go away! E.g., a huge dessert buffet of really finger lick'n good applications of the Hahn-Banach theorem is David G. Luenberger, 'Optimization by Vector Space Methods', and that book will be a nice source of methods for a lot that computing people might encounter. > You left out graph theory and combinatorics, both of which are extremely important to CS. For…
on “Ask HN: Learn Math the Hard Way” ↗…I mentioned it would seem that there is some practical value. For some of what can be done with applications, there is a long dessert buffet from the Hahn-Banach theorem and more in D. Luenberger, Optimization by Vector Space Methods. One heck of a toolkit. Make just one good application and retire, make a donation to a university, and get your name on a building. Here's another possibility: Attack integer linear programming problems…
on “The 1980 Math Olympiad Program: Where are they now?” ↗…good examples of good technical writing. E.g., for science, see the freshman physics texts by Sears, etc. For technology, see D. Knuth, if only his The TeXBook. For engineering, see, say, D. Luenberger, Optimization by Vector Space Methods. For math, see, say, any of the more respected texts for freshman calculus going back at least 50 years. Also see the texts of W. Rudin. The core problem I am addressing here is that the technical writing…
on “Amazon API Gateway – Build and Run Scalable Application Backends” ↗…be a deep subject, but often a relatively simple way out is via regular conditional probabilities as in Leo Breiman, 'Probability', ISBN 0-89871-296-3, SIAM, Philadelphia. For David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York. this has likely the easiest mathematical treatment of deterministic optimal control and also Kalman filtering and also the math needed for 'least action' in physics.…
on “The pitfalls of jargon and the damage it does to mathematics” ↗…Here is it: "Advanced pure and applied math.". Period. Sorry 'bout that. E.g., at Stanford, f'get about the Computer Science department, look up D. Luenberger, and study, say, David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York. which is fun and profit mostly from the Hahn-Banach theorem. Again, there is a fundamental problem: Only a tiny fraction of people can start really successful…
on “The Problem With Silicon Valley Is Itself” ↗…course by a star student of E. Cinlar, maybe I should like Kollar's course. Sorry, I didn't -- the quality looked too low. Better quality from Stanford? Sure, K. Chung, H. Royden, D. Luenberger (his 'Optimization by Vector Space Methods' is a beautifully, even elegantly, done one mile long applied math dessert buffet), D. Knuth. For courses in 'how to code', that is, introductory material in software, gotta be kidding! 'Coding' alone…
on “Online Education's Dirty Secret - Awful Retention” ↗…https://www.youtube.com/watch?v=w2itwFJCgFQ A broader view of some of what can be done with ordinary differential equations, and a long, gorgeous desert buffet of applied math: David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York, 1969. Generally, though, apparently the big glory days of ordinary differential equations were for the US DoD and NASA during the Cold War and the Space Race. Likely…
on “Show HN: Differential Equations Explained” ↗…the rationals can get as close as we please to pi! Banach spaces are not trivial or useless: E.g., based on the Hahn-Banach theorem, there is the grand applied math dessert buffet David G. Luenberger, Optimization by Vector Space Methods, John Wiley and Sons. A Hilbert space is a Banach space where the norm comes from an inner product. E.g., the set of all real valued random variables X such that E[X^2] is finite form a Hilbert space.…
on “Confession as an AI researcher; seeking advice” ↗The best introduction to optimization methods book I've ever read is "Optimization by Vector Space Methods" by Luenberger. This is the book that opened my eyes to the generality of least-squares methods, the actual geometry of the problem, and how to apply it extremely generally. I read this before Boyd and…
on “Programming books you might want to consider reading” ↗…complete, normed linear space. There is a nice chapter on Banach space in Walter Rudin, Real and Complex Analysis. There are some nice applications of the Hahn-Banach theorem in David G. Luenberger, Optimization by Vector Space Methods. In Patrick Billingsley, Convergence of Probability Measures. is a nice presentation of Ulam's result in measure theory Le Cam called tightness: Roughly, IIRC, for any probability measure P and any…
on “The Eccentric Lives of Steinhaus, Banach and Ulam (2014)” ↗…separation problem. Well, so as to have readers be more comfortable, I did omit mention of the Hahn-Banach theorem. For curious readers, there are lots of applications of that classic result in D. Luenberger, Optimization by Vector Space Methods. "Hard to work with ...". How 'bout hard to please?
on “Data Structure and Algorithms Interview Questions for Programmers” ↗…the subject of this thread had some math notation without such definitions. (2) My standard, quite clear, go-to source on Kalman filters is the section in (with TeX syntax) David G.\ Luenberger, {\it Optimization by Vector Space Methods,\/} John Wiley and Sons, Inc., New York, 1969.\ \ That's a really good example of some really good math and writing of math.
on “A Non-Mathematical Introduction to Kalman Filters for Programmers” ↗I love this one https://ia801706.us.archive.org/7/items/in.ernet.dli.2015.14... Luenberger, Optimization By Vector Space Methods. Although it is about a specific application, optimization, it is a good book to get a sense of infinite dimensional vector spaces. I would also recommend Halmos. His book surreptitiously introduces…
on “An introduction to functional analysis for science and engineering” ↗…0-471-06015-1, John Wiley and Sons, New York, 1977. ---------------------------------------- A grand applied mathematics dessert buffet, based on Banach space and the Hahn-Banach theorem is David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York, 1969. ======================================== Mathematical Analysis Relevant to Understanding Linearity ---------------------------------------- Long the first…
on “Linear algebra tutorial in four pages” ↗…general technique), integer programming (a way to see the importance of P versus NP), mathematical statistics, partial differential equations, mathematical finance, etc. For some ice cream, Luenberger, Optimization by Vector Space Methods or how to learn to love the Hahn-Banach theorem and use it to become rich, famous, and popular with girls!
on “Ask HN: How or where to begin learning mathematics from first principles?” ↗