Controlled Markov processes

Mentions per year

What Hacker News says

• graycat · May 29, 2015 · Recommends

  …Markov assumption, that is, that the past and future of the process are conditionally independent given the present. For a more detailed treatment, I'd recommend, say, E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes'.

  on “Pavlov.js – Reinforcement learning using Markov Decision Processes” ↗
• graycat · Sep 20, 2021 · Recommends

  …Dynamic Programming: Deterministic and Stochastic Models. George L. Nemhauser, Dynamic Programming. Stuart E. Dreyfus and Averill M. Law, The Art and Theory of Dynamic Programming. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes. Wendell H. Fleming and Raymond W. Rishel, Deterministic and Stochastic Optimal Control. There is more from R. T. Rockafellar and R. J.-B. Wets. And of course we should mention R. Bellman, the father…

  on “Decision Making Under Uncertainty with POMDPs.jl” ↗
• graycat · Aug 26, 2021 · Mentions

  …Dynamic Programming: Deterministic and Stochastic Models. George L. Nemhauser, Dynamic Programming. Stuart E. Dreyfus and Averill M. Law, The Art and Theory of Dynamic Programming. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes. Wendell H. Fleming and Raymond W. Rishel, Deterministic and Stochastic Optimal Control. There is more from R. T. Rockafellar and R. J.-B. Wets. And of course we should mention R. Bellman. IIRC,…

  on “The Mathematics of 2048: Optimal Play with Markov Decision Processes (2018)” ↗
• graycat · May 18, 2018 · Mentions

  …0-13-221581-0, Prentice-Hall, Englewood Cliffs, NJ, 1987.\ \ George L.\ Nemhauser, {\it Dynamic Programming,\/} ISBN 0-471-63150-7, John Wiley and Sons, New York, 1966.\ \ E.\ B.\ Dynkin and A.\ A.\ Yushkevich, {\it Controlled Markov Processes,\/} ISBN 0-387-90387-9, Springer-Verlag, Berlin, 1979.\ \ Dimitri P.\ Bertsekas and Steven E.\ Shreve, {\it Stochastic Optimal Control: The Discrete Time Case,\/} ISBN 0-12-093260-1, Academic Press, New…

  on “Solving dynamic programming interview problems” ↗
• HilbertSpace · Nov 29, 2010 · Mentions

  …processes since the Markov assumption is really what makes the crucial 'decomposition' into 'stages' justified. So, for much more on the Markov connection, there is E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes', ISBN 0-387-90387-9, Springer-Verlag, Berlin. Once I had a practical problem that could use these ideas, got a 90 second lecture on the subject from George L. Nemhauser as in 'Dynamic Programming',…

  on “Programmer Competency Matrix: Do you use it when hiring?” ↗
• NY_USA_Hacker · May 25, 2011 · Mentions

  …optimality. Also in that division at Brown, see the papers of Harold Kushner. As I recall, he wrote out a stochastic version of the Pontryagin maximum principle. For E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes', ISBN 0-387-90387-9, Springer-Verlag, Berlin. there are connections with economic planning. For Dimitri P. Bertsekas and Steven E. Shreve, 'Stochastic Optimal Control: The Discrete Time Case', ISBN…

  on “The pitfalls of jargon and the damage it does to mathematics” ↗
• graycat · Feb 1, 2015 · Mentions

  …0-12-093260-1, Academic Press. Wendell H. Fleming and Raymond W. Rishel, 'Deterministic and Stochastic Optimal Control', ISBN 0-387-90155-8, Springer-Verlag, Berlin, 1979. E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes', ISBN 0-387-90387-9, Springer-Verlag, Berlin. So, a lot of quite serious applied math research has been done, and a lot is known. Broadly stochastic dynamic programming makes the best decisions that…

  on “Super easy project planning and estimation” ↗
• graycat · Sep 18, 2018 · Mentions

  …then get "certainty equivalence" -- that is, get to f'get about the Gaussian, random, stochastic part and use just the expectations themselves. Huge speedup. For more, there is Dynkin and Yushkevich, Controlled Markov Processes -- one of the key assumptions that permits treating the spreadsheet columns one at a time is that the stochastic part obeys a Markov assumption (the past and future are conditionally independent given…

  on “David Patterson Says It’s Time for New Computer Architectures and Languages” ↗
• graycat · Jul 21, 2025 · Mentions

  …Stochastic Models,\/} ISBN 0-13-221581-0, Prentice-Hall.\ \ George L.\ Nemhauser, {\it Dynamic Programming,\/} ISBN 0-471-63150-7, John Wiley and Sons, New York\ \ E.\ B.\ Dynkin and A.\ A.\ Yushkevich, {\it Controlled Markov Processes,\/} ISBN 0-387-90387-9, Springer-Verlag, Berlin.\ \ Wendell H.\ Fleming and Raymond W.\ Rishel, {\it Deterministic and Stochastic Optimal Control,\/} ISBN 0-387-90155-8, Springer-Verlag, Berlin.\ \ Michael…

  on ““Dynamic programming” is not referring to “computer programming”” ↗
• NY_Entrepreneur · Aug 8, 2011 · Mentions

  …Dimitri P.\ Bertsekas and Steven E.\ Shreve, {\it Stochastic Optimal Control: The Discrete Time Case,\/} ISBN 0-12-093260-1, Academic Press, New York, 1978.\ \ E.\ B.\ Dynkin and A.\ A.\ Yushkevich, {\it Controlled Markov Processes,\/} ISBN 0-387-90387-9, Springer-Verlag, Berlin, 1979.\ \ Wendell H.\ Fleming and Raymond W.\ Rishel, {\it Deterministic and Stochastic Optimal Control,\/} ISBN 0-387-90155-8, Springer-Verlag, Berlin, 1979.\…

  on “Retiring a Great Interview Problem” ↗