Markov Decision Processes
by Martin L. Puterman · first published 1994
3mentions
2recommended by
3HN commenters
2011–2025first – last mentioned
67% recommend33% neutral
The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many s
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
The theory of multi-armed bandits arises from the study of Markov decision processes (MDP). This field is also closely studied in and related to Stochastic Control and Dynamic Programming. A good book on MDP's is "Markov Decision Processes: Discrete Stochastic Dynamic Programming by Martin…
on “Ask HN: Writing my first multi-armed bandit solution..?” ↗I would highly recommend Sergey Levine's course: http://rail.eecs.berkeley.edu/deeprlcourse/ For a more mathematical treatment, there's a beautiful book by Puterman: https://www.amazon.com/Markov-Decision-Processes-Stochastic-...
on “Spinning Up in Deep RL” ↗You should read more about Markov Decision Processes (MDPs) and Game Theory in general. The word "solve" is well defined in this context, and is used accurately by OP. You "solve" a game by finding the policy that maximizes the expected return. Obviously…
on “Yahtzeeql – Yahtzee solver that's mostly SQL” ↗