The Nature of Statistical Learning Theory
by Vladimir Naumovich Vapnik · first published 1995
8mentions
3recommended by
7HN commenters
2009–2023first – last mentioned
43% recommend57% neutral
The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning from the general point of view of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. These include: - the general setting of learning problems and the general model of minimizing the risk functional from empirical data - a comprehensive analysis of the empirical risk minimization
Affiliate links. As an Amazon Associate I earn from qualifying purchases.
If you happen to be much better at math than programming, starting with the theory could be much easier. This survey is the best introduction I have found. The alternative being to read Vapnik's "The Nature of Statistical Learning Theory", which is more longer and much more "Russian". Andrew Ng's notes are an excellent (non-theory) introduction to general machine learning. http://www.stanford.edu/class/cs229/materials.html
on “Introduction to Statistical Learning Theory” ↗…theory (available as a free manuscript online), which had a lot of the same kind of material with quite a good framework of motivation and explanation wrapped around its proofs. I also like Vapnik's _The Nature of Statistical Learning Theory_ which as I understand it is highly parallel to a much longer proof-heavy version of most of the same material. I much preferred this book to the approach in my undergraduate course in statistics. Again,…
on “The Feynman Lectures on Physics: Algebra (1963)” ↗…limits are so strict! I think MDL should be very helpful, but make sure you understand the danger of "manual overfitting" that I described in the book. I would also encourage you to read Vapnik's book the Nature of Statistical Learning Theory, which shows the relation b/t MDL and VC theory. Feel free to reach out to me at firstname dot lastname at gmail.com, I'm always happy to chat about these ideas.
on “Notes on a New Philosophy of Empirical Science (2011)” ↗…statistical inference, this is not true. I would like to demonstrate that in this area of science a good old principle is valid: Nothing is more practical than a good theory. -- From Vapnik's preface to The Nature of Statistical Learning Theory * Vapnik is not well-described as a "theory guy". That implies that he's not interested in connections between theory and practice, and this is most profoundly not the case. He has arguably been the…
on “Vladimir Vapnik Joins Facebook Research” ↗…the reference. From a simple Google, found at http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/vapnik... a review The Bactra Review: Occasional and eclectic book reviews by Cosma Shalizi The Nature of Statistical Learning Theory by V. N. Vapnik Okay, Vapnik is trying to do some mathematics to show some desirable mathematical properties of some of statistical learning theory, maybe in particular some of the algorithms. Good.…
on “Top Algorithms in Data Mining (2008) [pdf]” ↗…Elements of Statistical Learning Data: Mining, Inference, and Prediction, Second Edition, Springer, 2008. I find mention of Vapnik, mostly for separating hyperplanes, and references Vapnik, V. (1996). The Nature of Statistical Learning Theory, Springer, New York. Vapnik, V. (1998). Statistical Learning Theory, Wiley, New York. Looking at Hastie, et al., which seems relatively elementary, I wouldn't expect to find much on the Kuhn-Tucker…
on “Kenneth Arrow and the golden age of economic theory” ↗…Finds Global Minima of Deep Neural Networks. http://proceedings.mlr.press/v97/du19c.html [4] Zhang et al (2016). Understanding deep learning requires rethinking generalization. [5] Vapnik (1999). The nature of statistical learning theory. https://arxiv.org/abs/1611.03530
on “Learning to think critically about machine learning” ↗Thanks for the heads up. I found 38k citations for The Nature of Statistical Learning Theory: https://scholar.google.com/scholar?cluster=86085598803682809... And 25k for Statistical Learning Theory: https://scholar.google.com/scholar?cluster=86748554971781655... My initial estimation of 10k…
on “A map of one million scientific papers from the arXiv” ↗