统计学习理论是针对小样本情况研究统计学习规律的理论,是传统统汁学的重要发展和补充,为研究有限样本情况下机器学习的理论和方法提供了理论框架,其核心思想是通过控制学习机器的容量实现对推广能力的控制。在这一理论中发展出的支持向量机方法是一种新的通用学习机器,较以往方法表现出很多理论和实践上的优势。本书是该领域的权威著作,由该领域的创立者来讲述统计学习理论的本质,着重介绍了统计学习理论和支持向量机的关键思想、结论和方法,以及该领域的最新进展。
Four years have passed since the first edition of this book. These years were "fast time" in the development of new approaches in statistical inference inspired by learning theory.
During this time, new function estimation methods have been created where a high dimensionality of the unknown function does not always require a large number of observations in order to obtain a good estimate.The new methods control generalization using capacity factors that do not necessarily depend on dimensionality of the space.
These factors were known in the VC theory for many years. However,the practical significance of capacity control has become clear only recently after the appearance of support vector machines (SVM). In contrast to classical methods of statistics where in order to control performance one decreases the dimensionality of a feature space, the SVM dramatically increases dimensionality and relies on the so-called large margin factor.
In the first edition of this book general learning theory including SVM methods was introduced. At that time SVM methods oflearning were brand new, some of them were introduced for a first time. Now SVM margin control methods represents one of the most important directions both in theory and application of learning.
In the second edition of the book three new chapters devoted to the SVM methods were added. They include generalization of SVM method for estimating real-valued functions, direct methods of learning based on solving (using SVM) multidimensional integral equations, and extension of the empirical risk minimization principle and its application to SVM.
The years since the first edition of the book have also changed the general philosophy in our understanding the of nature of the induction problem.After many successful experiments with SVM, researchers became more determined in criticism of the classical philosophy of generalization based on the principle of Occam's razor.
This intellectual determination also is a very important part of scientific achievement. Note that the creation of the new methods of inference could have happened in the early 1970: All the necessary elements of the theory and the SVM algorithm were known. It took twenty-five years to reach this intellectual determination.
Now the analysis of generalization from the pure theoretical issues become a very practical subject, and this fact adds important details to ageneral picture of the developing computer learning problem described inthe first edition of the book.