Supervised Learning : Mathematical Foundations and Real-world Applications

This book discusses the relevance of probabilistic supervised learning, to the pursuit of automated and reliable prediction of an unknown that is in a state of relationship with another variable. The book provides methods for secured mechanistic learning of the function that represents this relationship between the output and input variables, where said learning is undertaken within the remit of real-world information that can be messy in different ways. For example, the available data may be highly multivariate or be high-dimensional, reflecting the nature of the output variable that could be a vector, or matrix, or even higher in dimension, as is often the case in a real-world application. Additionally, the data is noisy, and often it is small to moderately large in size in multiple applications. Another difficulty that regularly arises is that the training dataset – comprising pairs of values of the input and output –is such, that the sought function cannot be captured by a parametric shape, but is instead underlined by an inhomogeneous correlation structure. These difficulties notwithstanding, we desire a streamlined methodology that allows the learning of the inter variable relationship – to ultimately permit fast and reliable predictions of the output, at newly recorded values of the input. In fact, occasions arise when one seeks values of the input at which a new output value is recorded, and such a demand is also addressed in the book. The generic solution to the problem of secured supervised learning amidst real-world messiness, lies in treating the sought inter-variable relation as a (function-valued) random variable, which, being random, is ascribed a probability distribution. Then recalling that distributions on the space of functions are given by stochastic processes, the sought function is proposed to be a sample function of a stochastic process. This process is chosen as one that imposes minimal constraints on the sought function – identified as a Gau