gives examples of applications, presents the
basic tools of soft computing (neural networks, support vector
machines, and fuzzy logic models), reviews the classical
problems of approximation of multivariate functions, and
introduces the standard statistical approaches to regression
and classification that are based on the knowledge of probability-density
functions.
presents the basics of statistical learning theory
when there is no information about the probability distribution
but only experimental data. The VC dimension and structural
risk minimization are introduced. A description is given
of the SVM learning algorithm based on quadratic programming
that leads to parsimonious SVMs, that is, NNs or SVMs having
a small number of hidden layer neurons. This parsimony results
from sophisticated learning that matches model capacity to
data complexity. In this way, good generalization, meaning
the performance of the SVM on previously unseen data, is
assured.
deals with two early learning units - the perceptron
and the linear neuron (adaline) - as well as with single-layer
networks. Five different learning algorithms for the linear
activation function are presented. Despite the fact that
the linear neuron appears to be very simple, it is the constitutive
part of almost all models treated here and therefore is a
very important processing unit. The linear neuron can be
looked upon as a graphical (network) representation of classical
linear regression and linear classification (discriminant
analysis) schemes.
A
genuine neural network (a multilayer perceptron) - one that
comprises at least one hidden layer having neurons with nonlinear
activation functions - is introduced in . The
error-correction type of learning, introduced for single-layer
networks in chapter 3, is generalized, and the gradient-based
learning method known as error backpropagation is discussed
in detail here. Also shown are some of the generally accepted
heuristics while training multilayer perceptrons.
is concerned with regularization networks, which are
better known as radial basis function (RBF) networks. The
notion of ill-posed problems is discussed as well as how
regularization leads to networks whose activation functions
are radially symmetric. Details are provided on how to find a parsimonious
radial basis network by applying the orthogonal least squares approach. Also
explored is a linear programming approach to subset (basis function or support
vector) selection that, similar to the QP based algorithm for SVMs training,
leads to parsimonious NNs and SVMs.
Fuzzy
logic modeling is the subject of . Basic notions
of fuzzy modeling are introduced - fuzzy sets, relations, compositions
of fuzzy relations, fuzzy inference, and defuzzification. The
union, intersection, and Cartesian product of a family of sets
are described, and various properties are established. The
similarity between, and sometimes even the equivalence of,
RBF networks and fuzzy models is noted in detail. Finally,
fuzzy additive models (FAMs) are presented as a simple yet
powerful fuzzy modeling technique. FAMs are the most popular
type of fuzzy models in applications today.
presents three case studies that show the beauty and
strength of these modeling tools.
by applying neural
networks or fuzzy models are discussed at length.
focuses on the most popular classical approaches to
nonlinear optimization, which is the crucial part of learning
from data. It also describes the novel massive search algorithms
known as genetic algorithms or evolutionary computing.
contains specific mathematical topics and tools that
might be helpful for understanding the theoretical aspects
of soft models, although these concepts and tools are not
covered in great detail. It is supposed that the reader has
some knowledge of probability theory, linear algebra, and
vector calculus. Chapter 9 is designed only for easy reference
of properties and notation.
