Kernel Adaptive Filtering: A Comprehensive Introduction
Format: PDF / Kindle (mobi) / ePub
Online learning from a signal processing perspective
There is increased interest in kernel learning algorithms in neural networks and a growing need for nonlinear adaptive algorithms in advanced signal processing, communications, and controls. Kernel Adaptive Filtering is the first book to present a comprehensive, unifying introduction to online learning algorithms in reproducing kernel Hilbert spaces. Based on research being conducted in the Computational Neuro-Engineering Laboratory at the University of Florida and in the Cognitive Systems Laboratory at McMaster University, Ontario, Canada, this unique resource elevates the adaptive filtering theory to a new level, presenting a new design methodology of nonlinear adaptive filters.
Covers the kernel least mean squares algorithm, kernel affine projection algorithms, the kernel recursive least squares algorithm, the theory of Gaussian process regression, and the extended kernel recursive least squares algorithm
Presents a powerful model-selection method called maximum marginal likelihood
Addresses the principal bottleneck of kernel adaptive filters—their growing structure
Features twelve computer-oriented experiments to reinforce the concepts, with MATLAB codes downloadable from the authors' Web site
Concludes each chapter with a summary of the state of the art and potential future directions for original research
Kernel Adaptive Filtering is ideal for engineers, computer scientists, and graduate students interested in nonlinear adaptive systems for online applications (applications where the data stream arrives one sample at a time and incremental optimal solutions are desirable). It is also a useful guide for those who look for nonlinear adaptive filtering methodologies to solve practical problems.
one of the early pioneers of communications theory. Hammerstein and Wiener models are discussed in Wiener , Billings and Fakhouri , and Marmarelis . Volterra  and later Gabor  studied the use of Volterra series for nonlinear adaptive ﬁltering. The Wiener series was proposed and studied in Wiener  and Barrett  as an improvement over the Volterra series. For the use of time-lagged multilayer perceptrons, radialbasis function networks, and recurrent neural
that increasing the value of τ in equation (2.74) increases the dimension of the attractor in this chaotic system. This observation, and the simplicity of the equation, has led to the evolution of this equation into a standard model used to test algorithms for nonlinear modeling capability [Farmer and Sidorowich, 1987, Crowder, 1990, Platt, 1991, Martinetz et al., 1993, Mukherjee et al., 1997, Müller et al., 1997]. For more details, please refer to Glass and Mackey  and Beuter et al.
is a quadratic form of the 108 KERNEL RECURSIVE LEAST-SQUARES ALGORITHM test input with the posterior covariance matrix. The predictive uncertainty is large if 1) the magnitude of the test input is large or 2), the direction of the −1 ⎛ 1 ⎞ T test input aligns with the eigenvector of ⎜ 2 U ( i ) U ( i ) + σ w−2 I⎟ associated ⎝ σn ⎠ with the largest eigenvalue. If we assume that the posterior covariance matrix −1 ⎛ 1 ⎞ T is dominated by U(i)U(i)T, then the eigenvector of ⎜ 2 U ( i ) U ( i ) +
KRLS EX-KRLS MSE (dB) −8.12 −11.83 −19.89 −32.53 −44.92 ± ± ± ± ± 0.91 0.71 0.77 1.13 4.80 simulations with different channel parameters and independent training data. In Figure 5.8, the ensemble learning curves are shown by averaging 200 Monte Carlo simulations. The ﬁnal MSE is also listed in Table 5.3. As expected, the EX-KRLS has the best performance, followed by KRLS, EX-RLS, and NLMS. Part 4: In the last simulation, we compare the performance of SW-KRLS with EX-KRLS. Because the modeling
posterior converges quickly. Other non-Gaussian priors are also possible, but the price to pay is a much higher computational complexity. The posterior distribution of the output y(i + 1) given the input u(i + 1), and all past observations D(i) can be derived by the Bayes rule: p ( y ( i + 1) u ( i + 1) , D ( i )) = p ( y ( i + 1) y (1) , , y ( i ) , u (1) , , u ( i ) , u ( i + 1)) p ( y (1) , , y ( i ) , y ( i + 1) u (1) , , u ( i ) , u ( i + 1)) = p ( y (1) , , y ( i ) u (1) , , u ( i ) , u ( i