Kernel Adaptive Filtering: A Comprehensive Introduction

Kernel Adaptive Filtering: A Comprehensive Introduction

Language: English

Pages: 240

ISBN: 0470447532

Format: PDF / Kindle (mobi) / ePub

Online learning from a signal processing perspective

There is increased interest in kernel learning algorithms in neural networks and a growing need for nonlinear adaptive algorithms in advanced signal processing, communications, and controls. Kernel Adaptive Filtering is the first book to present a comprehensive, unifying introduction to online learning algorithms in reproducing kernel Hilbert spaces. Based on research being conducted in the Computational Neuro-Engineering Laboratory at the University of Florida and in the Cognitive Systems Laboratory at McMaster University, Ontario, Canada, this unique resource elevates the adaptive filtering theory to a new level, presenting a new design methodology of nonlinear adaptive filters.

  • Covers the kernel least mean squares algorithm, kernel affine projection algorithms, the kernel recursive least squares algorithm, the theory of Gaussian process regression, and the extended kernel recursive least squares algorithm

  • Presents a powerful model-selection method called maximum marginal likelihood

  • Addresses the principal bottleneck of kernel adaptive filters—their growing structure

  • Features twelve computer-oriented experiments to reinforce the concepts, with MATLAB codes downloadable from the authors' Web site

  • Concludes each chapter with a summary of the state of the art and potential future directions for original research

Kernel Adaptive Filtering is ideal for engineers, computer scientists, and graduate students interested in nonlinear adaptive systems for online applications (applications where the data stream arrives one sample at a time and incremental optimal solutions are desirable). It is also a useful guide for those who look for nonlinear adaptive filtering methodologies to solve practical problems.















one of the early pioneers of communications theory. Hammerstein and Wiener models are discussed in Wiener [1958], Billings and Fakhouri [1982], and Marmarelis [1993]. Volterra [1887] and later Gabor [1968] studied the use of Volterra series for nonlinear adaptive filtering. The Wiener series was proposed and studied in Wiener [1958] and Barrett [1963] as an improvement over the Volterra series. For the use of time-lagged multilayer perceptrons, radialbasis function networks, and recurrent neural

that increasing the value of τ in equation (2.74) increases the dimension of the attractor in this chaotic system. This observation, and the simplicity of the equation, has led to the evolution of this equation into a standard model used to test algorithms for nonlinear modeling capability [Farmer and Sidorowich, 1987, Crowder, 1990, Platt, 1991, Martinetz et al., 1993, Mukherjee et al., 1997, Müller et al., 1997]. For more details, please refer to Glass and Mackey [1988] and Beuter et al.

is a quadratic form of the 108 KERNEL RECURSIVE LEAST-SQUARES ALGORITHM test input with the posterior covariance matrix. The predictive uncertainty is large if 1) the magnitude of the test input is large or 2), the direction of the −1 ⎛ 1 ⎞ T test input aligns with the eigenvector of ⎜ 2 U ( i ) U ( i ) + σ w−2 I⎟ associated ⎝ σn ⎠ with the largest eigenvalue. If we assume that the posterior covariance matrix −1 ⎛ 1 ⎞ T is dominated by U(i)U(i)T, then the eigenvector of ⎜ 2 U ( i ) U ( i ) +

KRLS EX-KRLS MSE (dB) −8.12 −11.83 −19.89 −32.53 −44.92 ± ± ± ± ± 0.91 0.71 0.77 1.13 4.80 simulations with different channel parameters and independent training data. In Figure 5.8, the ensemble learning curves are shown by averaging 200 Monte Carlo simulations. The final MSE is also listed in Table 5.3. As expected, the EX-KRLS has the best performance, followed by KRLS, EX-RLS, and NLMS. Part 4: In the last simulation, we compare the performance of SW-KRLS with EX-KRLS. Because the modeling

posterior converges quickly. Other non-Gaussian priors are also possible, but the price to pay is a much higher computational complexity. The posterior distribution of the output y(i + 1) given the input u(i + 1), and all past observations D(i) can be derived by the Bayes rule: p ( y ( i + 1) u ( i + 1) , D ( i )) = p ( y ( i + 1) y (1) , , y ( i ) , u (1) , , u ( i ) , u ( i + 1)) p ( y (1) , , y ( i ) , y ( i + 1) u (1) , , u ( i ) , u ( i + 1)) = p ( y (1) , , y ( i ) u (1) , , u ( i ) , u ( i

Download sample