82、Understanding Radial Basis Function Networks and Kernel Methods

Understanding Radial Basis Function Networks and Kernel Methods

1. Introduction to Kernel Functions

Kernel functions play a crucial role in various machine - learning algorithms. Here, we’ll explore different types of kernel functions and then dive into radial basis function networks.

1.1 Fisher Kernel

The covariance matrix of the Fisher scores is related to the Fisher kernel. The Fisher kernel corresponds to a whitening of these scores. A simpler approach is to omit the Fisher information matrix and use the non - invariant kernel:
[k(x, x’) = g(\theta, x)^T g(\theta, x’)]
Hofmann (2000) has given an application of Fisher kernels to document retrieval.

1.2 Sigmoidal Kernel

The sigmoidal kernel is defined as: