Gaussian Process(高斯过程) GPSS暑校笔记六(英文版)——随机微分方程

本文探讨了为何使用随机微分方程(SDE)求解器来处理高斯过程,主要由于其能有效降低计算复杂性。内容涉及Ornstein-Uhlenbeck过程,它具有一个SDE的路径表示形式,并且通过高斯过程回归问题展示了等效的状态空间模型,可以通过Kalman滤波器/平滑器在O(n)时间内求解。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

Basic Ideas

Why use S§DE solvers for GPs?

  • The O(n3)O(n^3)O(n3) computational complexity is a challenge.
  • What do we get:
    • O(n)O(n)O(n) state-space methods for SDEs/SPDEs.
    • Sparse approximations developed for SPDEs.
    • Reduced rank Fourier/basis function approximations. Path to non-Gaussian processes.
  • Downsides:
    • We often need to approximate.
    • Mathematics can become messy

Stochastic differential equations and Gaussian processes

Ornstein-Uhlenbeck process

The mean and covariance functions:
m(x)=0k(x,x′)=σ2exp⁡(−λ∣x−x′∣) \begin{aligned} m(x) &=0 \\ k\left(x, x^{\prime}\right) &=\sigma^{2} \exp \left(-\lambda\left|x-x^{\prime}\right|\right) \end{aligned} m(x)k(x,x)=0=σ2exp(λxx)
This has a path representation as a stochastic differential equation (SDE):
df(t)dt=−λf(t)+w(t) \frac{d f(t)}{d t}=-\lambda f(t)+w(t) dtdf(t)=λf(t)+w(t)
where w(t)w(t)w(t) is a white noise process with xxx relabeled as ttt.

Prove:
FT:(iω)f^=−λf^+ω^f^=ω^λ+(iω)SpectralDensity:δ(ω)=E[∣w^∣2]w2+λ2=qw2+λ2IF:h(τ)=12π∫qw2+λ2exp⁡(iwτ)dτ \begin{aligned} FT: (i \omega) \hat{f} &= -\lambda \hat{f} + \hat{\omega} \\ \hat{f} &= \frac{\hat{\omega}}{\lambda +(i \omega) } \\ Spectral Density: \delta(\omega) &= \frac{{E}[|\hat{w}|^{2}]}{w^2+\lambda^2} = \frac{q}{w^2+\lambda^2}\\ IF:h(\tau) &= \frac{1}{2 \pi} \int \frac{q}{w^2+\lambda^2} \exp(iw\tau) d\tau\\ \end{aligned} FT:(iω)f^f^SpectralDensity:δ(ω)IF:h(τ)=λf^+ω^=λ+(iω)ω^=w2+λ2E[w^2]=w2+λ2q=2π1w2+λ2qexp(iwτ)dτ
Consider a Gaussian process regression problem:
f(x)∼GP(0,σ2exp⁡(−λ∣x−x′∣))yk=f(xk)+εk \begin{aligned} f(x) & \sim \mathrm{GP}\left(0, \sigma^{2} \exp \left(-\lambda\left|x-x^{\prime}\right|\right)\right) \\ y_{k} &=f\left(x_{k}\right)+\varepsilon_{k} \end{aligned} f(x)ykGP(0,σ2exp(λxx))=f(xk)+εk
this is equivalent to the state-space model:
df(t)dt=−λf(t)+w(t)yk=f(tk)+εk \begin{aligned} \frac{d f(t)}{d t} &=-\lambda f(t)+w(t) \\ y_{k} &=f\left(t_{k}\right)+\varepsilon_{k} \end{aligned} dtdf(t)yk=λf(t)+w(t)=f(tk)+εk
that is, with fk=f(tk)fk = f(t_k)fk=f(tk) we have a Gauss-Markov model
fk+1∼p(fk+1∣fk)yk∼p(yk∣fk) \begin{aligned} f_{k+1} & \sim p\left(f_{k+1} | f_{k}\right) \\ y_{k} & \sim p\left(y_{k} | f_{k}\right) \end{aligned} fk+1ykp(fk+1fk)p(ykfk)
Solvable in O(n)O(n)O(n) time using Kalman filter/smoother
ZZPGcV.png

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值