### 使用 LSTM 和 MDN 实现回归预测
混合密度网络（Mixture Density Network, MDN）是一种用于建模复杂分布的方法，它通过组合多个高斯分布来表示目标变量的概率分布。当与循环神经网络（Recurrent Neural Networks, RNNs），特别是长短期记忆网络（Long Short-Term Memory, LSTM）结合时，可以有效地处理时间序列数据并生成复杂的概率分布。
以下是基于 TensorFlow/Keras 的实现示例：
#### 数据准备
为了训练模型，通常需要将输入数据转换为适合 LSTM 处理的时间步格式。假设我们有一个时间序列数据集 `X` 和对应的标签 `Y`。
```python
import numpy as np
from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator
def create_dataset(data, look_back=10):
generator = TimeseriesGenerator(data, data[:, -1], length=look_back, batch_size=32)
X, y = [], []
for i in range(len(generator)):
xi, yi = generator[i]
X.append(xi)
y.append(yi)
return np.vstack(X), np.hstack(y)
# 假设 data 是一个形状为 (n_samples, n_features) 的 NumPy 数组
X_train, y_train = create_dataset(data[:train_split])
X_test, y_test = create_dataset(data[train_split:])
```
#### 构建 LSTM-MDN 模型
下面是一个简单的 LSTM-MDN 结构定义。MDN 部分由多个高斯分布组成，每个高斯分布有均值、标准差和权重参数。
```python
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM, Dense
class MixtureDensityNetwork(tf.keras.Model):
def __init__(self, num_components, input_dim, output_dim):
super(MixtureDensityNetwork, self).__init__()
self.num_components = num_components
self.input_layer = Input(shape=(None, input_dim))
self.lstm_layer = LSTM(64, return_sequences=False)
self.pi_layer = Dense(num_components, activation='softmax') # Mixing coefficients
self.mu_layer = Dense(num_components * output_dim) # Means
self.sigma_layer = Dense(num_components * output_dim, activation='softplus') # Standard deviations
def call(self, inputs):
x = self.lstm_layer(inputs)
pi = self.pi_layer(x)
mu = self.mu_layer(x)
sigma = self.sigma_layer(x)
return pi, mu, sigma
num_components = 5
input_dim = X_train.shape[-1]
output_dim = 1
model = MixtureDensityNetwork(num_components=num_components, input_dim=input_dim, output_dim=output_dim)
pi, mu, sigma = model(model.input_layer)
```
#### 定义损失函数
MDN 的核心在于其自定义的负对数似然损失函数，这使得它可以拟合任意形式的目标分布。
```python
def mdn_loss(pi, mu, sigma, y_true):
dist = tfp.distributions.MixtureSameFamily(
mixture_distribution=tfp.distributions.Categorical(probs=pi),
components_distribution=tfp.distributions.Normal(loc=mu, scale=sigma)
)
log_likelihood = dist.log_prob(tf.reshape(y_true, (-1,)))
loss = -tf.reduce_mean(log_likelihood)
return loss
loss_fn = lambda y_true: mdn_loss(pi, mu, sigma, y_true)
optimizer = tf.keras.optimizers.Adam()
@tf.function
def train_step(x, y):
with tf.GradientTape() as tape:
_, mu_pred, sigma_pred = model(x)
loss_value = loss_fn(y)
gradients = tape.gradient(loss_value, model.trainable_variables)
optimizer.apply_gradients(zip(gradients, model.trainable_variables))
return loss_value
```
#### 训练过程
使用上述定义好的模型和损失函数进行训练。
```python
epochs = 100
batch_size = 32
for epoch in range(epochs):
total_loss = 0.0
steps_per_epoch = len(X_train) // batch_size
dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train)).shuffle(buffer_size=1024).batch(batch_size)
for step, (x_batch_train, y_batch_train) in enumerate(dataset):
loss_value = train_step(x_batch_train, y_batch_train)
total_loss += loss_value
avg_loss = total_loss / steps_per_epoch
print(f"Epoch {epoch+1}, Loss: {avg_loss.numpy():.4f}")
```
#### PyTorch 版本
如果更倾向于使用 PyTorch，则可以通过类似的逻辑构建模型。
```python
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.distributions.normal import Normal
from torch.distributions.categorical import Categorical
from torch.distributions.mixture_same_family import MixtureSameFamily
class LSTMMDN(nn.Module):
def __init__(self, input_dim, hidden_dim, num_layers, num_components):
super(LSTMMDN, self).__init__()
self.lstm = nn.LSTM(input_dim, hidden_dim, num_layers, batch_first=True)
self.pi_fc = nn.Linear(hidden_dim, num_components)
self.mu_fc = nn.Linear(hidden_dim, num_components)
self.sigma_fc = nn.Linear(hidden_dim, num_components)
def forward(self, x):
lstm_out, _ = self.lstm(x)
out = lstm_out[:, -1, :] # 取最后一个时间步的输出
pi = F.softmax(self.pi_fc(out), dim=-1)
mu = self.mu_fc(out)
sigma = F.softplus(self.sigma_fc(out)) + 1e-8
return pi, mu, sigma
def mdn_loss(pi, mu, sigma, y_true):
mix = Categorical(pi)
comp = Normal(mu.unsqueeze(-1), sigma.unsqueeze(-1))
gmm = MixtureSameFamily(mix, comp)
return -gmm.log_prob(y_true).mean()
```
---
###
扫一扫
700
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
