官网实例详解4.19（lstm_seq2seq_restore.py）-keras学习笔记四

最新推荐文章于 2024-05-19 00:24:49 发布
原创 最新推荐文章于 2024-05-19 00:24:49 发布 · 733 阅读 · 2 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#深度学习 #keras

从磁盘恢复字符级序列到序列模型，并用它生成预测。

 

Keras实例目录

代码注释

'''Restore a character-level sequence to sequence model from disk and use it
to generate predictions.
从磁盘恢复字符级序列到序列模型，并用它生成预测。
This script loads the s2s.h5 model saved by lstm_seq2seq.py and generates
sequences from it.  It assumes that no changes have been made (for example:
latent_dim is unchanged, and the input data and model architecture are unchanged).
脚本加载lstm_seq2seq.py 保存的s2s.h5模型并从中生成序列。假定没有进行任何更改（例如：latent_dim是
不变的，并且输入数据和模型架构不变）。

See lstm_seq2seq.py for more details on the model architecture and how
it is trained.
有关模型架构的更多细节和如何训练（模型），请参考lstm_seq2seq.py
'''
from __future__ import print_function

from keras.models import Model, load_model
from keras.layers import Input
import numpy as np

batch_size = 64  # Batch size for training. 训练批次大小（每个批次包含样本数）
epochs = 100  # Number of epochs to train for.训练周期数
latent_dim = 256  # Latent dimensionality of the encoding space.编码空间的潜在维数
num_samples = 10000  # Number of samples to train on.训练集样本数
# Path to the data txt file on disk.
# 存储器数据文件路径（先下载，然后存放到和本脚本同级目录）
data_path = 'fra-eng/fra.txt'

# Vectorize the data.  We use the same approach as the training script.
# 数据向量化。使用与训练脚本相同的方法。
# NOTE: the data must be identical, in order for the character -> integer
# mappings to be consistent.
# 注意：为了使字符->整数映射一致，数据必须是相同的
# We omit encoding target_texts since they are not needed.
# 省略了target_texts的编码，因为不需要。
input_texts = []
target_texts = []
input_characters = set()
target_characters = set()
with open(data_path, 'r', encoding='utf-8') as f:
    lines = f.read().split('\n')
for line in lines[: min(num_samples, len(lines) - 1)]:
    input_text, target_text = line.split('\t')
    # We use "tab" as the "start sequence" character
    # 使用"tab"为开始序列字符
    # for the targets, and "\n" as "end sequence" character.
    # 使用“tab”作为目标的“开始序列”字符，而“\n”作为“结束序列”字符。
    target_text = '\t' + target_text + '\n'
    input_texts.append(input_text)
    target_texts.append(target_text)
    for char in input_text:
        if char not in input_characters:
            input_characters.add(char)
    for char in target_text:
        if char not in target_characters:
            target_characters.add(char)

input_characters = sorted(list(input_characters))
target_characters = sorted(list(target_characters))
num_encoder_tokens = len(input_characters)
num_decoder_tokens = len(target_characters)
max_encoder_seq_length = max([len(txt) for txt in input_texts])
max_decoder_seq_length = max([len(txt) for txt in target_texts])

print('Number of samples:', len(input_texts))
print('Number of unique input tokens:', num_encoder_tokens)
print('Number of unique output tokens:', num_decoder_tokens)
print('Max sequence length for inputs:', max_encoder_seq_length)
print('Max sequence length for outputs:', max_decoder_seq_length)

input_token_index = dict(
    [(char, i) for i, char in enumerate(input_characters)])
target_token_index = dict(
    [(char, i) for i, char in enumerate(target_characters)])

encoder_input_data = np.zeros(
    (len(input_texts), max_encoder_seq_length, num_encoder_tokens),
    dtype='float32')

for i, input_text in enumerate(input_texts):
    for t, char in enumerate(input_text):
        encoder_input_data[i, t, input_token_index[char]] = 1.

# Restore the model and construct the encoder and decoder.
# 恢复模型并构造编码器和解码器。
model = load_model('s2s.h5')

encoder_inputs = model.input[0]   # input_1
encoder_outputs, state_h_enc, state_c_enc = model.layers[2].output   # lstm_1
encoder_states = [state_h_enc, state_c_enc]
encoder_model = Model(encoder_inputs, encoder_states)

decoder_inputs = model.input[1]   # input_2
decoder_state_input_h = Input(shape=(latent_dim,), name='input_3')
decoder_state_input_c = Input(shape=(latent_dim,), name='input_4')
decoder_states_inputs = [decoder_state_input_h, decoder_state_input_c]
decoder_lstm = model.layers[3]
decoder_outputs, state_h_dec, state_c_dec = decoder_lstm(
    decoder_inputs, initial_state=decoder_states_inputs)
decoder_states = [state_h_dec, state_c_dec]
decoder_dense = model.layers[4]
decoder_outputs = decoder_dense(decoder_outputs)
decoder_model = Model(
    [decoder_inputs] + decoder_states_inputs,
    [decoder_outputs] + decoder_states)

# Reverse-lookup token index to decode sequences back to
# something readable.
# 反向查找分词索引（词的编号）将序列解码为可读的。
reverse_input_char_index = dict(
    (i, char) for char, i in input_token_index.items())
reverse_target_char_index = dict(
    (i, char) for char, i in target_token_index.items())


# Decodes an input sequence.  Future work should support beam search.
# 解码输入序列。未来的工作应该支持波束搜索。
def decode_sequence(input_seq):
    # Encode the input as state vectors.
    # 将输入（数据）编码为状态向量。
    states_value = encoder_model.predict(input_seq)

    # Generate empty target sequence of length 1.
    # 生成长度为1的空目标序列。
    target_seq = np.zeros((1, 1, num_decoder_tokens))
    # Populate the first character of target sequence with the start character.
    # 用开始字符填充目标序列的第一个字符。
    target_seq[0, 0, target_token_index['\t']] = 1.

    # Sampling loop for a batch of sequences
    # (to simplify, here we assume a batch of size 1).
    # 一批序列的采样循环（为了简化，这里假设一批次大小为1）。
    stop_condition = False
    decoded_sentence = ''
    while not stop_condition:
        output_tokens, h, c = decoder_model.predict(
            [target_seq] + states_value)

        # Sample a token
        # 采样
        sampled_token_index = np.argmax(output_tokens[0, -1, :])
        sampled_char = reverse_target_char_index[sampled_token_index]
        decoded_sentence += sampled_char

        # Exit condition: either hit max length
        # or find stop character.
        # 退出条件：到达最大长度或找到停止字符。
        if (sampled_char == '\n' or
           len(decoded_sentence) > max_decoder_seq_length):
            stop_condition = True

        # Update the target sequence (of length 1).
        # 更新目标序列（长度1）。
        target_seq = np.zeros((1, 1, num_decoder_tokens))
        target_seq[0, 0, sampled_token_index] = 1.

        # Update states
        # 更新状态
        states_value = [h, c]

    return decoded_sentence


for seq_index in range(100):
    # Take one sequence (part of the training set)
    # for trying out decoding.
    # 用一个序列（训练集的一部分）进行解码。
    input_seq = encoder_input_data[seq_index: seq_index + 1]
    decoded_sentence = decode_sequence(input_seq)
    print('-')
    print('Input sentence:', input_texts[seq_index])
    print('Decoded sentence:', decoded_sentence)

代码执行

C:\ProgramData\Anaconda3\python.exe E:/keras-master/examples/lstm_seq2seq_restore.py
Using TensorFlow backend.
Number of samples: 10000
Number of unique input tokens: 71
Number of unique output tokens: 94
Max sequence length for inputs: 16
Max sequence length for outputs: 59
-
Input sentence: Go.
Decoded sentence: Va !

-
Input sentence: Run!
Decoded sentence: Cours !

-
Input sentence: Run!
Decoded sentence: Cours !

-
Input sentence: Fire!
Decoded sentence: Au feu !

-
Input sentence: Help!
Decoded sentence: À l'aide !

-
Input sentence: Jump.
Decoded sentence: Saute.

-
Input sentence: Stop!
Decoded sentence: Arrête-toi !

-
Input sentence: Stop!
Decoded sentence: Arrête-toi !

-
Input sentence: Stop!
Decoded sentence: Arrête-toi !

-
Input sentence: Wait!
Decoded sentence: Attends !

-
Input sentence: Wait!
Decoded sentence: Attends !

-
Input sentence: Go on.
Decoded sentence: Continuez.

-
Input sentence: Go on.
Decoded sentence: Continuez.

-
Input sentence: Go on.
Decoded sentence: Continuez.

-
Input sentence: I see.
Decoded sentence: Je vois une lumière.

-
Input sentence: I try.
Decoded sentence: J'essaye.

-
Input sentence: I won!
Decoded sentence: J'ai demandé à dore.

-
Input sentence: I won!
Decoded sentence: J'ai demandé à dore.

-
Input sentence: Oh no!
Decoded sentence: Oh non !

-
Input sentence: Attack!
Decoded sentence: Attaquez !

-
Input sentence: Attack!
Decoded sentence: Attaquez !

-
Input sentence: Cheers!
Decoded sentence: À votre santé !

-
Input sentence: Cheers!
Decoded sentence: À votre santé !

-
Input sentence: Cheers!
Decoded sentence: À votre santé !

-
Input sentence: Cheers!
Decoded sentence: À votre santé !

-
Input sentence: Get up.
Decoded sentence: Lève-toi.

-
Input sentence: Go now.
Decoded sentence: Va doucement !

-
Input sentence: Go now.
Decoded sentence: Va doucement !

-
Input sentence: Go now.
Decoded sentence: Va doucement !

-
Input sentence: Got it!
Decoded sentence: Compris !

-
Input sentence: Got it!
Decoded sentence: Compris !

-
Input sentence: Got it?
Decoded sentence: Compris ?

-
Input sentence: Got it?
Decoded sentence: Compris ?

-
Input sentence: Got it?
Decoded sentence: Compris ?

-
Input sentence: Hop in.
Decoded sentence: Montez.

-
Input sentence: Hop in.
Decoded sentence: Montez.

-
Input sentence: Hug me.
Decoded sentence: Serre-moi dans tes bras !

-
Input sentence: Hug me.
Decoded sentence: Serre-moi dans tes bras !

-
Input sentence: I fell.
Decoded sentence: Je suis tombée.

-
Input sentence: I fell.
Decoded sentence: Je suis tombée.

-
Input sentence: I know.
Decoded sentence: Je sais.

-
Input sentence: I left.
Decoded sentence: Je suis parti.

-
Input sentence: I left.
Decoded sentence: Je suis parti.

-
Input sentence: I lost.
Decoded sentence: J'ai perdu.

-
Input sentence: I'm 19.
Decoded sentence: J'ai les chocontes.

-
Input sentence: I'm OK.
Decoded sentence: Je vais bien.

-
Input sentence: I'm OK.
Decoded sentence: Je vais bien.

-
Input sentence: Listen.
Decoded sentence: Écoutez !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: No way!
Decoded sentence: C'est exclu !

-
Input sentence: Really?
Decoded sentence: Vrai ?

-
Input sentence: Really?
Decoded sentence: Vrai ?

-
Input sentence: Really?
Decoded sentence: Vrai ?

-
Input sentence: Thanks.
Decoded sentence: Merci !

-
Input sentence: We try.
Decoded sentence: On essaye.

-
Input sentence: We won.
Decoded sentence: Nous avons réveillé.

-
Input sentence: We won.
Decoded sentence: Nous avons réveillé.

-
Input sentence: We won.
Decoded sentence: Nous avons réveillé.

-
Input sentence: We won.
Decoded sentence: Nous avons réveillé.

-
Input sentence: Ask Tom.
Decoded sentence: Demande-leur.

-
Input sentence: Awesome!
Decoded sentence: Faisalez-moi !

-
Input sentence: Be calm.
Decoded sentence: Sois calme !

-
Input sentence: Be calm.
Decoded sentence: Sois calme !

-
Input sentence: Be calm.
Decoded sentence: Sois calme !

-
Input sentence: Be cool.
Decoded sentence: Sois détendu !

-
Input sentence: Be fair.
Decoded sentence: Soyez équitables !

-
Input sentence: Be fair.
Decoded sentence: Soyez équitables !

-
Input sentence: Be fair.
Decoded sentence: Soyez équitables !

-
Input sentence: Be fair.
Decoded sentence: Soyez équitables !

-
Input sentence: Be fair.
Decoded sentence: Soyez équitables !

-
Input sentence: Be fair.
Decoded sentence: Soyez équitables !

-
Input sentence: Be kind.
Decoded sentence: Sois gentil.

-
Input sentence: Be nice.
Decoded sentence: Sois gentille !

-
Input sentence: Be nice.
Decoded sentence: Sois gentille !

-
Input sentence: Be nice.
Decoded sentence: Sois gentille !

-
Input sentence: Be nice.
Decoded sentence: Sois gentille !

-
Input sentence: Be nice.
Decoded sentence: Sois gentille !

-
Input sentence: Be nice.
Decoded sentence: Sois gentille !

-
Input sentence: Beat it.
Decoded sentence: Dégage !

-
Input sentence: Call me.
Decoded sentence: Appellez-moi !

-
Input sentence: Call me.
Decoded sentence: Appellez-moi !

-
Input sentence: Call us.
Decoded sentence: Appelle-nous !

-
Input sentence: Call us.
Decoded sentence: Appelle-nous !

-
Input sentence: Come in.
Decoded sentence: Entrez !

-
Input sentence: Come in.
Decoded sentence: Entrez !

-
Input sentence: Come in.
Decoded sentence: Entrez !

-
Input sentence: Come in.
Decoded sentence: Entrez !

-
Input sentence: Come on!
Decoded sentence: Allez !

-
Input sentence: Come on.
Decoded sentence: Viens !

-
Input sentence: Come on.
Decoded sentence: Viens !

-
Input sentence: Come on.
Decoded sentence: Viens !

-
Input sentence: Drop it!
Decoded sentence: Laissez-le tomber !

-
Input sentence: Drop it!
Decoded sentence: Laissez-le tomber !


Process finished with exit code 0

Keras详细介绍

英文：https://keras.io/

中文：http://keras-cn.readthedocs.io/en/latest/

实例下载

https://github.com/keras-team/keras

https://github.com/keras-team/keras/tree/master/examples

完整项目下载

方便没积分童鞋，请加企鹅452205574，共享文件夹。

包括：代码、数据集合（图片）、已生成model、安装库文件等。


自然语言处理之文本摘要：使用Keras实现Seq2Seq模型
zhubeibei168的博客
05-31 679
序列模型是处理序列数据的神经网络，如时间序列或文本序列。在NLP中，序列模型如循环神经网络(RNN)、长短期记忆网络(LSTM)和门控循环单元(GRU)被广泛使用，它们能够捕捉序列中的长期依赖关系，对于文本摘要、机器翻译等任务非常有效。编码器-解码器架构是Seq2Seq模型的核心，它通过将输入序列转换为固定长度的向量，然后使用这个向量来生成输出序列。这种架构特别适用于序列生成任务，如机器翻译、文本摘要等。Seq2Seq模型的性能可以通过调整超参数来优化。
lstm_seq2seq.py例子
01-28
keras 官方例子，深度学习专用，机器学习专用，代码简单，
keras seq2seq_在Keras中建立具有Luong注意的Seq2Seq LSTM以进行时间序列预测
weixin_26752765的博客
09-05 2604
keras seq2seqDo you want to try some other methods to solve your forecasting problem rather than traditional regression? There are many neural network architectures, which are frequently applied in NL...
lstm_seq2seq_restore.py例子
01-28
keras 官方例子，深度学习专用，机器学习专用，代码简单，
官网实例详解4.18（lstm_seq2seq.py）-keras学习笔记四
wyx100的专栏
06-27 2706
 Keras实例目录代码注释 代码执行 Keras详细介绍英文：https://keras.io/中文：http://keras-cn.readthedocs.io/en/latest/实例下载https://github.com/keras-team/kerashttps://github.com/keras-team/keras/tree/master/examples完整项目下载方便没积分童...
tensorflow lstm 预测_解析seq2seq原理+tensorflow实现
weixin_39688170的博客
12-08 723
1 写在前面最近抽时间写文章，将自己以前学过的知识总结一下，通过文章记录下来，以后忘记了也可以随时翻阅。本文主要介绍最基础的seq2seq模型，并尝试使用tensorflow实现。很多朋友都说pytorch很香，所以随后自己有时间也想入门pytorch，尝试写一个pytorch的版本seq2seq。2 seq2seq模型图1 seq2seq模型结合图1，我们介绍下seq2seq模型。seq2seq...
import os os.environ['KMP_DUPLICATE_LIB_OK'] = 'True' import torch import torch.nn as nn import torch.optim as optim import torch.utils.data as Data import numpy as np import pandas as pd from sklearn.preprocessing import StandardScaler from sklearn.model_selection import train_test_split import matplotlib.pyplot as plt from joblib import dump, load from time import time from mealpy.swarm_based import PSO from mealpy.evolutionary_based import GA from sko.SA import SA as SKO_SA from mealpy.swarm_based import ACOR from mealpy.swarm_based import WOA from mealpy.swarm_based import GWO # ==================== 1. 设备设置与随机种子 ==================== torch.manual_seed(100) device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # ==================== 2. 数据加载与预处理 ==================== data = pd.read_csv('D:/PycharmProjects/PythonProject3/liaotou4.csv') feature_columns = data.columns[1:-2] target_columns = data.columns[-2:] # 可多目标 X_all, y_all = data[feature_columns].values, data[target_columns].values # 特征归一化 scaler_x = StandardScaler() X_all = scaler_x.fit_transform(X_all) dump(scaler_x, 'scaler_x') # 目标归一化 scaler_y = StandardScaler() y_all = scaler_y.fit_transform(y_all) dump(scaler_y, 'scaler_y') # 构建序列数据 seq_len = 60 X_seq, y_seq = [], [] for i in range(len(X_all) - seq_len): X_seq.append(X_all[i:i+seq_len]) y_seq.append(y_all[i+seq_len]) X_seq = torch.tensor(np.array(X_seq), dtype=torch.float32) y_seq = torch.tensor(np.array(y_seq), dtype=torch.float32) # 数据集划分 # 数据集按时序划分 train_size = int(0.7 * len(X_seq)) train_x, test_x = X_seq[:train_size], X_seq[train_size:] train_y, test_y = y_seq[:train_size], y_seq[train_size:] batch_size = 64 train_loader = Data.DataLoader(Data.TensorDataset(train_x, train_y), batch_size=batch_size, shuffle=True, drop_last=True) test_loader = Data.DataLoader(Data.TensorDataset(test_x, test_y), batch_size=batch_size, drop_last=True) # ==================== 3. 定义模型 ==================== from torch.nn import TransformerEncoder, TransformerEncoderLayer class TransformerBiLSTM(nn.Module): def __init__(self, input_dim, hidden_layer_sizes, hidden_dim, num_layers, num_heads, output_dim, dropout_rate=0.5): super().__init__() self.transformer = TransformerEncoder( TransformerEncoderLayer(input_dim, num_heads, hidden_dim, dropout=dropout_rate, batch_first=True), num_layers ) self.num_layers = len(hidden_layer_sizes) self.bilstm_layers = nn.ModuleList() self.bilstm_layers.append(nn.LSTM(input_dim, hidden_layer_sizes[0], batch_first=True, bidirectional=True)) for i in range(1, self.num_layers): self.bilstm_layers.append(nn.LSTM(hidden_layer_sizes[i-1]*2, hidden_layer_sizes[i], batch_first=True, bidirectional=True)) self.linear = nn.Linear(hidden_layer_sizes[-1]*2, output_dim) def forward(self, input_seq): transformer_output = self.transformer(input_seq) bilstm_out = transformer_output for bilstm in self.bilstm_layers: bilstm_out, _ = bilstm(bilstm_out) predict = self.linear(bilstm_out[:, -1, :]) return predict # ==================== 4. VPPSO算法实现 ==================== def vppso( func, dim, bounds, N=12, N1=6, N2=6, T=16, a=0.3, c1=2.0, c2=2.0, b=1.0, verbose=True ): X = np.random.uniform([b[0] for b in bounds], [b[1] for b in bounds], (N, dim)) V = np.zeros((N, dim)) Pbest = X.copy() Pbest_f = np.array([func(x) for x in X]) gbest_idx = np.argmin(Pbest_f) Gbest = Pbest[gbest_idx].copy() Gbest_f = Pbest_f[gbest_idx] best_curve = [Gbest_f] for t in range(T): alpha_t = np.exp(-b * (t / T) ** b) for i in range(N): if i < N1: if np.random.rand() < a: V[i] = V[i] else: r1, r2, r3 = np.random.rand(3) V[i] = (V[i] * r1 * alpha_t + c1 * r2 * (Pbest[i] - X[i]) + c2 * r3 * (Gbest - X[i])) X[i] = X[i] + V[i] else: if np.random.rand() < 0.5: X[i] = Gbest + alpha_t * np.random.rand(dim) * np.abs(Gbest * alpha_t) else: X[i] = Gbest - alpha_t * np.random.rand(dim) * np.abs(Gbest * alpha_t) # 边界处理 for d in range(dim): if X[i, d] < bounds[d][0]: X[i, d] = bounds[d][0] if X[i, d] > bounds[d][1]: X[i, d] = bounds[d][1] # 适应度 F = np.array([func(x) for x in X]) for i in range(N): if i < N1: if F[i] < Pbest_f[i]: Pbest[i] = X[i].copy() Pbest_f[i] = F[i] if F[i] < Gbest_f: Gbest = X[i].copy() Gbest_f = F[i] else: if F[i] < Gbest_f: Gbest = X[i].copy() Gbest_f = F[i] best_curve.append(Gbest_f) if verbose and (t % 4 == 0 or t == T-1): print(f"Iter {t+1}/{T}, Best fitness: {Gbest_f}") return Gbest, Gbest_f, best_curve # ==================== 5. 定义超参数搜索空间与适应度函数 ==================== param_bounds = [ (32, 128), # hidden_layer_sizes[0] (32, 128), # hidden_layer_sizes[1] (64, 256), # hidden_dim (1, 4), # num_layers (1, 4), # num_heads (0.05, 0.5), # dropout_rate (0.00005, 0.005) # learning rate ] def eval_model_hyperparams(x): h1 = int(round(x[0])) h2 = int(round(x[1])) hidden_dim = int(round(x[2])) num_layers = int(round(x[3])) num_heads = int(round(x[4])) dropout = float(x[5]) lr = float(x[6]) try: model = TransformerBiLSTM( input_dim=X_seq.shape[2], hidden_layer_sizes=[h1, h2], hidden_dim=hidden_dim, num_layers=num_layers, num_heads=num_heads, output_dim=y_seq.shape[1], dropout_rate=dropout ).to(device) optimizer = optim.Adam(model.parameters(), lr) loss_function = nn.MSELoss(reduction='sum') best_mse = 1000. for epoch in range(4): # 搜索时只训练4个epoch model.train() for seq, labels in train_loader: seq, labels = seq.to(device), labels.to(device) optimizer.zero_grad() y_pred = model(seq) loss = loss_function(y_pred, labels) loss.backward() optimizer.step() model.eval() with torch.no_grad(): test_loss = 0. test_size = len(test_loader.dataset) for data, label in test_loader: data, label = data.to(device), label.to(device) pre = model(data) test_loss += loss_function(pre, label).item() test_av_mseloss = test_loss / test_size if test_av_mseloss < best_mse: best_mse = test_av_mseloss return best_mse except Exception as e: print("Exception in eval:", e) return 9999. def run_sa(): bounds = np.array(param_bounds) x0 = [(b[0] + b[1]) / 2 for b in param_bounds] sa = SKO_SA( func=lambda x: eval_model_hyperparams(np.clip(x, bounds[:, 0], bounds[:, 1])), x0=x0, T_max=50, T_min=1, L=30, max_stay_counter=20 ) best_param, best_loss = sa.run() return best_param, best_loss from mealpy import Problem problem = Problem( fit_func=eval_model_hyperparams, bounds=param_bounds, minmax="min" ) optimizer_dict = { 'GA': lambda: GA.BaseGA(problem, epoch=16, pop_size=12).solve().solution[:2], 'PSO': lambda: PSO.BasePSO(problem, epoch=16, pop_size=12).solve().solution[:2], 'ACO': lambda: ACOR.BaseACOR(problem, epoch=16, pop_size=12).solve().solution[:2], 'WOA': lambda: WOA.BaseWOA(problem, epoch=16, pop_size=12).solve().solution[:2], 'GWO': lambda: GWO.BaseGWO(problem, epoch=16, pop_size=12).solve().solution[:2], 'SA': run_sa, 'VPPSO': lambda: vppso(eval_model_hyperparams, len(param_bounds), param_bounds, N=12, N1=6, N2=6, T=16, a=0.3, c1=2.0, c2=2.0, b=1.0, verbose=False)[:2] } final_results = {} show_n = 100 # 展示前100个样本 alg_colors = { 'VPPSO': 'blue', 'GA': 'red', 'PSO': 'green', 'SA': 'purple', 'ACOR': 'orange', 'WOA': 'deepskyblue', 'GWO': 'brown' } for alg_name, alg_func in optimizer_dict.items(): print(f"\n------ 开始{alg_name}优化Transformer-BiLSTM超参数 ------") best_param, best_loss = alg_func() # 还原参数 h1 = int(round(best_param[0])) h2 = int(round(best_param[1])) hidden_dim = int(round(best_param[2])) num_layers = int(round(best_param[3])) num_heads = int(round(best_param[4])) dropout = float(best_param[5]) lr = float(best_param[6]) print(f'{alg_name}最优超参数: {best_param}, 验证loss: {best_loss}') # 训练模型 model = TransformerBiLSTM( input_dim=X_seq.shape[2], hidden_layer_sizes=[h1, h2], hidden_dim=hidden_dim, num_layers=num_layers, num_heads=num_heads, output_dim=y_seq.shape[1], dropout_rate=dropout ).to(device) optimizer = optim.Adam(model.parameters(), lr) loss_function = nn.MSELoss(reduction='sum') # 为加快总流程，只训练epochs=40，你可调整 def train_short(model, epochs): train_size = len(train_loader.dataset) test_size = len(test_loader.dataset) minimum_mse = 1000. best_model_wts = model.state_dict() for epoch in range(epochs): model.train() for seq, labels in train_loader: seq, labels = seq.to(device), labels.to(device) optimizer.zero_grad() y_pred = model(seq) loss = loss_function(y_pred, labels) loss.backward() optimizer.step() model.eval() with torch.no_grad(): test_loss = 0. for data, label in test_loader: data, label = data.to(device), label.to(device) pre = model(data) test_loss += loss_function(pre, label).item() test_av_mseloss = test_loss / test_size if test_av_mseloss < minimum_mse: minimum_mse = test_av_mseloss best_model_wts = model.state_dict() model.load_state_dict(best_model_wts) train_short(model, epochs=40) torch.save(model.state_dict(), f'best_model_{alg_name}.pt') # 预测前100个 model.eval() original_data = [] pre_data = [] with torch.no_grad(): for data, label in test_loader: data, label = data.to(device), label.to(device) y_pred = model(data) original_data.append(label.cpu().numpy()) pre_data.append(y_pred.cpu().numpy()) original_data = np.concatenate(original_data, axis=0) pre_data = np.concatenate(pre_data, axis=0) scaler_y = load('scaler_y') original_100 = scaler_y.inverse_transform(original_data)[:show_n, 0] pre_100 = scaler_y.inverse_transform(pre_data)[:show_n, 0] final_results[alg_name] = (original_100, pre_100) # ======================= 结果可视化对比 ====================== plt.figure(figsize=(14, 7)) plt.plot(final_results['VPPSO'][0], color='gray', label='真实值', linewidth=2, linestyle='--') for alg_name, (orig, pred) in final_results.items(): plt.plot(pred, color=alg_colors[alg_name], label=f'{alg_name}优化', alpha=0.85) plt.xlabel('样本编号') plt.ylabel('预测输出') plt.title('不同智能优化算法下Transformer-BiLSTM预测对比（前100样本）') plt.legend() plt.tight_layout() plt.show()
最新发布
07-30
我们首先需要构建一个基于Transformer和BiLSTM的深度学习模型。然后，我们将使用多种智能优化算法（如GA、PSO、SA、ACO、WOA、GWO等）来优化模型的超参数。最后，我们将对优化后的模型进行训练，并对预测结果进行...
import pandas as pd import numpy as np from sklearn.preprocessing import MinMaxScaler, OneHotEncoder import tensorflow from tensorflow.keras.preprocessing.text import Tokenizer from tensorflow.keras.preprocessing.sequence import pad_sequences from tensorflow.keras.models import Model from tensorflow.keras.layers import Input, Dense, Embedding, LSTM, Concatenate, Dropout, BatchNormalization from tensorflow.keras.optimizers import Adam from sklearn.model_selection import train_test_split from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau # 1.数据预处理与特征工程 # 加载数据集 df = pd.read_csv("training_data.csv") # 数值特征标准化 num_features = ['position', 'quality'] scaler = MinMaxScaler() df[num_features] = scaler.fit_transform(df[num_features]) # 序列特征编码 tokenizer = Tokenizer(char_level=True, num_words=4) # 仅A,C,G,T四种碱基 tokenizer.fit_on_texts(df['context']) sequences = tokenizer.texts_to_sequences(df['context']) max_length = max(len(seq) for seq in sequences) padded_sequences = pad_sequences(sequences, maxlen=max_length, padding='post') # 标签提取 labels = df['label'].values # 2.双输入混合模型架构 # 序列输入分支 sequence_input = Input(shape=(max_length,), name='sequence_input') embedding = Embedding(input_dim=5, output_dim=8, input_length=max_length)(sequence_input) # 5=4碱基+填充 lstm_out = LSTM(32, return_sequences=False)(embedding) # 数值特征输入分支 numeric_input = Input(shape=(len(num_features),), name='numeric_input') dense_numeric = Dense(16, activation='relu')(numeric_input) bn_numeric = BatchNormalization()(dense_numeric) # 合并分支 concatenated = Concatenate()([lstm_out, bn_numeric]) dense1 = Dense(64, activation='relu')(concatenated) dropout1 = Dropout(0.3)(dense1) dense2 = Dense(32, activation='relu')(dropout1) output = Dense(1, activation='sigmoid')(dense2) # 构建模型 model = Model(inputs=[sequence_input, numeric_input], outputs=output) model.compile( optimizer=Adam(learning_rate=0.001), loss='binary_crossentropy', metrics=['accuracy', 'AUC'] ) model.summary() # 3.模型训练与评估 # 划分训练集和测试集 X_seq_train, X_seq_test, X_num_train, X_num_test, y_train, y_test = train_test_split( padded_sequences, df[num_features].values, labels, test_size=0.2, stratify=labels ) # 回调函数 callbacks = [ EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True), ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=1e-6) ] # 训练模型 history = model.fit( [X_seq_train, X_num_train], y_train, validation_data=([X_seq_test, X_num_test], y_test), epochs=100, batch_size=64, callbacks=callbacks, class_weight={0: 1, 1: 2} # 处理类别不平衡 ) # 评估模型 test_loss, test_acc, test_auc = model.evaluate( [X_seq_test, X_num_test], y_test ) print(f"测试准确率: {test_acc:.4f}, AUC: {test_auc:.4f}") 请优化该代码 tensorflow.keras.preprocessing.text tensorflow.keras.preprocessing.sequence tensorflow.keras.models tensorflow.keras.layers tensorflow.keras.optimizers tensorflow.keras.callbacks 无法导入
07-19
根据引用[2][^2]，TensorFlow 2.x 已将 Keras 完全集成到 `tf.keras` 命名空间。请使用以下导入方式： ```python # 正确导入方式 (TensorFlow 2.x) import tensorflow as tf from tensorflow.keras.models import ...
帮我解决代码报错： If the `loss` argument is provided as a dict, the `loss_weight` argument should also be provided as a dict. Received: loss_weights=[0.7, 0.3]；代码如下：import pandas as pd import numpy as np import matplotlib.pyplot as plt from sklearn.preprocessing import MinMaxScaler from tensorflow.keras.models import Model from tensorflow.keras.layers import Input, LSTM, Dense, Dropout, Layer from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau from tensorflow.keras.optimizers import Adam from sklearn.model_selection import TimeSeriesSplit import joblib import tensorflow as tf from tensorflow.keras.models import Model from tensorflow.keras.layers import Input, LSTM, Dense, Dropout, Layer from tensorflow.keras.optimizers import Adam import warnings import tensorflow as tf warnings.filterwarnings('ignore') # ===== 自定义注意力层 ===== class AttentionLayer(Layer): def __init__(self, **kwargs): super(AttentionLayer, self).__init__(**kwargs) def build(self, input_shape): self.time_steps = input_shape[1] self.feature_dim = input_shape[2] # 创建可训练权重 self.W = self.add_weight( name='att_weight', shape=(self.feature_dim, self.feature_dim), initializer='glorot_uniform', trainable=True ) self.b = self.add_weight( name='att_bias', shape=(self.feature_dim,), initializer='zeros', trainable=True ) self.V = self.add_weight( name='att_v', shape=(self.feature_dim, 1), initializer='glorot_uniform', trainable=True ) super(AttentionLayer, self).build(input_shape) def call(self, inputs): # 计算注意力分数 score = tf.tanh(tf.matmul(inputs, self.W) + self.b) # (batch, time_steps, feature_dim) score = tf.matmul(score, self.V) # (batch, time_steps, 1) score = tf.squeeze(score, axis=-1) # (batch, time_steps) # 应用softmax获取注意力权重 alpha = tf.nn.softmax(score, axis=-1) # (batch, time_steps) alpha = tf.expand_dims(alpha, axis=-1) # (batch, time_steps, 1) # 加权求和 context = tf.reduce_sum(alpha * inputs, axis=1) # (batch, feature_dim) return context def compute_output_shape(self, input_shape): return (input_shape[0], input_shape[2]) # (batch_size, feature_dim) # ===== 模型构建函数 ===== def build_attention_lstm_model(input_shape): time_steps, num_features = input_shape inputs = Input(shape=input_shape) # LSTM层 lstm_out = LSTM(128, return_sequences=True)(inputs) lstm_out = Dropout(0.3)(lstm_out) # 使用自定义注意力层 attention_out = AttentionLayer()(lstm_out) # 红球分支 red_branch = Dense(64, activation='relu')(attention_out) red_branch = Dropout(0.2)(red_branch) red_output = Dense(33, activation='sigmoid', name='red_output')(red_branch) # 蓝球分支 blue_branch = Dense(32, activation='relu')(attention_out) blue_branch = Dropout(0.2)(blue_branch) blue_output = Dense(16, activation='sigmoid', name='blue_output')(blue_branch) model = Model(inputs=inputs, outputs=[red_output, blue_output]) optimizer = Adam(learning_rate=0.001) model.compile( optimizer=optimizer, loss={'red_output': 'binary_crossentropy', 'blue_output': 'binary_crossentropy'}, metrics={'red_output': 'binary_accuracy', 'blue_output': 'binary_accuracy'}, loss_weights=[0.7, 0.3] ) model.summary() return model # ===== 其他函数保持不变 ===== # step1_format_data, step2_process_data, create_features, prepare_data, # ===== 数据预处理函数 ===== def step1_format_data(): """格式化原始数据""" print("===== 步骤1: 格式化原始数据 =====") df = pd.read_excel('01hand.xlsx', sheet_name='Sheet1', header=None) # 提取A列和C列数据 new_df = pd.DataFrame({ 'A': pd.to_numeric(df.iloc[:, 0], errors='coerce'), 'B': pd.to_numeric(df.iloc[:, 2], errors='coerce') }).dropna() # 保存新文件 new_df.to_excel('01hand2.xlsx', index=False, header=False) print(f"新表格 '01hand2.xlsx' 创建成功! 包含 {len(new_df)} 行数据") def step2_process_data(): """数据去重和排序""" print("\n===== 步骤2: 数据去重和排序 =====") input_file = "01hand2.xlsx" output_file1 = "02resultA.xlsx" # 降序输出 output_file2 = "02resultB.xlsx" # 升序输出 # 读取数据并转换为长格式 df = pd.read_excel(input_file, header=None) all_values = df.stack().dropna().astype(str).tolist() # 确保数据长度是8的倍数 valid_length = len(all_values) - (len(all_values) % 8) if len(all_values) != valid_length: print(f"警告: 数据总量 {len(all_values)} 不符合8的倍数, 截断至 {valid_length} 个元素") all_values = all_values[:valid_length] # 转换数据格式 new_data = [] for i in range(0, len(all_values), 8): group = all_values[i:i+8] try: # 转换日期和数字 date = int(group[0]) numbers = [int(float(num)) if '.' in num else int(num) for num in group[1:]] new_data.append([date] + numbers) except: continue # 创建DataFrame并去重 columns = ['日期', '数字1', '数字2', '数字3', '数字4', '数字5', '数字6', '数字7'] df = pd.DataFrame(new_data, columns=columns) df = df.drop_duplicates(subset='日期').dropna() # 保存降序文件 df.sort_values('日期', ascending=False).to_excel(output_file1, index=False) print(f"降序文件保存至: {output_file1}") # 保存升序文件 df.sort_values('日期', ascending=True).to_excel(output_file2, index=False) print(f"升序文件保存至: {output_file2}") print(f"最终数据维度: {df.shape}") return df # ===== 特征工程函数 ===== def create_features(df, save_features=True): """创建模型特征并保存特征处理器""" print("\n===== 步骤3: 特征工程 =====") features = df[['日期']].copy() red_cols = ['数字1', '数字2', '数字3', '数字4', '数字5', '数字6'] # 基础特征 features['红球和值'] = df[red_cols].sum(axis=1) features['蓝球值'] = df['数字7'] features['奇偶比'] = df[red_cols].applymap(lambda x: x % 2).sum(axis=1) features['大小比'] = df[red_cols].applymap(lambda x: 1 if x > 16 else 0).sum(axis=1) # 窗口特征 (窗口大小10) window_size = 10 for col in ['红球和值', '奇偶比', '大小比']: features[f'{col}_MA{window_size}'] = features[col].rolling(window=window_size).mean() features[f'{col}_STD{window_size}'] = features[col].rolling(window=window_size).std() # 滞后特征 (滞后1-9期) for lag in range(1, 10): for col in red_cols + ['数字7']: features[f'{col}_lag{lag}'] = df[col].shift(lag) # 目标变量 (下一期开奖结果) red_targets = [] blue_targets = [] for i in range(len(df) - 1): next_row = df.iloc[i + 1] # 红球目标 (33选6) red_target = [1 if num in next_row[red_cols].values else 0 for num in range(1, 34)] # 蓝球目标 (16选1) blue_target = [1 if i == next_row['数字7'] else 0 for i in range(1, 17)] red_targets.append(red_target) blue_targets.append(blue_target) # 转换为numpy数组 red_targets = np.array(red_targets) blue_targets = np.array(blue_targets) # 移除无效数据 (前window_size行和最后一行) features = features.iloc[window_size:-1].reset_index(drop=True) red_targets = red_targets[window_size-1:-1] # 对齐索引 blue_targets = blue_targets[window_size-1:-1] # 保存特征处理器 feature_columns = features.drop(columns=['日期']).columns.tolist() joblib.dump(feature_columns, 'feature_columns.pkl') print(f"特征列名已保存: {len(feature_columns)}个特征") if save_features: features.to_excel('04_features.xlsx', index=False) print(f"特征工程完成, 维度: {features.shape}") return features, red_targets, blue_targets # ===== 模型构建函数 ===== def prepare_data(features, red_targets, blue_targets): """准备训练数据并保存数据处理器""" print("\n===== 步骤4: 数据准备 =====") scaler_X = MinMaxScaler() X_scaled = scaler_X.fit_transform(features.drop(columns=['日期'])) # 保存特征处理器 joblib.dump(scaler_X, 'scaler_X.save') print("特征缩放器已保存") # 创建时间序列数据 time_steps = 10 X_seq, y_red_seq, y_blue_seq = [], [], [] for i in range(time_steps, len(X_scaled)): X_seq.append(X_scaled[i-time_steps:i, :]) y_red_seq.append(red_targets[i-1]) # 使用当前时间步的目标 y_blue_seq.append(blue_targets[i-1]) X_seq = np.array(X_seq) y_red_seq = np.array(y_red_seq) y_blue_seq = np.array(y_blue_seq) print(f"时间序列数据形状: X={X_seq.shape}, y_red={y_red_seq.shape}, y_blue={y_blue_seq.shape}") # 保存历史数据用于预测 joblib.dump(X_scaled[-10:], 'historical_data.pkl') print("历史数据已保存用于预测") return X_seq, y_red_seq, y_blue_seq, scaler_X # train_model, plot_training_history, prepare_prediction_input, predict_next_period, main # ===== 模型训练函数 ===== def train_model(X, y_red, y_blue): """训练模型""" print("\n===== 步骤5: 模型训练 =====") best_models = [] tscv = TimeSeriesSplit(n_splits=3) for fold, (train_index, val_index) in enumerate(tscv.split(X)): print(f"\n===== 训练 Fold {fold+1}/3 =====") X_train, X_val = X[train_index], X[val_index] y_red_train, y_red_val = y_red[train_index], y_red[val_index] y_blue_train, y_blue_val = y_blue[train_index], y_blue[val_index] model = build_attention_lstm_model((X_train.shape[1], X_train.shape[2])) callbacks = [ EarlyStopping(monitor='val_loss', patience=15, restore_best_weights=True, verbose=1), ReduceLROnPlateau(monitor='val_loss', factor=0.3, patience=7, min_lr=1e-6, verbose=1) ] history = model.fit( X_train, {'red_output': y_red_train, 'blue_output': y_blue_train}, epochs=100, batch_size=32, validation_data=(X_val, {'red_output': y_red_val, 'blue_output': y_blue_val}), callbacks=callbacks, verbose=1 ) model.save(f'best_model_fold{fold+1}.h5') best_models.append(model) # 保存训练历史图 plot_training_history(history, fold+1) return best_models def plot_training_history(history, fold): """绘制训练历史图表""" plt.figure(figsize=(15, 10)) # 损失曲线 plt.subplot(2, 2, 1) plt.plot(history.history['loss'], label='训练损失') plt.plot(history.history['val_loss'], label='验证损失') plt.title(f'Fold {fold} - 总损失曲线') plt.ylabel('损失') plt.xlabel('Epoch') plt.legend() # 红球准确率 plt.subplot(2, 2, 2) plt.plot(history.history['red_output_binary_accuracy'], label='红球训练准确率') plt.plot(history.history['val_red_output_binary_accuracy'], label='红球验证准确率') plt.title(f'Fold {fold} - 红球准确率') plt.ylabel('准确率') plt.xlabel('Epoch') plt.legend() # 蓝球准确率 plt.subplot(2, 2, 3) plt.plot(history.history['blue_output_binary_accuracy'], label='蓝球训练准确率') plt.plot(history.history['val_blue_output_binary_accuracy'], label='蓝球验证准确率') plt.title(f'Fold {fold} - 蓝球准确率') plt.ylabel('准确率') plt.xlabel('Epoch') plt.legend() # 学习率 plt.subplot(2, 2, 4) if 'lr' in history.history: plt.plot(history.history['lr'], label='学习率') plt.title(f'Fold {fold} - 学习率变化') plt.ylabel('学习率') plt.xlabel('Epoch') plt.legend() plt.tight_layout() plt.savefig(f'training_history_fold{fold}.png') plt.close() # ===== 预测准备函数 ===== def prepare_prediction_input(df, features, scaler_X): """准备预测输入，确保特征一致性""" print("\n===== 准备预测输入 =====") # 加载特征列名 feature_columns = joblib.load('feature_columns.pkl') print(f"预期特征数量: {len(feature_columns)}") # 创建空DataFrame prediction_features = pd.DataFrame(columns=feature_columns) # 获取最后10行有效数据 last_10 = features.iloc[-10:] # 填充基础特征 red_cols = ['数字1', '数字2', '数字3', '数字4', '数字5', '数字6'] current_row = df.iloc[-1] prediction_features.loc[0, '红球和值'] = current_row[red_cols].sum() prediction_features.loc[0, '蓝球值'] = current_row['数字7'] prediction_features.loc[0, '奇偶比'] = current_row[red_cols].apply(lambda x: x % 2).sum() prediction_features.loc[0, '大小比'] = current_row[red_cols].apply(lambda x: 1 if x > 16 else 0).sum() # 填充窗口特征 window_size = 10 for col in ['红球和值', '奇偶比', '大小比']: col_values = features[col].iloc[-window_size:] prediction_features.loc[0, f'{col}_MA{window_size}'] = col_values.mean() prediction_features.loc[0, f'{col}_STD{window_size}'] = col_values.std() # 填充滞后特征 - 修正逻辑 for lag in range(1, 10): # 确保滞后索引有效 lag_index = -lag - 1 # 从当前行向前追溯 for col in red_cols + ['数字7']: feature_name = f'{col}_lag{lag}' if feature_name in feature_columns: if len(df) > lag: prediction_features.loc[0, feature_name] = df[col].iloc[lag_index] else: # 数据不足时使用平均值 prediction_features.loc[0, feature_name] = df[col].mean() # 处理缺失特征 missing_cols = set(feature_columns) - set(prediction_features.columns) for col in missing_cols: prediction_features[col] = 0 # 默认填充0 # 确保顺序一致 prediction_features = prediction_features[feature_columns] # 标准化 X_pred = scaler_X.transform(prediction_features) print(f"预测输入形状: {X_pred.shape}") return X_pred # ===== 预测函数 ===== def predict_next_period(models): """预测下一期开奖结果""" print("\n===== 步骤6: 预测 =====") # 加载历史数据和缩放器 scaler_X = joblib.load('scaler_X.save') historical_data = joblib.load('historical_data.pkl') # 获取特征列名 feature_columns = joblib.load('feature_columns.pkl') n_features = len(feature_columns) # 构建时间序列输入 time_steps = 10 if len(historical_data) >= time_steps: # 使用完整的历史数据 X_seq = historical_data[-time_steps:].reshape(1, time_steps, n_features) else: # 数据不足时复制最后一个时间点 X_seq = np.repeat(historical_data[-1:], time_steps, axis=0).reshape(1, time_steps, n_features) print(f"模型输入序列形状: {X_seq.shape}") # 集成模型预测 red_probs = np.zeros((1, 33)) blue_probs = np.zeros((1, 16)) total_weight = 0 for i, model in enumerate(models): r_prob, b_prob = model.predict(X_seq, verbose=0) weight = 0.3 + i * 0.2 # 加权集成 red_probs += r_prob * weight blue_probs += b_prob * weight total_weight += weight # 归一化概率 red_probs /= total_weight blue_probs /= total_weight # 获取预测结果 red_indices = np.argsort(red_probs[0])[::-1][:6] blue_indices = np.argsort(blue_probs[0])[::-1][:3] return ( [i+1 for i in red_indices], [red_probs[0][i] for i in red_indices], [i+1 for i in blue_indices], [blue_probs[0][i] for i in blue_indices] ) # ===== 主函数 ===== def main(): # 执行数据处理步骤 step1_format_data() df = step2_process_data() # 特征工程 features, red_targets, blue_targets = create_features(df) # 准备训练数据 X, y_red, y_blue, scaler_X = prepare_data(features, red_targets, blue_targets) # 训练模型 models = train_model(X, y_red, y_blue) # 预测 red_nums, red_probs, blue_nums, blue_probs = predict_next_period(models) # 打印结果 print("\n" + "="*50) print("双色球下一期预测结果") print("="*50) print("\n红球预测 (前6个):") for num, prob in zip(red_nums, red_probs): print(f"号码 {num:2d} : 概率 {prob:.4f}") print("\n蓝球预测 (前3个):") for num, prob in zip(blue_nums, blue_probs): print(f"号码 {num:2d} : 概率 {prob:.4f}") # 保存结果 result_df = pd.DataFrame({ '红球预测': red_nums, '红球概率': red_probs, '蓝球预测': blue_nums, '蓝球概率': blue_probs }) result_df.to_excel('prediction_results.xlsx', index=False) print("\n预测结果已保存至: prediction_results.xlsx") if __name__ == "__main__": main() # main.py import tensorflow as tf from tensorflow.keras.models import Model from tensorflow.keras.layers import Input, LSTM, Dense, Dropout, Layer from tensorflow.keras.optimizers import Adam # 内联定义自定义层（避免导入问题） class AttentionLayer(Layer): """自定义注意力层""" def __init__(self, **kwargs): super(AttentionLayer, self).__init__(**kwargs) def build(self, input_shape): self.time_steps = input_shape[1] self.feature_dim = input_shape[2] self.W = self.add_weight( name='att_weight', shape=(self.feature_dim, self.feature_dim), initializer='glorot_uniform', trainable=True ) self.b = self.add_weight( name='att_bias', shape=(self.feature_dim,), initializer='zeros', trainable=True ) self.V = self.add_weight( name='att_v', shape=(self.feature_dim, 1), initializer='glorot_uniform', trainable=True ) super(AttentionLayer, self).build(input_shape) def call(self, inputs): score = tf.tanh(tf.matmul(inputs, self.W) + self.b) score = tf.matmul(score, self.V) score = tf.squeeze(score, axis=-1) alpha = tf.nn.softmax(score, axis=-1) alpha = tf.expand_dims(alpha, axis=-1) context = tf.reduce_sum(alpha * inputs, axis=1) return context def compute_output_shape(self, input_shape): return (input_shape[0], input_shape[2]) # 模型构建函数 def build_attention_lstm_model(input_shape): inputs = Input(shape=input_shape) # LSTM层 lstm_out = LSTM(128, return_sequences=True)(inputs) lstm_out = Dropout(0.3)(lstm_out) # 使用自定义层 attention_out = AttentionLayer()(lstm_out) # 输出分支 red_branch = Dense(64, activation='relu')(attention_out) red_output = Dense(33, activation='sigmoid', name='red_output')(red_branch) blue_branch = Dense(32, activation='relu')(attention_out) blue_output = Dense(16, activation='sigmoid', name='blue_output')(blue_branch) model = Model(inputs=inputs, outputs=[red_output, blue_output]) model.compile( optimizer=Adam(0.001), loss={'red_output': 'binary_crossentropy', 'blue_output': 'binary_crossentropy'}, loss_weights={'red_output': 0.7, 'blue_output': 0.3} ) return model # 测试模型构建 if __name__ == "__main__": model = build_attention_lstm_model(input_shape=(10, 20)) model.summary() print("模型构建成功!")
07-17
步骤2：在`main.py`中（即最后面的部分）同样修改`build_attention_lstm_model`函数中的编译部分，将`loss_weights`改为字典形式。 但是，由于我们无法直接修改`main.py`（因为它是字符串形式在问题中），所以...
RNN、LSTM、Seq2seq、Transformer、BERT
weixin_43747748的博客
05-28 602
RNN LSTM Seq2seq with attention Transformer BERT
从LSTM到Seq2Seq
Just for fun的专栏
07-08 2585
http://x-algo.cn/index.php/2017/01/13/1609/LSTM(Long Short Term Memory)由于传统的RNN在进行几次链式法则求导后梯度会指数级缩小（如图），导致传播几层后出现梯度消失，无法处理“长期依赖”问题。国外学者刻意设计了一种RNN的变体来克服这个问题，即LSTM。基本的LSTM结构如图：(1) 输入门 itit ：控制当前输入和前一步输出...
Seq2seq模型以及Attention机制
weixin_42254244的博客
05-26 870
RNN和LSTM 1.RNN
Sequence to Sequence 实现机器翻译（keras demo）
6丁儿的猫
11-06 3596
最近在研究对话机器人，刚好看了几篇论文，参考keras demo理解。 原理请参考：- Sequence to Sequence Learning with Neural Networks https://arxiv.org/abs/1409.3215 - Learning Phrase Representations using RNN Encoder-Decoder for S
数据分析-循环神经网络-LSTM-案例
ITLiu的博客
04-01 5865
数据分析-RNN-LSTM
LSTM实例解析
m0_63024355的博客
05-19 2101
由于RNN（循环神经网络）模型的梯度消失现象，会导致RNN模型的失效，因此人们对RNN的隐含层神经元进行改造，便有了LSTM（长度期记忆）模型。至于对RNN的修改，可参考循环神经网络（RNN）-优快云博客本文来重点介绍这次实现过程中的代码。
转载：LSTM实例
weixin_41529093的博客
04-01 1054
仅作为记录，大佬请跳过。 文章目录源代码展示注参考 源代码 可直接运行 import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim # 2.1 准备数据 def prepare_sequence(seq, to_ix): idxs = [to_ix[w] for w in seq] return torch.tensor(idxs, dtype=to
Pytorch 的 LSTM 模型的简单示例
程序猿视角
05-28 8104
Pytorch 的 LSTM 模型的简单示例
matlab用lstm_lstm-seq2seq 源码解析
weixin_32528123的博客
01-12 1223
源代码地址：https://github.com/keras-team/keras/blob/master/examples/lstm_seq2seq.py相关博客：https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html可以先对博客中的内容进行阅读，下面主要记录，本...
【NLP】一文理解Seq2Seq
热门推荐
sikh_0529的博客
02-21 1万+
Seq2Seq技术，全称Sequence to Sequence，该技术突破了传统的固定大小输入问题框架，开通了将经典深度神经网络模型（DNNs）运用于在翻译，文本自动摘要和机器人自动问答以及一些回归预测任务上,并被证实在英语－法语翻译、英语－德语翻译以及人机短问快答的应用中有着不俗的表现。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值