[LSTM学习笔记9]How to Develop Generative LSTMs

一.生成模型概述

LSTM可以被用于生成模型，如Language Modeling：给定一个序列的大型语料库，用于生成风格类似的输出序列，常被用于文本生成，具体可以参看论文：

• Generaing Text with Recurrent Neural Networks,2011
• Generaing Sequence with Recurrent Neural Networks,2013

二.以形状生成问题为例演示

1.问题描述：
按逆时针方向绘制一个矩形，定义矩形四个点为：

• Bottom Left (BL): [0, 0]
• Bottom Right (BR): [1, 0]
• Top Right(TR): [1, 1]
• Top Left :[0,1]
  给定一个点，预测下一个点，最后把所有点绘制成形：
  
  2.代码：

from random import random
from numpy import array
from matplotlib import pyplot
from matplotlib.patches import PathPatch
from matplotlib.path import Path
from keras.models import Sequential
from keras.layers import LSTM
from keras.layers import Dense'

def random_rectangle():
    width, height = random(), random()
    points = list()
    # bottom left
    points.append([0.0, 0.0])
    # bottom right
    points.append([width, 0.0])
    # top right
    points.append([width, height])
    # top left
    points.append([0.0, height])
    return points

# plot a rectangle
def plot_rectangle(rect):
    # close the rectangle path
    rect.append(rect[0])
    # define path
    codes = [Path.MOVETO, Path.LINETO, Path.LINETO, Path.LINETO, Path.CLOSEPOLY]
    path = Path(rect, codes)
    axis = pyplot.gca()
    patch = PathPatch(path)
    # add shape to plot
    axis.add_patch(patch)
    axis.set_xlim(-0.1,1.1)
    axis.set_ylim(-0.1,1.1)
    pyplot.show()
# generate input and output sequences for one random rectangle
def get_samples():
    # generate rectangle
    rect = random_rectangle()
    X, y = list(), list()
    # create input output pairs for each coordinate
    for i in range(1, len (rect)):
        X.append(rect[i-1])
        y.append(rect[i])
    # convert input sequence shape to have 1 time step and 2 features
    X, y = array(X), array(y)
    X = X.reshape((X.shape[0], 1, 2))
    return X, y
# define model
model = Sequential()
model.add(LSTM(10, input_shape=(1, 2)))
model.add(Dense(2, activation='linear' ))
model.compile(loss= 'mae' , optimizer= 'adam' )
print (model.summary())

# fit model
for i in range(20000):
    X, y = get_samples()
    model.fit(X, y, epochs=1, verbose=0, shuffle=False)
    # use a fit LSTM model to generate a new rectangle from scratch

def generate_rectangle(model):
    rect = list()
    # use [0,0] to seed the generation process
    last = array([0.0,0.0]).reshape((1, 1, 2))
    rect.append([[y for y in x] for x in last[0]][0])
    # generate the remaining 3 coordinates
    for i in range(3):
        # predict the next coordinate
        yhat = model.predict(last, verbose=0)
        # use this output as input for the next prediction
        last = yhat.reshape((1, 1, 2))
        # store coordinate
        rect.append([[y for y in x] for x in last[0]][0])
    return rect

# generate new shapes from scratch
rect = generate_rectangle(model)
plot_rectangle(rect)

这里分别设置训练模型的次数，得到不同的模型，绘制出矩形可以看出，训练次数越多，最后的结果越像矩形：
500次训练得到的模型预测出的结果：

5000次训练得到的模型预测出的结果：

10000次训练得到的模型预测出的结果：

20000次训练得到的模型预测出的结果：

25000次训练得到的模型预测出的结果：(这里使用了Google TPU加速)

50000次训练得到的模型预测出的结果：(这里使用了Google TPU加速)

100000次训练得到的模型预测出的结果：(这里使用了Google TPU加速)