18、基于循环神经网络的人类活动识别：LSTM模型实现

基于循环神经网络的人类活动识别：LSTM模型实现

1. 整体算法流程

实现人类活动识别（HAR）的LSTM模型，整体算法（HumanAR.scala）的工作流程如下：
1. 加载数据
2. 定义超参数
3. 使用命令式编程和超参数设置LSTM模型
4. 进行批量训练，即选取批量大小的数据，将其输入到模型中，然后在某些迭代中评估模型并打印批量损失和准确率
5. 输出训练和测试误差的图表

以下是该流程的mermaid流程图：

graph LR
    A[加载数据] --> B[定义超参数]
    B --> C[设置LSTM模型]
    C --> D[批量训练]
    D --> E[输出图表]

2. 实现步骤

2.1 导入必要的库和包

package com.packt.ScalaML.HAR
import ml.dmlc.mxnet.Context
import LSTMNetworkConstructor.LSTMModel
import scala.collection.mutable.ArrayBuffer
import ml.dmlc.mxnet.optimizer.Adam
import ml.dmlc.mxnet.NDArray
import ml.dmlc.mxnet.optimizer.RMSProp
import org.sameersingh.scalaplot.MemXYSeries
import org.sameersingh.scalaplot.XYData
import org.sameersingh.scalaplot.XYChart
import org.sameersingh.scalaplot.Style._
import org.sameersingh.scalaplot.gnuplot.GnuplotPlotter
import org.sameersingh.scalaplot.jfreegraph.JFGraphPlotter

2.2 创建MXNet上下文

// Retrieves the name of this Context object
val ctx = Context.cpu()

2.3 加载和解析训练集与测试集

假设数据集已复制到 UCI_HAR_Dataset/ 目录，以下是加载数据的代码：

val datasetPath = "UCI_HAR_Dataset/"
val trainDataPath = s"$datasetPath/train/Inertial Signals"
val trainLabelPath = s"$datasetPath/train/y_train.txt"
val testDataPath = s"$datasetPath/test/Inertial Signals"
val testLabelPath = s"$datasetPath/test/y_test.txt"

val trainData = Utils.loadData(trainDataPath, "train")
val trainLabels = Utils.loadLabels(trainLabelPath)
val testData = Utils.loadData(testDataPath, "test")
val testLabels = Utils.loadLabels(testLabelPath)

loadData() 方法的实现：

def loadData(dataPath: String, name: String): Array[Array[Array[Float]]] =
{
    val dataSignalsPaths = INPUT_SIGNAL_TYPES.map( signal =>
s"$dataPath/${signal}${name}.txt" )
    val signals = dataSignalsPaths.map { path =>
      Source.fromFile(path).mkString.split("n").map { line =>
        line.replaceAll("  ", " ").trim().split(" ").map(_.toFloat) }
    }
    val inputDim = signals.length
    val numSamples = signals(0).length
    val timeStep = signals(0)(0).length
    (0 until numSamples).map { n =>
      (0 until timeStep).map { t =>
        (0 until inputDim).map( i => signals(i)(n)(t) ).toArray
      }
    .toArray
    }
    .toArray
  }

INPUT_SIGNAL_TYPES 数组：

private val INPUT_SIGNAL_TYPES = Array(
    "body_acc_x_",
    "body_acc_y_",
    "body_acc_z_",
    "body_gyro_x_",
    "body_gyro_y_",
    "body_gyro_z_",
    "total_acc_x_",
    "total_acc_y_",
    "total_acc_z_")

loadLabels() 方法的实现：

def loadLabels(labelPath: String): Array[Float] = {
       Source.fromFile(labelPath).mkString.split("n").map(_.toFloat - 1)
            }

LABELS 数组：

// Output classes: used to learn how to classify
private val LABELS = Array(
    "WALKING",
    "WALKING_UPSTAIRS",
    "WALKING_DOWNSTAIRS",
    "SITTING",
    "STANDING",
    "LAYING")

2.4 数据集探索性分析

val trainingDataCount = trainData.length // No. of training series
val testDataCount = testData.length // No. of testing series
val nSteps = trainData(0).length // No. of timesteps per series
val nInput = trainData(0)(0).length // No. of input parameters per timestep
println("Number of training series: "+ trainingDataCount)
println("Number of test series: "+ testDataCount)
println("Number of timestep per series: "+ nSteps)
println("Number of input parameters per timestep: "+ nInput)

输出结果：
| 数据类型 | 数量 |
| ---- | ---- |
| 训练系列数量 | 7352 |
| 测试系列数量 | 2947 |
| 每个系列的时间步长 | 128 |
| 每个时间步的输入参数数量 | 9 |

2.5 定义内部RNN结构和LSTM超参数

val nHidden = 128 // Number of features in a hidden layer
val nClasses = 6 // Total classes to be predicted
val learningRate = 0.001f
val trainingIters = trainingDataCount * 100  // iterate 100 times on trainset: total 7352000 iterations
val batchSize = 1500
val displayIter = 15000  // To show test set accuracy during training
val numLstmLayer = 3

2.6 LSTM网络构建

val model = LSTMNetworkConstructor.setupModel(nSteps, nInput, nHidden,
nClasses, batchSize, ctx = ctx)

setupModel() 方法的实现：

def setupModel(seqLen: Int, nInput: Int, numHidden: Int, numLabel: Int,
batchSize: Int, numLstmLayer: Int = 1, dropout: Float = 0f, ctx: Context =
Context.cpu()): LSTMModel = {
//get the symbolic model
    val sym = LSTMNetworkConstructor.getSymbol(seqLen, numHidden, numLabel,
numLstmLayer = numLstmLayer)
    val argNames = sym.listArguments()
    val auxNames = sym.listAuxiliaryStates()
// defining the initial argument and binding them to the model
    val initC = for (l <- 0 until numLstmLayer) yield (s"l${l}_init_c",
(batchSize, numHidden))
    val initH = for (l <- 0 until numLstmLayer) yield (s"l${l}_init_h",
(batchSize, numHidden))
    val initStates = (initC ++ initH).map(x => x._1 -> Shape(x._2._1,
x._2._2)).toMap
    val dataShapes = Map("data" -> Shape(batchSize, seqLen, nInput)) ++
initStates
    val (argShapes, outShapes, auxShapes) = sym.inferShape(dataShapes)
    val initializer = new Uniform(0.1f)
    val argsDict = argNames.zip(argShapes).map { case (name, shape) =>
       val nda = NDArray.zeros(shape, ctx)
       if (!dataShapes.contains(name) && name != "softmax_label") {
         initializer(name, nda)
       }
       name -> nda
    }.toMap
    val argsGradDict = argNames.zip(argShapes)
         .filter(x => x._1 != "softmax_label" && x._1 != "data")
         .map( x => x._1 -> NDArray.zeros(x._2, ctx) ).toMap
    val auxDict = auxNames.zip(auxShapes.map(NDArray.zeros(_, ctx))).toMap
    val exec = sym.bind(ctx, argsDict, argsGradDict, "write", auxDict,
null, null)
    val data = argsDict("data")
    val label = argsDict("softmax_label")
    LSTMModel(exec, sym, data, label, argsDict, argsGradDict)
}

getSymbol() 方法的实现：

private def getSymbol(seqLen: Int, numHidden: Int, numLabel: Int,
numLstmLayer: Int = 1,
                        dropout: Float = 0f): Symbol = {
                //symbolic training and label variables
                var inputX = Symbol.Variable("data")
                val inputY = Symbol.Variable("softmax_label")
                //the initial parameters and cells
                var paramCells = Array[LSTMParam]()
                var lastStates = Array[LSTMState]()
                //numLstmLayer is 1
                for (i <- 0 until numLstmLayer) {
                    paramCells = paramCells :+ LSTMParam(i2hWeight =
                    Symbol.Variable(s"l${i}_i2h_weight"),
                    i2hBias = Symbol.Variable(s"l${i}_i2h_bias"),
                    h2hWeight = Symbol.Variable(s"l${i}_h2h_weight"),
                    h2hBias = Symbol.Variable(s"l${i}_h2h_bias"))
                    lastStates = lastStates :+ LSTMState(c =
                    Symbol.Variable(s"l${i}_init_c"),
                    h = Symbol.Variable(s"l${i}_init_h"))
            }
            assert(lastStates.length == numLstmLayer)
            val lstmInputs = Symbol.SliceChannel()(inputX)(Map("axis"
            > 1, "num_outputs" -> seqLen,
            "squeeze_axis" -> 1))
            var hiddenAll = Array[Symbol]()
            var dpRatio = 0f
            var hidden: Symbol = null
//for each one of the 128 inputs, create a LSTM Cell
            for (seqIdx <- 0 until seqLen) {
                  hidden = lstmInputs.get(seqIdx)
// stack LSTM, where numLstmLayer is 1 so the loop will be executed only one time
                  for (i <- 0 until numLstmLayer) {
                        if (i == 0) dpRatio = 0f else dpRatio = dropout
//for each one of the 128 inputs, create a LSTM Cell
                        val nextState = lstmCell(numHidden, inData =
hidden,
                          prevState = lastStates(i),
                          param = paramCells(i),
                          seqIdx = seqIdx, layerIdx = i, dropout =
                        dpRatio)
                    hidden = nextState.h // has no effect
                    lastStates(i) = nextState // has no effect
              }
// adding dropout before softmax has no effect- dropout is 0 due to numLstmLayer == 1
              if (dropout > 0f) hidden = Symbol.Dropout()()(Map("data" ->
hidden, "p" -> dropout))
// store the lstm cells output layers
                  hiddenAll = hiddenAll :+ hidden
    }
    val finalOut = hiddenAll.reduce(_+_)
    val fc = Symbol.FullyConnected()()(Map("data" -> finalOut, "num_hidden" ->
numLabel))
    Symbol.SoftmaxOutput()()(Map("data" -> fc, "label" -> inputY))
}

LSTMState 和 LSTMParam 的定义：

final case class LSTMState(c: Symbol, h: Symbol)
final case class LSTMParam(i2hWeight: Symbol, i2hBias: Symbol, h2hWeight: Symbol,
h2hBias: Symbol)

lstmCell() 方法的实现：

private def lstmCell( numHidden: Int, inData: Symbol, prevState:
LSTMState, param: LSTMParam,
                        seqIdx: Int, layerIdx: Int, dropout: Float = 0f):
LSTMState = {
        val inDataa = {
              if (dropout > 0f) Symbol.Dropout()()(Map("data" -> inData,
"p" -> dropout))
              else inData
                }
        val i2h =
Symbol.FullyConnected(s"t${seqIdx}_l${layerIdx}_i2h")()(Map("data" ->
inDataa,"weight"                             -> param.i2hWeight,"bias" ->
param.i2hBias,"num_hidden" -> numHidden * 4))
        val h2h =
Symbol.FullyConnected(s"t${seqIdx}_l${layerIdx}_h2h")()(Map("data" ->
prevState.h,"weight" -> param.h2hWeight,"bias" ->
param.h2hBias,"num_hidden" -> numHidden * 4))
        val gates = i2h + h2h
        val
sliceGates=Symbol.SliceChannel(s"t${seqIdx}_l${layerIdx}_slice")(gates)(Map
("num_outputs"
       -> 4))
        val ingate = Symbol.Activation()()(Map("data" -> sliceGates.get(0),
"act_type" -> "sigmoid"))
        val inTransform = Symbol.Activation()()(Map("data" ->
sliceGates.get(1), "act_type" -> "tanh"))
        val forgetGate = Symbol.Activation()()(Map("data" ->
sliceGates.get(2), "act_type" -> "sigmoid"))
        val outGate = Symbol.Activation()()(Map("data" ->
sliceGates.get(3), "act_type" -> "sigmoid"))
        val nextC = (forgetGate * prevState.c) + (ingate * inTransform)
        val nextH = outGate * Symbol.Activation()()(Map("data" -> nextC,
"act_type" -> "tanh"))
        LSTMState(c = nextC, h = nextH)
  }

2.7 设置优化器

val opt = new RMSProp(learningRate = learningRate)
val paramBlocks = model.symbol.listArguments()
      .filter(x => x != "data" && x != "softmax_label")
      .zipWithIndex.map { case (name, idx) =>
        val state = opt.createState(idx, model.argsDict(name))
        (idx, model.argsDict(name), model.gradDict(name), state, name)
      }
    .toArray

2.8 训练LSTM网络

在开始训练LSTM网络之前，先定义一些变量来跟踪训练性能：

val testLosses = ArrayBuffer[Float]()
val testAccuracies = ArrayBuffer[Float]()
val trainLosses = ArrayBuffer[Float]()
val trainAccuracies = ArrayBuffer[Float]()

然后，开始以批量大小进行训练步骤：

var step = 1
while (step * batchSize <= trainingIters) {
    val (batchTrainData, batchTrainLabel) = {
        val idx = ((step - 1) * batchSize) % trainingDataCount
        if (idx + batchSize <= trainingDataCount) {
          val datas = trainData.drop(idx).take(batchSize)
          val labels = trainLabels.drop(idx).take(batchSize)
          (datas, labels)
        } else {
          val right = (idx + batchSize) - trainingDataCount
          val left = trainingDataCount - idx
          val datas = trainData.drop(idx).take(left) ++ trainData.take(right)
          val labels = trainLabels.drop(idx).take(left) ++ trainLabels.take(right)
          (datas, labels)
    }
}
model.data.set(batchTrainData.flatten.flatten)
model.label.set(batchTrainLabel)
model.exec.forward(isTrain = true)
model.exec.backward()
paramBlocks.foreach {
    case (idx, weight, grad, state, name) => opt.update(idx, weight, grad, state)
    }
val (acc, loss) = getAccAndLoss(model.exec.outputs(0), batchTrainLabel)
trainLosses += loss / batchSize
trainAccuracies += acc / batchSize

getAccAndLoss() 方法的实现：

def getAccAndLoss(pred: NDArray, label: Array[Float], dropNum: Int = 0): (Float, Float) = {
    val shape = pred.shape
    val maxIdx = NDArray.argmax_channel(pred).toArray
    val acc = {
      val sum = maxIdx.drop(dropNum).zip(label.drop(dropNum)).foldLeft(0f){
case (acc, elem) =>
        if (elem._1 == elem._2) acc + 1 else acc
      }
      sum
    }
    val loss = pred.toArray.grouped(shape(1)).drop(dropNum).zipWithIndex.map { case (array, idx) =>
        array(maxIdx(idx).toInt)
      }.map(-Math.log(_)).sum.toFloat
 (acc, loss)
}

在某些步骤评估网络以加快训练：

if ( (step * batchSize % displayIter == 0) || (step == 1) || (step * batchSize > trainingIters) ) {
        println(s"Iter ${step * batchSize}, Batch Loss = ${\"%.6f\".format(loss / batchSize)}, Accuracy = ${acc / batchSize}")
    }

部分训练输出示例：
| 迭代次数 | 批量损失 | 准确率 |
| ---- | ---- | ---- |
| 1500 | 1.189168 | 0.14266667 |
| 15000 | 0.479527 | 0.53866667 |
| 30000 | 0.293270 | 0.83933336 |
|… |… |… |

2.9 评估模型

训练完成后，评估测试集：

val (testLoss, testAcc) = test(testDataCount, batchSize, testData, testLabels, model)
println(s"TEST SET DISPLAY STEP:  Batch Loss = ${\"%.6f\".format(testLoss)}, Accuracy = $testAcc")
testAccuracies += testAcc
testLosses += testLoss
step += 1
}
val (finalLoss, accuracy) = test(testDataCount, batchSize, testData, testLabels, model)
println(s"FINAL RESULT: Batch Loss= $finalLoss, Accuracy= $accuracy")

test() 方法的实现：

def test(testDataCount: Int, batchSize: Int, testDatas: Array[Array[Array[Float]]],
      testLabels: Array[Float], model: LSTMModel): (Float, Float) = {
    var testLoss, testAcc = 0f
    for (begin <- 0 until testDataCount by batchSize) {
      val (testData, testLabel, dropNum) = {
        if (begin + batchSize <= testDataCount) {
          val datas = testDatas.drop(begin).take(batchSize)
          val labels = testLabels.drop(begin).take(batchSize)
          (datas, labels, 0)
        } else {
          val right = (begin + batchSize) - testDataCount
          val left = testDataCount - begin
          val datas = testDatas.drop(begin).take(left) ++ testDatas.take(right)
          val labels = testLabels.drop(begin).take(left) ++ testLabels.take(right)
          (datas, labels, right)
        }
      }
      model.data.set(testData.flatten.flatten)
      model.label.set(testLabel)
      model.exec.forward(isTrain = false)
      val (acc, loss) = getAccAndLoss(model.exec.outputs(0), testLabel)
      testLoss += loss
      testAcc += acc
    }
    (testLoss / testDataCount, testAcc / testDataCount)
  }

释放资源：

model.exec.dispose()

部分测试输出示例：
| 测试步骤 | 批量损失 | 准确率 |
| ---- | ---- | ---- |
| 1 | 0.065859 | 0.9138107 |
| 2 | 0.077047 | 0.912114 |
| 3 | 0.069186 | 0.90566677 |
|… |… |… |

最后，将训练和测试的准确率和误差可视化：

// visualize
val xTrain = (0 until trainLosses.length * batchSize by batchSize).toArray.map(_.toDouble)
val yTrainL = trainLosses.toArray.map(_.toDouble)
val yTrainA = trainAccuracies.toArray.map(_.toDouble)
val xTest = (0 until testLosses.length * displayIter by displayIter).toArray.map(_.toDouble)
val yTestL = testLosses.toArray.map(_.toDouble)
val yTestA = testAccuracies.toArray.map(_.toDouble)
var series = new MemXYSeries(xTrain, yTrainL, "Train losses")
val data = new XYData(series)
series = new MemXYSeries(xTrain, yTrainA, "Train accuracies")
data += series
series = new MemXYSeries(xTest, yTestL, "Test losses")
data += series
series = new MemXYSeries(xTest, yTestA, "Test accuracies")
data += series
val chart = new XYChart("Training session's progress over iterations!", data)
chart.showLegend = true
val plotter = new JFGraphPlotter(chart)
plotter.gui()

以下是整个训练和评估流程的mermaid流程图：

graph LR
    A[初始化变量] --> B[训练循环]
    B --> C{是否达到训练迭代次数}
    C -- 否 --> D[获取批量数据]
    D --> E[前向传播]
    E --> F[反向传播]
    F --> G[更新参数]
    G --> H[计算损失和准确率]
    H --> I{是否评估}
    I -- 是 --> J[评估测试集]
    J --> K[记录测试损失和准确率]
    K --> B
    I -- 否 --> B
    C -- 是 --> L[最终评估测试集]
    L --> M[输出最终结果]
    M --> N[释放资源]
    N --> O[可视化结果]

通过以上步骤，我们实现了一个基于LSTM的人类活动识别模型，并取得了较好的准确率。整个过程涵盖了数据加载、模型构建、训练和评估等关键步骤，为相关领域的研究和应用提供了一个可行的解决方案。