mshadow简介

Tensor Data Structure

mshadow基础的数据结构是Tensor. 以下是mashadow/tensor.h的简化版本.

  typedef unsigned index_t;
template<int dimension>
struct Shape {
  index_t shape_[dimension];
};
template<typename Device, int dimension, typename DType = float>
struct Tensor {
  DType *dptr_;
  Shape<dimension> shape_;
  Stream<Device> stream_;
  index_t stride_;
};
// this is how shape object declaration look like
Shape<2> shape2;
// this is how tensor object declaration look like
// you can
Tensor<cpu, 2> ts2;
Tensor<gpu, 3, float> ts3;   

Tensor

struct Shape<2> {
  index_t shape_[2];
};
struct Tensor<cpu, 2, float> {
  float *dptr_;
  Shape<2> shape_;
  index_t stride_;
};

• Tensor

float data[9] = {0, 1, 2, 3, 4, 5, 6, 7, 8};
Tensor<cpu, 2> ts;
ts.dptr_ = data;
ts.shape_ = mshadow::Shape2(3, 2);
ts.stride_ = 3;
// now: ts[0][0] == 0, ts[0][1] == 1 , ts[1][0] == 3, ts[1][1] == 4
for (index_t i = 0; i < ts.size(0); ++i) {
  for (index_t j = 0; j < ts.size(1); ++j) {
    printf("ts[%u][%u]=%f\n", i, j, ts[i][j]);
  }
}

结果是一个3*2的矩阵,其中data[2],data[5],data[8]被当作padding被忽略了(译注:值还在,只是没法通过ts访问到了,而且ts[0]和ts[1]是不连续的内存了). 如果希望获得连续的内存, 需要把stride_设置成shape_[1] (译注:因为依然是2*3的矩阵,data[2],data[5],data[8]依然无法通过ts访问)
NOTICE: We highly recommend use stream in gpu mode, there will be an error thrown out if no stream is set. Check basic_stream.cu for more detail.

Memory Allocation

mshadow内一个重要的设计是只要设置好dptr_, shape_和stride_, 就可以把tensor作为whitebox(译注:不是blakcbox??):
* Tensor

// create a 5 x 3 tensor on the device, and allocate space
Tensor<gpu, 2> ts2(Shape2(5, 3));
AllocSpace(&ts2);
// allocate 5 x 3 x 2 tensor on the host, initialized by 0
Tensor<cpu, 3> ts3 = NewTensor<cpu>(Shape3(5,3,2), 0.0f);
// free space
FreeSpace(&ts2); FreeSpace(&ts3);

所有内存分配都是显示的,任何操作过程中都不存在隐式内存分配和释放. 这意味着Tensor

Elementwise Operations

mshadow中所有操作符(比如+,-,*,/,+=)都是元素级别的. 考虑如下SGD中权重更新代码:

void UpdateSGD(Tensor<cpu, 2> weight, Tensor<cpu, 2> grad, float eta, float lambda) {
  weight -= eta * (grad + lambda * weight);
}

编译阶段,这段代码会转换成如下形式:

void UpdateSGD(Tensor<cpu,2> weight, Tensor<cpu,2> grad, float eta, float lambda) {
  for (index_t y = 0; y < weight.size(0); ++y) {
    for (index_t x = 0; x < weight.size(1); ++x) {
      weight[y][x] -= eta * (grad[y][x] + lambda * weight[y][x]);
    }
  }
}

如上所示, 转换后的代码没有内存分配操作. 对于Tensor

One code for both CPU and GPU

因为Tensor

template<typename xpu>
void UpdateSGD(Tensor<xpu, 2> weight, const Tensor<xpu, 2> &grad,
               float eta, float lambda) {
  weight -= eta * (grad + lambda * weight);
}

Matrix Multiplications

我们可以实现一段简单的代码,实现矩阵点乘, 编译阶段将被转换成对MKL和cuBLAS这里标准库的调用:

template<typename xpu>
void Backprop(Tensor<xpu, 2> gradin,
              const Tensor<xpu, 2> &gradout,
              const Tensor<xpu, 2> &netweight) {
  gradin = dot(gradout, netweight.T());
}

这段代码编译后既可以在cpu上运行,也可以在gpu上运行

User Define Operator

应用中我们很可能会自定义函数. 例如mshadow里没有元素级别的sigmoid变化,我们可以通过如下代码增加这个功能:

struct sigmoid {
  MSHADOW_XINLINE static float Map(float a) {
    return 1.0f / (1.0f + expf(-a));
  }
};
template<typename xpu>
void ExampleSigmoid(Tensor<xpu, 2> out, const Tensor<xpu, 2> &in) {
  out = F<sigmoid>(in * 2.0f) + 1.0f;
}

转换后的CPU代码如下所示:

template<typename xpu>
void ExampleSigmoid(Tensor<xpu, 2> out, const Tensor<xpu, 2> &in) {
  for (index_t y = 0; y < out.size(0); ++y) {
    for(index_t x = 0; x < out.size(1); ++x) {
      out[y][x] = sigmoid::Map(in[y][x] * 2.0f) + 1.0f;
    }
  }
}

操作可以使用表达式的组合. 我们不仅可以 out = F<sigmoid>(in), 也可以 out = F<sigmoid>+2.0
或者 out = F<sigmoid>(F<sigmoid>(in))
这段代码同样可以转换成CUDA kernel的版本,在GPU上运行,参考defop.cpp.

Complete Example

以下的代码来自basic.cpp,展示了mshadow的常见用法.

// header file to use mshadow
#include "mshadow/tensor.h"
// this namespace contains all data structures, functions
using namespace mshadow;
// this namespace contains all operator overloads
using namespace mshadow::expr;

int main(void) {
  // intialize tensor engine before using tensor operation, needed for CuBLAS
  InitTensorEngine<cpu>();
  // assume we have a float space
  float data[20];
  // create a 2 x 5 x 2 tensor, from existing space
  Tensor<cpu, 3> ts(data, Shape3(2,5,2));
    // take first subscript of the tensor
  Tensor<cpu, 2> mat = ts[0];
  // Tensor object is only a handle, assignment means they have same data content
  // we can specify content type of a Tensor, if not specified, it is float bydefault
  Tensor<cpu, 2, float> mat2 = mat;

  // shape of matrix, note size order is the same as numpy
  printf("%u X %u matrix\n", mat.size(0), mat.size(1));

  // initialize all element to zero
  mat = 0.0f;
  // assign some values
  mat[0][1] = 1.0f; mat[1][0] = 2.0f;
  // elementwise operations
  mat += (mat + 10.0f) / 10.0f + 2.0f;

  // print out matrix, note: mat2 and mat1 are handles(pointers)
  for (index_t i = 0; i < mat.size(0); ++i) {
    for (index_t j = 0; j < mat.size(1); ++j) {
      printf("%.2f ", mat2[i][j]);
    }
    printf("\n");
  }
  // shutdown tensor enigne after usage
  ShutdownTensorEngine<cpu>();
  return 0;
}

参考

[mshadow@github]https://github.com/dmlc/mshadow/tree/master/guide