TensorFlow多GPU并行计算

1. TensorFlow指定特定GPU或者CPU进行计算：

说明：示例计算机为单CPU（编号为0），单GPU（编号为0），安装的TensorFlow为GPU版。

本文的结构如下：

• 默认为GPU #0
• 指定CPU #0
• 指定GPU #1
• 指定GPU #0 + CPU #0

1.1 默认为GPU #0

In [1]: import tensorflow as tf

I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally

In [2]: with tf.Session() as sess:
   ...:     matrix1=tf.constant([[3.,3.]])
   ...:     matrix2=tf.constant([[2.],[2.]])
   ...:     product=tf.matmul(matrix1,matrix2)
   ...:     result=sess.run(product)
   ...:     print result
   ...:

I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties: 
name: GeForce GTX 970
major: 5 minor: 2 memoryClockRate (GHz) 1.266
pciBusID 0000:01:00.0
Total memory: 4.00GiB
Free memory: 3.62GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:838] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 970, pci bus id: 0000:01:00.0)
[[ 12.]]

1.2 指定GPU #0

In [1]: import tensorflow as tf

I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so local