TFRecords读写

TFRecords读写实践

最新推荐文章于 2019-10-15 16:33:06 发布

原创最新推荐文章于 2019-10-15 16:33:06 发布 · 置顶 · 220 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#TFRecords #读写

开发技术专栏收录该内容

6 篇文章

订阅专栏

本文介绍如何使用TensorFlow中的TFRecords格式进行数据的读写操作。TFRecords是一种高效的文件格式，适用于大规模数据集。文中提供了将图像数据转换为TFRecords格式的示例代码，并展示了如何从TFRecords中读取这些数据。

数据庞大的时候，建议转成TFRecords
读
def read_tfrecords(file_list):
reader = tf.TFRecordReader()
file_queue = tf.train.string_input_producer(file_list)
_,serialized_example = reader.read(file_queue)

features = tf.parse_single_example(
serialized_example,
features={
‘label’:tf.FixedLenFeature([],tf.int64)
‘features’:tf.FixedLenFeature([784],tf.int64)
}
)
return features[‘label’],features[‘features’]

file_list = []
file_list.append(path)
label,img = read_tfrecords(file_list)
label_batch,img_batch = tf.train.shuffle_batch([label,img],batch_size=10000,capacity=20000,min_after_dequeue=10000)

写,来自tensorlayer里的example
def data_to_tfrecord(images, labels, filename):
“”” Save data into TFRecord “””
if os.path.isfile(filename):
print(“%s exists” % filename)
return
print(“Converting data into %s …” % filename)
# cwd = os.getcwd()
writer = tf.python_io.TFRecordWriter(filename)
for index, img in enumerate(images):
img_raw = img.tobytes()
## Visualize a image
# tl.visualize.frame(np.asarray(img, dtype=np.uint8), second=1, saveable=False, name=’frame’, fig_idx=1236)
label = int(labels[index])
# print(label)
## Convert the bytes back to image as follow:
# image = Image.frombytes(‘RGB’, (32, 32), img_raw)
# image = np.fromstring(img_raw, np.float32)
# image = image.reshape([32, 32, 3])
# tl.visualize.frame(np.asarray(image, dtype=np.uint8), second=1, saveable=False, name=’frame’, fig_idx=1236)
example = tf.train.Example(
features=tf.train.Features(
feature={
“label”: tf.train.Feature(int64_list=tf.train.Int64List(value=[label])),
‘img_raw’: tf.train.Feature(bytes_list=tf.train.BytesList(value=[img_raw])),
}))
writer.write(example.SerializeToString()) # Serialize To String
writer.close()