Hadoop-SequenceFile code

最新推荐文章于 2025-08-15 12:37:27 发布

SAN_YUN

最新推荐文章于 2025-08-15 12:37:27 发布

阅读量137

点赞数

CC 4.0 BY-SA版权

分类专栏： hadoop 文章标签：大数据 java

本文链接：https://blog.youkuaiyun.com/SAN_YUN/article/details/84525363

hadoop 专栏收录该内容

12 篇文章

订阅专栏

本文提供了一个使用Hadoop SequenceFile的示例程序，演示了如何创建SequenceFile并将数据写入其中。通过具体代码展示了SequenceFile的基本用法，包括配置设置、文件系统的获取以及数据的写入过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

写的例子：

package mytest.mapreduce;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.SequenceFile;
import org.apache.hadoop.io.Text;

public class MySequenceTester {

	private static final String data[] = {
			"One, two, buckle my shoe",
			"Three, four, shut the door",
			"Five, six, pick up sticks",
			"Seven, eight, lay them straight",
			"Nine, ten, a big fat hen"
	};

	public static void main(String[] args) throws IOException {
		Configuration conf = new Configuration();
		FileSystem fs = FileSystem.get(conf);
		Path path = new Path("/duitang/data/test/client-request.log.seq");

		IntWritable key = new IntWritable();
		Text value = new Text();

		SequenceFile.Writer writer = SequenceFile.createWriter(fs, conf, path, key.getClass(), value.getClass());
		try {

			for (int i = 0; i < 100; i++) {
				key.set(100 - i);
				value.set(data[i % data.length]);
				System.out.printf("[%s]\t%s\t%s\n", writer.getLength(), key, value);
				writer.append(key, value);
			}

		} finally {
			IOUtils.closeStream(writer);
		}
	}
}