storm trident实战 filter,function的使用

最新推荐文章于 2019-07-11 11:22:00 发布

hui1207968974

最新推荐文章于 2019-07-11 11:22:00 发布

阅读量218

点赞数

分类专栏： storm 文章标签： storm trident filter function

storm 专栏收录该内容

7 篇文章

订阅专栏

本文介绍 Apache Storm 中 Trident 模块的 filter 和 function 的使用方法。filter 通过返回 true 或 false 来决定是否保留 tuple，而 function 则可以接收并输出一个或多个 tuples。示例代码展示了如何实现简单的过滤逻辑及对 tuple 的字段进行处理。

一、Storm trident filter

filter通过返回true和false。来判断是否对信息过滤。

1.1 代码

	public static void main(String[] args) throws InterruptedException,
			AlreadyAliveException, InvalidTopologyException,
			AuthorizationException {
		FixedBatchSpout spout = new FixedBatchSpout(new Fields("a", "b"),
				1, new Values(1, 2), new Values(4, 1),
				new Values(3, 0));
		spout.setCycle(false);
		TridentTopology topology = new TridentTopology();
		topology.newStream("spout", spout)
				.each(new Fields("a"), new MyFilter())
				.each(new Fields("a", "b"), new PrintFilterBolt(),new Fields(""));
		Config config = new Config();
		config.setNumWorkers(2);
		config.setNumAckers(1);
		config.setDebug(false);
		StormSubmitter.submitTopology("trident_filter", config,
				topology.build());
	}

MyFilter：

import org.apache.storm.trident.operation.BaseFilter;
import org.apache.storm.trident.tuple.TridentTuple;

public class MyFilter extends BaseFilter {

    /**
	 * 
	 */
	private static final long serialVersionUID = 1L;

	@Override
	public boolean isKeep(TridentTuple tuple) {
		return tuple.getInteger(0) == 1;
	}
	
}

PrintFilterBolt：

public class PrintFilterBolt extends BaseFunction {
	/**
	 * 
	 */
	private static final long serialVersionUID = 1L;

	@Override
	public void execute(TridentTuple tuple, TridentCollector collector) {
		int firstIndex = tuple.getInteger(0);
		int secondIndex = tuple.getInteger(1);
		List<Integer> list = new ArrayList<Integer>();
		list.add(firstIndex);
		list.add(secondIndex);
		System.out.println("after storm filter opertition change is : "
				+ list.toString());
	}

}

运行结果：

2016-12-22 13:16:09.079 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-12-22 13:16:09.088 o.a.s.d.executor [INFO] Prepared bolt $spoutcoord-spout-spout:(2)
2016-12-22 13:16:09.736 STDIO [INFO] after storm filter opertition change is : [1, 2]

二、Storm trident function

函数的作用是接收一个tuple(需指定接收tuple的哪个字段)，输出0个或多个tuples。输出的新字段值会被追加到原始输入tuple的后面，如果一个function不输出tuple，那就意味这这个tuple被过滤掉了。

2.1 代码

public static void main(String[] args) throws InterruptedException,
			AlreadyAliveException, InvalidTopologyException,
			AuthorizationException {

		FixedBatchSpout spout = new FixedBatchSpout(new Fields("a", "b", "c"),
				1, new Values(1, 2, 3), new Values(4, 1, 6),
				new Values(3, 0, 8));
		spout.setCycle(false);
		TridentTopology topology = new TridentTopology();
		topology.newStream("spout", spout)
				.each(new Fields("b"), new MyFunction(), new Fields("d"))
				.each(new Fields("a", "b", "c", "d"), new PrintFunctionBolt(),
						new Fields(""));
		Config config = new Config();
		config.setNumWorkers(2);
		config.setNumAckers(1);
		config.setDebug(false);
		StormSubmitter.submitTopology("trident_function", config,
				topology.build());
	}

MyFunction：

public class MyFunction extends BaseFunction {

    /**
	 * 
	 */
	private static final long serialVersionUID = 1L;
	
	public void execute(TridentTuple tuple, TridentCollector collector) {
        for(int i=0; i < tuple.getInteger(0); i++) {
            collector.emit(new Values(i));
        }
    }

}

PrintFunctionBolt：

public class PrintFunctionBolt extends BaseFunction {
	/**
	 * 
	 */
	private static final long serialVersionUID = 1L;

	@Override
	public void execute(TridentTuple tuple, TridentCollector collector) {
		int firstIndex = tuple.getInteger(0);
		int secondIndex = tuple.getInteger(1);
		int threeIndex = tuple.getInteger(2);
		int fourIndex = tuple.getInteger(3);
		List<Integer> list = new ArrayList<Integer>();
		list.add(firstIndex);
		list.add(secondIndex);
		list.add(threeIndex);
		list.add(fourIndex);
		System.out.println("after storm function opertition change is : " +list.toString());
	}

}

运行效果：

2016-12-22 13:22:34.365 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.80.130/192.168.80.130:2181, sessionid = 0x159285f1109000c, negotiated timeout = 20000
2016-12-22 13:22:34.366 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-12-22 13:22:34.374 o.a.s.d.executor [INFO] Prepared bolt $spoutcoord-spout-spout:(2)
2016-12-22 13:22:34.415 STDIO [INFO] after storm function opertition change is : [1, 2, 3, 0]
2016-12-22 13:22:34.415 STDIO [INFO] after storm function opertition change is : [1, 2, 3, 1]
2016-12-22 13:22:34.442 STDIO [INFO] after storm function opertition change is : [4, 1, 6, 0]