hbase shell使用filter

最新推荐文章于 2024-03-30 14:39:25 发布

wangyh19870928

最新推荐文章于 2024-03-30 14:39:25 发布

阅读量241

点赞数

CC 4.0 BY-SA版权

文章标签：大数据 shell

本文链接：https://blog.youkuaiyun.com/wangyh19870928/article/details/84684402

本文详细介绍了如何在HBase中使用单个filter和FilterList进行数据过滤，包括IntSingleColumnValueFilter、SingleColumnValueFilter、CompareFilter、SubstringComparator等组件的应用实例。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

单个filter:

import org.apache.hadoop.hbase.filter.CompareFilter
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter
import org.apache.hadoop.hbase.filter.SubstringComparator
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.filter.FilterList
import 'ict.gavin.IntSingleColumnValueFilter' //注意加引号

filter=IntSingleColumnValueFilter.new(Bytes.toBytes('cf3'),Bytes.toBytes('dport'), CompareFilter::CompareOp.valueOf('EQUAL'),1)
scan 'hive_test_b', {FILTER=>filter, COLUMNS=>['cf3:dport']}

--------------------------------------------------------------------------------------

filterlist的情况：

import org.apache.hadoop.hbase.filter.CompareFilter
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter
import org.apache.hadoop.hbase.filter.SubstringComparator
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.filter.FilterList

filter1 = SingleColumnValueFilter.new(
Bytes.toBytes('c'),
Bytes.toBytes('PDR'),
CompareFilter::CompareOp.valueOf('EQUAL'),
SubstringComparator.new('\\w')
)
filter1.setFilterIfMissing(true)
filter2 = SingleColumnValueFilter.new(
Bytes.toBytes('c'),
Bytes.toBytes('PDR'),
CompareFilter::CompareOp.valueOf('EQUAL'),
SubstringComparator.new('.+')
)
filter2.setFilterIfMissing(true)
filter3 = SingleColumnValueFilter.new(
Bytes.toBytes('c'),
Bytes.toBytes('PDR'),
CompareFilter::CompareOp.valueOf('EQUAL'),
SubstringComparator.new('.*')
)
filter3.setFilterIfMissing(true)
filterlist= FilterList.new(FilterList::Operator::MUST_PASS_ONE,[filter1, filter2,filter3])

scan 'BoardConfig',{FILTER=>filterlist,COLUMNS=>['o:URL','c:PDR']}