hbase过滤器filter及自定义filter

hbase过滤器filter及自定义filter

1.filter源码实现：

hbase的filter定义在protobuf中（filter.proto文件）。如：

message QualifierFilter {
    required CompareFilter compare_filter = 1;
}

message RandomRowFilter {
    required float chance = 1;
}

message RowFilter {
    required CompareFilter compare_filter = 1;
}

message SingleColumnValueExcludeFilter {
    required SingleColumnValueFilter single_column_value_filter = 1;
}

message SingleColumnValueFilter {
    optional bytes column_family = 1;
    optional bytes column_qualifier = 2;
    required CompareType compare_op = 3;
    required Comparator comparator = 4;
    optional bool filter_if_missing = 5;
    optional bool latest_version_only = 6;
}

等。。。各种内置filter定义。
具体filter实现在org.apache.hadoop.hbase.filter包中。
如：

image

2.自定义实现filter：

需要继承filterBase类，FilterBase类说明：

/**
 * Abstract base class to help you implement new Filters.  Common "ignore" or NOOP type
 * methods can go here, helping to reduce boiler plate in an ever-expanding filter
 * library.
 * <p>
 * If you could instantiate FilterBase, it would end up being a "null" filter -
 * that is one that never filters anything.
 */
public abstract class FilterBase extends Filter {
需要重写的方法：

    //为每个新行重置过滤器
    @Override
    public void reset() throws IOException {
    }
    
    //检查行键，offset为偏移量，不是0
    @Override
    public boolean filterRowKey(byte[] buffer, int offset, int length) throws IOException {
        return false;
    }
    
    //true，则用于结束扫描操作
    @Override
    public boolean filterAllRemaining() throws IOException {
        return false;
    }
    @Override
    public Cell transformCell(Cell v) throws IOException {
        // Old filters based off of this class will override KeyValue transform(KeyValue).
        // Thus to maintain compatibility we need to call the old version.
        return transform(KeyValueUtil.ensureKeyValue(v));
    }
    @Override
    public void filterRowCells(List<Cell> ignored) throws IOException {
    }
    @Override
    public boolean hasFilterRow() {
        return false;
    }
    @Override
    public boolean filterRow() throws IOException {
        return false;
    }
    public Cell getNextCellHint(Cell currentKV) throws IOException {
        // Old filters based off of this class will override KeyValue getNextKeyHint(KeyValue).
        // Thus to maintain compatibility we need to call the old version.
        return getNextKeyHint(KeyValueUtil.ensureKeyValue(currentKV));
    }
    public boolean isFamilyEssential(byte[] name) throws IOException {
        return true;
    }
    public static Filter createFilterFromArguments(ArrayList<byte[]> filterArguments) {
        throw new IllegalArgumentException("This method has not been implemented");
    }
    public String toString() {
        return this.getClass().getSimpleName();
    }
    public byte[] toByteArray() throws IOException {
        return new byte[0];
    }
    boolean areSerializedFieldsEqual(Filter other) {
        return true;
    }

}

filter内部方法执行顺序：

image

3.使用自定义filter：

打成jar包，要在hbase-env.sh中指明路径。
export  HBASE_CLASSPATH="/hbase/target/hbase-customfilter.jar"，然后就可以在客户端中使用它了。

也可以在源码中直接添加filter，有必要的话。