HBase和Phoenix的类型转换

在Phoenix中字段的类型所保存的byte数组和HBase中用Bytes工具了转换的byte数组可能会对不上,比如Bytes.toBytes(1)的结果和Phoenix中定义为INTEGER类的byte[]就不一样。因为Phoenix中的INTEGER是带符号的,要用无符号的话需要定义UNSIGNED_INT类型,UNSIGENT_INT类型的byte[]值和Bytes.toBytes(1)是一样的,但是UNSIGENT_INT无法存负数。

所以如果直接定义INTEGER类型的话是不能直接简单地用Bytes.toBytes来转换得到的,由于这种情况的存在就不能直接操作HBase然后直接在Phoenix查看,反之亦然。

所一这里定义了一套将byte数组转换为Phoenix可以用的方法,具体准换的代码是从Phoenix的core包中抠出来的。

先定义一个类型枚举类:

/**
 * Phoenix中的类型
 */
public enum PhType {
    DEFAULT(-1, "DEFAULT"),
    UNSIGNED_INT(4, "UNSIGNED_INT"),
    UNSIGNED_BIGINT(8, "UNSIGNED_BIGINT"),
    UNSIGNED_TINYINT(1, "UNSIGNED_TINYINT"),
    UNSIGNED_SMAILLINT(2, "UNSIGNED_SMAILLINT"),
    UNSIGNED_FLOAT(4, "UNSIGNED_FLOAT"),
    UNSIGNED_DOUBLE(8, "UNSIGNED_DOUBLE"),
    INTEGER(4, "INTEGER"),
    BIGINT(8, "BIGINT"),
    TINYINT(1, "TINYINT"),
    SMAILLINT(2, "SMAILLINT"),
    FLOAT(4, "FLOAT"),
    DOUBLE(8, "DOUBLE"),
    DECIMAL(-1, "DECIMAL"),
    BOOLEAN(1, "BOOLEAN"),
    TIME(8, "TIME"),                       //对应Phoenix的UNSIGNED_TIME
    DATE(8, "DATE"),                       //对应Phoenix的UNSIGNED_DATE
    TIMESTAMP(12, "TIMESTAMP"),            //对应Phoenix的UNSIGNED_TIMESTAMP
    VARCHAR(-1, "VARCHAR"),
    VARBINARY(-1, "VARBINARY");

    /**
     * -1:长度可变
     */
    private int len;
    private String type;

    PhType(int len, String type) {
        this.len = len;
        this.type = type;
    }

    public int getLen() {
        return len;
    }

    public String getType() {
        return this.type;
    }

    public static PhType getType(String type) {
        if (type == null) return DEFAULT;
        PhType phType;
        if (type.equalsIgnoreCase(UNSIGNED_INT.type)) {
            phType = UNSIGNED_INT;
        } else if (type.equalsIgnoreCase(UNSIGNED_BIGINT.type)) {
            phType = UNSIGNED_BIGINT;
        } else if (type.equalsIgnoreCase(UNSIGNED_TINYINT.type)) {
            phType = UNSIGNED_TINYINT;
        } else if (type.equalsIgnoreCase(UNSIGNED_SMAILLINT.type)) {
            phType = UNSIGNED_SMAILLINT;
        } else if (type.equalsIgnoreCase(UNSIGNED_FLOAT.type)) {
            phType = UNSIGNED_FLOAT;
        } else if (type.equalsIgnoreCase(UNSIGNED_DOUBLE.type)) {
            phType = UNSIGNED_DOUBLE;
        } else if (type.equalsIgnoreCase(INTEGER.type)) {
            phType = INTEGER;
        } else if (type.equalsIgnoreCase(BIGINT.type)) {
            phType = BIGINT;
        } else if (type.equalsIgnoreCase(TINYINT.type)) {
            phType = TINYINT;
        } else if (type.equalsIgnoreCase(SMAILLINT.type)) {
            phType = SMAILLINT;
        } else if (type.equalsIgnoreCase(FLOAT.type)) {
            phType = FLOAT;
        } else if (type.equalsIgnoreCase(DOUBLE.type)) {
            phType = DOUBLE;
        } else if (type.equalsIgnoreCase(BOOLEAN.type)) {
            phType = BOOLEAN;
        } else if (type.equalsIgnoreCase(TIME.type)) {
            phType = TIME;
        } else if (type.equalsIgnoreCase(DATE.type)) {
            phType = DATE;
        } else if (type.equalsIgnoreCase(TIMESTAMP.type)) {
            phType = TIMESTAMP;
        } else if (type.equalsIgnoreCase(VARCHAR.type)) {
            phType = VARCHAR;
        } else if (type.equalsIgnoreCase(VARBINARY.type)) {
            phType = VARBINARY;
        } else if (type.equalsIgnoreCase(DECIMAL.type)) {
            phType = DECIMAL;
        } else {
            phType = DEFAULT;
        }
        return phType;
    }
}

然后编写对以上枚举中所有类型的转换:

import com.google.common.math.LongMath;
import org.apache.hadoop.hbase.util.Bytes;

import java.math.BigDecimal;
import java.math.BigInteger;
import java.math.MathContext;
import java.math.RoundingMode;
import java.sql.Timestamp;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

/**
 * 针对Phoenix的类型转换
 */
public class PhTypeUtil {
    private static String dateFormatPattern = "yyyy-MM-dd";
    private static String timeFormatPattern = "HH:mm:ss";
    private static String datetimeFormatPattern = "yyyy-MM-dd HH:mm:ss";
    private static String timestampFormatPattern = "yyyy-MM-dd HH:mm:ss.SSS";

    public static byte[] toBytes(Object v, PhType phType) {
        if (v == null) return null;
        byte[] b = null;
        if (phType == PhType.INTEGER) {
            b = new byte[Bytes.SIZEOF_INT];
            encodeInt(((Number) v).intValue(), b, 0);
        } else if (phType == PhType.UNSIGNED_INT) {
            b = new byte[Bytes.SIZEOF_INT];
            encodeUnsignedInt(((Number) v).intValue(), b, 0);
        } else if (phType == PhType.BIGINT) {
            b = new byte[Bytes.SIZEOF_LONG];
            encodeLong(((Number) v).longValue(), b, 0);
        } else if (phType == PhType.UNSIGNED_BIGINT) {
            b = new byte[Bytes.SIZEOF_LONG];
            encodeUnsignedLong(((Number) v).longValue(), b, 0);
        } else if (phType == PhType.SMAILLINT) {
            b = new byte[Bytes.SIZEOF_SHORT];
            encodeShort(((Number) v).shortValue(), b, 0);
        } else if (phType == PhType.UNSIGNED_SMAILLINT) {
            b = new byte[Bytes.SIZEOF_SHORT];
            encodeUnsignedShort(((Number) v).shortValue(), b, 0);
        } else if (phType == PhType.TINYINT) {
            b = new byte[Bytes.SIZEOF_BYTE];
            encodeByte(((Number) v).byteValue(), b, 0);
        } else if (phType == PhType.UNSIGNED_TINYINT) {
            b = new byte[Bytes.SIZEOF_BYTE];
            encodeUnsignedByte(((Number) v).byteValue(), b, 0);
        } else if (phType == PhType.FLOAT) {
            b = new byte[Bytes.SIZEOF_FLOAT];
            encodeFloat(((Number) v).floatValue(), b, 0);
        } else if (phType == PhType.UNSIGNED_FLOAT) {
            b = new byte[Bytes.SIZEOF_FLOAT];
            encodeUnsignedFloat(((Number) v).floatValue(), b, 0);
        } else if (phType == PhType.DOUBLE) {
            b = new byte[Bytes.SIZEOF_DOUBLE];
            encodeDouble(((Number) v).doubleValue(), b, 0);
        } else if (phType == PhType.UNSIGNED_DOUBLE) {
            b = new byte[Bytes.SIZEOF_DOUBLE];
            encodeUnsignedDouble(((Number) v).doubleValue(), b, 0);
        } else if (phType == PhType.BOOLEAN) {
            if ((Boolean) v) {
                b = new byte[]{1};
            } else {
                b = new byte[]{0};
            }
        } else if (phType == PhType.TIME || phType == PhType.DATE) {
            b = new byte[Bytes.SIZEOF_LONG];
            encodeDate(v, b, 0);
        } else if (phType == PhType.TIMESTAMP) {
            b = new byte[Bytes.SIZEOF_LONG + Bytes.SIZEOF_INT];
            encodeTimestamp(v, b, 0);
        } else if (phType == PhType.VARBINARY) {
            b = (byte[]) v;
        } else if (phType == PhType.VARCHAR || phType == PhType.DEFAULT) {
            b = Bytes.toBytes(v.toString());
        } else if (phType == PhType.DECIMAL) {
            b = encodeDecimal(v);
        }
        return b;
    }

    public static Object toObject(byte[] b, PhType phType) {
        if (b == null) return null;
        Object v = null;
        if (phType == PhType.INTEGER) {
            v = decodeInt(b, 0);
        } else if (phType == PhType.UNSIGNED_INT) {
            v = decodeUnsignedInt(b, 0);
        } else if (phType == PhType.BIGINT) {
            v = decodeLong(b, 0);
        } else if (phType == PhType.UNSIGNED_BIGINT) {
            v = decodeUnsignedLong(b, 0);
        } else if (phType == PhType.SMAILLINT) {
            v = decodeShort(b, 0);
        } else if (phType == PhType.UNSIGNED_SMAILLINT) {
            v = decodeUnsignedShort(b, 0);
        } else if (phType == PhType.TINYINT) {
            v = decodeByte(b, 0);
        } else if (phType == PhType.UNSIGNED_TINYINT) {
            v = decodeUnsignedByte(b, 0);
        } else if (phType == PhType.FLOAT) {
            v = decodeFloat(b, 0);
        } else if (phType == PhType.UNSIGNED_FLOAT) {
            v = decodeUnsignedFloat(b, 0);
        } else if (phType == PhType.DOUBLE) {
            v = decodeDouble(b, 0);
        } else if (phType == PhType.UNSIGNED_DOUBLE) {
            v = decodeUnsignedDouble(b, 0);
        } else if (phType == PhType.BOOLEAN) {
            checkForSufficientLength(b, 0, Bytes.SIZEOF_BOOLEAN);
            if (b[0] == 1) {
                v = true;
            } else if (b[0] == 0) {
                v = false;
            }
        } else if (phType == PhType.TIME || phType == PhType.DATE) {
            v = new Date(decodeUnsignedLong(b, 0));
        } else if (phType == PhType.TIMESTAMP) {
            long millisDeserialized = decodeUnsignedLong(b, 0);
            Timestamp ts = new Timestamp(millisDeserialized);
            int nanosDeserialized = decodeUnsignedInt(b, Bytes.SIZEOF_LONG);
            ts.setNanos(nanosDeserialized < 1000000 ? ts.getNanos() + nanosDeserialized : nanosDeserialized);
            v = ts;
        } else if (phType == PhType.VARBINARY) {
            v = b;
        } else if (phType == PhType.VARCHAR || phType == PhType.DEFAULT) {
            v = Bytes.toString(b);
        } else if (phType == PhType.DECIMAL) {
            v = decodeDecimal(b, 0, b.length);
        }

        return v;
    }


    public static int decodeInt(byte[] bytes, int o) {
        checkForSufficientLength(bytes, o, Bytes.SIZEOF_INT);
        int v;
        v = bytes[o] ^ 0x80; // Flip sign bit back
        for (int i = 1; i < Bytes.SIZEOF_INT; i++) {
            v = (v << 8) + (bytes[o + i] & 0xff);
        }
        return v;
    }

    public static int encodeInt(int v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
        b[o + 0] = (byte) ((v >> 24) ^ 0x80); // Flip sign bit so that INTEGER is binary comparable
        b[o + 1] = (byte) (v >> 16);
        b[o + 2] = (byte) (v >> 8);
        b[o + 3] = (byte) v;
        return Bytes.SIZEOF_INT;
    }

    public static int decodeUnsignedInt(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);

        int v = Bytes.toInt(b, o);
        if (v < 0) {
            throw new RuntimeException();
        }
        return v;
    }

    public static int encodeUnsignedInt(int v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
        if (v < 0) {
            throw new RuntimeException();
        }
        Bytes.putInt(b, o, v);
        return Bytes.SIZEOF_INT;
    }

    public static long decodeLong(byte[] bytes, int o) {
        checkForSufficientLength(bytes, o, Bytes.SIZEOF_LONG);
        long v;
        byte b = bytes[o];
        v = b ^ 0x80; // Flip sign bit back
        for (int i = 1; i < Bytes.SIZEOF_LONG; i++) {
            b = bytes[o + i];
            v = (v << 8) + (b & 0xff);
        }
        return v;
    }

    public static int encodeLong(long v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
        b[o + 0] = (byte) ((v >> 56) ^ 0x80); // Flip sign bit so that INTEGER is binary comparable
        b[o + 1] = (byte) (v >> 48);
        b[o + 2] = (byte) (v >> 40);
        b[o + 3] = (byte) (v >> 32);
        b[o + 4] = (byte) (v >> 24);
        b[o + 5] = (byte) (v >> 16);
        b[o + 6] = (byte) (v >> 8);
        b[o + 7] = (byte) v;
        return Bytes.SIZEOF_LONG;
    }

    public static long decodeUnsignedLong(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
        long v = 0;
        for (int i = o; i < o + Bytes.SIZEOF_LONG; i++) {
            v <<= 8;
            v ^= b[i] & 0xFF;
        }

        if (v < 0) {
            throw new RuntimeException();
        }
        return v;
    }

    public static int encodeUnsignedLong(long v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
        if (v < 0) {
            throw new RuntimeException();
        }
        Bytes.putLong(b, o, v);
        return Bytes.SIZEOF_LONG;
    }

    public static short decodeShort(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
        int v;
        v = b[o] ^ 0x80; // Flip sign bit back
        for (int i = 1; i < Bytes.SIZEOF_SHORT; i++) {
            v = (v << 8) + (b[o + i] & 0xff);
        }
        return (short) v;
    }

    public static int encodeShort(short v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
        b[o + 0] = (byte) ((v >> 8) ^ 0x80); // Flip sign bit so that Short is binary comparable
        b[o + 1] = (byte) v;
        return Bytes.SIZEOF_SHORT;
    }

    public static short decodeUnsignedShort(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
        short v = Bytes.toShort(b, o);
        if (v < 0) {
            throw new RuntimeException();
        }
        return v;
    }

    public static int encodeUnsignedShort(short v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
        if (v < 0) {
            throw new RuntimeException();
        }
        Bytes.putShort(b, o, v);
        return Bytes.SIZEOF_SHORT;
    }

    public static byte decodeByte(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_BYTE);
        int v;
        v = b[o] ^ 0x80; // Flip sign bit back
        return (byte) v;
    }

    public static int encodeByte(byte v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_BYTE);
        b[o] = (byte) (v ^ 0x80); // Flip sign bit so that Short is binary comparable
        return Bytes.SIZEOF_BYTE;
    }

    public static byte decodeUnsignedByte(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_BYTE);
        byte v = b[o];
        if (v < 0) {
            throw new RuntimeException();
        }
        return v;
    }

    public static int encodeUnsignedByte(byte v, byte[] b, int o) {
        if (v < 0) {
            throw new RuntimeException();
        }
        Bytes.putByte(b, o, v);
        return Bytes.SIZEOF_BYTE;
    }

    public static float decodeFloat(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
        int value;
        value = Bytes.toInt(b, o);
        value--;
        value ^= (~value >> Integer.SIZE - 1) | Integer.MIN_VALUE;
        return Float.intBitsToFloat(value);
    }


    public static int encodeFloat(float v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_FLOAT);
        int i = Float.floatToIntBits(v);
        i = (i ^ ((i >> Integer.SIZE - 1) | Integer.MIN_VALUE)) + 1;
        Bytes.putInt(b, o, i);
        return Bytes.SIZEOF_FLOAT;
    }

    public static float decodeUnsignedFloat(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_FLOAT);
        float v = Bytes.toFloat(b, o);
        if (v < 0) {
            throw new RuntimeException();
        }
        return v;
    }

    public static int encodeUnsignedFloat(float v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_FLOAT);
        if (v < 0) {
            throw new RuntimeException();
        }
        Bytes.putFloat(b, o, v);
        return Bytes.SIZEOF_FLOAT;
    }

    public static double decodeDouble(byte[] bytes, int o) {
        checkForSufficientLength(bytes, o, Bytes.SIZEOF_LONG);
        long l;
        l = Bytes.toLong(bytes, o);
        l--;
        l ^= (~l >> Long.SIZE - 1) | Long.MIN_VALUE;
        return Double.longBitsToDouble(l);
    }


    public static int encodeDouble(double v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
        long l = Double.doubleToLongBits(v);
        l = (l ^ ((l >> Long.SIZE - 1) | Long.MIN_VALUE)) + 1;
        Bytes.putLong(b, o, l);
        return Bytes.SIZEOF_LONG;
    }

    public static double decodeUnsignedDouble(byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_DOUBLE);
        double v = Bytes.toDouble(b, o);
        if (v < 0) {
            throw new RuntimeException();
        }
        return v;
    }

    public static int encodeUnsignedDouble(double v, byte[] b, int o) {
        checkForSufficientLength(b, o, Bytes.SIZEOF_DOUBLE);
        if (v < 0) {
            throw new RuntimeException();
        }
        Bytes.putDouble(b, o, v);
        return Bytes.SIZEOF_DOUBLE;
    }

    public static int encodeDate(Object v, byte[] b, int o) {
        if (v instanceof Date) {
            encodeUnsignedLong(((Date) v).getTime(), b, 0);
        } else if (v instanceof String) {
            String dateStr = (String) v;
            int len = dateStr.length();
            Date date = null;
            try {
                if (len == 10 && dateStr.charAt(4) == '-' && dateStr.charAt(7) == '-') {
                    SimpleDateFormat format = new SimpleDateFormat(dateFormatPattern);
                    date = format.parse(dateStr);
                } else if (len == 8 && dateStr.charAt(2) == ':' && dateStr.charAt(5) == ':') {
                    SimpleDateFormat format = new SimpleDateFormat(timeFormatPattern);
                    date = format.parse(dateStr);
                } else if (len == 19 && dateStr.charAt(4) == '-' && dateStr.charAt(7) == '-'
                        && dateStr.charAt(13) == ':' && dateStr.charAt(16) == ':') {
                    SimpleDateFormat format = new SimpleDateFormat(datetimeFormatPattern);
                    date = format.parse(dateStr);
                } else if (len == 23 && dateStr.charAt(4) == '-' && dateStr.charAt(7) == '-'
                        && dateStr.charAt(13) == ':' && dateStr.charAt(16) == ':'
                        && dateStr.charAt(19) == '.') {
                    SimpleDateFormat format = new SimpleDateFormat(timestampFormatPattern);
                    date = format.parse(dateStr);
                }
                if (date != null) {
                    encodeUnsignedLong(date.getTime(), b, 0);
                }
            } catch (ParseException e) {
                throw new RuntimeException(e);
            }
        }
        return Bytes.SIZEOF_LONG;
    }

    public static int encodeTimestamp(Object v, byte[] b, int o) {
        if (v instanceof Timestamp) {
            Timestamp ts = (Timestamp) v;
            encodeUnsignedLong(ts.getTime(), b, o);
            Bytes.putInt(b, Bytes.SIZEOF_LONG, ts.getNanos() % 1000000);
        } else {
            encodeDate(v, b, o);
        }
        return Bytes.SIZEOF_LONG + Bytes.SIZEOF_INT;
    }

    public static byte[] encodeDecimal(Object object) {
        if (object == null) {
            return new byte[0];
        }
        BigDecimal v = (BigDecimal) object;
        v = v.round(DEFAULT_MATH_CONTEXT).stripTrailingZeros();
        int len = getLength(v);
        byte[] result = new byte[Math.min(len, 21)];
        decimalToBytes(v, result, 0, len);
        return result;
    }


    public static BigDecimal decodeDecimal(byte[] bytes, int offset, int length) {
        if (length == 1 && bytes[offset] == ZERO_BYTE) {
            return BigDecimal.ZERO;
        }
        int signum = ((bytes[offset] & 0x80) == 0) ? -1 : 1;
        int scale;
        int index;
        int digitOffset;
        long multiplier = 100L;
        int begIndex = offset + 1;
        if (signum == 1) {
            scale = (byte) (((bytes[offset] & 0x7F) - 65) * -2);
            index = offset + length;
            digitOffset = POS_DIGIT_OFFSET;
        } else {
            scale = (byte) ((~bytes[offset] - 65 - 128) * -2);
            index = offset + length - (bytes[offset + length - 1] == NEG_TERMINAL_BYTE ? 1 : 0);
            digitOffset = -NEG_DIGIT_OFFSET;
        }
        length = index - offset;
        long l = signum * bytes[--index] - digitOffset;
        if (l % 10 == 0) { // trailing zero
            scale--; // drop trailing zero and compensate in the scale
            l /= 10;
            multiplier = 10;
        }
        // Use long arithmetic for as long as we can
        while (index > begIndex) {
            if (l >= MAX_LONG_FOR_DESERIALIZE || multiplier >= Long.MAX_VALUE / 100) {
                multiplier = LongMath.divide(multiplier, 100L, RoundingMode.UNNECESSARY);
                break; // Exit loop early so we don't overflow our multiplier
            }
            int digit100 = signum * bytes[--index] - digitOffset;
            l += digit100 * multiplier;
            multiplier = LongMath.checkedMultiply(multiplier, 100);
        }

        BigInteger bi;
        // If still more digits, switch to BigInteger arithmetic
        if (index > begIndex) {
            bi = BigInteger.valueOf(l);
            BigInteger biMultiplier = BigInteger.valueOf(multiplier).multiply(ONE_HUNDRED);
            do {
                int digit100 = signum * bytes[--index] - digitOffset;
                bi = bi.add(biMultiplier.multiply(BigInteger.valueOf(digit100)));
                biMultiplier = biMultiplier.multiply(ONE_HUNDRED);
            } while (index > begIndex);
            if (signum == -1) {
                bi = bi.negate();
            }
        } else {
            bi = BigInteger.valueOf(l * signum);
        }
        // Update the scale based on the precision
        scale += (length - 2) * 2;
        BigDecimal v = new BigDecimal(bi, scale);
        return v;
    }


    private static int getLength(BigDecimal v) {
        int signum = v.signum();
        if (signum == 0) { // Special case for zero
            return 1;
        }
        return (signum < 0 ? 2 : 1) + (v.precision() + 1 + (v.scale() % 2 == 0 ? 0 : 1)) / 2;
    }

    private static final int MAX_PRECISION = 38;
    private static final MathContext DEFAULT_MATH_CONTEXT = new MathContext(MAX_PRECISION, RoundingMode.HALF_UP);
    private static final Integer MAX_BIG_DECIMAL_BYTES = 21;
    private static final byte ZERO_BYTE = (byte) 0x80;
    private static final byte NEG_TERMINAL_BYTE = (byte) 102;
    private static final int EXP_BYTE_OFFSET = 65;
    private static final int POS_DIGIT_OFFSET = 1;
    private static final int NEG_DIGIT_OFFSET = 101;
    private static final BigInteger MAX_LONG = BigInteger.valueOf(Long.MAX_VALUE);
    private static final BigInteger MIN_LONG = BigInteger.valueOf(Long.MIN_VALUE);
    private static final BigInteger ONE_HUNDRED = BigInteger.valueOf(100);
    private static final long MAX_LONG_FOR_DESERIALIZE = Long.MAX_VALUE / 1000;


    private static int decimalToBytes(BigDecimal v, byte[] result, final int offset, int length) {
        int signum = v.signum();
        if (signum == 0) {
            result[offset] = ZERO_BYTE;
            return 1;
        }
        int index = offset + length;
        int scale = v.scale();
        int expOffset = scale % 2 * (scale < 0 ? -1 : 1);
        int multiplyBy;
        BigInteger divideBy;
        if (expOffset == 0) {
            multiplyBy = 1;
            divideBy = ONE_HUNDRED;
        } else {
            multiplyBy = 10;
            divideBy = BigInteger.TEN;
        }
        // Normalize the scale based on what is necessary to end up with a base 100 decimal (i.e. 10.123e3)
        int digitOffset;
        BigInteger compareAgainst;
        if (signum == 1) {
            digitOffset = POS_DIGIT_OFFSET;
            compareAgainst = MAX_LONG;
            scale -= (length - 2) * 2;
            result[offset] = (byte) ((-(scale + expOffset) / 2 + EXP_BYTE_OFFSET) | 0x80);
        } else {
            digitOffset = NEG_DIGIT_OFFSET;
            compareAgainst = MIN_LONG;
            // Scale adjustment shouldn't include terminal byte in length
            scale -= (length - 2 - 1) * 2;
            result[offset] = (byte) (~(-(scale + expOffset) / 2 + EXP_BYTE_OFFSET + 128) & 0x7F);
            if (length <= MAX_BIG_DECIMAL_BYTES) {
                result[--index] = NEG_TERMINAL_BYTE;
            } else {
                // Adjust length and offset down because we don't have enough room
                length = MAX_BIG_DECIMAL_BYTES;
                index = offset + length;
            }
        }
        BigInteger bi = v.unscaledValue();
        // Use BigDecimal arithmetic until we can fit into a long
        while (bi.compareTo(compareAgainst) * signum > 0) {
            BigInteger[] dandr = bi.divideAndRemainder(divideBy);
            bi = dandr[0];
            int digit = dandr[1].intValue();
            result[--index] = (byte) (digit * multiplyBy + digitOffset);
            multiplyBy = 1;
            divideBy = ONE_HUNDRED;
        }
        long l = bi.longValue();
        do {
            long divBy = 100 / multiplyBy;
            long digit = l % divBy;
            l /= divBy;
            result[--index] = (byte) (digit * multiplyBy + digitOffset);
            multiplyBy = 1;
        } while (l != 0);

        return length;
    }

    private static void checkForSufficientLength(byte[] b, int offset, int requiredLength) {
        if (b.length < offset + requiredLength) {
            throw new RuntimeException
                    ("Expected length of at least " + requiredLength + " bytes, but had " + (b.length - offset));
        }
    }
}

其中调用public static byte[] toBytes(Object v, PhType phType) 和

public static Object toObject(byte[] b, PhType phType)两个方法就可以实现java类型到Phoenix类型的换转。


                
<think>好的,用户的问题是如何通过HBase访问Phoenix数据,需要配置方法示例代码。首先,我需要回忆HBasePhoenix的关系。Phoenix是构建在HBase之上的SQL层,提供SQL查询功能。用户可能已经知道Phoenix可以通过SQL操作HBase数据,但现在想反过来,用HBase的API访问Phoenix处理过的数据。接下来,我需要确定两者的数据存储结构是否兼容。Phoenix在创建表时,会使用特定的命名空间列族,比如将列映射为HBase的列限定符。因此,HBase直接访问Phoenix表的数据是可行的,但需要注意表名、列族列名的对应关系。然后,配置方面可能需要确保HBasePhoenix的版本兼容。用户提到的引用中提到了Phoenix5.0.0-5.1.2,所以需要确认对应的HBase版本,比如2.1.x是否兼容。另外,集群部署时,两者的JAR包需要正确放置,比如将Phoenix的clientjar添加到HBase的类路径中,这样HBase的API才能正确解析Phoenix的数据结构。 用户可能需要具体的步骤,比如如何配置环境,如何通过HBase Shell或Java API访问数据。例如,在HBase Shell中使用scan命令查看Phoenix创建的表,或者用Java API读取数据。需要注意Phoenix的表在HBase中的存储方式,比如命名空间可能为SYSTEM或用户自定义,列族通常是0等。 示例代码部分,用户可能需要JavaAPI的示例,展示如何连接HBase并读取Phoenix表的数据。需要确保代码中正确指定表名、列族列名,并处理可能的版本问题。同时,需要提醒用户注意依赖的版本匹配,避免出现类冲突或方法不兼容的情况。 另外,用户引用中提到Bulk LoadSpark集成,可能用户有大数据量处理的需求,可以建议使用BulkLoad方式将数据导入HBase,再通过Phoenix查询,或者反之。但当前问题集中在HBase访问Phoenix数据,所以需要专注于直接的访问方法。最后,需要验证步骤,比如通过HBaseShell列出表,查看数据是否存在,或者编写简单的Java程序读取数据。同时,常见问题如类路径错误、表不存在或权限问题也需要提及,帮助用户排查。</think>### HBase访问Phoenix数据的配置与实现方法 HBase与Apache Phoenix的集成主要通过**共享底层存储**实现,Phoenix的表数据实际存储在HBase中。以下是具体配置步骤示例代码: --- #### 一、环境配置要点 1. **版本匹配** PhoenixHBase需版本兼容(如Phoenix 5.1.2对应HBase 2.1.x)[^2]。 2. **部署Phoenix客户端库** 将Phoenix的`phoenix-client-hbase-2.1-5.1.2.jar`复制到HBase的`lib`目录: ```bash cp /path/to/phoenix-client.jar $HBASE_HOME/lib/ ``` 3. **重启HBase集群** 确保所有节点加载Phoenix依赖库。 --- #### 二、通过HBase Shell访问Phoenix表 1. **查看Phoenix创建的表** Phoenix表默认存储在`SYSTEM`命名空间或用户自定义命名空间: ```bash hbase> list_namespace_tables 'SYSTEM' # 查看系统表 hbase> scan 'SYSTEM:STATS' # 示例:扫描Phoenix元数据表 ``` 2. **访问用户表** 假设Phoenix中创建了表`MY_TABLE`: ```sql CREATE TABLE MY_TABLE (ID INTEGER PRIMARY KEY, NAME VARCHAR); ``` 在HBase Shell中对应的操作为: ```bash hbase> scan 'MY_TABLE' # 直接扫描表 ``` --- #### 三、Java API访问示例 ```java import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.client.Connection; import org.apache.hadoop.hbase.client.ConnectionFactory; import org.apache.hadoop.hbase.client.Get; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.Table; public class HBasePhoenixAccess { public static void main(String[] args) throws Exception { Configuration config = HBaseConfiguration.create(); config.set("hbase.zookeeper.quorum", "hadoop001:2181"); try (Connection conn = ConnectionFactory.createConnection(config)) { Table table = conn.getTable(TableName.valueOf("MY_TABLE")); Get get = new Get(Bytes.toBytes("rowkey1")); // Phoenix的ROWKEY为二进制格式 Result result = table.get(get); // 解析列值(Phoenix列名转换为HBase列限定符) byte[] nameValue = result.getValue( Bytes.toBytes("0"), // Phoenix默认列族为0 Bytes.toBytes("NAME") // 对应Phoenix列名 ); System.out.println("NAME: " + Bytes.toString(nameValue)); } } } ``` --- #### 四、关键注意事项 1. **列名与列族映射** Phoenix的列默认映射到HBase的列族`0`,列名转换为全大写[^3]。 2. **数据类型兼容性** Phoenix的`VARCHAR`在HBase中存储为`STRING`类型,需确保编解码一致。 3. **命名空间同步** 若Phoenix表创建在自定义命名空间(如`MY_SCHEMA`),HBase中需通过`MY_SCHEMA:TABLE_NAME`访问。 ---
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值