Phoenix4.5适配CDH5.4

问题描述

Error: org.apache.hadoop.hbase.DoNotRetryIOException: SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;

at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:89)

at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1524)

at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16093)

at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7234)

at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1776)

at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1758)

at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32209)

at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2034)

at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)

at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)

at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)

at java.lang.Thread.run(Thread.java:724)

Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;

at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.buildDeletedTable(MetaDataEndpointImpl.java:1107)

at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.loadTable(MetaDataEndpointImpl.java:1211)

at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1404)

... 10 more (state=08000,code=101)

问题解决

Cloudera doesn't officially support Apache phoenix, it's still in cloudera Labs, so you cannot find any Cloudera Pheonix tar.gz files in cloudera repository, The only place where you can find Phoenix in Cloudera repository is in the parcel repository, However parcel can be used only if you install through cloudera manager, latest available version of cloudera Phoenix is 4.3.0.

If you wanted to execute Phoenix 4.4 or 4.5 version on Cloudera Hadoop distribution, you need to re-build phoenix libraries using CDH dependency jars. You cannot simply use apache Phoenix tar.gz

编译

1、Download Apache phoenix 4.5 source from Apache  repository . (Skip this step if you are downloading from the above github page)

2、Rebuild the source code using CDH dependency jars - Update pom.xml and 2 source files as follows(My CDH version is 5.4.2)

diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/pom.xml phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/pom.xml

28c28

< <!-- <module>phoenix-pig</module> -->

---

> <module>phoenix-pig</module>

37a38,41

> <id>apache release</id>

> <url>https://repository.apache.org/content/repositories/releases/</url>

> </repository>

> <repository>

42,43c46,50

< <id>cloudera</id>

< <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>

---

> <id>apache snapshot</id>

> <url>https://repository.apache.org/content/repositories/snapshots/</url>

> <snapshots>

> <enabled>true</enabled>

> </snapshots>

45d51

<

54d59

<

77,81c82,83

< <hbase.version>1.0.0-cdh5.4.2</hbase.version>

< <hadoop-two.version>2.6.0-cdh5.4.2</hadoop-two.version>

/< <hadoop.version>2.6.0-cdh5.4.2</hadoop.version>

< <pig.version>0.12.0</pig.version>

< <flume.version>1.5.0-cdh5.4.2</flume.version>

---

> <hbase.version>1.0.1</hbase.version>

> <hadoop-two.version>2.5.1</hadoop-two.version>

84a87,88

> <hadoop.version>2.5.1</hadoop.version>

> <pig.version>0.13.0</pig.version>

97a102

> <flume.version>1.4.0</flume.version>

449,450c454

<

< <dependency>

---

> <dependency>

454c458

< </dependency>

---

> </dependency>

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java

84c84

< rss.getServerName(), metaEntries,1);

---

> rss.getServerName(), metaEntries);

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java

291c291

< daughterRegions.getSecond().getRegionInfo(), server.getServerName(),1);

---

> daughterRegions.getSecond().getRegionInfo(), server.getServerName());

978c978

< }

---

> }

\ No newline at end of file

3、Above build will create new Jars under target directory of each sub component.

4、Extract phoenix-4.5.0-HBase-1.0-bin.tar.gz and replace the below Phoenix Jars with new Jars

• phoenix-4.5.0-HBase-1.0-client.jar
• phoenix-4.5.0-HBase-1.0-server-without-antlr.jar
• phoenix-4.5.0-HBase-1.0-client-minimal.jar
• phoenix-assembly-4.5.0-HBase-1.0-tests.jar
• phoenix-4.5.0-HBase-1.0-client-without-hbase.jar
• phoenix-core-4.5.0-HBase-1.0.jar
• phoenix-4.5.0-HBase-1.0-server.jar

5、Replace  phoenix-4.5.0-HBase-1.0-server.jar  and  phoenix-core-4.5.0-HBase-1.0.jar  in hbase lib location and restart hbase. (In 4.7 only phoenix-4.7.0-cdh5.X.1-server.jar to be copied to hbase lib)

6、Execute phoenix command from the new updated directory.

参考资料

https://stackoverflow.com/questions/31849454/using-phoenix-with-cloudera-hbase-installed-from-repo