sqoop导hbase zookeeper出现的问题

最新推荐文章于 2022-09-08 11:03:13 发布

tttjjjlll

最新推荐文章于 2022-09-08 11:03:13 发布

阅读量868

点赞数

CC 4.0 BY-SA版权

分类专栏： sqoop，hbase 文章标签： hbase sqoop

本文链接：https://blog.youkuaiyun.com/tttjjjlll/article/details/84487966

sqoop，hbase 专栏收录该内容

1 篇文章

订阅专栏

本文记录了在测试环境中使用SQoop将数据从Oracle导入HBase时遇到的连接拒绝错误。该问题源于RegionServers错误地向Master报告其IP地址为127.0.0.1。通过正确配置/etc/hosts文件解决了此问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

由于测试环境的改变，使用sqoop从oracle往hbase导数时一直出现下列错误：


13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/dw/hadoop-2.0.0-cdh4.3.0/lib/native
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-358.el6.x86_64
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hadoop
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/10/12 10:16:54 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 7576@master1
13/10/12 10:16:54 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:54 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:54 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/10/12 10:16:54 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
13/10/12 10:16:55 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:55 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:56 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:56 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:56 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/10/12 10:16:56 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
13/10/12 10:16:57 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:57 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:58 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:58 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:59 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:59 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

导致这个问题的原因是：Master认为Region servers的IP为127.0.0.1 即localhost;Region servers错误的告诉Master它们的IP为127.0.0.1。

解决办法：
在Region servers上修改/etc/hosts
127.0.0.1 fully.qualified.regionservername regionservername localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

修改为：
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

上述问题解决。