Installation. Hbase-0.98.7-hadoop2 SetUp

本文指导如何顺利安装并配置Hadoop和HBase,包括选择合适的Hadoop版本、下载与解压HBase包、调整环境变量和配置文件、复制至从机以及启动HBase服务,最终通过网页验证集群状态。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

裝完hadoop后搭建hbase就方便好多了~


去官网下载 hbase-0.98.7-hadoop2-bin.tar.gz 到写这篇文章的时候, 应该是stable版的最新

这里要插播一下, 为什么回选择hadoop-2.2.0 

而不是hadoop-2.4.1甚至hadoop-2.5.x

原因是这样的: 其实hbase-0.98.7里面的很多相关的jar包默认都是使用hadoop-2.2.0的, 所以真的搭建运行起来 还是配合着比较好, 我是这样想的.

而且我看有些安装教程, 有些步骤是要将hadoop的jar包替换掉hbase的jar包, 因为hbase的更新相对于hadoop要慢一点, 所以jar包也回旧一些


好, 直接来步骤.

1. 还是要将下载好的hbase的.tar.gz移动到/opt/目录下, 然后执行:

  • cd /opt
  • sudo tar -zxvf hbase-0.98.7-hadoop2-bin.tar.gz 
PS: 如果在搭建hadoop的时候要专门新增了使用者来管理hadoop的话, 记得要更改hbase的使用者, 详情以下网站:
http://blog.youkuaiyun.com/chhj0103121429/article/details/40711371
http://darbylu.blogspot.tw/

2. 到hbase目录下修改环境文件和设定各种参数:
  • cd /opt/hbase-0.98.7-hadoop2/conf
  • sudo nano hbase-env.sh
在最后添加以下几行(要改成你机子对应的实际情况!)
export JAVA_HOME=/usr/lib/jvm/java-7-oracle
export HADOOP_HOME=/opt/hadoop-2.2.0
export HBASE_HOME=/opt/hbase-0.98.7-hadoop2
export HBASE_LOG_DIR=/opt/hbase-0.98.7-hadoop2/log
export HBASE_PID_DIR=/opt/hbase-0.98.7-hadoop2/pid
export HBASE_MANAGES_ZK=true

修改hbase-site.xml, 其中(hbase.rootdir, hbase.zookeeper.quorum的value要改成你对应的主机和从机名)
  • sudo nano hbase-site.xml //  http://hbase.apache.org/book/config.files.html  这个网页有默认设置, 及告诉你该设置有何用处
<configuration>
   <property>
        <name>hbase.rootdir</name>
        <value>hdfs://master:9000/hbase</value>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>master,slave1,slave2</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2222</value>
    </property>
    <property>
        <name>hbase.tmp.dir</name>
        <value>/tmp/hbase-${user.name}</value>
    </property>
</configuration>

设置regionservers
  • sudo nano regionservers
slave1
slave2

3. 将hbase这个目录下所有子目录及其文件复制到其它从机上
  • ssh slave1
  • cd /opt
  • sudo mkdir hbase-0.98.7-hadoop2
  • exit
  • scp -r /opt/hbase-0.98.7-hadoop2/* slave1:/opt/hbase-0.98.7-hadoop2
重复操作, 确保复制到各从机上.

4. 回到主机上, 进入hbase目录下:
  • cd /opt/hbase-0.98.7-hadoop2
  • bin/start-hbase.sh

5. 验证及查看, 输入jps(红色部分是新增的, 其余是已经打开的hadoop的进程):
  • NameNode
  • HQuorumPeer
  • ResourceManager
  • HMaster
  • SecondaryNameNode
  • Jps
在从机上输入jps:
  • Jps
  • NodeManager
  • DataNode
  • HRegionServer
  • HQuorumPeer
在网页上输入: http://master:60010/ 可以通过网页看到各节点情况~!
安装完咯. 工作环境搭建好了~~!!!

[root@localhost sqoop-1.4.7.bin__hadoop-2.6.0]# bin/sqoop export --connect jdbc:mysql://localhost:3306/dbtaobao --username root --password 123456 --table user_log --export-dir '/user/hive/warehouse/dbtaobao.db/inner_user_log' --fields-terminated-by ','; Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 2025-06-06 00:44:19,902 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 2025-06-06 00:44:20,015 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-06-06 00:44:20,281 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-06-06 00:44:20,287 INFO tool.CodeGenTool: Beginning code generation Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2025-06-06 00:44:22,137 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_log` AS t LIMIT 1 2025-06-06 00:44:22,270 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_log` AS t LIMIT 1 2025-06-06 00:44:22,293 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hadoop/hadoop-3.1.3 注: /tmp/sqoop-root/compile/2647ebe9a3777fcaa95ca65a919294ec/user_log.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 2025-06-06 00:44:26,610 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/2647ebe9a3777fcaa95ca65a919294ec/user_log.jar 2025-06-06 00:44:26,629 INFO mapreduce.ExportJobBase: Beginning export of user_log 2025-06-06 00:44:26,629 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-06-06 00:44:27,018 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-06-06 00:44:28,689 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:28,961 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2025-06-06 00:44:28,966 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-06-06 00:44:28,966 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2025-06-06 00:44:29,494 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2025-06-06 00:44:29,762 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-06-06 00:44:29,762 INFO impl.MetricsSystemImpl: JobTracker metrics system started 2025-06-06 00:44:30,031 INFO input.FileInputFormat: Total input files to process : 1 2025-06-06 00:44:30,053 INFO input.FileInputFormat: Total input files to process : 1 2025-06-06 00:44:30,136 INFO mapreduce.JobSubmitter: number of splits:4 2025-06-06 00:44:30,284 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-06-06 00:44:30,660 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local583218486_0001 2025-06-06 00:44:30,660 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2025-06-06 00:44:31,271 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/local/1749141870870/libjars <- /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/* 2025-06-06 00:44:31,278 WARN fs.FileUtil: Command 'ln -s /tmp/hadoop-root/mapred/local/1749141870870/libjars /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/*' failed 1 with: ln: 无法创建符号链接"/usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/*": 没有那个文件或目录 2025-06-06 00:44:31,278 WARN mapred.LocalDistributedCacheManager: Failed to create symlink: /tmp/hadoop-root/mapred/local/1749141870870/libjars <- /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/* 2025-06-06 00:44:31,278 INFO mapred.LocalDistributedCacheManager: Localized file:/tmp/hadoop/mapred/staging/root583218486/.staging/job_local583218486_0001/libjars as file:/tmp/hadoop-root/mapred/local/1749141870870/libjars 2025-06-06 00:44:31,569 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 2025-06-06 00:44:31,571 INFO mapreduce.Job: Running job: job_local583218486_0001 2025-06-06 00:44:31,614 INFO mapred.LocalJobRunner: OutputCommitter set in config null 2025-06-06 00:44:31,631 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter 2025-06-06 00:44:31,798 INFO mapred.LocalJobRunner: Waiting for map tasks 2025-06-06 00:44:31,802 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000000_0 2025-06-06 00:44:32,000 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,006 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:355065+59179,/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:414244+59179 2025-06-06 00:44:32,012 INFO Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file 2025-06-06 00:44:32,012 INFO Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start 2025-06-06 00:44:32,012 INFO Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length 2025-06-06 00:44:32,050 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,260 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,299 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:32,328 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000001_0 2025-06-06 00:44:32,357 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,359 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:0+118355 2025-06-06 00:44:32,387 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,524 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:32,553 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000002_0 2025-06-06 00:44:32,566 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,567 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:118355+118355 2025-06-06 00:44:32,585 INFO mapreduce.Job: Job job_local583218486_0001 running in uber mode : false 2025-06-06 00:44:32,587 INFO mapreduce.Job: map 0% reduce 0% 2025-06-06 00:44:32,616 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,745 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,799 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:32,866 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000003_0 2025-06-06 00:44:32,901 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,903 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:236710+118355 2025-06-06 00:44:32,930 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:33,004 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:33,034 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:33,049 INFO mapred.LocalJobRunner: map task executor complete. 2025-06-06 00:44:33,050 WARN mapred.LocalJobRunner: job_local583218486_0001 java.lang.Exception: java.io.IOException: java.lang.ClassNotFoundException: user_log at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) Caused by: java.io.IOException: java.lang.ClassNotFoundException: user_log at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:74) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: user_log at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:70) ... 10 more 2025-06-06 00:44:33,590 INFO mapreduce.Job: Job job_local583218486_0001 failed with state FAILED due to: NA 2025-06-06 00:44:33,599 INFO mapreduce.Job: Counters: 0 2025-06-06 00:44:33,623 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2025-06-06 00:44:33,625 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 4.6274 seconds (0 bytes/sec) 2025-06-06 00:44:33,626 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2025-06-06 00:44:33,626 INFO mapreduce.ExportJobBase: Exported 0 records. 2025-06-06 00:44:33,626 ERROR mapreduce.ExportJobBase: Export job failed! 2025-06-06 00:44:33,626 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) [root@localhost sqoop-1.4.7.bin__hadoop-2.6.0]# 给我解决方案
06-07
从hdfs到mysql报错Warning: /usr/local/sqoop/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/local/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /usr/local/sqoop/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 错误: 找不到或无法加载主类 org.apache.hadoop.hbase.util.GetJavaProperty 2025-06-05 17:41:55,556 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6 2025-06-05 17:41:55,583 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-06-05 17:41:55,664 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-06-05 17:41:55,664 INFO tool.CodeGenTool: Using existing jar: /tmp/sqoop-hadoop/compile/8b77c7c2bee6c4dd7f3856618f5f8556/user_action.jar 2025-06-05 17:41:55,668 INFO mapreduce.ExportJobBase: Beginning export of user_action 2025-06-05 17:41:55,668 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-06-05 17:41:55,827 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-06-05 17:41:56,366 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Thu Jun 05 17:41:56 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-06-05 17:41:56,900 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2025-06-05 17:41:56,902 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-06-05 17:41:56,904 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2025-06-05 17:41:56,967 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2025-06-05 17:41:57,035 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-06-05 17:41:57,035 INFO impl.MetricsSystemImpl: JobTracker metrics system started 2025-06-05 17:41:57,104 INFO input.FileInputFormat: Total input files to process : 1 2025-06-05 17:41:57,111 INFO input.FileInputFormat: Total input files to process : 1 2025-06-05 17:41:57,139 INFO mapreduce.JobSubmitter: number of splits:4 2025-06-05 17:41:57,159 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-06-05 17:41:57,236 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1705881514_0001 2025-06-05 17:41:57,236 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2025-06-05 17:41:57,374 INFO mapred.LocalDistributedCacheManager: Creating symlink: /usr/local/hadoop/tmp/mapred/local/1749116517294/libjars <- /usr/local/sqoop/libjars/* 2025-06-05 17:41:57,376 INFO mapred.LocalDistributedCacheManager: Localized file:/tmp/hadoop/mapred/staging/hadoop1705881514/.staging/job_local1705881514_0001/libjars as file:/usr/local/hadoop/tmp/mapred/local/1749116517294/libjars 2025-06-05 17:41:57,408 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 2025-06-05 17:41:57,409 INFO mapreduce.Job: Running job: job_local1705881514_0001 2025-06-05 17:41:57,413 INFO mapred.LocalJobRunner: OutputCommitter set in config null 2025-06-05 17:41:57,416 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter 2025-06-05 17:41:57,455 INFO mapred.LocalJobRunner: Waiting for map tasks 2025-06-05 17:41:57,456 INFO mapred.LocalJobRunner: Starting task: attempt_local1705881514_0001_m_000000_0 2025-06-05 17:41:57,492 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-05 17:41:57,499 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:11692413+1948736,/user/hive/warehouse/dblab.db/user_action/000000_0:13641149+1948737 2025-06-05 17:41:57,507 INFO Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file 2025-06-05 17:41:57,507 INFO Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start 2025-06-05 17:41:57,507 INFO Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length 2025-06-05 17:41:57,515 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Thu Jun 05 17:41:57 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-06-05 17:41:57,543 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-05 17:41:57,548 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-05 17:41:57,551 INFO mapred.LocalJobRunner: Starting task: attempt_local1705881514_0001_m_000001_0 2025-06-05 17:41:57,552 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-05 17:41:57,553 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:0+3897471 2025-06-05 17:41:57,562 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Thu Jun 05 17:41:57 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-06-05 17:41:57,574 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-05 17:41:57,577 INFO mapred.LocalJobRunner: Starting task: attempt_local1705881514_0001_m_000002_0 2025-06-05 17:41:57,578 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-05 17:41:57,579 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:3897471+3897471 2025-06-05 17:41:57,583 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Thu Jun 05 17:41:57 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-06-05 17:41:57,603 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-05 17:41:57,607 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-05 17:41:57,614 INFO mapred.LocalJobRunner: Starting task: attempt_local1705881514_0001_m_000003_0 2025-06-05 17:41:57,616 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-05 17:41:57,617 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:7794942+3897471 2025-06-05 17:41:57,623 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Thu Jun 05 17:41:57 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-06-05 17:41:57,638 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-05 17:41:57,640 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-05 17:41:57,641 INFO mapred.LocalJobRunner: map task executor complete. 2025-06-05 17:41:57,642 WARN mapred.LocalJobRunner: job_local1705881514_0001 java.lang.Exception: java.io.IOException: java.lang.ClassNotFoundException: user_action at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) Caused by: java.io.IOException: java.lang.ClassNotFoundException: user_action at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:70) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: user_action at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:66) ... 10 more 2025-06-05 17:41:58,413 INFO mapreduce.Job: Job job_local1705881514_0001 running in uber mode : false 2025-06-05 17:41:58,414 INFO mapreduce.Job: map 0% reduce 0% 2025-06-05 17:41:58,416 INFO mapreduce.Job: Job job_local1705881514_0001 failed with state FAILED due to: NA 2025-06-05 17:41:58,422 INFO mapreduce.Job: Counters: 0 2025-06-05 17:41:58,432 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2025-06-05 17:41:58,434 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 1.5124 seconds (0 bytes/sec) 2025-06-05 17:41:58,436 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2025-06-05 17:41:58,436 INFO mapreduce.ExportJobBase: Exported 0 records. 2025-06-05 17:41:58,437 ERROR tool.ExportTool: Error during export: Export job failed!
06-06
数据从 hive 复制到 mysql 中报错hadoop@t3l-VirtualBox:/usr/local/sqoop$ ./bin/sqoop export --connect jdbc:mysql://localhost:3306/dblab --username root --password '*-+' --table user_action --export-dir '/user/hive/warehouse/dblab.db/user_action' --fields-terminated-by '\t'; Warning: /usr/local/sqoop/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/local/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /usr/local/sqoop/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 错误: 找不到或无法加载主类 org.apache.hadoop.hbase.util.GetJavaProperty 2025-08-08 16:00:57,770 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6 2025-08-08 16:00:57,835 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-08-08 16:00:58,042 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-08-08 16:00:58,042 INFO tool.CodeGenTool: Beginning code generation Fri Aug 08 16:00:58 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-08-08 16:00:59,101 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_action` AS t LIMIT 1 2025-08-08 16:00:59,171 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_action` AS t LIMIT 1 2025-08-08 16:00:59,210 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop 注: /tmp/sqoop-hadoop/compile/e5381a60ac3daedca9188e7ac7d80584/user_action.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 2025-08-08 16:01:01,847 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/e5381a60ac3daedca9188e7ac7d80584/user_action.jar 2025-08-08 16:01:01,853 INFO mapreduce.ExportJobBase: Beginning export of user_action 2025-08-08 16:01:01,854 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-08-08 16:01:02,095 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-08-08 16:01:03,097 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-08-08 16:01:03,334 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2025-08-08 16:01:03,337 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-08-08 16:01:03,342 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2025-08-08 16:01:03,692 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2025-08-08 16:01:03,859 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-08-08 16:01:03,859 INFO impl.MetricsSystemImpl: JobTracker metrics system started 2025-08-08 16:01:04,058 INFO input.FileInputFormat: Total input files to process : 1 2025-08-08 16:01:04,065 INFO input.FileInputFormat: Total input files to process : 1 2025-08-08 16:01:04,168 INFO mapreduce.JobSubmitter: number of splits:4 2025-08-08 16:01:04,275 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-08-08 16:01:04,481 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1968709802_0001 2025-08-08 16:01:04,485 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2025-08-08 16:01:04,820 INFO mapred.LocalDistributedCacheManager: Creating symlink: /usr/local/hadoop/tmp/mapred/local/1754640064639/libjars <- /usr/local/sqoop/libjars/* 2025-08-08 16:01:04,827 WARN fs.FileUtil: Command 'ln -s /usr/local/hadoop/tmp/mapred/local/1754640064639/libjars /usr/local/sqoop/libjars/*' failed 1 with: ln: 无法创建符号链接'/usr/local/sqoop/libjars/*': 没有那个文件或目录 2025-08-08 16:01:04,827 WARN mapred.LocalDistributedCacheManager: Failed to create symlink: /usr/local/hadoop/tmp/mapred/local/1754640064639/libjars <- /usr/local/sqoop/libjars/* 2025-08-08 16:01:04,827 INFO mapred.LocalDistributedCacheManager: Localized file:/tmp/hadoop/mapred/staging/hadoop1968709802/.staging/job_local1968709802_0001/libjars as file:/usr/local/hadoop/tmp/mapred/local/1754640064639/libjars 2025-08-08 16:01:04,958 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 2025-08-08 16:01:04,959 INFO mapreduce.Job: Running job: job_local1968709802_0001 2025-08-08 16:01:04,962 INFO mapred.LocalJobRunner: OutputCommitter set in config null 2025-08-08 16:01:04,990 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter 2025-08-08 16:01:05,123 INFO mapred.LocalJobRunner: Waiting for map tasks 2025-08-08 16:01:05,130 INFO mapred.LocalJobRunner: Starting task: attempt_local1968709802_0001_m_000000_0 2025-08-08 16:01:05,245 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-08-08 16:01:05,253 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:0+3897281 2025-08-08 16:01:05,259 INFO Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file 2025-08-08 16:01:05,259 INFO Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start 2025-08-08 16:01:05,259 INFO Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length 2025-08-08 16:01:05,277 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Fri Aug 08 16:01:05 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-08-08 16:01:05,392 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-08-08 16:01:05,396 INFO mapred.LocalJobRunner: Starting task: attempt_local1968709802_0001_m_000001_0 2025-08-08 16:01:05,398 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-08-08 16:01:05,398 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:3897281+3897281 2025-08-08 16:01:05,422 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Fri Aug 08 16:01:05 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-08-08 16:01:05,509 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-08-08 16:01:05,515 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-08-08 16:01:05,523 INFO mapred.LocalJobRunner: Starting task: attempt_local1968709802_0001_m_000002_0 2025-08-08 16:01:05,527 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-08-08 16:01:05,544 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:7794562+3897281 2025-08-08 16:01:05,557 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Fri Aug 08 16:01:05 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-08-08 16:01:05,635 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-08-08 16:01:05,648 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-08-08 16:01:05,652 INFO mapred.LocalJobRunner: Starting task: attempt_local1968709802_0001_m_000003_0 2025-08-08 16:01:05,655 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-08-08 16:01:05,674 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:11691843+3897281 2025-08-08 16:01:05,697 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false Fri Aug 08 16:01:05 CST 2025 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2025-08-08 16:01:05,768 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-08-08 16:01:05,782 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-08-08 16:01:05,790 INFO mapred.LocalJobRunner: map task executor complete. 2025-08-08 16:01:05,792 WARN mapred.LocalJobRunner: job_local1968709802_0001 java.lang.Exception: java.io.IOException: java.lang.ClassNotFoundException: user_action at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) Caused by: java.io.IOException: java.lang.ClassNotFoundException: user_action at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:70) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: user_action at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:66) ... 10 more 2025-08-08 16:01:05,962 INFO mapreduce.Job: Job job_local1968709802_0001 running in uber mode : false 2025-08-08 16:01:05,965 INFO mapreduce.Job: map 0% reduce 0% 2025-08-08 16:01:05,967 INFO mapreduce.Job: Job job_local1968709802_0001 failed with state FAILED due to: NA 2025-08-08 16:01:05,973 INFO mapreduce.Job: Counters: 0 2025-08-08 16:01:06,018 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2025-08-08 16:01:06,022 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 2.6143 seconds (0 bytes/sec) 2025-08-08 16:01:06,036 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2025-08-08 16:01:06,036 INFO mapreduce.ExportJobBase: Exported 0 records. 2025-08-08 16:01:06,036 ERROR tool.ExportTool: Error during export: Export job failed!
08-09
java.lang.Exception: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file hdfs://192.168.8.201:8020/apps/hbase/data/data/gt_dw/profile_gid_lbs_locvalue/7ef0422f73082b2d140d755a08ab6904/lbs/75c83b238e0b4be496eecf33eed5e5c3     at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) ~[hadoop-mapreduce-client-common-2.7.2.jar:na]     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) ~[hadoop-mapreduce-client-common-2.7.2.jar:na] Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file hdfs://192.168.8.201:8020/apps/hbase/data/data/gt_dw/profile_gid_lbs_locvalue/7ef0422f73082b2d140d755a08ab6904/lbs/75c83b238e0b4be496eecf33eed5e5c3     at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:463) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:506) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at com.glab.fz.etl.hfile.util.HFileInputFormat$HFileRecordReader.initialize(HFileInputFormat.java:60) ~[classes/:na]     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548) ~[hadoop-mapreduce-client-core-2.7.2.jar:na]     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786) ~[hadoop-mapreduce-client-core-2.7.2.jar:na]     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) ~[hadoop-mapreduce-client-core-2.7.2.jar:na]     at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) ~[hadoop-mapreduce-client-common-2.7.2.jar:na]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_431]     at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_431]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_431]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_431]     at java.lang.Thread.run(Thread.java:750) ~[na:1.8.0_431] Caused by: java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.     at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65) ~[hadoop-common-2.7.2.jar:na]     at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:193) ~[hadoop-common-2.7.2.jar:na]     at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:178) ~[hadoop-common-2.7.2.jar:na]     at org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getDecompressor(Compression.java:327) ~[hbase-common-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.compress.Compression.decompress(Compression.java:422) ~[hbase-common-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultDecodingContext.prepareDecoding(HFileBlockDefaultDecodingContext.java:91) ~[hbase-common-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFileBlock.unpack(HFileBlock.java:507) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1255) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1261) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.<init>(HFileReaderV2.java:147) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFileReaderV3.<init>(HFileReaderV3.java:73) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]     at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:453) ~[hbase-server-0.98.13-hadoop2.jar:0.98.13-hadoop2]
06-16
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值