Warning: $HADOOP_HOME is deprecated解决方法

本文介绍了启动Hadoop1.2.1时出现的警告信息及其原因,并提供了两种解决方案:一是注释掉配置文件中的相关代码;二是通过设置环境变量来抑制警告。建议采用第二种方案。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

启动Hadoop时报了一个警告信息,我的系统是CentOS,安装的Hadoop版本是hadoop1.2.1,具体警告信息如下:

[root@m1 hadoop-1.2.1]# bin/start-all.sh 
Warning: $HADOOP_HOME is deprecated.

网上的说法是因为Hadoop本身对HADOOP_HOME做了判断,具体在bin/hadoop和bin/hadoop-config.sh里。在hadoop-config.sh里有如下的配置:

if [ "$HADOOP_HOME_WARN_SUPPRESS" = "" ] && [ "$HADOOP_HOME" != "" ]; then
  echo "Warning: \$HADOOP_HOME is deprecated." 1>&2
  echo 1>&2
fi

对于这个警告问题,解决方法如下:
1. 注释掉hadoop-config.sh里的上面给出的这段if fi配置(不推荐
2. 在当前用户目录下的 .bash_profile 里增加一个环境变量:
export HADOOP_HOME_WARN_SUPPRESS=1
注:修改完 .bash_profile 后需要执行 source 操作使其生效
用到的命令:

cd ~
vi .bash_profile //使用Vi命令修改.bash_profile
source .bash_profile //修改完.bash_profile后需要执行source操作使其生效

转载于:https://www.cnblogs.com/wuyida/p/6300490.html

[root@master script]# sqoop export \ > --connect jdbc:mysql://192.168.196.130:3306/mydatabase \ > --username root \ > --password 123456 \ > --table student_count \ > --export-dir /user/hadoop/output/preprocessed \ > --input-fields-terminated-by ',' \ > --input-lines-terminated-by '\n' \ > --m 1 Warning: /usr/local/src/sqoop/../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /usr/local/src/sqoop/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/local/src/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2025-03-22 22:55:17,097 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 2025-03-22 22:55:17,178 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-03-22 22:55:17,274 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-03-22 22:55:17,278 INFO tool.CodeGenTool: Beginning code generation Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2025-03-22 22:55:17,832 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_count` AS t LIMIT 1 2025-03-22 22:55:17,860 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_count` AS t LIMIT 1 2025-03-22 22:55:17,865 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/src/hadoop 注: /tmp/sqoop-root/compile/4440f43d1ce107ad5106d71abed5c63e/student_count.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 2025-03-22 22:55:18,993 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/4440f43d1ce107ad5106d71abed5c63e/student_count.jar 2025-03-22 2
03-24
[root@localhost sqoop-1.4.7.bin__hadoop-2.6.0]# bin/sqoop export --connect jdbc:mysql://localhost:3306/dbtaobao --username root --password 123456 --table user_log --export-dir '/user/hive/warehouse/dbtaobao.db/inner_user_log' --fields-terminated-by ','; Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. Warning: /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. 2025-06-06 00:44:19,902 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 2025-06-06 00:44:20,015 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 2025-06-06 00:44:20,281 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2025-06-06 00:44:20,287 INFO tool.CodeGenTool: Beginning code generation Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2025-06-06 00:44:22,137 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_log` AS t LIMIT 1 2025-06-06 00:44:22,270 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_log` AS t LIMIT 1 2025-06-06 00:44:22,293 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hadoop/hadoop-3.1.3 注: /tmp/sqoop-root/compile/2647ebe9a3777fcaa95ca65a919294ec/user_log.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 2025-06-06 00:44:26,610 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/2647ebe9a3777fcaa95ca65a919294ec/user_log.jar 2025-06-06 00:44:26,629 INFO mapreduce.ExportJobBase: Beginning export of user_log 2025-06-06 00:44:26,629 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-06-06 00:44:27,018 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-06-06 00:44:28,689 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:28,961 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2025-06-06 00:44:28,966 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-06-06 00:44:28,966 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2025-06-06 00:44:29,494 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2025-06-06 00:44:29,762 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2025-06-06 00:44:29,762 INFO impl.MetricsSystemImpl: JobTracker metrics system started 2025-06-06 00:44:30,031 INFO input.FileInputFormat: Total input files to process : 1 2025-06-06 00:44:30,053 INFO input.FileInputFormat: Total input files to process : 1 2025-06-06 00:44:30,136 INFO mapreduce.JobSubmitter: number of splits:4 2025-06-06 00:44:30,284 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 2025-06-06 00:44:30,660 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local583218486_0001 2025-06-06 00:44:30,660 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2025-06-06 00:44:31,271 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/local/1749141870870/libjars <- /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/* 2025-06-06 00:44:31,278 WARN fs.FileUtil: Command 'ln -s /tmp/hadoop-root/mapred/local/1749141870870/libjars /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/*' failed 1 with: ln: 无法创建符号链接"/usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/*": 没有那个文件或目录 2025-06-06 00:44:31,278 WARN mapred.LocalDistributedCacheManager: Failed to create symlink: /tmp/hadoop-root/mapred/local/1749141870870/libjars <- /usr/sqoop/sqoop-1.4.7.bin__hadoop-2.6.0/libjars/* 2025-06-06 00:44:31,278 INFO mapred.LocalDistributedCacheManager: Localized file:/tmp/hadoop/mapred/staging/root583218486/.staging/job_local583218486_0001/libjars as file:/tmp/hadoop-root/mapred/local/1749141870870/libjars 2025-06-06 00:44:31,569 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 2025-06-06 00:44:31,571 INFO mapreduce.Job: Running job: job_local583218486_0001 2025-06-06 00:44:31,614 INFO mapred.LocalJobRunner: OutputCommitter set in config null 2025-06-06 00:44:31,631 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter 2025-06-06 00:44:31,798 INFO mapred.LocalJobRunner: Waiting for map tasks 2025-06-06 00:44:31,802 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000000_0 2025-06-06 00:44:32,000 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,006 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:355065+59179,/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:414244+59179 2025-06-06 00:44:32,012 INFO Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file 2025-06-06 00:44:32,012 INFO Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start 2025-06-06 00:44:32,012 INFO Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length 2025-06-06 00:44:32,050 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,260 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,299 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:32,328 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000001_0 2025-06-06 00:44:32,357 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,359 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:0+118355 2025-06-06 00:44:32,387 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,524 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:32,553 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000002_0 2025-06-06 00:44:32,566 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,567 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:118355+118355 2025-06-06 00:44:32,585 INFO mapreduce.Job: Job job_local583218486_0001 running in uber mode : false 2025-06-06 00:44:32,587 INFO mapreduce.Job: map 0% reduce 0% 2025-06-06 00:44:32,616 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,745 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:32,799 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:32,866 INFO mapred.LocalJobRunner: Starting task: attempt_local583218486_0001_m_000003_0 2025-06-06 00:44:32,901 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2025-06-06 00:44:32,903 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dbtaobao.db/inner_user_log/000000_0:236710+118355 2025-06-06 00:44:32,930 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:33,004 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2025-06-06 00:44:33,034 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 2025-06-06 00:44:33,049 INFO mapred.LocalJobRunner: map task executor complete. 2025-06-06 00:44:33,050 WARN mapred.LocalJobRunner: job_local583218486_0001 java.lang.Exception: java.io.IOException: java.lang.ClassNotFoundException: user_log at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) Caused by: java.io.IOException: java.lang.ClassNotFoundException: user_log at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:74) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: user_log at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:70) ... 10 more 2025-06-06 00:44:33,590 INFO mapreduce.Job: Job job_local583218486_0001 failed with state FAILED due to: NA 2025-06-06 00:44:33,599 INFO mapreduce.Job: Counters: 0 2025-06-06 00:44:33,623 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2025-06-06 00:44:33,625 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 4.6274 seconds (0 bytes/sec) 2025-06-06 00:44:33,626 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2025-06-06 00:44:33,626 INFO mapreduce.ExportJobBase: Exported 0 records. 2025-06-06 00:44:33,626 ERROR mapreduce.ExportJobBase: Export job failed! 2025-06-06 00:44:33,626 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) [root@localhost sqoop-1.4.7.bin__hadoop-2.6.0]# 给我解决方案
06-07
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值