hadoop跑数命令

最新推荐文章于 2025-11-24 15:49:50 发布

原创最新推荐文章于 2025-11-24 15:49:50 发布 · 1.6k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#sqoop #hadoop #hive

数据仓库专栏收录该内容

23 篇文章

订阅专栏

该博客内容涉及使用Hadoop文件系统（Hadoopfs）上传SQL脚本到用户目录，并通过shell脚本（etl_comm.sh）执行YARN上的ETL作业。作业包括债券、股票、期货、基金交易详情等初始化数据处理，涉及fahs和fa45两个项目目录。作业执行的日志被重定向到app3.log文件中。

/home/ctmagent/etl_app/fahs/hql/dwd/trade/incre--hs�������ϴ�·��
/home/ctmagent/etl_app/fa45/hql/dwd/trade/incre--yss�������ϴ�·��
cd /home/ctmagent/etl_app/fahs/hql/dwd/trade/incre
cd /home/ctmagent/etl_app/fa45/hql/dwd/trade/incre
hadoop fs -put -f bond_trade_detail_init.sql /user/etl/fahs/dwd/
hadoop fs -put -f future_trade_detail_init.sql /user/etl/fahs/dwd/
hadoop fs -put -f financial_investment_trade_det_init.sql /user/etl/fahs/dwd/
hadoop fs -put -f stock_trade_detail_init.sql /user/etl/fahs/dwd/
hadoop fs -put -f fund_trade_detail_init.sql /user/etl/fahs/dwd/
hadoop fs -put -f stock_trade_detail_init.sql /user/etl/fa45/dwd/
hadoop fs -put -f fund_trade_detail_init.sql /user/etl/fa45/dwd/
hadoop fs -put -f bond_trade_detail_init.sql /user/etl/fa45/dwd/
--hadoop�ϴ�
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fahs/dwd/future_trade_detail_init.sql--ͨ��shell���ýű�
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fahs/dwd/stock_trade_detail_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fahs/dwd/bond_trade_detail_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fahs/dwd/future_trade_detail_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fahs/dwd/fund_trade_detail_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fahs/dwd/financial_investment_trade_det_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fa45/dwd/stock_trade_detail_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fa45/dwd/fund_trade_detail_init.sql
sh ~/etl_app/comm/bin/etl_comm.sh /user/etl/fa45/dwd/bond_trade_detail_init.sql
yarn logs -applicationId application_1633244180597_12530 >./app3.log