一、安装Hadoop
安装hadoop参考此文,关键点是安装JDK和Hadoop的配置,为避免引用文章变收费,我把关键信息摘录如下:
jdk安装和配置就不说了(我本机安装了1.8/15/17/21,以17为主),hadoop安装过程:
1.设置SSH免密
1. 打开系统偏好设置,在输入框输入sharing(共享),勾选”远程连接“:
2.打开终端,依次输入如下命令:
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/id_rsa.pub
3.输入【ssh localhost】验证,无需输入密码则成功
2.下载并解压hadoop安装包
到官网下载需要的版本,我下载的是3.3.6,然后解压到用户下自己用户名的目录下:
3.修改配置文件
(1)vim ~/.zprofile(路径修改为自己安装包的路径)
# Hadoop
export HADOOP_HOME=/Users/wangxiaoxuan/hadoop-3.3.6/
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/nativ"
输入【source ~/.zprofile】使内容生效
(2)cd /Users/wangxiaoxuan/hadoop-3.3.6/etc/hadoop
vim hadoop-env.sh
export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk-1.8.jdk/Contents/Home"
(3)vim core-site.xml,将标签<configuration></configuration>中内容替换如下,路径需换成自己的路径