Ubuntu18.04安装伪分布式Hadoop3.3.1

最新推荐文章于 2022-11-19 22:55:12 发布

原创

最新推荐文章于 2022-11-19 22:55:12 发布 · 887 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#分布式 #linux #hadoop

这篇教程详细介绍了如何在Ubuntu 18.04系统中为用户hadoop创建新账户，安装升级必要的软件如vim和ssh，并设置无密码登录。接着，教程指导安装Java 8，配置环境变量，并通过wget下载Hadoop 3.3.1，解压并更改文件所有权。然后，教程详细讲述了配置Hadoop伪分布式模式的步骤，包括修改多个配置文件如core-site.xml、hdfs-site.xml等，并启动HDFS和YARN服务。最后，验证服务启动成功并通过运行MapReduce示例任务来测试配置。

Hadoop3.3.1 Pseudo Distributed Mode on Ubuntu 18.04

The operating system in this tutorial is Ubuntu18.04.6. The steps to install Ubuntu18.04 is omitted.

1. Create user hadoop

Open the terminal and type in command below to create new user:

sudo useradd -m hadoop -s /bin/bash

This command creates a log-in user hadoop and uses /bin/bash as shell.

Set up password for user hadoop:

sudo passwd hadoop

Give sudo permission to user hadoop:

sudo adduser hadoop sudo

Switch Linux login user (via Ubuntu UI) to hadoop to process steps below.

upgrade apt

sudo apt-get update

install vim

sudo apt-get install vim

install ssh, set up ssh none-key login

sudo apt-get install openssh-server

ssh localhost

exit localhost

exit

authorize the key

cd ~/.ssh/
ssh-keygen -t rsa
cat ./id_rsa.pub >> ./authorized_keys

2. Install Java

sudo apt-get install openjdk-8-jre openjdk-8-jdk

change environment variables

cd ~
vim ~/.bashrc

add details below to it

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
</