Apache Storm 的安装

最新推荐文章于 2025-12-31 01:33:12 发布

原创最新推荐文章于 2025-12-31 01:33:12 发布 · 552 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#apache #storm #zookeeper

BIGDATA 专栏收录该内容

35 篇文章

订阅专栏

本文档详细介绍了如何在Linux环境下安装Java、ZooKeeper及Apache Storm，并配置相关参数。包括创建必要的文件夹、配置文件，启动服务等步骤。

安装Java

> sudo apt-get install openjdk-8-jdk

安装ZooKeeper

> wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz
> tar -zxf zookeeper-3.4.9.tar.gz 
> cd zookeeper-3.4.9.tar.gz 

# Create storage folder
> sudo mkdir /var/zookeeper
> sudo chmod -R 777 /var/zookeeper

# Create config file in conf/zoo.cfg
> vim zoo.cfg
    tickTime=2000
    dataDir=/var/zookeeper
    clientPort=2181

# Running:
> ./bin/zkServer.sh start

# Connecting to ZooKeeper:
> ./bin/zkCli.sh -server 127.0.0.1:2181
> help
> ls /
> ctrl+c for exit

安装Apache Storm

> wget http://mirrors.hust.edu.cn/apache/storm/apache-storm-1.1.0/apache-storm-1.1.0.tar.gz
> tar -zxf apache-storm-1.1.0.tar.gz 
> cd apache-storm-1.1.0/

Note: The Storm release contains a file at conf/storm.yaml that configures the Storm daemons.

Configurations

storm.zookeeper.servers:
- "127.0.0.1:2181"

storm.zookeeper.port: 2181

> sudo mkdir /var/storm
> sudo chmod -R 777 /var/storm
storm.local.dir: "/var/storm"

nimbus.seeds: The worker nodes need to know which machines are the candidate of master in order to download topology jars and confs.
nimbus.seeds : ["localhost"]

supervisor.slots.ports: For each worker machine, you configure how many workers run on that machine with this config. Each worker uses a single port for receiving messages, and this setting defines which ports are open for use.
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703


The health check directory location can be configured with:
storm.health.check.dir: "/var/storm/healthchecks"
storm.health.check.timeout.ms: 2000

Running:

> ./bin/storm nimbus
> ./bin/storm supervisor
> ./bin/storm ui (ui can be access from http://{ui host}:8080)

Options:
> sudo ln -s /vagrant/apache-storm-1.1.0/bin/storm /usr/local/bin/

Use Python streamparse

streamparse http://streamparse.readthedocs.io/en/stable/

# 安装 Leiningen
> wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein
> sudo cp lein /usr/local/bin/
> sudo chmod a+x /usr/local/bin/lein
> lein
> lein version

# 链接 storm命令
> sudo ln -s /vagrant/apache-storm-1.1.0/bin/storm /usr/local/bin/
> storm version