Ubuntu上安装配置Strom

最新推荐文章于 2022-06-29 10:54:36 发布

原创最新推荐文章于 2022-06-29 10:54:36 发布 · 1.2k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#Strom #安装 #配置

Strom 专栏收录该内容

1 篇文章

订阅专栏

本文详细介绍了如何在Ubuntu环境下安装配置ZeroMQ、JZMQ及ZooKeeper，并指导完成Storm集群的部署，包括各节点的配置与启动。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1.安装配置ZeroMQ和JZMQ

sudo apt-get install libtool autoconf automake
sudo apt-get install uuid-dev g++
sudo apt-get install make
sudo apt-get install pkg-config
sudo apt-get install git

zeromq

wget http://download.zeromq.org/zeromq-2.1.7.tar.gz
tar -xzf zeromq-2.1.7.tar.gz
cd zeromq-2.1.7
./configure
make
sudo make install

CentOS 可能需要安装libuuid-devel yum install libuuid-devel

git clone https://github.com/nathanmarz/jzmq.git
cd jzmq
./autogen.sh
./configure

make
make install

安装jzmq时遇到No rule to make target `classdist_noinst.stamp'的make错误
具体的make错误信息：

make[1]: *** No rule to make target `classdist_noinst.stamp',
needed by `org/zeromq/ZMQ.class'. Stop.

解决方法：手动创建classdist_noinst.stamp空文件。
touch src/classdist_noinst.stamp
cd src
javac -d . org/zeromq/*.java
cd ..
make
make install

2.安装配置zookeeper

下载zookeeper 3.4.5

http://www.apache.org/dyn/closer.cgi/zookeeper/

具体安装过程可以参考

http://www.cnblogs.com/linjiqin/archive/2013/03/16/2962597.html

3. 安装和启动Strom

官方wiki的安装说明：

https://github.com/nathanmarz/storm/wiki/Setting-up-a-Storm-cluster

unzip storm-0.8.2.zip
mv storm-0.8.2 /var/storm

在4个节点上都执行上述过程，192.168.26.226-192.168.26.229

修改storm.yaml，并在其他几点上copy修改完毕的yaml文件
cd /var/storm/conf
scp 192.168.26.226:/var/storm/conf/storm.yaml .

文件内容修改为：

########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
    - "192.168.26.226"
    - "192.168.26.227"
    - "192.168.26.228"
nimbus.host: "192.168.26.226"
ui.port: 8080
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703
topology.debug: true
supervisor.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true"
storm.local.dir: "/var/storm-0.9.0-rc2/data"
#
#
# ##### These may optionally be filled in:
#    
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
drpc.servers:
    - "192.168.26.227"
    - "192.168.26.228"
    - "192.168.26.229"
drpc.worker.threads: 1024
drpc.queue.size: 10240
topology.max.spout.pending: 20480
topology.max.task.parallelism: 1024

启动Strom集群

cd /var/storm
192.168.26.226
nohup bin/storm nimbus &
nohup bin/storm ui > ui.out 2>&1 &

192.168.26.227-229
nohup bin/storm supervisor > supervisor.out 2>&1 &
bin/storm drpc &

然后可以在http://192.168.26.226:8080上查看Strom集群的运行状况

提交任务可以使用命令

bin/storm jar credit-realtime.jar com.dianping.credit.realtime.topology.CreditTopology Test