Spark群集——Mesosphere方案

#安装mesosphere repo

sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm 

#下载apache mesos

wget http://downloads.mesosphere.io/master/centos/7/mesos-0.21.1-1.1.centos701406.x86_64.rpm

sudo rpm -Uvh mesos-0.21.1-1.1.centos701406.x86_64.rpm

#安装marathon

sudo yum -y install marathon

#安装chronos

sudo yum -y install chronos

ZooKeeper

On each node, replacing the IP addresses below with each master's IP address, set /etc/mesos/zk to:

zk://1.1.1.1:2181,2.2.2.2:2181,3.3.3.3:2181/mesos 

Quorum

Set /etc/mesos-master/quorum to:

2 

----------------------------------------

root     19973     1  0 18:51 ?        00:00:00 /usr/sbin/mesos-master --zk=zk://xd-1:2181,xd-2:2181,xd-3:2181/mesos --port=5050 --log_dir=/var/log/mesos --quorum=2 --work_dir=/var/lib/mesos
root     19983 19973  0 18:51 ?        00:00:00 logger -p user.info -t mesos-master[19973]
root     19984 19973  0 18:51 ?        00:00:00 logger -p user.err -t mesos-master[19973]
root     20081     1 28 18:51 ?        00:00:06 java -Djava.library.path=/usr/local/lib -Djava.util.logging.SimpleFormatter.format=%2$s%5$s%6$s%n -Xmx512m -cp /usr/local/bin/marathon mesosphere.marathon.Main --zk zk://xd-1:2181,xd-2:2181,xd-3:2181/marathon --master zk://xd-1:2181,xd-2:2181,xd-3:2181/mesos
root     20257 13510  0 18:51 pts/4    00:00:00 grep --color=auto mesos

-------------------------------------------

Verifying Installation

If the packages were installed and configured correctly, you should be able to access the Mesos console athttp://<master-ip>:5050 and the Marathon console at http://<master-ip>:8080 (where <master-ip>is any of the master IP addresses).

Another quick way to verify your cluster works is to launch a task through mesos-execute from any of the newly provisioned nodes:

MASTER=$(mesos-resolve `cat /etc/mesos/zk`)
mesos-execute --master=$MASTER --name="cluster-test" --command="sleep 5" 

Besides the console output, which will show a task being created and changing status to RUNNING and thenFINISHED, you should also see a newly terminated framework on the frameworks page of the Mesos console.

====================

启动Master
[dev@xd-ui ~]$ tree /etc/mesos-master/
/etc/mesos-master/
├── hostname    -->xd-ui
├── quorum    -->1
└── work_dir    -->/alidata1/mesos
重启Mesos Master:
[dev@xd-ui ~]$sudo service mesos-master restart

配置各个节点的Slaves
[dev@spark-1 ~]$ tree /etc/mesos-slave/
/etc/mesos-slave/
├── hostname    xd-1
└── work_dir    /alidata1/mesos


在各个节点上重启各个Mesos Slaves:
[dev@xd-ui ~]$sudo service mesos-slave restart

检查各个节点是否起来：

[dev@xd-ui ~]$sudo ps -ef | grep mesos

/usr/sbin/mesos-slave --master=zk://xd-1:2181,xd-2:2181,xd-3:2181/mesos --log_dir=/var/log/mesos --hostname=xd-3 --

 

work_dir=/alidata1/mesos
/usr/sbin/mesos-slave --master=zk://xd-1:2181,xd-2:2181,xd-3:2181/mesos --log_dir=/var/log/mesos --hostname=xd-2 --work_dir=/alidata1/mesos

/usr/sbin/mesos-slave --master=zk://xd-1:2181,xd-2:2181,xd-3:2181/mesos --log_dir=/var/log/mesos --hostname=xd-1 --work_dir=/alidata1/mesos


修改:${SPARK_HOME}/conf/spark-env.sh：
export MESOS_NATIVE_LIBRARY=/usr/local/lib/libmesos.so
export SPARK_EXECUTOR_URI=http://xd-gw/download/spark-1.2.0-dist.tar.gz
#export MASTER=zk://xd-1:2181,xd-2:2181,xd-3:2181/mesos
export MASTER=mesos://xd-ui:5050


修改spark-1.2.0-dist.tar.gz里面spark-1.2.0-dist/bin/spark-class或者conf/spark-env.sh
在首行添加：
export JAVA_HOME=/usr/share/jdk1.7.0_45

 

#启动spark shell， 输入测试：

val data = 1 to 10000; val distData = sc.parallelize(data); distData.filter(_< 10).collect()

如下：

[dev@xd-ui spark-1.2.0-dist]$ ./bin/spark-shell
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/01/22 16:07:46 INFO SecurityManager: Changing view acls to: dev
15/01/22 16:07:46 INFO SecurityManager: Changing modify acls to: dev
15/01/22 16:07:46 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(dev); users with modify permissions: Set(dev)
15/01/22 16:07:46 INFO HttpServer: Starting HTTP Server
15/01/22 16:07:46 INFO Utils: Successfully started service 'HTTP class server' on port 50256.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.2.0
      /_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_45)
Type in expressions to have them evaluated.

scala> val data = 1 to 10000; val distData = sc.parallelize(data); distData.filter(_< 10).collect()
data: scala.collection.immutable.Range.Inclusive = Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170...

====================

slaves

feameworks

 

 

Spark监控页面（4040端口）

 

Spark task

 

参考资料：

                 

http://mesosphere.com/downloads/details/index.html

http://mesosphere.com/docs/tutorials/run-spark-on-mesos/

http://mesosphere.com/docs/getting-started/datacenter/install/

http://www.ulimit.top/sparkji-qun-pei-zhi-mesospherefang-an/

http://mesos.apache.org/gettingstarted/

http://spark.apache.org/docs/latest/running-on-mesos.html#configuration