Spark实例演示之SparkPi

1.贴上源码

object SparkPi {

def main(args: Array[String]) {

val conf = new SparkConf().setAppName("Spark Pi").setMaster("spark://hadoop1:7077").setJars(List("/home/kaiseu/MyProject/IdeaProjects/week2/out/artifacts/week2/week2.jar"))

val spark = new SparkContext(conf)

val slices = if (args.length > 0) args(0).toInt else 2

val n = 100000 * slices

val count = spark.parallelize(1 to n, slices).map { i =>

val x = Math.random * 2 - 1

val y = Math.random * 2 - 1

if (x * x + y * y < 1) 1 else 0

}.reduce(_ + _)

println("Pi is rounghly " + 4.0 * count / n)

spark.stop()

}

}

2 启动spark集群

到spark安装目录：/app/hadoop/spark100

注意：在启动的时候命令必须写成："./start-all.sh" 来表明是当前目录下的"start-all.sh"，因为在配置Hadoop的bin 目录下也有一个"start-all.sh"文件！

使用jps查看进程

3.通过spark集群web页面访问"http://hadoop1:8080"

4.运行结果