Spark的缓存Cache

1、启动spark-shell

./spark-shell --master spark://hdp-1:7077 --executor-memory 500m --total-executor-cores 1

2、读文件

var lines = sc.textFile("hdfs://hdp-1:9000/aaa.txt")

3、将读到的文件放入缓存

var cached = lines.cache

4、出发action

cached.count
lines.count

5、观察spark的job执行时长

6、缓存应用扩展

 cached.filter(_.contains("lucas")).count