本文探讨了如何在Apache Spark中集成Redis作为数据源，通过获取列表并生成RDD的方式提高数据处理效率。文中还提供了多个参考资料，包括StackOverflow上的讨论、GitHub上的项目及Apache Spark官方文档。
Spark Redis(1) Old Notes
I do not think I will directly use this to generate RDD. I can fetch the list and generate RDD from response list.
References:
http://stackoverflow.com/questions/28006517/redis-on-sparktask-not-serializable
https://github.com/sasha-polev/redis-spark
https://github.com/apache/spark/tree/master/core/src/main/scala/org/apache/spark/rdd
https://github.com/apache/spark/tree/master/examples/src/main/scala/org/apache/spark/examples
https://gist.github.com/bigsnarfdude/81a7422a0de1e3f99a83 writes to redis
I do not think I will directly use this to generate RDD. I can fetch the list and generate RDD from response list.
References:
http://stackoverflow.com/questions/28006517/redis-on-sparktask-not-serializable
https://github.com/sasha-polev/redis-spark
https://github.com/apache/spark/tree/master/core/src/main/scala/org/apache/spark/rdd
https://github.com/apache/spark/tree/master/examples/src/main/scala/org/apache/spark/examples
https://gist.github.com/bigsnarfdude/81a7422a0de1e3f99a83 writes to redis
扫一扫
685
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
