本文探讨了R与Hadoop结合使用的局限性，指出R的优势在于丰富的统计和可视化库，但这些库通常是非分布式的，并且操作内存中的数据。因此，在面对大数据集时，R的性能可能受限，且无法通过Hadoop直接提升。文章还提到，尽管一些分布式数据库供应商声称提供解决方案，但实际上它们只能在小数据集上并行运行R。
Referer: http://www.quora.com/How-can-R-and-Hadoop-be-used-together/answer/Jay-Kreps?srid=OVd9&share=1
Another way to answer this question is that they don't really integrate very well.
The advantage of R is not its syntax but rather the incredible library of primitives for visualization and statistics. These libraries are fundamentally non-distributed, and almost always operate on data resident in memory. So, for example, if you are finding R's glm method slow (or completely infeasible) on a particular dataset, there is really no way to make it run faster with Hadoop.
The reason it is important to point this out is that
1. Transparently distributed R is every data geeks wet dream.
2. I have sat through numerous presentations from distributed database vendors claiming to provide this.
What hadoop and database vendors can provide is the ability to run R in parallel on lots of little data sets. Virtually none of the libraries will work on a data set larger than memory.
Referer: http://www.quora.com/How-can-R-and-Hadoop-be-used-together/answer/Jay-Kreps?srid=OVd9&share=1
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
