
spark
sisiel
厚积薄发
展开
-
spark报错:value xxx is not a member of Any
df1.rdd.filter(x=>x(1).contains("腾讯")).take(5)遇到这个问题此时可以先做类型转换:df1.rdd.filter(x=>x(1).toString.contains("腾讯")).take(5)对于变量类,可以使用asInstanceOfval a: Any = "123"val b = a.asInstanceOf[String]...原创 2021-01-08 13:59:58 · 4802 阅读 · 0 评论 -
spark:rdd 转换dataframe报错: Array takes type parameters;toDF is not a member of org.apache.spark.rdd
tdf4.select("id").rdd.map(x=>Array(x(0),cmls)).toDF("id","cmls").show()试图构造上述dataframe,其中cmls是个array[any]此时报错:value toDF is not a member of org.apache.spark.rdd.RDD[Array[Any]]尝试了网上的import sqlContext.implicits._. 不行后,发现是由于这个方法只针对下面几种格式:RD...原创 2020-12-22 14:03:14 · 991 阅读 · 0 评论