java.lang.IllegalArgumentException: requirement failed: Column features must be of type org.apache.s

最新推荐文章于 2024-01-07 04:30:00 发布

原创

最新推荐文章于 2024-01-07 04:30:00 发布 · 2.8k 阅读

0 ·

CC 4.0 BY-SA版权

在使用Spark进行lr模型训练时遇到IllegalArgumentException，原因是features列的数据类型应为org.apache.spark.ml.linalg.Vector，但实际上为org.apache.spark.mllib.linalg.Vector。错误发生在fit方法调用过程中。解决方案是确保在Spark 2.x及以上版本中，正确导入org.apache.spark.ml.linalg.Vector而非org.apache.spark.mllib.linalg.Vector。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

lr训练模型报错：

val model1 = lr.fit(training)

java.lang.IllegalArgumentException: requirement failed: Column features must be of type org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7 but was actually org.apache.spark.mllib.linalg.VectorUDT@f71b0bce.
at scala.Predef$.require(Predef.scala:224)
at org.apache.spark.ml.util.SchemaUtils$.checkColumnType(SchemaUtils.scala:42)
at org.apache.spark.ml.PredictorParams$class.validateAndTransformSchema(Predictor.scala:51)
at org.apache.spark.ml.classification.Classifier.org$apache$spark$ml$classification$ClassifierParams$$super$validateAndTransformSchema(Classifier.scala:58)
at org.apache.spark.ml.classification.ClassifierParams$class.validateAndTransformSchema(Classifier.scala:42)
at org.apache.spark.ml.classification.ProbabilisticClassifier.org$apache$spark$ml$classification$ProbabilisticClassifierParams$$super$validateAndTransformSchema(ProbabilisticClassifier.scala:53)<