写 spark sql时踩到一个巨坑
val markLabel = udf((userId: String, orderId: String) => if (blackBroadcast.value.contains(userId) || orderBroadcast.value.contains(orderId)) 1.0 else 0.0); val labelData = extData.withColumn("label", markLabel(extData("user_accountId"), extData("order_orderId")));
Run状态是OK的,结果也对,但是打包的时候就老报下面的错误,网上找了一圈也没找到答案,
[ERROR] E:\Code\scalaCode\RiskManager\05_DataAnaly\src\main\scala\RiskManager\DataAnaly.scala:51: error: No TypeTag available for String [INFO] val markLabel = udf((userId: String, orderId: String) => if (blackBroadcast.value.contains(userId) || orderBroadcast.value.contains(orderId)) 1.0 else 0.0);
最后发现了原来是这个原因,引入了一个叫play_json的包
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play-json_2.10</artifactId>
<version>2.4.0-M1</version>
</dependency>
去掉就能正常打包了!!!!