输入源是kafka的sparkstreaming出现的错误

最新推荐文章于 2025-05-27 20:12:36 发布

邓禹

最新推荐文章于 2025-05-27 20:12:36 发布

阅读量242

点赞数 1

分类专栏： spark

本文链接：https://blog.youkuaiyun.com/weixin_44803002/article/details/103194198

版权

spark 专栏收录该内容

4 篇文章

订阅专栏

19/11/22 08:50:24 ERROR StreamingContext: Error starting the context, marking it as stopped
java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
	at scala.Predef$.require(Predef.scala:224)
	at org.apache.spark.streaming.DStreamGraph.validate(DStreamGraph.scala:168)
	at org.apache.spark.streaming.StreamingContext.validate(StreamingContext.scala:513)
	at org.apache.spark.streaming.StreamingContext.liftedTree1$1(StreamingContext.scala:573)
	at org.apache.spark.streaming.StreamingContext.start(StreamingContext.scala:572)
	at com.zpark.sparkstreaming.SparkStream_02$.main(SparkStream_02.scala:30)
	at com.zpark.sparkstreaming.SparkStream_02.main(SparkStream_02.scala)
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
	at scala.Predef$.require(Predef.scala:224)
	at org.apache.spark.streaming.DStreamGraph.validate(DStreamGraph.scala:168)
	at org.apache.spark.streaming.StreamingContext.validate(StreamingContext.scala:513)
	at org.apache.spark.streaming.StreamingContext.liftedTree1$1(StreamingContext.scala:573)
	at org.apache.spark.streaming.StreamingContext.start(StreamingContext.scala:572)
	at com.zpark.sparkstreaming.SparkStream_02$.main(SparkStream_02.scala:30)
	at com.zpark.sparkstreaming.SparkStream_02.main(SparkStream_02.scala)

错误已经提示你：没有注册输出操作，所以没有要执行的操作

所以我们要注意是否输出DStream
例如对

val kafkaDStream: ReceiverInputDStream[(String, String)] = KafkaUtils.createStream(ssc,"hdp-1:2181","text",Map("text"->2))

使用foreachRDD方法转换成RDD在使用行动算子输出。问题就解决了！