confd java_Spark Streaming：打印JavaInputDStream

目前我正在使用Spark Streaming以及从Kafka读取消息的可能性 . 使用Kafka Producer，我将消息发送到主题，并希望在Spark Streaming的帮助下阅读此主题 .

我使用以下Java代码来查询消息：

package apache_spark_streaming;

import java.util.*;

import org.apache.spark.SparkConf;

import org.apache.spark.streaming.Durations;

import org.apache.spark.streaming.api.java.JavaInputDStream;

import org.apache.spark.streaming.api.java.JavaStreamingContext;

import org.apache.spark.streaming.kafka010.*;

import org.apache.kafka.clients.consumer.ConsumerRecord;

public final class Spark_Kafka_Example {

private static final String BOOTSTRAP_SERVERS_CONNECTION = "XXXXX";

private static final String SPARK_CONNECTION = "spark://XXXXX:7077";

private static final String TOPIC_NAME = "KafkaTesting1";

private static final Set TOPIC_1 = new HashSet<>(Arrays.asList(TOPIC_NAME.split(",")));

public static Map getProperties() {

try {

Map kafkaParams = new HashMap<>();

kafkaParams.put("bootstrap.servers", BOOTSTRAP_SERVERS_CONNECTION);

kafkaParams.put("key.deserializer", org.apache.kafka.common.serialization.StringDeserializer.class.getName());

kafkaParams.put("value.deserializer", org.apache.kafka.common.serialization.StringDeserializer.class.getName());

kafkaParams.put("group.id", "Stream Testing");

kafkaParams.put("auto.offset.reset", "earliest");

kafkaParams.put("enable.auto.commit", false);

return kafkaParams;

}

catch (Exception e) {

e.printStackTrace();

}

return null;

}

public static void main(String[] args) throws Exception {

// Create context with a 2 seconds batch interval

SparkConf sparkConf = new SparkConf().setAppName("Kafka Example").setMaster(SPARK_CONNECTION);

JavaStreamingContext sc = new JavaStreamingContext(sparkConf, Durations.seconds(2));

JavaInputDStream> stream = KafkaUtils.createDirectStream(

sc

, LocationStrategies.PreferConsistent()

, ConsumerStrategies.Subscribe(TOPIC_1, getProperties())

);

stream.print();

sc.start();

sc.awaitTermination();

}

}

我的问题是我不知道如何在命令行上输出消息 . 也许我只有一个理解问题如何正确使用JavaInputDStreams .

目前我只使用print()函数作为输出：

17/07/10 16:59:20 INFO JobScheduler：添加了1499698760000 ms的工作时间

我希望你能帮我解决这个“问题” .

Updated 我试过了

stream.foreachRDD(consumerRecordJavaRDD -> {

consumerRecordJavaRDD.foreach(stringStringConsumerRecord -> {

//.to get topic name: stringStringConsumerRecord.topic()

//To get value : stringStringConsumerRecord.value()

} }