You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Imre Nagi <im...@gmail.com> on 2016/03/16 04:58:08 UTC
Get Pair of Topic and Message from Kafka + Spark Streaming
Hi,
I'm just trying to process the data that come from the kafka source in my
spark streaming application. What I want to do is get the pair of topic and
message in a tuple from the message stream.
Here is my streams:
val streams = KafkaUtils.createDirectStream[String, Array[Byte],
> StringDecoder, DefaultDecoder](ssc,kafkaParameter,
> Array["topic1", "topic2])
I have done several things, but still failed when i did some
transformations from the streams to the pair of topic and message. I hope
somebody can help me here.
Thanks,
Imre