You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Kevin Tran <ke...@gmail.com> on 2016/09/25 22:59:26 UTC

Extract timestamp from Kafka message

Hi Everyone,
Does anyone know how could we extract timestamp from Kafka message in Spark
streaming ?

JavaPairInputDStream<String, String> messagesDStream =
KafkaUtils.createDirectStream(
       ssc,
       String.class,
       String.class,
       StringDecoder.class,
       StringDecoder.class,
       kafkaParams,
       topics
   );


Thanks,
Kevin.

Re: Extract timestamp from Kafka message

Posted by Alonso Isidoro Roman <al...@gmail.com>.

hum, i think you have to embed the timestamp within the message...

Alonso Isidoro Roman
[image: https://]about.me/alonso.isidoro.roman
<https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links>

2016-09-26 0:59 GMT+02:00 Kevin Tran <ke...@gmail.com>:

> Hi Everyone,
> Does anyone know how could we extract timestamp from Kafka message in
> Spark streaming ?
>
> JavaPairInputDStream<String, String> messagesDStream =
> KafkaUtils.createDirectStream(
>        ssc,
>        String.class,
>        String.class,
>        StringDecoder.class,
>        StringDecoder.class,
>        kafkaParams,
>        topics
>    );
>
>
> Thanks,
> Kevin.
>
>
>
>
>
>
>