You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by vr spark <vr...@gmail.com> on 2018/01/17 05:05:59 UTC
spark streaming kafka not displaying data in local eclipse
Hi,
I have a simple Java program to read data from kafka using spark streaming.
When i run it from eclipse on my mac, it is connecting to the zookeeper,
bootstrap nodes,
But its not displaying any data. it does not give any error.
it just shows
18/01/16 20:49:15 INFO Executor: Finished task 96.0 in stage 0.0 (TID 0).
1412 bytes result sent to driver
18/01/16 20:49:15 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID
2, localhost, partition 1, ANY, 5832 bytes)
18/01/16 20:49:15 INFO Executor: Running task 1.0 in stage 0.0 (TID 2)
18/01/16 20:49:15 INFO TaskSetManager: Finished task 96.0 in stage 0.0 (TID
0) in 111 ms on localhost (1/97)
18/01/16 20:49:15 INFO KafkaRDD: Computing topic data_stream, partition 16
offsets 25624028 -> 25624097
18/01/16 20:49:15 INFO VerifiableProperties: Verifying properties
18/01/16 20:49:15 INFO VerifiableProperties: Property auto.offset.reset is
overridden to largest
18/01/16 20:49:15 INFO VerifiableProperties: Property
fetch.message.max.bytes is overridden to 20971520
18/01/16 20:49:15 INFO VerifiableProperties: Property group.id is
overridden to VR-Test-Group
18/01/16 20:49:15 INFO VerifiableProperties: Property zookeeper.connect is
overridden to zk.kafka-cluster...xxxx:8091
18/01/16 20:49:25 INFO JobScheduler: Added jobs for time 1516164560000 ms
18/01/16 20:49:36 INFO JobScheduler: Added jobs for time 1516164570000 ms
18/01/16 20:49:45 INFO JobScheduler: Added jobs for time 1516164580000 ms
18/01/16 20:49:55 INFO JobScheduler: Added jobs for time 1516164590000 ms
18/01/16 20:50:07 INFO JobScheduler: Added jobs for time 1516164600000 ms
18/01/16 20:50:15 INFO JobScheduler: Added jobs for time 1516164610000 ms
But when i export it as jar and run it in a remote spark cluster , it does
display the actual data.
Please suggest what could be wrong.
thanks
VR