You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by kant kodali <ka...@gmail.com> on 2017/05/16 07:59:34 UTC
How to print data to console in structured streaming using Spark
2.1.0 and Zeppelin 0.7.1?
Hi All,
I have the following code.
val ds = sparkSession.readStream()
.format("kafka")
.option("kafka.bootstrap.servers",bootstrapServers))
.option("subscribe", topicName)
.option("checkpointLocation", hdfsCheckPointDir)
.load();
val ds1 = ds.select($"value")
val query = ds1.writeStream.outputMode("append").format("console").start()
query.awaitTermination()
There are no errors when I execute this code however I don't see any data
being printed out to console? When I run my standalone test Kafka consumer
jar I can see that it is receiving messages. so I am not sure what is going
on with above code? any ideas?
Thanks!