You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Arwin Tio <ar...@hotmail.com> on 2020/12/28 23:16:03 UTC

Using Streaming Listener in a Structured Streaming job

In a Structured Streaming job, the listener that is supported is StreamingQueryListener.

spark.streams().addListener(
  new StreamingQueryListener() {
    ...
  }
);

However, there is no straightforward way to use StreamingListener.

I have done it like this:

StreamingContext streamingContext = new StreamingContext(spark.sparkContext(), new Duration(1000))

streamingContext.addStreamingListener(
  new StreamingListener() {
    ...
  }
)
However, this is not working for me.  Is there a way to use StreamingListener in a Structured Streaming query?

Thanks,

Arwin