You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Ajit Dongre <Aj...@walmartlabs.com> on 2020/03/02 08:17:58 UTC

Customize Kafka Streaming meta info on Spark UI

Hi,

I am developing Beam Streaming pipeline for KafkaIO using Spark runner.

On Spark Streaming UI, I am not able to get information about partitions and offsets involved in particular streaming batch.  It displays information about watermark etc.

Generally Spark streaming UI displays below kind of meta information for each batch.
This meta data contains information about partitions and corresponding offsets present in that streaming batch as displayed in below image.

[Batch details]


Want to know is it possible to customize this meta information ?

Regards,
Ajit Dongre