You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Hendrik Dev <he...@gmail.com> on 2018/02/13 14:20:37 UTC
Retrieve batch metadata via the spark monitoring api
I use Spark 2.2.1 with streaming and when i open the Spark Streaming
UI i can see input metadata for each of my batches. In my case i
stream from Kafka and in the metadata section i find useful
informations about my topic, partitions and offsets.
Assume the url for this batch looks like
http://localhost:4040/streaming/batch/?id=1566530100001
When i now query spark with the monitoring API there seems to be no
way to retrieve the above mentioned metadata.
Have i overlooked something or is there currently with spark 2.2.1
really no way to retrieve the input metadata via the REST API?
Thx
Hendrik
--
Hendrik Saly (salyh, hendrikdev22)
@hendrikdev22
PGP: 0x22D7F6EC
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org