You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by aitozi <gj...@gmail.com> on 2017/10/30 11:59:53 UTC

Re: Telling if a job has caught up with Kafka

Hi, rmetzger0

Sorry to reply to this old question, i found that we use the kafka client
0.9 in class kafkaThread which lead to the lose of many other detail metrics
add in kafka client 10 like per partition consumer lag mentioned by this
issuse https://issues.apache.org/jira/browse/FLINK-7945.  i will try to add
the detail metrics 


rmetzger0 wrote
> Sorry for joining this discussion late, but there is already a metric for
> the offset lag in our 0.9+ consumers.
> Its called the "records-lag-max":
> https://kafka.apache.org/documentation/#new_consumer_fetch_monitoring and
> its exposed via Flink's metrics system.
> The only issue is that it only shows the maximum lag across all
> partitions,
> not detailed per-partition metrics.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/