Streams monitoring

[Brian Putt <pu...@gmail.com>]

Hello,

We have multiple stream services that we're looking to monitor when they've
been disconnected from the broker so that we can restart the services.

I've looked at https://issues.apache.org/jira/browse/KAFKA-6520 and am
wondering if anyone has suggestions on what we can do today to help ensure
our services don't go idle.

As an example, we'll have our streaming services running and we'll
stop/start the kafka brokers. The services will remain running, but they're
not actually pulling any data.

We could look at time since last record received, but that's not a
guarantee as there's always a possibility that data was legitimately turned
off upstream.

Thanks,

Brian

Re: Streams monitoring

["Matthias J. Sax" <ma...@confluent.io>]

What about monitoring consumer lag?

-Matthias

On 7/10/19 5:19 PM, Brian Putt wrote:
> Hello,
> 
> We have multiple stream services that we're looking to monitor when they've
> been disconnected from the broker so that we can restart the services.
> 
> I've looked at https://issues.apache.org/jira/browse/KAFKA-6520 and am
> wondering if anyone has suggestions on what we can do today to help ensure
> our services don't go idle.
> 
> As an example, we'll have our streaming services running and we'll
> stop/start the kafka brokers. The services will remain running, but they're
> not actually pulling any data.
> 
> We could look at time since last record received, but that's not a
> guarantee as there's always a possibility that data was legitimately turned
> off upstream.
> 
> Thanks,
> 
> Brian
>