You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Greg Fodor <gf...@gmail.com> on 2016/04/25 09:13:08 UTC
Kafka Streams file handle leak
Apologies for a second posting to the list, but the ASF JIRA is
preventing me from creating KAFKA- tickets so I wanted to report this
issue. This seems to be a critical bug with KStreams, the .lock files
in the state store directories do not seem to be having their file
handles freed (despite the locks being freed), so on a complex job the
number of file handles in use goes up rapidly as the locks are taken
for the cleanup routine at the end of the thread run loop. I've
created a PR that seems to resolve this issue:
https://github.com/apache/kafka/pull/1267
Re: Kafka Streams file handle leak
Posted by Greg Fodor <gf...@gmail.com>.
Great, thanks:
https://issues.apache.org/jira/browse/KAFKA-3619
On Mon, Apr 25, 2016 at 11:13 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Thanks for reporting that Greg.
>
> I have just added you as a Kafka contributor so you should be able to
> create JIRAs now. Could you create a new JIRA for this and change the RB
> title with the JIRA number?
>
> Thanks!
>
> Guozhang
>
> On Mon, Apr 25, 2016 at 12:13 AM, Greg Fodor <gf...@gmail.com> wrote:
>
>> Apologies for a second posting to the list, but the ASF JIRA is
>> preventing me from creating KAFKA- tickets so I wanted to report this
>> issue. This seems to be a critical bug with KStreams, the .lock files
>> in the state store directories do not seem to be having their file
>> handles freed (despite the locks being freed), so on a complex job the
>> number of file handles in use goes up rapidly as the locks are taken
>> for the cleanup routine at the end of the thread run loop. I've
>> created a PR that seems to resolve this issue:
>>
>> https://github.com/apache/kafka/pull/1267
>>
>
>
>
> --
> -- Guozhang
Re: Kafka Streams file handle leak
Posted by Guozhang Wang <wa...@gmail.com>.
Thanks for reporting that Greg.
I have just added you as a Kafka contributor so you should be able to
create JIRAs now. Could you create a new JIRA for this and change the RB
title with the JIRA number?
Thanks!
Guozhang
On Mon, Apr 25, 2016 at 12:13 AM, Greg Fodor <gf...@gmail.com> wrote:
> Apologies for a second posting to the list, but the ASF JIRA is
> preventing me from creating KAFKA- tickets so I wanted to report this
> issue. This seems to be a critical bug with KStreams, the .lock files
> in the state store directories do not seem to be having their file
> handles freed (despite the locks being freed), so on a complex job the
> number of file handles in use goes up rapidly as the locks are taken
> for the cleanup routine at the end of the thread run loop. I've
> created a PR that seems to resolve this issue:
>
> https://github.com/apache/kafka/pull/1267
>
--
-- Guozhang