You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@iceberg.apache.org by Sud <su...@gmail.com> on 2020/09/17 22:56:47 UTC

Connection leak (unclosed reader) leading to s3a timeout exceptions

HI Iceberg-dev's

I am investigating the connection leak issue we are seeing after upgrading
to the latest iceberg.
I have narrowed down investigation to following PR and testing fix now

https://github.com/apache/iceberg/commit/7060c928390c59e24dc207ec86f99132f6c1a828#diff-9726b2a5391d8755f6c5c849cddc478f

looks like method generateOffsetToStartPos  created new reader but its not
closed.

Has anyone seen a similar issue with the latest release?

I will submit PR once I am done with testing.



--
Thanks

Re: Connection leak (unclosed reader) leading to s3a timeout exceptions

Posted by Ryan Blue <rb...@netflix.com.INVALID>.

Thanks for the quick fix!

On Thu, Sep 17, 2020 at 4:25 PM Sud <su...@gmail.com> wrote:

> submitted PR with fix please review
>
> https://github.com/apache/iceberg/pull/1474/files
>
> On Thu, Sep 17, 2020 at 3:56 PM Sud <su...@gmail.com> wrote:
>
>> HI Iceberg-dev's
>>
>> I am investigating the connection leak issue we are seeing after
>> upgrading to the latest iceberg.
>> I have narrowed down investigation to following PR and testing fix now
>>
>>
>> https://github.com/apache/iceberg/commit/7060c928390c59e24dc207ec86f99132f6c1a828#diff-9726b2a5391d8755f6c5c849cddc478f
>>
>> looks like method generateOffsetToStartPos  created new reader but
>> its not closed.
>>
>> Has anyone seen a similar issue with the latest release?
>>
>> I will submit PR once I am done with testing.
>>
>>
>>
>> --
>> Thanks
>>
>

-- 
Ryan Blue
Software Engineer
Netflix

Re: Connection leak (unclosed reader) leading to s3a timeout exceptions

Posted by Sud <su...@gmail.com>.

submitted PR with fix please review

https://github.com/apache/iceberg/pull/1474/files

On Thu, Sep 17, 2020 at 3:56 PM Sud <su...@gmail.com> wrote:

> HI Iceberg-dev's
>
> I am investigating the connection leak issue we are seeing after upgrading
> to the latest iceberg.
> I have narrowed down investigation to following PR and testing fix now
>
>
> https://github.com/apache/iceberg/commit/7060c928390c59e24dc207ec86f99132f6c1a828#diff-9726b2a5391d8755f6c5c849cddc478f
>
> looks like method generateOffsetToStartPos  created new reader but
> its not closed.
>
> Has anyone seen a similar issue with the latest release?
>
> I will submit PR once I am done with testing.
>
>
>
> --
> Thanks
>