You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Mark Davis <mo...@protonmail.com> on 2020/08/26 14:05:07 UTC
Resource leak in DataSourceNode?
Hi,
I am trying to investigate a problem with non-released resources in my application.
I have a stateful application which submits Flink DataSetjobs using code very similar to the code in CliFrontend.
I noticed what I am getting a lot of non-closed connections to my data store (HBase in my case). The connections are held by the application not the jobs themselves.
I am using HBaseRowDataInputFormat and it seems that HBase connections opened in the configure() method during the job graph creation(before the jobs is executed) are not closed. My search lead me to the method DataSourceNode.computeOperatorSpecificDefaultEstimates(DataStatistics) where I see that a format is not closed after being configured.
Is that correct? How can I overcome this issue?
My application is long running that is probably why I observe the resource leak. Would I spawn a new JVM to run jobs this problem would not be noticeable.
Thank you!
Cheers,
Marc
Re: Resource leak in DataSourceNode?
Posted by Robert Metzger <rm...@apache.org>.
Hi Mark,
from the discussion in the JIRA ticket, it seems that we've found somebody
in the community who's going to fix this.
I don't think calling close() is necessary in the DataSourceNode. The
problem is that the connection should not be established in configure() but
in open().
Thanks again for the bug report. The fix will be in the next release of the
1.11 line.
On Sun, Aug 30, 2020 at 5:34 PM Mark Davis <mo...@protonmail.com> wrote:
> Hi Robert,
>
> Thank you for confirming that there is an issue.
> I do not have a solution for it and would like to hear the committer
> insights what is wrong there.
>
> I think there are actually two issues - the first one is the HBase
> InputFormat does not close a connection in close().
> Another is DataSourceNode not calling the close() method.
>
> Cheers,
> Mark
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, August 27, 2020 3:30 PM, Robert Metzger <rm...@apache.org>
> wrote:
>
> Hi Mark,
>
> Thanks a lot for your message and the good investigation! I believe you've
> found a bug in Flink. I filed an issue for the problem:
> https://issues.apache.org/jira/browse/FLINK-19064.
>
> Would you be interested in opening a pull request to fix this?
> Otherwise, I'm sure a committer will pick up the issue soon.
>
> I'm not aware of a simple workaround for the problem.
>
> Best,
> Robert
>
>
> On Wed, Aug 26, 2020 at 4:05 PM Mark Davis <mo...@protonmail.com> wrote:
>
>> Hi,
>>
>> I am trying to investigate a problem with non-released resources in my
>> application.
>>
>> I have a stateful application which submits Flink DataSet jobs using
>> code very similar to the code in CliFrontend.
>> I noticed what I am getting a lot of non-closed connections to my data
>> store (HBase in my case). The connections are held by the application not
>> the jobs themselves.
>>
>> I am using HBaseRowDataInputFormat and it seems that HBase connections
>> opened in the configure() method during the job graph creation(before the
>> jobs is executed) are not closed. My search lead me to the method
>> DataSourceNode.computeOperatorSpecificDefaultEstimates(DataStatistics)
>> where I see that a format is not closed after being configured.
>>
>> Is that correct? How can I overcome this issue?
>>
>> My application is long running that is probably why I observe the
>> resource leak. Would I spawn a new JVM to run jobs this problem would not
>> be noticeable.
>>
>> Thank you!
>>
>> Cheers,
>> Marc
>>
>
>
Re: Resource leak in DataSourceNode?
Posted by Mark Davis <mo...@protonmail.com>.
Hi Robert,
Thank you for confirming that there is an issue.
I do not have a solution for it and would like to hear the committer insights what is wrong there.
I think there are actually two issues - the first one is the HBase InputFormat does not close a connection in close().
Another is DataSourceNode not calling the close() method.
Cheers,
Mark
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, August 27, 2020 3:30 PM, Robert Metzger <rm...@apache.org> wrote:
> Hi Mark,
>
> Thanks a lot for your message and the good investigation! I believe you've found a bug in Flink. I filed an issue for the problem: https://issues.apache.org/jira/browse/FLINK-19064.
>
> Would you be interested in opening a pull request to fix this?
> Otherwise, I'm sure a committer will pick up the issue soon.
>
> I'm not aware of a simple workaround for the problem.
>
> Best,
> Robert
>
> On Wed, Aug 26, 2020 at 4:05 PM Mark Davis <mo...@protonmail.com> wrote:
>
>> Hi,
>>
>> I am trying to investigate a problem with non-released resources in my application.
>>
>> I have a stateful application which submits Flink DataSetjobs using code very similar to the code in CliFrontend.
>> I noticed what I am getting a lot of non-closed connections to my data store (HBase in my case). The connections are held by the application not the jobs themselves.
>>
>> I am using HBaseRowDataInputFormat and it seems that HBase connections opened in the configure() method during the job graph creation(before the jobs is executed) are not closed. My search lead me to the method DataSourceNode.computeOperatorSpecificDefaultEstimates(DataStatistics) where I see that a format is not closed after being configured.
>>
>> Is that correct? How can I overcome this issue?
>>
>> My application is long running that is probably why I observe the resource leak. Would I spawn a new JVM to run jobs this problem would not be noticeable.
>>
>> Thank you!
>>
>> Cheers,
>> Marc
Re: Resource leak in DataSourceNode?
Posted by Robert Metzger <rm...@apache.org>.
Hi Mark,
Thanks a lot for your message and the good investigation! I believe you've
found a bug in Flink. I filed an issue for the problem:
https://issues.apache.org/jira/browse/FLINK-19064.
Would you be interested in opening a pull request to fix this?
Otherwise, I'm sure a committer will pick up the issue soon.
I'm not aware of a simple workaround for the problem.
Best,
Robert
On Wed, Aug 26, 2020 at 4:05 PM Mark Davis <mo...@protonmail.com> wrote:
> Hi,
>
> I am trying to investigate a problem with non-released resources in my
> application.
>
> I have a stateful application which submits Flink DataSet jobs using code
> very similar to the code in CliFrontend.
> I noticed what I am getting a lot of non-closed connections to my data
> store (HBase in my case). The connections are held by the application not
> the jobs themselves.
>
> I am using HBaseRowDataInputFormat and it seems that HBase connections
> opened in the configure() method during the job graph creation(before the
> jobs is executed) are not closed. My search lead me to the method
> DataSourceNode.computeOperatorSpecificDefaultEstimates(DataStatistics)
> where I see that a format is not closed after being configured.
>
> Is that correct? How can I overcome this issue?
>
> My application is long running that is probably why I observe the resource
> leak. Would I spawn a new JVM to run jobs this problem would not be
> noticeable.
>
> Thank you!
>
> Cheers,
> Marc
>