You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "junmuz (via GitHub)" <gi...@apache.org> on 2026/06/17 10:22:07 UTC
[PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
junmuz opened a new pull request, #8268:
URL: https://github.com/apache/paimon/pull/8268
### Purpose
Fixes the Jdbc connection leaks that happen when Flink TMs restart.
Issue: #8267
### Tests
There are some tests added specially the `testConnectionClosedWhenPoolClosedDuringAction` that simulates the scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "junmuz (via GitHub)" <gi...@apache.org>.
junmuz commented on PR #8268:
URL: https://github.com/apache/paimon/pull/8268#issuecomment-4777412547
Thanks @nickdelnano for raising the PR. Yeah I agree tracking that PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8268:
URL: https://github.com/apache/paimon/pull/8268#issuecomment-4776191091
I think #8323 is better to Paimon ecosystem. Let's track #8323
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed pull request #8268: [core] Fix JDBC connection leak in ClientPool and JdbcCatalog
URL: https://github.com/apache/paimon/pull/8268
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "junmuz (via GitHub)" <gi...@apache.org>.
junmuz commented on PR #8268:
URL: https://github.com/apache/paimon/pull/8268#issuecomment-4735839863
@JingsongLi Can I get a review on the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "junmuz (via GitHub)" <gi...@apache.org>.
junmuz commented on PR #8268:
URL: https://github.com/apache/paimon/pull/8268#issuecomment-4769917734
> I found one remaining leak race in `ClientPool.run`. The check at lines 71-72 is not atomic with returning the borrowed client to the deque: a thread can observe `this.clients != null`, then another thread calls `close()`, sets `this.clients = null` and drains the deque, and then the first thread executes `clients.addFirst(client)` on the old deque. That client was not included in the drain and is no longer reachable from the pool, so it is still left unclosed. This is the same close-during-action case this PR is trying to handle, just with a narrower interleaving. The return-to-pool path and `close()` need to be made mutually exclusive, or the code needs a post-add closed check/removal that closes the client if the pool was closed concurrently.
@JingsongLi Thanks for the comment. I have tried addressing that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8268:
URL: https://github.com/apache/paimon/pull/8268#issuecomment-4747337737
I found one remaining leak race in `ClientPool.run`. The check at lines 71-72 is not atomic with returning the borrowed client to the deque: a thread can observe `this.clients != null`, then another thread calls `close()`, sets `this.clients = null` and drains the deque, and then the first thread executes `clients.addFirst(client)` on the old deque. That client was not included in the drain and is no longer reachable from the pool, so it is still left unclosed. This is the same close-during-action case this PR is trying to handle, just with a narrower interleaving. The return-to-pool path and `close()` need to be made mutually exclusive, or the code needs a post-add closed check/removal that closes the client if the pool was closed concurrently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix JDBC connection leak in ClientPool and JdbcCatalog [paimon]
Posted by "nickdelnano (via GitHub)" <gi...@apache.org>.
nickdelnano commented on PR #8268:
URL: https://github.com/apache/paimon/pull/8268#issuecomment-4771887642
@junmuz I took a different approach in https://github.com/apache/paimon/pull/8323 to reduce jdbc client connections via sharing per JVM. That approach can reduce connections by a large factor for high parallelism jobs. It would affect some of the logic in this PR as with shared connections we would not explicitly call close.
We would still need the fix of 54a23e4289fc05af3f153ebfeefffa77833d5d19 from this branch, which I have ported there.
Let me know if you have any thoughts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org