You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "muralibasani (via GitHub)" <gi...@apache.org> on 2026/04/04 09:34:15 UTC
[PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
muralibasani opened a new pull request, #21964:
URL: https://github.com/apache/kafka/pull/21964
Fixes below issues.
- There is a resource leak in closeQuietly
- Add a waitForCondition in testConcurrentCacheDeletedFileExists to wait for all pending tasks are completed
- testDeleteInvalidIndexFilesOnInit wrapped with try/resources. There is a resource leak in new RemoteIndexCache.. which was not closed earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-4981310624
@apoorvmittal10 possible for you to take a look at this one too ? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : storage : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-4463014960
@smjn or @AndrewJSchofield possible for a review ? A small cleanup in storage module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4121703757
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
+ } finally {
+ // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
+ // will be deleted at the end of test.
+ try {
+ Utils.delete(logDir);
+ } catch (IOException ioe) {
+ // ignore
+ }
+ // Verify no lingering threads. It is important to have this as the very last statement in the @AfterEach
+ // because this may throw an exception and prevent cleanup after it
+ TestUtils.assertNoLeakedThreadsWithNameAndDaemonStatus(REMOTE_LOG_INDEX_CACHE_CLEANER_THREAD, true);
Review Comment:
I think there is no resource leak in closeQuietly.
close method seems to call cleanerScheduler.shutdown() first and that calls executor.shutdown() before it blocks in awaitTermination. so even if close throws after that point, the executor is already shutdown, and it's thread will exit. There is a fix since #20998 for retry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4121675371
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
+ } finally {
+ // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
+ // will be deleted at the end of test.
+ try {
+ Utils.delete(logDir);
+ } catch (IOException ioe) {
+ // ignore
+ }
+ // Verify no lingering threads. It is important to have this as the very last statement in the @AfterEach
+ // because this may throw an exception and prevent cleanup after it
+ TestUtils.assertNoLeakedThreadsWithNameAndDaemonStatus(REMOTE_LOG_INDEX_CACHE_CLEANER_THREAD, true);
Review Comment:
Indeed, it's not good inside. If closing cache throws, then it will be hidden.
I moved it back out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : storage : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-4189264657
@Shekharrajak would you like to take a look ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-4982574305
@smjn Can you take it up please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub) via jira" <ji...@kafka.apache.org>.
apoorvmittal10 commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4562256552
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -535,9 +535,9 @@ public void testReloadCacheAfterClose() throws IOException, RemoteStorageExcepti
cache.close();
Review Comment:
It's fine then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4121703757
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
+ } finally {
+ // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
+ // will be deleted at the end of test.
+ try {
+ Utils.delete(logDir);
+ } catch (IOException ioe) {
+ // ignore
+ }
+ // Verify no lingering threads. It is important to have this as the very last statement in the @AfterEach
+ // because this may throw an exception and prevent cleanup after it
+ TestUtils.assertNoLeakedThreadsWithNameAndDaemonStatus(REMOTE_LOG_INDEX_CACHE_CLEANER_THREAD, true);
Review Comment:
Yes, I have updated pr desc. .There is no resource leak in closeQuietly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub) via jira" <ji...@kafka.apache.org>.
apoorvmittal10 commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4556925294
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -535,9 +535,9 @@ public void testReloadCacheAfterClose() throws IOException, RemoteStorageExcepti
cache.close();
Review Comment:
Do you still require this or shall be handled in `cleanup` as mentioned by you in comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub) via jira" <ji...@kafka.apache.org>.
apoorvmittal10 merged PR #21964:
URL: https://github.com/apache/kafka/pull/21964
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-5869150379
> It checks the scheduler's queue size (pendingTaskSize()), which hits 0 as soon as a worker dequeues the task but does it guarantee that cleanup actually finished?
I removed the expiredIdxPendingForDeletion() == 0 wait from testConcurrentCacheDeletedFileExists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub) via jira" <ji...@kafka.apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-6804294917
> I do not find the added check, am I missing something?
Sorry, those edits got lost when I squashed. I've pushed them now: cache.close() is guarded with a null check.
I think in my other PRs, I will push new commits, instead of amends. Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub) via jira" <ji...@kafka.apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-6804259845
> @muralibasani Please avoid doing forced pushes, it makes it harder to track changes. Though this PR is very small hence will not be that concerning but thought to let you know for future.
Sure, will do. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : storage : Fix resource leak, cleanup [kafka]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r3037904547
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -1044,6 +1049,11 @@ public void testConcurrentCacheDeletedFileExists() throws InterruptedException,
"Time index file should not be present on disk at " + remoteIndexCacheDir.toPath());
TestUtils.waitForCondition(() -> getIndexFileFromRemoteCacheDir(cache, LogFileUtils.DELETED_FILE_SUFFIX).isEmpty(),
"Index file marked for deletion should not be present on disk at " + remoteIndexCacheDir.toPath());
+
+ // Wait for all scheduled cleanup tasks to complete so the cleaner thread is idle before @AfterEach
+ // shuts it down. This prevents a race between an in-flight task and scheduler shutdown.
+ TestUtils.waitForCondition(() -> cache.expiredIdxPendingForDeletion() == 0,
Review Comment:
👍🏼
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4121713091
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
Review Comment:
Yes, it could throw npe. Added a non-null check now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub) via jira" <ji...@kafka.apache.org>.
apoorvmittal10 commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-6804040467
@muralibasani Please avoid doing forced pushes, it makes it harder to track changes. Though this PR is very small hence will not be that concerning but thought to let you know for future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4115539955
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
+ } finally {
+ // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
+ // will be deleted at the end of test.
+ try {
+ Utils.delete(logDir);
+ } catch (IOException ioe) {
+ // ignore
+ }
+ // Verify no lingering threads. It is important to have this as the very last statement in the @AfterEach
+ // because this may throw an exception and prevent cleanup after it
+ TestUtils.assertNoLeakedThreadsWithNameAndDaemonStatus(REMOTE_LOG_INDEX_CACHE_CLEANER_THREAD, true);
Review Comment:
Any reason of moving this inside the try block now? Can live outside a earlier, right?
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -1044,6 +1049,11 @@ public void testConcurrentCacheDeletedFileExists() throws InterruptedException,
"Time index file should not be present on disk at " + remoteIndexCacheDir.toPath());
TestUtils.waitForCondition(() -> getIndexFileFromRemoteCacheDir(cache, LogFileUtils.DELETED_FILE_SUFFIX).isEmpty(),
"Index file marked for deletion should not be present on disk at " + remoteIndexCacheDir.toPath());
+
+ // Wait for all scheduled cleanup tasks to complete so the cleaner thread is idle before @AfterEach
+ // shuts it down. This prevents a race between an in-flight task and scheduler shutdown.
+ TestUtils.waitForCondition(() -> cache.expiredIdxPendingForDeletion() == 0,
Review Comment:
It checks the scheduler's queue size (pendingTaskSize()), which hits 0 as soon as a worker dequeues the task but does it guarantee that cleanup actually finished?
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
+ } finally {
+ // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
+ // will be deleted at the end of test.
+ try {
+ Utils.delete(logDir);
+ } catch (IOException ioe) {
+ // ignore
+ }
+ // Verify no lingering threads. It is important to have this as the very last statement in the @AfterEach
+ // because this may throw an exception and prevent cleanup after it
+ TestUtils.assertNoLeakedThreadsWithNameAndDaemonStatus(REMOTE_LOG_INDEX_CACHE_CLEANER_THREAD, true);
Review Comment:
If cache.close throws then still there can be a leak, correct?
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
Review Comment:
Do we require null check for cache? If setup i.e. before each fails before cache is instantiated then can there be a NPE here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-5821187130
> @smjn Can you take it up please?
@apoorvmittal10 possible to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #21964:
URL: https://github.com/apache/kafka/pull/21964#issuecomment-6126555902
@apoorvmittal10 or @smjn can you take another look ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "apoorvmittal10 (via GitHub) via jira" <ji...@kafka.apache.org>.
apoorvmittal10 commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4541465768
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -127,20 +127,25 @@ public void setup() throws IOException, RemoteStorageException {
}
@AfterEach
- public void cleanup() throws InterruptedException {
+ public void cleanup() throws Exception {
reset(rsm);
- // the files created for the test will be deleted automatically on thread exit since we use temp dir
- Utils.closeQuietly(cache, "RemoteIndexCache created for unit test");
- // best effort to delete the per-test resource. Even if we don't delete, it is ok because the parent directory
- // will be deleted at the end of test.
+ // Close the cache directly instead of using closeQuietly to ensure any exception during close
+ // (e.g. failure to shut down the cleaner scheduler) is properly propagated rather than silently swallowed,
+ // which could leave the cleaner thread alive and cause the leaked thread assertion below to fail.
try {
- Utils.delete(logDir);
- } catch (IOException ioe) {
- // ignore
+ cache.close();
Review Comment:
I do not find the added check, am I missing something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19483 : Fix resource leak, cleanup [kafka]
Posted by "muralibasani (via GitHub) via jira" <ji...@kafka.apache.org>.
muralibasani commented on code in PR #21964:
URL: https://github.com/apache/kafka/pull/21964#discussion_r4559710825
##########
storage/src/test/java/org/apache/kafka/storage/internals/log/RemoteIndexCacheTest.java:
##########
@@ -535,9 +535,9 @@ public void testReloadCacheAfterClose() throws IOException, RemoteStorageExcepti
cache.close();
Review Comment:
Yes, it is still needed. The close in cleanup() is only a safety net if the test fails earlier.
If you prefer, I can add a comment above this line like "Close the cache before reloading to simulate a restart."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org