You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "arturobernalg (via GitHub)" <gi...@apache.org> on 2026/03/06 09:27:09 UTC

[PR] Fix lease timeout race to prevent pool entry leak [httpcomponents-core]

arturobernalg opened a new pull request, #649:
URL: https://github.com/apache/httpcomponents-core/pull/649

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix lease timeout race to prevent pool entry leak [httpcomponents-core]

Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#issuecomment-4013162004

   @arturobernalg Can we also try to get this test case to work? 
   
   Interestingly enough, `OFFLOCK` is the only pool implementation that appears to be able to handle the test correctly. Kudos to you, But it can also be my test case is flawed, so let's be diligent. 
   
   ```
       @ParameterizedTest(name = "{0}")
       @MethodSource("pools")
       @org.junit.jupiter.api.Timeout(60)
       void testInterruptDoesNotLeakLeasedEntries(final PoolCase poolCase) throws Exception {
           final ManagedConnPool<String, DummyConn> pool = poolCase.supplier.get();
   
           final String route = "route-1";
   
           final int concurrentThreads = 10;
   
           final ExecutorService executorService = Executors.newFixedThreadPool(concurrentThreads);
           final AtomicReference<Exception> unexpectedException = new AtomicReference<>();
           try {
               final Queue<Future<?>> taskQueue = new ConcurrentLinkedQueue<>();
               for (int i = 0; i < concurrentThreads * 100; i++) {
                   taskQueue.add(executorService.submit(() -> {
                       final Future<PoolEntry<String, DummyConn>> f = pool.lease(route, null, TIMEOUT, null);
                       try {
                           final PoolEntry<String, DummyConn> entry =
                                   f.get(TIMEOUT.getDuration(), TIMEOUT.getTimeUnit());
                           pool.release(entry, true);
                       } catch (final InterruptedException ex) {
                           Thread.currentThread().interrupt();
                       } catch (final ExecutionException | TimeoutException ex) {
                           f.cancel(true);
                           unexpectedException.compareAndSet(null, ex);
                       }
                   }));
   
               }
   
               for (;;) {
                   final Future<?> future = taskQueue.poll();
                   if (future != null) {
                       future.cancel(true);
                   } else {
                       break;
                   }
               }
   
               executorService.shutdown();
               Assertions.assertTrue(executorService.awaitTermination(TIMEOUT.getDuration(), TIMEOUT.getTimeUnit()));
               Assertions.assertNull(unexpectedException.get());
   
               final PoolStats stats = pool.getStats(route);
               Assertions.assertEquals(0, stats.getLeased());
           } finally {
               executorService.shutdownNow();
               pool.close(CloseMode.GRACEFUL);
           }
       }
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix connection lease timeout race to prevent connection pool entry leak [httpcomponents-core]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#issuecomment-4016275289

   > cherry
   
   on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix connection lease timeout race to prevent connection pool entry leak [httpcomponents-core]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#issuecomment-4016317950

   > > cherry
   > 
   > on it.
   
   Done


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix lease timeout race to prevent pool entry leak [httpcomponents-core]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on code in PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#discussion_r2897095691


##########
httpcore5/src/test/java/org/apache/hc/core5/pool/TestStrictConnPoolLeaseTimeoutRace.java:
##########
@@ -0,0 +1,153 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation.  For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.core5.pool;
+
+import java.io.IOException;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.function.Supplier;
+import java.util.stream.Stream;
+
+import org.apache.hc.core5.http.SocketModalCloseable;
+import org.apache.hc.core5.io.CloseMode;
+import org.apache.hc.core5.util.TimeValue;
+import org.apache.hc.core5.util.Timeout;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+class TestConnPoolLeaseTimeout {
+
+    private static final Timeout TIMEOUT = Timeout.ofSeconds(30);
+
+    static final class DummyConn implements SocketModalCloseable {
+
+        private volatile Timeout socketTimeout;
+
+        @Override
+        public Timeout getSocketTimeout() {
+            return socketTimeout;
+        }
+
+        @Override
+        public void setSocketTimeout(final Timeout timeout) {
+            this.socketTimeout = timeout;
+        }
+
+        @Override
+        public void close(final CloseMode closeMode) {
+        }
+
+        @Override
+        public void close() throws IOException {
+        }
+    }
+
+    static final class PoolCase {
+        final String name;
+        final Supplier<ManagedConnPool<String, DummyConn>> supplier;
+
+        PoolCase(final String name, final Supplier<ManagedConnPool<String, DummyConn>> supplier) {
+            this.name = name;
+            this.supplier = supplier;
+        }
+
+        @Override
+        public String toString() {
+            return name;
+        }
+    }
+
+    static Stream<PoolCase> pools() {
+        return Stream.of(
+                new PoolCase("STRICT", () -> new StrictConnPool<>(1, 1)),
+                new PoolCase("LAX", () -> new LaxConnPool<>(1)),
+                new PoolCase("OFFLOCK", () -> new RouteSegmentedConnPool<>(
+                        1,
+                        1,
+                        TimeValue.NEG_ONE_MILLISECOND,
+                        PoolReusePolicy.LIFO,
+                        new DefaultDisposalCallback<>()))
+        );
+    }
+
+    @ParameterizedTest(name = "{0}")

Review Comment:
   @ok2c I just updated the test with your suggestions.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix lease timeout race to prevent pool entry leak [httpcomponents-core]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#issuecomment-4011338385

   > You see how much smaller and cleaner the fix is when applied at the right place. Let me poke at your changes a bit more over the weekend but fundamentally looks OK.
   
   
   
   > @arturobernalg You see how much smaller and cleaner the fix is when applied at the right place. Let me poke at your changes a bit more over the weekend but fundamentally looks OK.
   
   @ok2c Indeed. make more sense


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix lease timeout race to prevent pool entry leak [httpcomponents-core]

Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#issuecomment-4011112519

   @arturobernalg You see how much smaller and cleaner the fix is when applied at the right place. Let me poke at your changes a bit more over the weekend but fundamentally looks OK.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix connection lease timeout race to prevent connection pool entry leak [httpcomponents-core]

Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg merged PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Re: [PR] Fix lease timeout race to prevent pool entry leak [httpcomponents-core]

Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on code in PR #649:
URL: https://github.com/apache/httpcomponents-core/pull/649#discussion_r2896983681


##########
httpcore5/src/test/java/org/apache/hc/core5/pool/TestStrictConnPoolLeaseTimeoutRace.java:
##########
@@ -0,0 +1,153 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation.  For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.core5.pool;
+
+import java.io.IOException;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.function.Supplier;
+import java.util.stream.Stream;
+
+import org.apache.hc.core5.http.SocketModalCloseable;
+import org.apache.hc.core5.io.CloseMode;
+import org.apache.hc.core5.util.TimeValue;
+import org.apache.hc.core5.util.Timeout;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+
+class TestConnPoolLeaseTimeout {
+
+    private static final Timeout TIMEOUT = Timeout.ofSeconds(30);
+
+    static final class DummyConn implements SocketModalCloseable {
+
+        private volatile Timeout socketTimeout;
+
+        @Override
+        public Timeout getSocketTimeout() {
+            return socketTimeout;
+        }
+
+        @Override
+        public void setSocketTimeout(final Timeout timeout) {
+            this.socketTimeout = timeout;
+        }
+
+        @Override
+        public void close(final CloseMode closeMode) {
+        }
+
+        @Override
+        public void close() throws IOException {
+        }
+    }
+
+    static final class PoolCase {
+        final String name;
+        final Supplier<ManagedConnPool<String, DummyConn>> supplier;
+
+        PoolCase(final String name, final Supplier<ManagedConnPool<String, DummyConn>> supplier) {
+            this.name = name;
+            this.supplier = supplier;
+        }
+
+        @Override
+        public String toString() {
+            return name;
+        }
+    }
+
+    static Stream<PoolCase> pools() {
+        return Stream.of(
+                new PoolCase("STRICT", () -> new StrictConnPool<>(1, 1)),
+                new PoolCase("LAX", () -> new LaxConnPool<>(1)),
+                new PoolCase("OFFLOCK", () -> new RouteSegmentedConnPool<>(
+                        1,
+                        1,
+                        TimeValue.NEG_ONE_MILLISECOND,
+                        PoolReusePolicy.LIFO,
+                        new DefaultDisposalCallback<>()))
+        );
+    }
+
+    @ParameterizedTest(name = "{0}")

Review Comment:
   @arturobernalg Let's improve the test case a bit
   
   ```
       @ParameterizedTest(name = "{0}")
       @MethodSource("pools")
       @org.junit.jupiter.api.Timeout(60)
       void testLeaseTimeoutDoesNotLeakLeasedEntries(final PoolCase poolCase) throws Exception {
           final ManagedConnPool<String, DummyConn> pool = poolCase.supplier.get();
   
           final String route = "route-1";
           final Timeout requestTimeout = Timeout.ofMicroseconds(1);
   
           final int concurrentThreads = 10;
           final CountDownLatch countDownLatch = new CountDownLatch(concurrentThreads);
           final AtomicLong n = new AtomicLong(concurrentThreads * 100);
   
           final ExecutorService executorService = Executors.newFixedThreadPool(concurrentThreads);
           final AtomicReference<Exception> unexpectedException = new AtomicReference<>();
           try {
               for (int i = 0; i < concurrentThreads; i++) {
                   executorService.execute(() -> {
                       try {
                           while (n.decrementAndGet() > 0) {
                               final Future<PoolEntry<String, DummyConn>> f = pool.lease(route, null, requestTimeout, null);
                               try {
                                   final PoolEntry<String, DummyConn> entry =
                                           f.get(requestTimeout.getDuration(), requestTimeout.getTimeUnit());
                                   pool.release(entry, true);
                               } catch (final InterruptedException ex) {
                                   Thread.currentThread().interrupt();
                                   unexpectedException.compareAndSet(null, ex);
                               } catch (final TimeoutException ex) {
                                   f.cancel(true);
                               } catch (final ExecutionException ex) {
                                   f.cancel(true);
                                   if (!(ex.getCause() instanceof TimeoutException)) {
                                       unexpectedException.compareAndSet(null, ex);
                                   }
                               }
                           }
                       } finally {
                           countDownLatch.countDown();
                       }
                   });
               }
   
               Assertions.assertTrue(countDownLatch.await(TIMEOUT.getDuration(), TIMEOUT.getTimeUnit()));
               Assertions.assertTrue(n.get() <= 0);
               Assertions.assertNull(unexpectedException.get());
   
               final PoolStats stats = pool.getStats(route);
               Assertions.assertEquals(0, stats.getLeased());
   
           } finally {
               executorService.shutdownNow();
               executorService.awaitTermination(TIMEOUT.getDuration(), TIMEOUT.getTimeUnit());
               pool.close(CloseMode.GRACEFUL);
           }
       }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org