You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2023/07/13 11:17:20 UTC
[GitHub] [pulsar] lhotari opened a new issue, #20797: [Tests] Thread resource leaks in unit tests
lhotari opened a new issue, #20797:
URL: https://github.com/apache/pulsar/issues/20797
Thread leaks:
```
❯ grep -shEr "created [0-9]+ new threads" pulsar-broker | awk -F "Summary: " '{ print $2 }' | awk '{ print $(NF-2), $0}' |sort -rn | cut -f2- -d' '
Tests in class org.apache.pulsar.broker.service.ServerCnxTest created 849 new threads
Tests in class org.apache.pulsar.broker.service.ClusterMigrationTest created 743 new threads
Tests in class org.apache.pulsar.broker.transaction.AuthenticatedTransactionProducerConsumerTest created 100 new threads
Tests in class org.apache.pulsar.broker.loadbalance.LoadBalancerTest created 92 new threads
Tests in class org.apache.pulsar.broker.loadbalance.MultiBrokerLeaderElectionTest created 89 new threads
Tests in class org.apache.pulsar.broker.loadbalance.MultiBrokerLeaderElectionExpirationTest created 81 new threads
Tests in class org.apache.pulsar.broker.transaction.pendingack.PendingAckInMemoryDeleteTest created 78 new threads
Tests in class org.apache.pulsar.broker.transaction.buffer.TransactionStablePositionTest created 78 new threads
Tests in class org.apache.pulsar.broker.service.MessageCumulativeAckTest created 62 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorTest created 61 new threads
Tests in class org.apache.pulsar.broker.service.BrokerBookieIsolationTest created 51 new threads
Tests in class org.apache.pulsar.broker.loadbalance.SimpleLoadManagerImplTest created 51 new threads
Tests in class org.apache.pulsar.broker.loadbalance.ModularLoadManagerImplTest created 47 new threads
Tests in class org.apache.pulsar.broker.service.TopicOwnerTest created 40 new threads
Tests in class org.apache.pulsar.broker.service.BrokerServiceTest created 37 new threads
Tests in class org.apache.pulsar.broker.service.BrokerBkEnsemblesTests created 32 new threads
Tests in class org.apache.pulsar.broker.SLAMonitoringTest created 31 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorAdminTlsTest created 26 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorAdminTlsWithKeyStoreTest created 25 new threads
Tests in class org.apache.pulsar.broker.service.persistent.PersistentSubscriptionTest created 25 new threads
Tests in class org.apache.pulsar.broker.service.PeerReplicatorTest created 24 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorSubscriptionTest created 23 new threads
Tests in class org.apache.pulsar.broker.service.MaxMessageSizeTest created 23 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorTopicPoliciesTest created 22 new threads
Tests in class org.apache.pulsar.broker.service.OneWayReplicatorTest created 20 new threads
Tests in class org.apache.pulsar.broker.service.schema.SchemaServiceTest created 18 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorTlsTest created 18 new threads
Tests in class org.apache.pulsar.broker.service.ReplicatorRemoveClusterTest created 18 new threads
Tests in class org.apache.pulsar.broker.service.BacklogQuotaManagerTest created 17 new threads
Tests in class org.apache.pulsar.broker.service.schema.PartitionedTopicsSchemaTest created 16 new threads
Tests in class org.apache.pulsar.broker.loadbalance.SimpleBrokerStartTest created 13 new threads
Tests in class org.apache.pulsar.broker.loadbalance.LeaderElectionServiceTest created 13 new threads
Tests in class org.apache.pulsar.broker.loadbalance.impl.BundleSplitterTaskTest created 13 new threads
Tests in class org.apache.pulsar.broker.auth.AuthorizationTest created 12 new threads
Tests in class org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumersTest created 11 new threads
Tests in class org.apache.pulsar.broker.loadbalance.extensions.scheduler.UnloadSchedulerTest created 10 new threads
Tests in class org.apache.pulsar.broker.intercept.MangedLedgerInterceptorImplTest created 10 new threads
Tests in class org.apache.pulsar.broker.service.OpportunisticStripingTest created 9 new threads
Tests in class org.apache.pulsar.broker.service.ManagedLedgerCompressionTest created 9 new threads
Tests in class org.apache.pulsar.broker.loadbalance.extensions.AntiAffinityNamespaceGroupExtensionTest created 8 new threads
Tests in class org.apache.pulsar.broker.stats.ManagedCursorMetricsTest created 7 new threads
Tests in class org.apache.pulsar.broker.service.BusyWaitServiceTest created 7 new threads
Tests in class org.apache.pulsar.broker.loadbalance.extensions.BrokerRegistryTest created 7 new threads
Tests in class org.apache.pulsar.broker.web.WebServiceTest created 6 new threads
Tests in class org.apache.pulsar.broker.service.PersistentMessageFinderTest created 6 new threads
Tests in class org.apache.pulsar.broker.loadbalance.extensions.channel.ServiceUnitStateChannelTest created 6 new threads
Tests in class org.apache.pulsar.broker.BrokerAdditionalServletTest created 6 new threads
Tests in class org.apache.pulsar.broker.service.PersistentFailoverE2ETest created 5 new threads
Tests in class org.apache.pulsar.broker.service.NonPersistentTopicE2ETest created 5 new threads
Tests in class org.apache.pulsar.broker.service.InactiveTopicDeleteTest created 5 new threads
Tests in class org.apache.pulsar.broker.service.ExclusiveProducerTest created 5 new threads
```
_Originally posted by @lhotari in https://github.com/apache/pulsar/issues/20673#issuecomment-1633958559_
In some cases, it's possible that detected thread leaks are false positives.
One reason for that is that broker shutdown is asynchronous in tests because of performance reasons. Broker shutdown can be made to wait for completion in tests by setting `svcConfig.setBrokerShutdownTimeoutMs(5000L);`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20797: [Tests] Thread resource leaks in unit tests
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #20797:
URL: https://github.com/apache/pulsar/issues/20797#issuecomment-1634081930
Simple way to reproduce the thread leak in ServerCnxTest by executing a single test method:
```
mvn -DredirectTestOutputToFile=false -DtestRetryCount=0 test -pl pulsar-broker -Dtest=ServerCnxTest#testConnectCommandWithEnum
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari closed issue #20797: [Tests] Thread resource leaks in unit tests
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari closed issue #20797: [Tests] Thread resource leaks in unit tests
URL: https://github.com/apache/pulsar/issues/20797
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #20797: [Tests] Thread resource leaks in unit tests
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #20797:
URL: https://github.com/apache/pulsar/issues/20797#issuecomment-1634097234
I found the root cause. Fixing this. /cc @dave2wave
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org