You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "pdolif (via GitHub)" <gi...@apache.org> on 2025/08/15 18:49:31 UTC
[PR] [fix][test] Fix ConcurrentModificationException in Ipv4Proxy [pulsar]
pdolif opened a new pull request, #24632:
URL: https://github.com/apache/pulsar/pull/24632
Fixes #24627
### Motivation
As shared in #24627, ZkSessionExpireTest is flaky. In the cleanup method, the main thread tries to close a PulsarService but then remains in WAITING state, and the test runs into the one hour timeout.
The logs shared in the issue contain a ConcurrentModificationException:
```
java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1095)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1049)
at org.apache.pulsar.broker.service.Ipv4Proxy.disconnectFrontChannels(Ipv4Proxy.java:87)
at org.apache.pulsar.broker.service.ZkSessionExpireTest.testTopicUnloadAfterSessionRebuild(ZkSessionExpireTest.java:149)
```
The ZkSessionExpireTest calls `Ipv4Proxy.disconnectFrontChannels()` to inject a ZK session expire error:
```java
metadataZKProxy.rejectAllConnections();
metadataZKProxy.disconnectFrontChannels();
```
The method `disconnectFrontChannels()` looks as follows:
```java
for (Channel channel : frontChannels) {
channel.close();
}
```
When `channel.close()` is called, it triggers `FrontendHandler.channelInactive()`, which removes the channel from the `frontChannels` list:
```java
@Override
public void channelInactive(ChannelHandlerContext ctx) {
frontChannels.remove(ctx.channel());
...
}
```
This can trigger the ConcurrentModificationException.
If the ConcurrentModificationException is thrown, the test execution stops. We don't reach the point of the test where the `rejectAllConnections` is reverted. Therefore, the cleanup method cannot close the PulsarService properly.
I was able to replicate the ConcurrentModificationException by inserting a `Thread.sleep()` into `disconnectFrontChannels()`:
```java
public void disconnectFrontChannels() throws InterruptedException {
for (Channel channel : frontChannels) {
Thread.sleep(5);
channel.close();
}
}
```
<!-- Explain here the context, and why you're making that change. What is the problem you're trying to solve. -->
### Modifications
Prevent ConcurrentModificationException by iterating over a copy of `frontChannels`.
### Verifying this change
- [x] Make sure that the change passes the CI checks.
When adding the `Thread.sleep()`, the ConcurrentModificationException was always thrown. After changing the for-loop to iterate over a copy of `frontChannels`, even with the `Thread.sleep()`, no ConcurrentModificationException occurs.
### Does this pull request potentially affect one of the following parts:
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: https://github.com/pdolif/pulsar/pull/13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][test] Fix ConcurrentModificationException in Ipv4Proxy [pulsar]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #24632:
URL: https://github.com/apache/pulsar/pull/24632#issuecomment-3192632725
## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/24632?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 74.24%. Comparing base ([`ba4f715`](https://app.codecov.io/gh/apache/pulsar/commit/ba4f715de39d3b087b57d8339490849d6d8108ec?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`ffd6384`](https://app.codecov.io/gh/apache/pulsar/commit/ffd63841ec61837422f106d8f2436603b6405751?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
[](https://app.codecov.io/gh/apache/pulsar/pull/24632?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
```diff
@@ Coverage Diff @@
## master #24632 +/- ##
============================================
+ Coverage 73.56% 74.24% +0.68%
- Complexity 32894 33153 +259
============================================
Files 1882 1882
Lines 146854 146854
Branches 16866 16866
============================================
+ Hits 108030 109035 +1005
+ Misses 30153 29157 -996
+ Partials 8671 8662 -9
```
| [Flag](https://app.codecov.io/gh/apache/pulsar/pull/24632/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [inttests](https://app.codecov.io/gh/apache/pulsar/pull/24632/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `26.64% <ø> (?)` | |
| [systests](https://app.codecov.io/gh/apache/pulsar/pull/24632/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `23.31% <ø> (+0.01%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/apache/pulsar/pull/24632/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.74% <ø> (+0.62%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
[see 148 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/24632/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][test] Fix ConcurrentModificationException in Ipv4Proxy [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari merged PR #24632:
URL: https://github.com/apache/pulsar/pull/24632
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org