You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/09/01 09:23:51 UTC
[GitHub] [pulsar] HQebupt opened a new pull request, #17394: [fix][ml]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
HQebupt opened a new pull request, #17394:
URL: https://github.com/apache/pulsar/pull/17394
### Motivation
There is a redundant releases in `org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl#beforeAddEntry` at line:853.
It will cause the error log as follows when there is an exception in `ManagedLedgerInterceptor#beforeAddEntry`.
```
2022-09-01T16:07:04,851 - ERROR - [test-OrderedScheduler-1-0:SafeRunnable@38] - Unexpected throwable caught
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:83) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:147) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:101) ~[netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:90) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.beforeAddEntry(ManagedLedgerImpl.java:854) ~[classes/:?]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.internalAsyncAddEntry(ManagedLedgerImpl.java:774) ~[classes/:?]
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$asyncAddEntry$4(ManagedLedgerImpl.java:769) ~[classes/:?]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[classes/:?]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) ~[bookkeeper-common-4.15.0.jar:4.15.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
### Modifications
- Remove duplicate release in `ManagedLedgerInterceptor`
### Verifying this change
- [x] Make sure that the change passes the CI checks.
This change is a trivial rework / code cleanup without any test coverage.
### Does this pull request potentially affect one of the following parts:
If `yes` was chosen, please highlight the changes
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
- The rest endpoints: (no)
- The admin cli options: (no)
- Anything that affects deployment: (no)
### Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?
- [x] `doc-not-needed`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1234420463
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on a diff in pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
Jason918 commented on code in PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#discussion_r960480279
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/intercept/MangedLedgerInterceptorImplTest.java:
##########
@@ -256,6 +257,29 @@ public void testFindPositionByIndex() throws Exception {
ledger.close();
}
+ @Test
+ public void testBeforeAddEntryWithException() throws Exception {
+ final int MOCK_BATCH_SIZE = 2;
+ final String ledgerAndCursorName = "testBeforeAddEntryWithException";
+
+ ManagedLedgerInterceptor interceptor =
+ new MockManagedLedgerInterceptorImpl(getBrokerEntryMetadataInterceptors(), null);
Review Comment:
Better just use an anonymous class here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
mattisonchao commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1244819267
It looks like we got many conflicts when cherry-picking it to branch-2.9.
Would you mind pushing a PR to branch-2.9? (To avoid cherry-picking involving bugs)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1235381793
@codelipenghui @Technoboy- PTAL, thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AnonHxy merged pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
AnonHxy merged PR #17394:
URL: https://github.com/apache/pulsar/pull/17394
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on a diff in pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
AnonHxy commented on code in PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#discussion_r963215655
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/intercept/MangedLedgerInterceptorImplTest.java:
##########
@@ -256,6 +257,29 @@ public void testFindPositionByIndex() throws Exception {
ledger.close();
}
+ @Test
+ public void testBeforeAddEntryWithException() throws Exception {
Review Comment:
> There is no such error log with the bug fix in the test.
This will not prevent regression. The test will alwalys pass no matter duplicate release or not.
How about verifying the `Bytebuf.refCnt()` like below ? @HQebupt
```suggestion
ByteBuf buffer = Unpooled.wrappedBuffer("message".getBytes());
ManagedLedger ledger = factory.open(ledgerAndCursorName, config);
CountDownLatch countDownLatch = new CountDownLatch(1);
ledger.asyncAddEntry(buffer, new AsyncCallbacks.AddEntryCallback() {
@Override
public void addComplete(Position position, ByteBuf entryData, Object ctx) {
countDownLatch.countDown();
}
@Override
public void addFailed(ManagedLedgerException exception, Object ctx) {
countDownLatch.countDown();
}
}, null);
countDownLatch.await();
assertEquals(buffer.refCnt(), 1);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1234188225
@HQebupt Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1235165735
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][ml]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1234010145
@Jason918 @HQebupt @codelipenghui @Technoboy- @AnonHxy PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1235108900
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1234987617
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1239263492
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on a diff in pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on code in PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#discussion_r963602235
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/intercept/MangedLedgerInterceptorImplTest.java:
##########
@@ -256,6 +257,29 @@ public void testFindPositionByIndex() throws Exception {
ledger.close();
}
+ @Test
+ public void testBeforeAddEntryWithException() throws Exception {
Review Comment:
Great! I made a minor modification. PTAL @AnonHxy thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on a diff in pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
AnonHxy commented on code in PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#discussion_r962513000
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/intercept/MangedLedgerInterceptorImplTest.java:
##########
@@ -256,6 +257,29 @@ public void testFindPositionByIndex() throws Exception {
ledger.close();
}
+ @Test
+ public void testBeforeAddEntryWithException() throws Exception {
Review Comment:
I wander how this test can verify the duplicate release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on a diff in pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on code in PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#discussion_r963195717
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/intercept/MangedLedgerInterceptorImplTest.java:
##########
@@ -256,6 +257,29 @@ public void testFindPositionByIndex() throws Exception {
ledger.close();
}
+ @Test
+ public void testBeforeAddEntryWithException() throws Exception {
Review Comment:
There is no such error log with the bug fix in the test. Since the following error can not interrupt the normal execution, so it is hard to cover the error in a test case. Do you have any idea?
```
ERROR - [test-OrderedScheduler-1-0:SafeRunnable@38] - Unexpected throwable caught
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] HQebupt commented on pull request #17394: [fix][storage]fix OpAddEntry release error when exception in ManagedLedgerInterceptor
Posted by GitBox <gi...@apache.org>.
HQebupt commented on PR #17394:
URL: https://github.com/apache/pulsar/pull/17394#issuecomment-1238877464
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org