You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/04/13 19:26:41 UTC
[GitHub] [pulsar] devinbost opened a new pull request #10217: Adding more permits debug statements to better diagnose permit issues
devinbost opened a new pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217
This PR just adds debug statements whenever the broker updates its permits in `broker.service.Consumer` and in `PersistentDispatcherMultipleConsumers`. It also adds batch sizes to some of the debug log statements.
That will help with the forensic analysis of problematic permit behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822322409
I am trying to cherry pick to branch-7.2 and see if it builds properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
eolivelli merged pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-820136551
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-821852351
> /pulsarbot run-failure-checks
@devinbost please rebase your changes. The master branch contains fixes to CI issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-821801231
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822024711
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-821801288
@lhotari are you okay with this PR now ?
it looks like your comment has been addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-821848018
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-821012634
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822335349
@devinbost picked to branch-2.7 as c243d185261e964010a122b65f9c730bffbb2f0c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on a change in pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on a change in pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#discussion_r612940010
##########
File path: docker/build.sh
##########
@@ -23,3 +23,4 @@ cd $ROOT_DIR/docker
mvn -f ../dashboard/pom.xml package -Pdocker
mvn package -Pdocker
+PersistentDispatcherMultipleConsumersPersistentDispatcherMultipleConsumers
Review comment:
Whoops. Not sure how that got there. Should be fixed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on a change in pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
lhotari commented on a change in pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#discussion_r612935875
##########
File path: docker/build.sh
##########
@@ -23,3 +23,4 @@ cd $ROOT_DIR/docker
mvn -f ../dashboard/pom.xml package -Pdocker
mvn package -Pdocker
+PersistentDispatcherMultipleConsumersPersistentDispatcherMultipleConsumers
Review comment:
Please remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-821724858
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822042241
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822113883
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-818996925
@merlimat @eolivelli Since it's not easy to get debug logs from functions due to https://github.com/apache/pulsar/issues/3259, I added more logs to the broker instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822147411
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
merlimat commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-819081766
@devinbost There are few checkstyle violations reported in the CI jobs:
```
Error: src/main/java/org/apache/pulsar/broker/service/PulsarCommandSenderImpl.java:[254] (sizes) LineLength: Line is longer than 120 characters (found 130).
5726
Error: src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java:[189,119] (whitespace) OperatorWrap: '+' should be on a new line.
5727
Error: src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java:[212,104] (whitespace) OperatorWrap: '+' should be on a new line.
5728
Error: src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java:[496,98] (whitespace) OperatorWrap: '+' should be on a new line.
5729
Error: src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java:[529,17] (whitespace) WhitespaceAround: 'if' is not followed by whitespace.
5730
Error: src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java:[530,106] (whitespace) OperatorWrap: '+' should be on a new line.
5731
Error: src/main/java/org/apache/pulsar/broker/service/Consumer.java:[239,21] (whitespace) WhitespaceAround: 'if' is not followed by whitespace.
5732
Error: src/main/java/org/apache/pulsar/broker/service/Consumer.java:[240,109] (whitespace) OperatorWrap: '+' should be on a new line.
5733
Error: src/main/java/org/apache/pulsar/broker/service/Consumer.java:[257,9] (whitespace) WhitespaceAround: 'if' is not followed by whitespace.
5734
Error: src/main/java/org/apache/pulsar/broker/service/Consumer.java:[258,114] (whitespace) OperatorWrap: '+' should be on a new line.
5735
Error: src/main/java/org/apache/pulsar/broker/service/Consumer.java:[531,117] (whitespace) OperatorWrap: '+' should be on a new line.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-819764573
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-820428765
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822397831
> Looks like we missed a flaky test. @lhotari
> org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest created 1 new threads
> 2021-04-17 20:21:28,701 - INFO - [main:FastThreadLocalCleanupListener@53] - Cleaning up FastThreadLocal thread local state.
> Error: Tests run: 9, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 382.475 s <<< FAILURE! - in TestSuite
> Error: testGenericObjectSink(org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest) Time elapsed: 47.627 s <<< FAILURE!
> java.lang.AssertionError: Sink was restarted, probably an error occurred expected [true] but found [false]
@devinbost this is fixed in #10248
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822025225
Looks like we missed a flaky test. @lhotari
org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest created 1 new threads
2021-04-17 20:21:28,701 - INFO - [main:FastThreadLocalCleanupListener@53] - Cleaning up FastThreadLocal thread local state.
Error: Tests run: 9, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 382.475 s <<< FAILURE! - in TestSuite
Error: testGenericObjectSink(org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest) Time elapsed: 47.627 s <<< FAILURE!
java.lang.AssertionError: Sink was restarted, probably an error occurred expected [true] but found [false]
at org.testng.Assert.fail(Assert.java:99)
at org.testng.Assert.failNotEquals(Assert.java:1037)
at org.testng.Assert.assertTrue(Assert.java:45)
at org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest.testGenericObjectSink(PulsarGenericObjectSinkTest.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO]
[INFO] Results:
[INFO]
Error: Failures:
Error: org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest.testGenericObjectSink(org.apache.pulsar.tests.integration.io.PulsarGenericObjectSinkTest)
[INFO] Run 1: PASS
Error: Run 2: PulsarGenericObjectSinkTest.testGenericObjectSink:142 Sink was restarted, probably an error occurred expected [true] but found [false]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-819200571
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on pull request #10217: Adding more permits debug statements to better diagnose permit issues
Posted by GitBox <gi...@apache.org>.
devinbost commented on pull request #10217:
URL: https://github.com/apache/pulsar/pull/10217#issuecomment-822054891
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org