You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/03 19:19:01 UTC

[GitHub] [pulsar] lifepuzzlefun opened a new pull request, #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

lifepuzzlefun opened a new pull request, #18730:
URL: https://github.com/apache/pulsar/pull/18730

   
   
   <!-- Either this PR fixes an issue, -->
   
   Fixes #18705
   
   ### Motivation
   
   awaitUntilShadowReplicatorReady check is returned when ShadowReplicator is null
   
   so the replicator is not ready when producer send message.
   
   and the consumer wont get any message.
   
   ### Modifications
   
   <!-- Describe the modifications you've done. -->
   
   ### Verifying this change
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   ### Does this pull request potentially affect one of the following parts:
   
   *If the box was checked, please highlight the changes*
   
   - [ ] Dependencies (add or upgrade a dependency)
   - [ ] The public API
   - [ ] The schema
   - [ ] The default values of configurations
   - [ ] The threading model
   - [ ] The binary protocol
   - [ ] The REST endpoints
   - [ ] The admin CLI options
   - [ ] Anything that affects deployment
   
   ### Documentation
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   - [ ] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
   - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lifepuzzlefun commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
lifepuzzlefun commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336431237

   @labuladong hi, after check the issue and try send more message by producer. I finally understand what you see and what do you mean. I add some comment under the issue #18731 . For this fix, i think you find another problem but not the root case for this test. I'll explain a bit more.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336708668

   /pulsarbot run-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] labuladong commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
labuladong commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336373658

   Nice catch! This is one cause of the flaky test, and I think the root cause may be retention policy, see #18731


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336718470

   # [Codecov](https://codecov.io/gh/apache/pulsar/pull/18730?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#18730](https://codecov.io/gh/apache/pulsar/pull/18730?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a77883b) into [master](https://codecov.io/gh/apache/pulsar/commit/3df506de099f27d4c3b7fbf2064db881ffda0402?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3df506d) will **decrease** coverage by `9.20%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pulsar/pull/18730/graphs/tree.svg?width=650&height=150&src=pr&token=acYqCpsK9J&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/18730?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18730      +/-   ##
   ============================================
   - Coverage     46.30%   37.09%   -9.21%     
   + Complexity    10379     1969    -8410     
   ============================================
     Files           703      209     -494     
     Lines         68795    14421   -54374     
     Branches       7373     1573    -5800     
   ============================================
   - Hits          31853     5350   -26503     
   + Misses        33358     8484   -24874     
   + Partials       3584      587    -2997     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.09% <ø> (-9.21%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/18730?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...g/apache/pulsar/client/impl/ConnectionHandler.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0Nvbm5lY3Rpb25IYW5kbGVyLmphdmE=) | `50.00% <0.00%> (-5.32%)` | :arrow_down: |
   | [...va/org/apache/pulsar/client/impl/ProducerBase.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VyQmFzZS5qYXZh) | `32.69% <0.00%> (-1.93%)` | :arrow_down: |
   | [...lsar/broker/loadbalance/impl/ThresholdShedder.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL1RocmVzaG9sZFNoZWRkZXIuamF2YQ==) | | |
   | [...r/broker/service/schema/AlwaysSchemaValidator.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3NjaGVtYS9BbHdheXNTY2hlbWFWYWxpZGF0b3IuamF2YQ==) | | |
   | [...apache/bookkeeper/mledger/ManagedLedgerConfig.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFuYWdlZC1sZWRnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Jvb2trZWVwZXIvbWxlZGdlci9NYW5hZ2VkTGVkZ2VyQ29uZmlnLmphdmE=) | | |
   | [...java/org/apache/pulsar/proxy/server/WebServer.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLXByb3h5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9wdWxzYXIvcHJveHkvc2VydmVyL1dlYlNlcnZlci5qYXZh) | | |
   | [.../pulsar/broker/systopic/SystemTopicClientBase.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zeXN0b3BpYy9TeXN0ZW1Ub3BpY0NsaWVudEJhc2UuamF2YQ==) | | |
   | [...e/bookkeeper/mledger/impl/NullLedgerOffloader.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFuYWdlZC1sZWRnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Jvb2trZWVwZXIvbWxlZGdlci9pbXBsL051bGxMZWRnZXJPZmZsb2FkZXIuamF2YQ==) | | |
   | [...r/service/AbstractDispatcherMultipleConsumers.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL0Fic3RyYWN0RGlzcGF0Y2hlck11bHRpcGxlQ29uc3VtZXJzLmphdmE=) | | |
   | [...nsaction/buffer/impl/TransactionBufferDisable.java](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9idWZmZXIvaW1wbC9UcmFuc2FjdGlvbkJ1ZmZlckRpc2FibGUuamF2YQ==) | | |
   | ... and [487 more](https://codecov.io/gh/apache/pulsar/pull/18730/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lifepuzzlefun commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
lifepuzzlefun commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336222788

   @codelipenghui @coderzc @labuladong please take a look thank you ~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] labuladong commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
labuladong commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336384935

   I have changed and tested it, if you try a few more times, the test will still fail.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Jason918 merged pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
Jason918 merged PR #18730:
URL: https://github.com/apache/pulsar/pull/18730


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lifepuzzlefun commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
lifepuzzlefun commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336383992

   > Nice catch! This is one cause of the flaky test, and I think the root cause may be retention policy, see #18731
   
   thanks for reply. i tried you code. if you replace you ut `Thread.sleep(1000 * 3)` add the same line in my pr. the ut code also passed. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lifepuzzlefun commented on pull request #18730: [fix][test] Flaky-test: ShadowTopicTest.testShadowTopicConsuming

Posted by GitBox <gi...@apache.org>.
lifepuzzlefun commented on PR #18730:
URL: https://github.com/apache/pulsar/pull/18730#issuecomment-1336223388

   ![image](https://user-images.githubusercontent.com/13600283/205458148-f1b57048-b0dc-4af8-bec0-6d2c7db61764.png)
   
   add break point on line `consumer.receive(60, TimeUnit.SECONDS)` can get the replicator is null.
   also when enable debug log show the started ShadowReplicator is on the latest message so the message producer send is passed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org