You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "shounakmk219 (via GitHub)" <gi...@apache.org> on 2024/08/07 16:07:13 UTC
[PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
shounakmk219 opened a new pull request, #13771:
URL: https://github.com/apache/pinot/pull/13771
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on PR #13771:
URL: https://github.com/apache/pinot/pull/13771#issuecomment-2294414099
@klsince Can you help verify the test logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "shounakmk219 (via GitHub)" <gi...@apache.org>.
shounakmk219 commented on code in PR #13771:
URL: https://github.com/apache/pinot/pull/13771#discussion_r1724451147
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java:
##########
@@ -169,14 +193,60 @@ private void runQueries(double qps, boolean shouldFail)
assertTrue((failCount == 0 && !shouldFail) || (failCount != 0 && shouldFail));
}
+ private void verifyQuotaUpdate(long quotaQps) {
+ String query = "SELECT COUNT(*) FROM " + getTableName();
+ if (quotaQps == 0) {
+ TestUtils.waitForCondition(aVoid -> !executeQueryOnBroker(query).getExceptions().elements().hasNext()
Review Comment:
> looks like the two stmts are same. is this intended to run the query twice back to back?
Yes, basically making sure consecutive queries pass which indicated quota is not applied.
> perhaps we can add a broker restful API to get runtime quotas for a table, which can help simplify the test here and can be helpful for debugging in prod as well.
That's a good idea, will use this approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "KKcorps (via GitHub)" <gi...@apache.org>.
KKcorps merged PR #13771:
URL: https://github.com/apache/pinot/pull/13771
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "shounakmk219 (via GitHub)" <gi...@apache.org>.
shounakmk219 commented on code in PR #13771:
URL: https://github.com/apache/pinot/pull/13771#discussion_r1724783017
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java:
##########
@@ -169,14 +193,60 @@ private void runQueries(double qps, boolean shouldFail)
assertTrue((failCount == 0 && !shouldFail) || (failCount != 0 && shouldFail));
}
+ private void verifyQuotaUpdate(long quotaQps) {
+ String query = "SELECT COUNT(*) FROM " + getTableName();
+ if (quotaQps == 0) {
+ TestUtils.waitForCondition(aVoid -> !executeQueryOnBroker(query).getExceptions().elements().hasNext()
Review Comment:
PR to introduce the debug endpoints on broker https://github.com/apache/pinot/pull/13864
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "klsince (via GitHub)" <gi...@apache.org>.
klsince commented on code in PR #13771:
URL: https://github.com/apache/pinot/pull/13771#discussion_r1724067958
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java:
##########
@@ -169,14 +193,60 @@ private void runQueries(double qps, boolean shouldFail)
assertTrue((failCount == 0 && !shouldFail) || (failCount != 0 && shouldFail));
}
+ private void verifyQuotaUpdate(long quotaQps) {
+ String query = "SELECT COUNT(*) FROM " + getTableName();
+ if (quotaQps == 0) {
+ TestUtils.waitForCondition(aVoid -> !executeQueryOnBroker(query).getExceptions().elements().hasNext()
Review Comment:
looks like the two stmts are same. is this intended to run the query twice back to back?
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java:
##########
@@ -169,14 +193,60 @@ private void runQueries(double qps, boolean shouldFail)
assertTrue((failCount == 0 && !shouldFail) || (failCount != 0 && shouldFail));
}
+ private void verifyQuotaUpdate(long quotaQps) {
+ String query = "SELECT COUNT(*) FROM " + getTableName();
+ if (quotaQps == 0) {
+ TestUtils.waitForCondition(aVoid -> !executeQueryOnBroker(query).getExceptions().elements().hasNext()
Review Comment:
perhaps we can add a broker restful API to get runtime quotas for a table, which can help simplify the test here and can be helpful for debugging in prod as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13771:
URL: https://github.com/apache/pinot/pull/13771#issuecomment-2273911864
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13771?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 61.81%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`bb49686`)](https://app.codecov.io/gh/apache/pinot/commit/bb49686105da81931c10a9ec94ad54519d594622?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 845 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #13771 +/- ##
============================================
+ Coverage 61.75% 61.81% +0.06%
- Complexity 207 1315 +1108
============================================
Files 2436 2557 +121
Lines 133233 141058 +7825
Branches 20636 21936 +1300
============================================
+ Hits 82274 87194 +4920
- Misses 44911 47208 +2297
- Partials 6048 6656 +608
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `27.78% <ø> (-33.93%)` | :arrow_down: |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `61.81% <ø> (+0.18%)` | :arrow_up: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `61.79% <ø> (+0.04%)` | :arrow_up: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `61.78% <ø> (+34.05%)` | :arrow_up: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `61.81% <ø> (+0.06%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `61.81% <ø> (+0.06%)` | :arrow_up: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.21% <ø> (-0.68%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13771/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `27.79% <ø> (+0.06%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/13771?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #13771:
URL: https://github.com/apache/pinot/pull/13771#discussion_r1707533463
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java:
##########
@@ -125,12 +132,15 @@ public void testDatabaseQueryQuotaWithTableQueryQuotaWithExtraBroker()
addQueryQuotaToTableConfig(10);
// Add one more broker such that quota gets distributed equally among them
brokerStarter = startOneBroker(2);
+ _brokerBaseApiUrl = LOCAL_HOST + ":" + brokerStarter.getPort();
// to allow change propagation to QueryQuotaManager
Thread.sleep(1000);
Review Comment:
Sleep is not robust, especially in the CI environment where machines have very limited resource. One way to make it more robust is to use `TestUtils.waitForCondition()` and keep trying until it meets the expectation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "shounakmk219 (via GitHub)" <gi...@apache.org>.
shounakmk219 commented on PR #13771:
URL: https://github.com/apache/pinot/pull/13771#issuecomment-2293833114
Hey @Jackie-Jiang I could not find any direct way of getting the signal that query quota rate limiter change is propagated so right now I am checking for the delay between 2 successful requests and ensure it lies within the expected changed rate (with 20% margin).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "shounakmk219 (via GitHub)" <gi...@apache.org>.
shounakmk219 commented on code in PR #13771:
URL: https://github.com/apache/pinot/pull/13771#discussion_r1724451147
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/QueryQuotaClusterIntegrationTest.java:
##########
@@ -169,14 +193,60 @@ private void runQueries(double qps, boolean shouldFail)
assertTrue((failCount == 0 && !shouldFail) || (failCount != 0 && shouldFail));
}
+ private void verifyQuotaUpdate(long quotaQps) {
+ String query = "SELECT COUNT(*) FROM " + getTableName();
+ if (quotaQps == 0) {
+ TestUtils.waitForCondition(aVoid -> !executeQueryOnBroker(query).getExceptions().elements().hasNext()
Review Comment:
> looks like the two stmts are same. is this intended to run the query twice back to back?
Yes, basically making sure consecutive queries pass which indicates quota is not applied.
> perhaps we can add a broker restful API to get runtime quotas for a table, which can help simplify the test here and can be helpful for debugging in prod as well.
That's a good idea, will use this approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Flaky test fix. Query only 1 broker to test quota split [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on PR #13771:
URL: https://github.com/apache/pinot/pull/13771#issuecomment-2292930061
Any progress on this fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org