You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2020/08/06 20:17:02 UTC
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16301
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
test_multiple_sort_run_bytes_limits seems to become flaky in
ubuntu-16.04-dockerised-tests. This flakiness may come from accuracy
change in query estimates or mem_limit specified in the test does not
fit anymore. This patch tune the parameter of the first and the second
test case of test_multiple_sort_run_bytes_limits to pass the assertion.
The assertion is also changed a bit to allow easier debugging in case if
test regression occurs again in the future.
Testing:
- Run and pass test_sort.py
Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
---
M tests/query_test/test_sort.py
1 file changed, 6 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/16301/1
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6269/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 20:25:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
test_multiple_sort_run_bytes_limits seems to become flaky in
ubuntu-16.04-dockerised-tests. This flakiness may come from accuracy
change in query estimates, the mem_limit specified in the test does not
fit anymore, or query concurrency in mini cluster that may disturb
expected memory allocation. This patch remove the second test case of
test_multiple_sort_run_bytes_limits due to variability in several test
run in the past. It does not compromise the test itself because the
basic feature of sort_run_bytes_limit is still verifiable by the
remaining test cases. The assertion is also changed a bit to allow
easier debugging in case test regression occurs again in the future.
Testing:
- Run and pass test_sort.py
Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Reviewed-on: http://gerrit.cloudera.org:8080/16301
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/query_test/test_sort.py
1 file changed, 5 insertions(+), 5 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Attila Jeges, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16301
to look at the new patch set (#3).
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
test_multiple_sort_run_bytes_limits seems to become flaky in
ubuntu-16.04-dockerised-tests. This flakiness may come from accuracy
change in query estimates, the mem_limit specified in the test does not
fit anymore, or query concurrency in mini cluster that may disturb
expected memory allocation. This patch remove the second test case of
test_multiple_sort_run_bytes_limits due to variability in several test
run in the past. It does not compromise the test itself because the
basic feature of sort_run_bytes_limit is still verifiable by the
remaining test cases. The assertion is also changed a bit to allow
easier debugging in case test regression occurs again in the future.
Testing:
- Run and pass test_sort.py
Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
---
M tests/query_test/test_sort.py
1 file changed, 5 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/16301/3
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Gabor Kaszab (Code Review)" <ge...@cloudera.org>.
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4:
Apparently, the job failed during dataload that is unrelated to this change. I re-launched it, let's see what happens.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 06:54:45 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6262/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 06:54:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16301/1/tests/query_test/test_sort.py
File tests/query_test/test_sort.py:
http://gerrit.cloudera.org:8080/#/c/16301/1/tests/query_test/test_sort.py@90
PS1, Line 90: ' - SpilledRuns:.*'
> nit: Perhaps you could use a more complete regex pattern here:
Also, please use raw strings for regex patterns, e.g.:
r'\s+\- SpilledRuns: %s' % spilled_runs
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Aug 2020 11:02:29 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 10 Aug 2020 20:31:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 2:
I reran the dockerised test with the new parameter and still failed to get the expected result.
https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2921/testReport/junit/query_test.test_sort/TestQueryFullSort/test_multiple_sort_run_bytes_limits_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/
I thought it might overspill (SpilledRuns > 3), but turns out, the query finish with only 2 SpilledRuns.
It seems test case 2 is hard to get right since other concurrent running queries might disturb the memory allocation for this test.
At this point, I lean towards removing test case 2 to stabilize the build going forward.
In my opinion, test case 1 and 3 is sufficient to verify sort_run_bytes_limit feature.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sat, 08 Aug 2020 01:15:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 5: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Aug 2020 07:07:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6259/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 10 Aug 2020 20:31:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4: Code-Review+2
The infrastructure change that led to the broken dataload was fixed, see IMPALA-10072. I am restarting the test.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 20:24:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6842/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sat, 08 Aug 2020 02:01:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6827/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Aug 2020 16:09:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 20:25:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Sahil Takiar (Code Review)" <ge...@cloudera.org>.
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 5:
Looks like it failed due to IMPALA-9923. Going to retry.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Aug 2020 02:02:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 2:
(1 comment)
Hi Attila, I change the assertion as you suggest, but not quite the same.
http://gerrit.cloudera.org:8080/#/c/16301/1/tests/query_test/test_sort.py
File tests/query_test/test_sort.py:
http://gerrit.cloudera.org:8080/#/c/16301/1/tests/query_test/test_sort.py@90
PS1, Line 90: '\s+\- SpilledRuns: .*', quer
> Also, please use raw strings for regex patterns, e.g.:
I kept the assertion in L92, but remove the one in L91.
The intention is that in case the assertion is fail, we can see the comparison between the expected result vs actual result in the stacktrace like this
query_test/test_sort.py:91: in test_multiple_sort_run_bytes_limits
assert "SpilledRuns: " + spilled_runs in m.group()
E assert ('SpilledRuns: ' + '3') in '\n - SpilledRuns: 2 (2)'
E + where '\n - SpilledRuns: 2 (2)' = <built-in method group of _sre.SRE_Match object at 0x7fe017df5e68>()
E + where <built-in method group of _sre.SRE_Match object at 0x7fe017df5e68> = <_sre.SRE_Match object at 0x7fe017df5e68>.group
Whereas previously, the actual result is not clearly shown because right hand side of the expression is the entire runtime_profile.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Aug 2020 15:50:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6814/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Aug 2020 20:41:22 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Attila Jeges, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16301
to look at the new patch set (#2).
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
test_multiple_sort_run_bytes_limits seems to become flaky in
ubuntu-16.04-dockerised-tests. This flakiness may come from accuracy
change in query estimates or mem_limit specified in the test does not
fit anymore. This patch tune the parameter of the first and the second
test case of test_multiple_sort_run_bytes_limits to pass the assertion.
The assertion is also changed a bit to allow easier debugging in case if
test regression occurs again in the future.
Testing:
- Run and pass test_sort.py
Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
---
M tests/query_test/test_sort.py
1 file changed, 5 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/16301/2
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Sahil Takiar (Code Review)" <ge...@cloudera.org>.
Sahil Takiar has removed a vote on this change.
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6272/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Aug 2020 02:03:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 5: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6269/
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Aug 2020 01:34:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4:
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6262/
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 12:12:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16301/1/tests/query_test/test_sort.py
File tests/query_test/test_sort.py:
http://gerrit.cloudera.org:8080/#/c/16301/1/tests/query_test/test_sort.py@90
PS1, Line 90: ' - SpilledRuns:.*'
nit: Perhaps you could use a more complete regex pattern here:
'\s+\- SpilledRuns: %s.*' % spilled_runs
and then you can remove the extra check in L92.
You can also use re.search() instead of re.findall() since you don't need to scan the whole runtime profile after the first match.
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Aug 2020 10:02:09 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 3: Code-Review+2
Thanks for the explanation!
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sun, 09 Aug 2020 11:47:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10054: Fix flakiness in test multiple sort run bytes limits
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16301 )
Change subject: IMPALA-10054: Fix flakiness in test_multiple_sort_run_bytes_limits
......................................................................
Patch Set 4: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6259/
--
To view, visit http://gerrit.cloudera.org:8080/16301
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I84a8b579c943cddba4432cf183f7f002ef8ec6ad
Gerrit-Change-Number: 16301
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Aug 2020 01:49:57 +0000
Gerrit-HasComments: No