You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org> on 2022/06/25 00:21:37 UTC
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18671
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
IMPALA-8957 fixed the flakiness for test by adding a delay via
DEBUG_ACTION BPRS_BEFORE_ADD_ROWS in BlockingPlanRootSink::Send().
BPRS_BEFORE_ADD_ROWS should be used for spool_query_results as false,
BPRS_BEFORE_ADD_BATCH should be used for spool_query_results as true.
Extended dimension is added for TestFetchAndSpooling with
spool_query_results as true. Now the default value of pool_query_results
is set as true so spool_query_results as false is not tested in the test
case.
Since the test uses BPRS_BEFORE_ADD_ROWS if spool_query_results is
not specified as true, it makes DEBUG_ACTION BPRS_BEFORE_ADD_ROWS to be
used for spool_query_results as true. This causes the test flaky since
no delay in BufferedPlanRootSink::Send().
Another issue is that the test uses bool() to convert string to boolean
value, but the function returns true for any non empty string.
This patch changed the extended dimension setting for
spool_query_results as false, and made the test to use the right
DEBUG_ACTION for spool_query_results as true and false.
Also reverted the previous fixing which disabled the test for S3
testing environment.
Testing:
- Ran the test more than 300 times on local machine.
Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
---
M tests/query_test/test_fetch.py
1 file changed, 4 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/18671/1
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
IMPALA-8957 fixed the flakiness for test by adding a delay via
DEBUG_ACTION BPRS_BEFORE_ADD_ROWS in BlockingPlanRootSink::Send().
test_rows_sent_counters uses DEBUG_ACTION BPRS_BEFORE_ADD_BATCH when
spool_query_results is on, and uses BPRS_BEFORE_ADD_ROWS when
spool_query_results is off with assumption that result spooling is
disabled by default.
IMPALA-9856 enabled result spooling by default.
Following two issues were introduced for the test when result spooling
was enabled by default.
1) spool_query_results as false is not covered in the test since
extended dimension is added with spool_query_results as true.
2) Since the test uses BPRS_BEFORE_ADD_ROWS if spool_query_results is
not specified as true, it makes DEBUG_ACTION BPRS_BEFORE_ADD_ROWS to be
used for spool_query_results as true. This causes the test flaky since
no delay to be added in BufferedPlanRootSink::Send().
There is another bug in the test. It uses bool() to convert string to
boolean value, but the function returns true for any non empty string.
This patch changed the extended dimension setting for
spool_query_results as false, and made the test to use the right
DEBUG_ACTION for spool_query_results as true and false.
Also reverted the previous fixing which disabled the test for S3
testing environment.
Testing:
- Ran the test more than 10000 times without failure on Jenkins.
Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
---
M tests/query_test/test_fetch.py
1 file changed, 5 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/18671/2
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10879/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 25 Jun 2022 00:41:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2:
hit IMPALA-11160
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 06:11:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 00:41:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8271/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 01:25:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 10:54:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8272/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 06:12:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
IMPALA-8957 fixed the flakiness for test by adding a delay via
DEBUG_ACTION BPRS_BEFORE_ADD_ROWS in BlockingPlanRootSink::Send().
test_rows_sent_counters uses DEBUG_ACTION BPRS_BEFORE_ADD_BATCH when
spool_query_results is on, and uses BPRS_BEFORE_ADD_ROWS when
spool_query_results is off with assumption that result spooling is
disabled by default.
IMPALA-9856 enabled result spooling by default.
Following two issues were introduced for the test when result spooling
was enabled by default.
1) spool_query_results as false is not covered in the test since
extended dimension is added with spool_query_results as true.
2) Since the test uses BPRS_BEFORE_ADD_ROWS if spool_query_results is
not specified as true, it makes DEBUG_ACTION BPRS_BEFORE_ADD_ROWS to be
used for spool_query_results as true. This causes the test flaky since
no delay to be added in BufferedPlanRootSink::Send().
There is another bug in the test. It uses bool() to convert string to
boolean value, but the function returns true for any non empty string.
This patch changed the extended dimension setting for
spool_query_results as false, and made the test to use the right
DEBUG_ACTION for spool_query_results as true and false.
Also reverted the previous fixing which disabled the test for S3
testing environment.
Testing:
- Ran the test more than 10000 times without failure on Jenkins.
Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Reviewed-on: http://gerrit.cloudera.org:8080/18671
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/query_test/test_fetch.py
1 file changed, 5 insertions(+), 6 deletions(-)
Approvals:
Quanlong Huang: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10880/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 00:58:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10927: Deflaky TestFetchAndSpooling.test rows sent counters
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18671 )
Change subject: IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
......................................................................
Patch Set 2: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8271/
--
To view, visit http://gerrit.cloudera.org:8080/18671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Gerrit-Change-Number: 18671
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 27 Jun 2022 06:09:37 +0000
Gerrit-HasComments: No