You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org> on 2021/03/24 22:07:21 UTC
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17228
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
New test case TestDecimalOverflowExprs::test_ctas_exprs was added
in the first patch for IMPALA-10564. But it failed in S3 build with
Parquet format since the table was not successfully created when
CTAS query failed.
This patch fixed the test failure by skipping checking if NULL is
inserted into table after CTAS failed for S3 build with Parquet.
Testing:
- Reproduced the test failure in local box with defaultFS as s3a.
Verified the fixing was working with defaultFS as s3a.
- Passed EE_TEST.
Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
---
M tests/query_test/test_decimal_queries.py
1 file changed, 15 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/17228/1
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG@10
PS2, Line 10: But it failed in S3 build with
: Parquet format since the table was not successfully created when
: CTAS query failed.
> That sounds like a bug though. We shouldn't end up with a corrupt table no
Filed Jira IMPALA-10607 to track this issue. Will fix it later.
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 01:14:34 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/17228/2/tests/query_test/test_decimal_queries.py
File tests/query_test/test_decimal_queries.py:
http://gerrit.cloudera.org:8080/#/c/17228/2/tests/query_test/test_decimal_queries.py@a186
PS2, Line 186:
> Do we still need to do this check here to ensure that the table we tried to
Added back
http://gerrit.cloudera.org:8080/#/c/17228/2/tests/query_test/test_decimal_queries.py@175
PS2, Line 175: if not ('parquet' in str(vector.get_value('table_format')) and IS_S3):
> Could you add a comment here explaining why we skip this check for s3?
Done
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 01:04:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7002/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 15:55:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8441/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 01:27:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8436/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 22:53:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17228/1/tests/query_test/test_decimal_queries.py
File tests/query_test/test_decimal_queries.py:
http://gerrit.cloudera.org:8080/#/c/17228/1/tests/query_test/test_decimal_queries.py@28
PS1, Line 28: from tests.util.filesystem_utils import WAREHOUSE, get_fs_path, IS_S3
> flake8: F401 'tests.util.filesystem_utils.WAREHOUSE' imported but unused
Done
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 22:31:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 01:38:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
New test case TestDecimalOverflowExprs::test_ctas_exprs was added
in the first patch for IMPALA-10564. But it failed in S3 build with
Parquet format since the table was not successfully created when
CTAS query failed.
This patch fixed the test failure by skipping checking if NULL is
inserted into table after CTAS failed for S3 build with Parquet.
Testing:
- Reproduced the test failure in local box with defaultFS as s3a.
Verified the fixing was working with defaultFS as s3a.
- Passed EE_TEST.
Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
---
M tests/query_test/test_decimal_queries.py
1 file changed, 15 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/17228/2
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
New test case TestDecimalOverflowExprs::test_ctas_exprs was added
in the first patch for IMPALA-10564. But it failed in S3 build with
Parquet format since the table was not successfully created when
CTAS query failed.
This patch fixed the test failure by skipping checking if NULL is
inserted into table after CTAS failed for S3 build with Parquet.
Testing:
- Reproduced the test failure in local box with defaultFS as s3a.
Verified the fixing was working with defaultFS as s3a.
- Passed EE_TEST.
Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
---
M tests/query_test/test_decimal_queries.py
1 file changed, 21 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/17228/3
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
New test case TestDecimalOverflowExprs::test_ctas_exprs was added
in the first patch for IMPALA-10564. But it failed in S3 build with
Parquet format since the table was not successfully created when
CTAS query failed.
This patch fixed the test failure by skipping checking if NULL is
inserted into table after CTAS failed for S3 build with Parquet.
Testing:
- Reproduced the test failure in local box with defaultFS as s3a.
Verified the fixing was working with defaultFS as s3a.
- Passed EE_TEST.
Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Reviewed-on: http://gerrit.cloudera.org:8080/17228
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/query_test/test_decimal_queries.py
1 file changed, 21 insertions(+), 9 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 3:
Build failure was caused by irrelevant errors: "generate_junitxml.buildall.all-tests" failed.
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 06:00:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 21:45:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG@10
PS2, Line 10: But it failed in S3 build with
: Parquet format since the table was not successfully created when
: CTAS query failed.
Why does this work differently in for this case?
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 22:41:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8435/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 22:27:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7000/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 01:38:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG@10
PS2, Line 10: But it failed in S3 build with
: Parquet format since the table was not successfully created when
: CTAS query failed.
> Why does this work differently in for this case?
When tried to read the table after CTAS, got following error "Query aborted:Parquet file s3a://impala-test-uswest2-1/test-warehouse/test_ctas_exprs_7304e515.db/overflowed_decimal_tbl_1/b74f0ce129189cf1-4c3c5bd600000000_1609291350_data.0.parq has an invalid file length: 4". I guess the data is partially flushed in s3.
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 22:52:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/17228/1/tests/query_test/test_decimal_queries.py
File tests/query_test/test_decimal_queries.py:
http://gerrit.cloudera.org:8080/#/c/17228/1/tests/query_test/test_decimal_queries.py@28
PS1, Line 28: from tests.util.filesystem_utils import WAREHOUSE, get_fs_path, IS_S3
flake8: F401 'tests.util.filesystem_utils.WAREHOUSE' imported but unused
http://gerrit.cloudera.org:8080/#/c/17228/1/tests/query_test/test_decimal_queries.py@28
PS1, Line 28: from tests.util.filesystem_utils import WAREHOUSE, get_fs_path, IS_S3
flake8: F401 'tests.util.filesystem_utils.get_fs_path' imported but unused
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 22:08:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 15:55:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17228/2//COMMIT_MSG@10
PS2, Line 10: But it failed in S3 build with
: Parquet format since the table was not successfully created when
: CTAS query failed.
> When tried to read the table after CTAS, got following error "Query aborted
That sounds like a bug though. We shouldn't end up with a corrupt table no matter how the CTAS finished.
It might be tricky to fix, and I guess its low priority since the table isn't expected to have meaningful contents anyways, so its okay to go ahead with this patch to unbreak the builds, but you should at least file a JIRA for it
http://gerrit.cloudera.org:8080/#/c/17228/2/tests/query_test/test_decimal_queries.py
File tests/query_test/test_decimal_queries.py:
http://gerrit.cloudera.org:8080/#/c/17228/2/tests/query_test/test_decimal_queries.py@a186
PS2, Line 186:
Do we still need to do this check here to ensure that the table we tried to insert into doesn't end up with the null value?
http://gerrit.cloudera.org:8080/#/c/17228/2/tests/query_test/test_decimal_queries.py@175
PS2, Line 175: if not ('parquet' in str(vector.get_value('table_format')) and IS_S3):
Could you add a comment here explaining why we skip this check for s3?
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 00:37:48 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10564 (part 2): Fixed test ctas exprs failure for S3 build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17228 )
Change subject: IMPALA-10564 (part 2): Fixed test_ctas_exprs failure for S3 build
......................................................................
Patch Set 3: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7000/
--
To view, visit http://gerrit.cloudera.org:8080/17228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia627ca70ed41764e86be348a0bc19e330b3334d2
Gerrit-Change-Number: 17228
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 25 Mar 2021 05:55:05 +0000
Gerrit-HasComments: No