You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Attila Jeges (Code Review)" <ge...@cloudera.org> on 2019/08/20 19:51:59 UTC
[Impala-ASF-CR] IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
Attila Jeges has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14108
Change subject: IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
......................................................................
IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 4 in a series of steps to coordinate the
introduction of HMS API changes to Hive3 and Impala. For more
information see IMPALA-8842 part 1.
Step 4 replaces *V2 calls with *. The *V2 names were introduced
temporarily and will be removed from the HMS API in the near future.
TESTING:
E2E tests were added to make sure that column statistics are
differentiated by engine for partitioned and non-partitioned tables.
The tests are executed for transactional and non-transactional tables.
Change-Id: I28e383156c48c7038e8bc235a3f3e5cc58a4fc01
---
M bin/impala-config.sh
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M tests/metadata/test_hms_integration.py
3 files changed, 109 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/14108/1
--
To view, visit http://gerrit.cloudera.org:8080/14108
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I28e383156c48c7038e8bc235a3f3e5cc58a4fc01
Gerrit-Change-Number: 14108
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
[Impala-ASF-CR] IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14108 )
Change subject: IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 1:
> (2 comments)
Thanks for taking a look. Currently this patch is blocked by a Hive3 bug, hopefully it will be fixed soon and we can move this forward.
--
To view, visit http://gerrit.cloudera.org:8080/14108
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28e383156c48c7038e8bc235a3f3e5cc58a4fc01
Gerrit-Change-Number: 14108
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Sep 2019 15:12:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14108 )
Change subject: IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4310/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14108
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28e383156c48c7038e8bc235a3f3e5cc58a4fc01
Gerrit-Change-Number: 14108
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Aug 2019 20:32:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has abandoned this change. ( http://gerrit.cloudera.org:8080/14108 )
Change subject: IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Abandoned
--
To view, visit http://gerrit.cloudera.org:8080/14108
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I28e383156c48c7038e8bc235a3f3e5cc58a4fc01
Gerrit-Change-Number: 14108
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14108 )
Change subject: IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14108/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14108/1//COMMIT_MSG@7
PS1, Line 7: IMPALA-8842 part 2: (Hive3) Use 'engine' field in HMS stat API
Can you mention that CDP_BUILD_NUMBER is dumped, or maybe even move it to a separate patch? Knowing version bumps by looking at the title is useful when investigating broken evergreen Jenkins jobs, as the runs show the list of new commits compared to the previous run and version bumps often lead to test failures.
For the same reason it would be good to add in the TESTING sections if a CDP build was run. Core tests seem quite green these days.
http://gerrit.cloudera.org:8080/#/c/14108/1/tests/metadata/test_hms_integration.py
File tests/metadata/test_hms_integration.py:
http://gerrit.cloudera.org:8080/#/c/14108/1/tests/metadata/test_hms_integration.py@157
PS1, Line 157: self.impala.client.execute(
: 'set DEFAULT_TRANSACTIONAL_TYPE=insert_only')
I am not sure how long is the effect of this SET - can you check whether it actually affects the next create table + whether it leaks and affects other table creations?
--
To view, visit http://gerrit.cloudera.org:8080/14108
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28e383156c48c7038e8bc235a3f3e5cc58a4fc01
Gerrit-Change-Number: 14108
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 23 Aug 2019 17:35:04 +0000
Gerrit-HasComments: Yes