You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Attila Jeges (Code Review)" <ge...@cloudera.org> on 2019/08/09 15:18:08 UTC
[Impala-ASF-CR] WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Attila Jeges has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14043
Change subject: WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the introduction
of HMS API changes to Hive and Impala:
1. Change in Hive.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive.
Remove *V2 methods from API.
This patch-set is in WIP state as Step 1 is not yet complete.
TODO: Once Step 1 is complete, make the new build available to Impala
via the native-toolchain. Update the CDP_BUILD_NUMBER and
CDP_HIVE_VERSION environment variables in bin/impala-config.sh
accordingly.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
---
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
6 files changed, 90 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/1
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
[Impala-ASF-CR] WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4206/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Aug 2019 15:37:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4262/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 15:57:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 2:
> Uploaded patch set 2.
Rebased
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Aug 2019 15:57:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 3:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/4253/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Aug 2019 21:45:06 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4798/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 15:42:45 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4260/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 15:11:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14
PS7, Line 14: This change is Step 2 in a series of steps to coordinate the
: introduction of HMS API changes to Hive3 and Impala. Step 4 is also
: Impala related, it will be covered in IMPALA-8842 part 2.
:
: The steps are as follows:
:
: 1. Change in Hive3.
: We push new APIs so Impala can use them. New APIs will simply call old
: existing methods so there should not be any change of functionality
: there. Since there were many incompatible changes, new APIs are tagged
: method_name_V2.
:
: 2. Change in Impala.
: Push changes to use new methods *V2.
:
: 3. Change in Hive3.
: Push patch with complete functionality. *V2 methods contains the new
: logic. The old existing methods are not used anymore by Impala at this
: point, hence they can be removed. For every method_name_V2, I will
: create a corresponding method method_name that calls the former one.
:
: 4. Change in Impala
: Replace *V2 calls by *.
:
: 5. Change in Hive3.
: Remove *V2 methods from API
> I also wanted to explain why there's no change in functionality yet.
I am okay with this. This should not block the commit.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 18:24:08 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 3:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/4250/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Aug 2019 19:34:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4254/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Aug 2019 22:29:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4207/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Aug 2019 16:37:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14
PS7, Line 14: This change is Step 2 in a series of steps to coordinate the
: introduction of HMS API changes to Hive3 and Impala. Step 4 is also
: Impala related, it will be covered in IMPALA-8842 part 2.
:
: The steps are as follows:
:
: 1. Change in Hive3.
: We push new APIs so Impala can use them. New APIs will simply call old
: existing methods so there should not be any change of functionality
: there. Since there were many incompatible changes, new APIs are tagged
: method_name_V2.
:
: 2. Change in Impala.
: Push changes to use new methods *V2.
:
: 3. Change in Hive3.
: Push patch with complete functionality. *V2 methods contains the new
: logic. The old existing methods are not used anymore by Impala at this
: point, hence they can be removed. For every method_name_V2, I will
: create a corresponding method method_name that calls the former one.
:
: 4. Change in Impala
: Replace *V2 calls by *.
:
: 5. Change in Hive3.
: Remove *V2 methods from API
I am not sure why we need this in the commit message. The commit message should give details what the patch does and why. The coordination of patches between Hive and Impala can be done out-of-band and provides no value by recording the steps in the git commit message.
http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml
File shaded-deps/pom.xml:
http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml@43
PS7, Line 43: <exclusions>
: <exclusion>
: <groupId>org.apache.hive.shims</groupId>
: <artifactId>hive-shims-0.20</artifactId>
: </exclusion>
: </exclusions>
Is there a particular reason why this needs to be excluded for this patch?
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 17:20:01 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14
PS7, Line 14: This change is Step 2 in a series of steps to coordinate the
: introduction of HMS API changes to Hive3 and Impala. Step 4 is also
: Impala related, it will be covered in IMPALA-8842 part 2.
:
: The steps are as follows:
:
: 1. Change in Hive3.
: We push new APIs so Impala can use them. New APIs will simply call old
: existing methods so there should not be any change of functionality
: there. Since there were many incompatible changes, new APIs are tagged
: method_name_V2.
:
: 2. Change in Impala.
: Push changes to use new methods *V2.
:
: 3. Change in Hive3.
: Push patch with complete functionality. *V2 methods contains the new
: logic. The old existing methods are not used anymore by Impala at this
: point, hence they can be removed. For every method_name_V2, I will
: create a corresponding method method_name that calls the former one.
:
: 4. Change in Impala
: Replace *V2 calls by *.
:
: 5. Change in Hive3.
: Remove *V2 methods from API
> I am not sure why we need this in the commit message. The commit message sh
I recorded the steps in the commit msg because I wanted to explain why we need to break IMPALA-8842 down to part 1 and part2.
http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml
File shaded-deps/pom.xml:
http://gerrit.cloudera.org:8080/#/c/14043/7/shaded-deps/pom.xml@43
PS7, Line 43: <exclusions>
: <exclusion>
: <groupId>org.apache.hive.shims</groupId>
: <artifactId>hive-shims-0.20</artifactId>
: </exclusion>
: </exclusions>
> Is there a particular reason why this needs to be excluded for this patch?
This is mentioned briefly in the Commit Message.
hive-shims-0.20 brings in Hadoop2 dependencies which will break the Impala build.
This is a workaround to avoid build failures until we find a way to stop Hadoop2 dependencies from leaking out.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 17:45:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 6: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@7
PS4, Line 7: IMPALA-8842 part 1: (Hive3) Use 'engine' field in H
> Done
Thanks, I understand it now, I think that it is ok this way.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 15:32:59 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 4: Code-Review+1
(3 comments)
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@7
PS4, Line 7: IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2
nit: I would prefer something more descriptive like "(Hive3) Use 'engine' field in HMS stat API"
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@7
PS4, Line 7: part 1
What does part 1 mean? Will there be additional changes after part 1 step 4?
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@40
PS4, Line 40:
About testing: there were EE tests in https://gerrit.cloudera.org/#/c/14032/4/tests/metadata/test_hms_integration.py
Shouldn't that work at this point? Please add a comment if the tests do not work yet or add the tests if they do.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 13:12:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4258/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 10:29:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the
introduction of HMS API changes to Hive3 and Impala:
1. Change in Hive3.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive3.
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive3.
Remove *V2 methods from API.
This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml
to work around the Hive3 Hadoop2 dependency.
TESTING:
Only Step 1 has been merged in into Hive3 at this point, which means
that there's no change in Impala and Hive3 functionality yet.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
---
M bin/impala-config.sh
M fe/pom.xml
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M shaded-deps/pom.xml
9 files changed, 106 insertions(+), 17 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/5
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 15:42:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14
PS7, Line 14: This change is Step 2 in a series of steps to coordinate the
: introduction of HMS API changes to Hive3 and Impala. Step 4 is also
: Impala related, it will be covered in IMPALA-8842 part 2.
:
: The steps are as follows:
:
: 1. Change in Hive3.
: We push new APIs so Impala can use them. New APIs will simply call old
: existing methods so there should not be any change of functionality
: there. Since there were many incompatible changes, new APIs are tagged
: method_name_V2.
:
: 2. Change in Impala.
: Push changes to use new methods *V2.
:
: 3. Change in Hive3.
: Push patch with complete functionality. *V2 methods contains the new
: logic. The old existing methods are not used anymore by Impala at this
: point, hence they can be removed. For every method_name_V2, I will
: create a corresponding method method_name that calls the former one.
:
: 4. Change in Impala
: Replace *V2 calls by *.
:
: 5. Change in Hive3.
: Remove *V2 methods from API
> I recorded the steps in the commit msg because I wanted to explain why we n
I also wanted to explain why there's no change in functionality yet.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 18:01:04 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the
introduction of HMS API changes to Hive3 and Impala. Step 4 is also
Impala related, it will be covered in IMPALA-8842 part 2.
The steps are as follows:
1. Change in Hive3.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive3.
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive3.
Remove *V2 methods from API.
This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml
to work around the Hive3 Hadoop2 dependency.
TESTING:
Only Step 1 has been merged in into Hive3 at this point, which means
that there's no change in Impala and Hive3 functionality yet.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Reviewed-on: http://gerrit.cloudera.org:8080/14043
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/impala-config.sh
M fe/pom.xml
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M shaded-deps/pom.xml
9 files changed, 106 insertions(+), 17 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the
introduction of HMS API changes to Hive3 and Impala. Step 4 is also
Impala related, it will be covered in IMPALA-8842 part 2.
The steps are as follows:
1. Change in Hive3.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive3.
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive3.
Remove *V2 methods from API.
This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml
to work around the Hive3 Hadoop2 dependency.
TESTING:
Only Step 1 has been merged in into Hive3 at this point, which means
that there's no change in Impala and Hive3 functionality yet.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
---
M bin/impala-config.sh
M fe/pom.xml
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M shaded-deps/pom.xml
9 files changed, 106 insertions(+), 17 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/6
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/7//COMMIT_MSG@14
PS7, Line 14: This change is Step 2 in a series of steps to coordinate the
: introduction of HMS API changes to Hive3 and Impala. Step 4 is also
: Impala related, it will be covered in IMPALA-8842 part 2.
:
: The steps are as follows:
:
: 1. Change in Hive3.
: We push new APIs so Impala can use them. New APIs will simply call old
: existing methods so there should not be any change of functionality
: there. Since there were many incompatible changes, new APIs are tagged
: method_name_V2.
:
: 2. Change in Impala.
: Push changes to use new methods *V2.
:
: 3. Change in Hive3.
: Push patch with complete functionality. *V2 methods contains the new
: logic. The old existing methods are not used anymore by Impala at this
: point, hence they can be removed. For every method_name_V2, I will
: create a corresponding method method_name that calls the former one.
:
: 4. Change in Impala
: Replace *V2 calls by *.
:
: 5. Change in Hive3.
: Remove *V2 methods from API
> I recorded the steps in the commit msg because I wanted to explain why we n
Please let me know if you are okay with this commit msg, as II am planning to merge this in in the next couple of hours.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 17:58:03 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
......................................................................
Patch Set 4:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@7
PS4, Line 7: IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2
> nit: I would prefer something more descriptive like "(Hive3) Use 'engine' f
Done
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@7
PS4, Line 7: part 1
> What does part 1 mean? Will there be additional changes after part 1 step 4
This is step 2 in a series of steps that involves both Hive and Impala.
IMPALA-8842 covers the Impala related steps, that is: step 2 and step 4.
so step 2 is part 1 of IMPALA-8842, while step 4 will be part 2.
If you think this is too confusing, I could remove "part 1" from the summary. Or maybe I should remove "Step 2" from the end of the summary line?
http://gerrit.cloudera.org:8080/#/c/14043/4//COMMIT_MSG@40
PS4, Line 40:
> About testing: there were EE tests in https://gerrit.cloudera.org/#/c/14032
So far only step 1 has been merged in on the Hive side:
"New APIs will simply call old existing methods so there should not be any change of functionality there."
Since there's no new functionality, no new tests are needed. I'll add E2E tests in Step 4.
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 14:29:07 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: (Hive3) Use 'engine' field in HMS stat API
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Aug 2019 19:51:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
IMPALA-8842: Accommodate HIVE-22046 - Step 2.
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the introduction
of HMS API changes to Hive and Impala:
1. Change in Hive.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive.
Remove *V2 methods from API.
This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml
to work around the hive hadoop2 dependency.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
---
M bin/impala-config.sh
M fe/pom.xml
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M shaded-deps/pom.xml
9 files changed, 106 insertions(+), 17 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/3
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/14043 )
Change subject: WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
......................................................................
WIP: IMPALA-8842: Accommodate HIVE-22046 - Step 2.
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the introduction
of HMS API changes to Hive and Impala:
1. Change in Hive.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive.
Remove *V2 methods from API.
This patch-set is in WIP state as Step 1 is not yet complete.
TODO: Once Step 1 is complete, make the new build available to Impala
via the native-toolchain. Update the CDP_BUILD_NUMBER and
CDP_HIVE_VERSION environment variables in bin/impala-config.sh
accordingly.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
---
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
6 files changed, 90 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/2
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/14043 )
Change subject: IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
......................................................................
IMPALA-8842 part 1: Accommodate HIVE-22046 - Step 2.
HIVE-22046 added 'engine' column to TAB_COL_STATS and PART_COL_STATS
HMS tables. The new column is used to differentiate among column stats
computed by different engines. The related HMS API calls were changed
accordingly.
This change is Step 2 in a series of steps to coordinate the
introduction of HMS API changes to Hive and Impala:
1. Change in Hive.
We push new APIs so Impala can use them. New APIs will simply call old
existing methods so there should not be any change of functionality
there. Since there were many incompatible changes, new APIs are tagged
method_name_V2.
2. Change in Impala.
Push changes to use new methods *V2.
3. Change in Hive
Push patch with complete functionality. *V2 methods contains the new
logic. The old existing methods are not used anymore by Impala at this
point, hence they can be removed. For every method_name_V2, I will
create a corresponding method method_name that calls the former one.
4. Change in Impala
Replace *V2 calls by *.
5. Change in Hive.
Remove *V2 methods from API.
This change also adds exclusions to fe/pom.xml and shaded-deps/pom.xml
to work around the hive hadoop2 dependency.
Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
---
M bin/impala-config.sh
M fe/pom.xml
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M shaded-deps/pom.xml
9 files changed, 106 insertions(+), 17 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14043/4
--
To view, visit http://gerrit.cloudera.org:8080/14043
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9a73f5eeac8e84d63b22aaed5dfbcd8ea39f0af4
Gerrit-Change-Number: 14043
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>