You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/07/19 07:16:11 UTC
[GitHub] [hive] umamaheswararao opened a new pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
umamaheswararao opened a new pull request #1279:
URL: https://github.com/apache/hive/pull/1279
https://issues.apache.org/jira/browse/HIVE-16490
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] umamaheswararao commented on a change in pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on a change in pull request #1279:
URL: https://github.com/apache/hive/pull/1279#discussion_r458272410
##########
File path: shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
##########
@@ -1223,11 +1223,10 @@ public static boolean isHdfsEncryptionSupported() {
private final Configuration conf;
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
- DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);
-
this.conf = conf;
- this.keyProvider = dfs.getClient().getKeyProvider();
this.hdfsAdmin = new HdfsAdmin(uri, conf);
+ // HDFS-11687 added getKeyProvider method in hdfsAdmin from Hadoop-2.9.0
Review comment:
Done. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashutoshc commented on a change in pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
ashutoshc commented on a change in pull request #1279:
URL: https://github.com/apache/hive/pull/1279#discussion_r457659517
##########
File path: shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
##########
@@ -1223,11 +1223,31 @@ public static boolean isHdfsEncryptionSupported() {
private final Configuration conf;
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
- DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);
-
this.conf = conf;
- this.keyProvider = dfs.getClient().getKeyProvider();
this.hdfsAdmin = new HdfsAdmin(uri, conf);
+ this.keyProvider = getKeyProvider();
+ }
+
+ private KeyProvider getKeyProvider() throws IOException {
+ if (isMethodExist(HdfsAdmin.class, "getKeyProvider")) {
Review comment:
this is not needed. Hive's minimum version required for Hadoop is 3.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] umamaheswararao commented on a change in pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on a change in pull request #1279:
URL: https://github.com/apache/hive/pull/1279#discussion_r458272410
##########
File path: shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
##########
@@ -1223,11 +1223,10 @@ public static boolean isHdfsEncryptionSupported() {
private final Configuration conf;
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
- DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);
-
this.conf = conf;
- this.keyProvider = dfs.getClient().getKeyProvider();
this.hdfsAdmin = new HdfsAdmin(uri, conf);
+ // HDFS-11687 added getKeyProvider method in hdfsAdmin from Hadoop-2.9.0
Review comment:
@belugabehr , Thank you for review! Done. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] closed pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #1279:
URL: https://github.com/apache/hive/pull/1279
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] umamaheswararao commented on pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on pull request #1279:
URL: https://github.com/apache/hive/pull/1279#issuecomment-662179244
My latest commit has only removal of a comment: https://github.com/apache/hive/pull/1279/commits/2db5d300789bbc8325cae33eeb47840130f525d6
The patch already has green tests in previous runs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1279:
URL: https://github.com/apache/hive/pull/1279#issuecomment-695860130
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on a change in pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1279:
URL: https://github.com/apache/hive/pull/1279#discussion_r458131677
##########
File path: shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
##########
@@ -1223,11 +1223,10 @@ public static boolean isHdfsEncryptionSupported() {
private final Configuration conf;
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
- DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);
-
this.conf = conf;
- this.keyProvider = dfs.getClient().getKeyProvider();
this.hdfsAdmin = new HdfsAdmin(uri, conf);
+ // HDFS-11687 added getKeyProvider method in hdfsAdmin from Hadoop-2.9.0
Review comment:
Please remove this comment. Keep this information in JIRA. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] umamaheswararao commented on a change in pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on a change in pull request #1279:
URL: https://github.com/apache/hive/pull/1279#discussion_r457661391
##########
File path: shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
##########
@@ -1223,11 +1223,31 @@ public static boolean isHdfsEncryptionSupported() {
private final Configuration conf;
public HdfsEncryptionShim(URI uri, Configuration conf) throws IOException {
- DistributedFileSystem dfs = (DistributedFileSystem)FileSystem.get(uri, conf);
-
this.conf = conf;
- this.keyProvider = dfs.getClient().getKeyProvider();
this.hdfsAdmin = new HdfsAdmin(uri, conf);
+ this.keyProvider = getKeyProvider();
+ }
+
+ private KeyProvider getKeyProvider() throws IOException {
+ if (isMethodExist(HdfsAdmin.class, "getKeyProvider")) {
Review comment:
That is great. I will remove that then. Thanks for pointing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] umamaheswararao commented on pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on pull request #1279:
URL: https://github.com/apache/hive/pull/1279#issuecomment-661339873
Thanks @ashutoshc for the review!. I have updated it. Please take a look. Thank you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #1279: HIVE-16490. Hive should not use getKeyProvider from DFSClient directly.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1279:
URL: https://github.com/apache/hive/pull/1279#issuecomment-695860130
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org