You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Rushabh S Shah (JIRA)" <ji...@apache.org> on 2017/04/20 20:12:04 UTC
[jira] [Comment Edited] (HADOOP-14333) HADOOP-14104 changed
DFSClient API isHDFSEncryptionEnabled, impacted hacky hive code
[ https://issues.apache.org/jira/browse/HADOOP-14333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977379#comment-15977379 ]
Rushabh S Shah edited comment on HADOOP-14333 at 4/20/17 8:11 PM:
------------------------------------------------------------------
If I understood the hive code correctly, {{isEncryptionEnabled}} method is getting called indirectly by CryptoProcessor.
The javadoc for the {{CryptoProcessor}} says:
{noformat}
/**
* This class processes HADOOP commands used for HDFS encryption. It is meant to be run
* only by Hive unit & queries tests.
*/
public class CryptoProcessor implements CommandProcessor {
{noformat}
So I assume its only test code that is breaking. In that case I wouldn't change anything from HADOOP-14104 just to fix hive tests failure.
bq. removing the throws added by HADOOP-14104 is a quicker solution to unblock hive
Secondly removing throws and returning false in case of Exception is _wrong_ since there can be any exception while calling getServerDefaults (like socket timeout) and we will return false even if encryption is enabled.
It will be good to hear from members who are working on hive.
PS: I was referring to code from hive git repository master branch.
was (Author: shahrs87):
If I understood the hive code correctly, {{isEncryptionEnabled}} method is getting called indirectly by CryptoProcessor.
The javadoc for the {{CryptoProcessor}} says:
{noformat}
/**
* This class processes HADOOP commands used for HDFS encryption. It is meant to be run
* only by Hive unit & queries tests.
*/
public class CryptoProcessor implements CommandProcessor {
{noformat}
So I assume its only test code that is breaking. In that case I wouldn't change anything from HADOOP-14104 just to fix hive tests failure.
bq. removing the throws added by HADOOP-14104 is a quicker solution to unblock hive
Secondly removing throws and returning false in case of Exception is _wrong_ since there can be any exception while calling getServerDefaults (like socket timeout) and we will return false even if encryption is enabled.
It will be good to hear from members who are working on hive.
> HADOOP-14104 changed DFSClient API isHDFSEncryptionEnabled, impacted hacky hive code
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-14333
> URL: https://issues.apache.org/jira/browse/HADOOP-14333
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
>
> Though Hive should be fixed not to access DFSClient which is private to HADOOP, removing the throws added by HADOOP-14104 is a quicker solution to unblock hive.
> Hive code
> {code}
> private boolean isEncryptionEnabled(DFSClient client, Configuration conf) {
> try {
> DFSClient.class.getMethod("isHDFSEncryptionEnabled");
> } catch (NoSuchMethodException e) {
> // the method is available since Hadoop-2.7.1
> // if we run with an older Hadoop, check this ourselves
> return !conf.getTrimmed(DFSConfigKeys.DFS_ENCRYPTION_KEY_PROVIDER_URI, "").isEmpty();
> }
> return client.isHDFSEncryptionEnabled();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org