You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Mark Memory <gh...@gmail.com> on 2016/05/19 12:45:18 UTC

how can I access HDFS(namenode HA) by using hive-common-1.2.1.jar??

Hi guys,

who knows why class HiveConf has no HADOOPCONF enum type in hive common jar
now?
I write code using hive-common-1.2.1.jar HiveConf class to access HDFS(HA
namenode), and I get an error below. I realized my code didn't
config HADOOPCONF so it can't connect to HDFS, but there is no HADOOPCONF
in hive-common-1.2.1.jar, I found previous version of hive-common has the
HADOOPCONF [
http://www.docjar.com/html/api/org/apache/hadoop/hive/conf/HiveConf.java.html
].

My question is how can I access HDFS by using hive-common-1.2.1.jar??

Thank you !!!

*the error is:*

Caused by: java.lang.IllegalArgumentException:
java.net.UnknownHostException: cluster

at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374)

at
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:312)

at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:178)

at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:665)

*My code is:*

hiveConf.setVar(HiveConf.ConfVars.HADOOPBIN, "/opt/modules/hadoop/bin");

hiveConf.setVar(HiveConf.ConfVars.HADOOPFS, "hdfs://cluster");

hiveConf.setVar(HiveConf.ConfVars.LOCALSCRATCHDIR, "/opt/modules/hive/temp
");

    hiveConf.setVar(HiveConf.ConfVars.DOWNLOADED_RESOURCES_DIR,
"/opt/modules/hive/temp");

    hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY, false);

hiveConf.setVar(HiveConf.ConfVars.METASTOREWAREHOUSE, "/warehouse");

hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, "thrift://127.0.0.1:9083");

    hiveConf.setVar(HiveConf.ConfVars.METASTORE_CONNECTION_DRIVER,
"com.mysql.jdbc.Driver");

    hiveConf.setVar(HiveConf.ConfVars.METASTORECONNECTURLKEY, "jdbc:mysql://
192.168.5.29:3306/hive?createDatabaseIfNotExist=true");

    hiveConf.setVar(HiveConf.ConfVars.METASTORE_CONNECTION_USER_NAME,
"hive");

hiveConf.setVar(HiveConf.ConfVars.METASTOREPWD, "123456");

hiveConf.setVar(HiveConf.ConfVars.HIVEHISTORYFILELOC, "/opt/modules/hive/
temp");