[jira] [Resolved] (HIVE-14369) submit a task with hive on spark to other yarn cluster failed

["Shohei Okumiya (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-14369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shohei Okumiya resolved HIVE-14369.
-----------------------------------
    Fix Version/s: NA
       Resolution: Won't Fix

We have discontinued Hive on Spark and EoLed Hive 3. HIVE-26134

> submit a task with hive on spark to other yarn cluster failed
> -------------------------------------------------------------
>
>                 Key: HIVE-14369
>                 URL: https://issues.apache.org/jira/browse/HIVE-14369
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Xiaomeng Huang
>            Priority: Major
>             Fix For: NA
>
>
> In our environment, we have two hadoop clusters with HA,  named hivecluster and sparkcluster.
> hivecluster is a HA hadoop cluster for hive, which has large hard disk.
> sparkcluster is a HA hadoop cluster for spark, which has large memory.
> e.g. below is a hdfs-site.xml of hivecluster:
> {code}
> <property>
>         <name>dfs.ha.namenodes.hivecluster</name>
>         <value>nn1,nn2</value>
> </property>
> <property>
>         <name>dfs.namenode.rpc-address.hivecluster.nn1</name>
>         <value>10.17.21.32:9000</value>
> </property>
> <property>
>         <name>dfs.namenode.rpc-address.hivecluster.nn2</name>
>         <value>10.17.21.77:9000</value>
> </property>
> <property>
>         <name>dfs.namenode.http-address.hivecluster.nn1</name>
>         <value>10.17.21.32:50070</value>
> </property>
> <property>
>         <name>dfs.namenode.http-address.hivecluster.nn2</name>
>         <value>10.17.21.77:50070</value>
> </property>
> {code}
> Firstly, I created a hive table located as hdfs://hivecluster/hive/warehouse/xxx
> If I use hive on mr, it will run successfully.
> But if I use hive on spark to submit a task to yarn cluster of sparkcluster, it says:
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
> yarn日志显示：
> Diagnostics: java.lang.IllegalArgumentException: java.net.UnknownHostException: hivecluster
> Failing this attempt. Failing the application.
> {code}
> I didn't set host of hivecluster into hdfs-site.xml of sparkcluster



--
This message was sent by Atlassian Jira
(v8.20.10#820010)