[jira] [Comment Edited] (FALCON-2011) Document for creating cluster entity in HA enabled cluster

["Rahul Pathak (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316525#comment-15316525 ] 

Rahul Pathak edited comment on FALCON-2011 at 2/9/17 9:14 PM:
--------------------------------------------------------------

It was problem with proxyusers.hosts in core-site of source cluster.

I have set all proxyusers.hosts and proxyusers.groups config to *

Correct configuration is :

In source cluster entity
{code}<interface type="execute" endpoint="source-resourcemanager1:8050" version="2.2.0"/> (same as yarn.resourcemanager.address which is set in yarn-site.xml)
{code}
In target cluster entity
{code}<interface type="execute" endpoint="target-resourcemanager1:8050" version="2.2.0"/>
(same as yarn.resourcemanager.address which is set in yarn-site.xml)
{code}
In oozie-site on target cluster:
{code}
oozie.service.HadoopAccessorService.hadoop.configurations=*={{hadoop_conf_dir}},source-resourcemanager1:8050=/etc/hadoop_primary/conf,source-resourcemanager2:8050=/etc/hadoop_primary/conf,src-namenode1:8020=/etc/hadoop_primary/conf,src-namenode2:8020=/etc/hadoop_primary/conf
{code}


was (Author: rahulp):
It was problem with proxyusers.hosts in core-site of source cluster.

I have set all proxyusers.hosts and proxyusers.groups config to *

Correct configuration is :

In source cluster entity
{code}<interface type="execute" endpoint="source-resourcemanager1:8050" version="2.2.0"/> (same as yarn.resourcemanager.address which is set in yarn-site.xml)
{code}
In target cluster entity
{code}<interface type="execute" endpoint="target-resourcemanager1:8050" version="2.2.0"/>
(same as yarn.resourcemanager.address which is set in yarn-site.xml)
{code}
In oozie-site on target cluster:
{code}
oozie.service.HadoopAccessorService.hadoop.configurations=*={{hadoop_conf_dir}},source-resourcemanager1:8050=/etc/hadoop_primary/conf,source-resourcemanager2:8050=/etc/hadoop_primary/conf,src1.hwxblr.com:8020=/etc/hadoop_primary/conf,src2.hwxblr.com:8020=/etc/hadoop_primary/conf
{code}

> Document for creating cluster entity in HA enabled cluster
> ----------------------------------------------------------
>
>                 Key: FALCON-2011
>                 URL: https://issues.apache.org/jira/browse/FALCON-2011
>             Project: Falcon
>          Issue Type: Task
>          Components: client, docs
>    Affects Versions: 0.6.1
>         Environment: Hortonworks 2.3.4 as source and target cluster(2 different clusters). 
> HA enabled for both RM and NN in both the clusters.
> Trying to implement https://falcon.apache.org/HiveIntegration.html
>            Reporter: Rahul Pathak
>            Priority: Blocker
>              Labels: documentation
>
> I have tried below cluster entity definitions without any success:
> For source cluster one at a time:
> interface type="execute" endpoint="src-resourceManager1:8050"
> interface type="execute" endpoint="src-resourceManager2:8050"
> interface type="execute" endpoint="src-resourceManager1:8032"
> interface type="execute" endpoint="src-resourceManager2:8032"
> On target cluster one at a time:
> interface type="execute" endpoint="tgt-resourceManager1:8050"
> interface type="execute" endpoint="tgt-resourceManager2:8050"
> interface type="execute" endpoint="tgt-resourceManager1:8032"
> interface type="execute" endpoint="tgt-resourceManager2:8032"
> With different combination of below on target oozie server
> {code}
> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> 
> <value>*=/etc/hadoop/conf,src-namenode1:8020=/etc/src_hadoop/conf,src-resourceManager1:8050=/etc/src_hadoop/conf,src-namenode2:8020=/etc/src_hadoop/conf,src-resourceManager2:8050=/etc/src_hadoop/conf</value> 
> {code}
> *And*
> {code}
> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> 
> <value>*=/etc/hadoop/conf,src-namenode1:8020=/etc/src_hadoop/conf,src-resourceManager1:8032=/etc/src_hadoop/conf,src-namenode2:8020=/etc/src_hadoop/conf,src-resourceManager2:8032=/etc/src_hadoop/conf</value> 
> {code}
> I get below oozie exception while running table-export 
> {code}
> JA009: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses
> {code}
> *However it has worked fine before enabling HA on both the clusters.*



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)