[jira] [Commented] (SAMZA-519) Support for Yarn High Availability (HA RM)

["Alexander Schurman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998617#comment-14998617 ] 

Alexander Schurman commented on SAMZA-519:
------------------------------------------

Hello Jon, 

Our product (Pentaho Data Integration) was not failing either until we encounter a customer that implemented AUTH_TO_LOCAL property for Kerberos basically to accept main principal actions from internal cluster nodes only. 

To be more specific for you the property is: *hadoop.security.auth_to_local*
{noformat}
  <property>
    <name>hadoop.security.auth_to_local</name>
    <value>
RULE:[2:$1/$2@$0](hdfs/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/hdfs/
RULE:[2:$1/$2@$0](mapred/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/mapred/
RULE:[2:$1/$2@$0](yarn/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/yarn/
RULE:[2:$1/$2@$0](hive/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/hive/
RULE:[2:$1/$2@$0](hue/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/hue/
RULE:[2:$1/$2@$0](zookeeper/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/zookeeper/
RULE:[2:$1/$2@$0](impala/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/impala/
RULE:[2:$1/$2@$0](oozie/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/oozie/
RULE:[2:$1/$2@$0](solr/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/solr/
RULE:[2:$1/$2@$0](httpfs/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/httpfs/
RULE:[2:$1/$2@$0](flume/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/flume/
RULE:[2:$1/$2@$0](HTTP/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/HTTP/
RULE:[2:$1/$2@$0](sentry/node[1-9].fqdn@REALM.COM)s/(.*)@REALM.COM/sentry/
RULE:[1:$1@$0](^hdfs@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^mapred@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^yarn@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^hive@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^hue@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^HTTP@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^httpfs@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^impala@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^solr@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^oozie@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^sentry@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^flume@.*$)s/^.*$/nobody/
RULE:[1:$1@$0](^zookeeper@.*$)s/^.*$/nobody/
RULE:[2:$1/$2@$0](^.*$)s/^.*$/nobody/
DEFAULT</value>
  </property>
{noformat}

Please take a look to LAST rule that was recommended from Cloudera to rejact any other Principal activity from external nodes 
{noformat}RULE:[2:$1/$2@$0](^.*$)s/^.*$/nobody/{noformat}

This is where the issue started to be VISIBLE, and neither Map Reduce or our YARN application was working. After the Fix from Cloudera was provided, Map Reduce started to work.

Hope this helps you



> Support for Yarn High Availability (HA RM)
> ------------------------------------------
>
>                 Key: SAMZA-519
>                 URL: https://issues.apache.org/jira/browse/SAMZA-519
>             Project: Samza
>          Issue Type: New Feature
>          Components: yarn
>    Affects Versions: 0.8.1
>         Environment: Yarn 2.6.0 - 2 RMs with HA enabled
>            Reporter: Jon Bringhurst
>            Assignee: Jon Bringhurst
>             Fix For: 0.10.0
>
>
> Samza appears to have trouble working with Yarn HA.
> The AM starts up, but it tries to connect to 0.0.0.0 instead of the active RM.
> When "yarn.resourcemanager.hostname" is added back into the NM config, the AM works again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)