You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Yesha Vora (JIRA)" <ji...@apache.org> on 2017/08/21 18:10:06 UTC

[jira] [Created] (AMBARI-21770) spark1 shuffle service not configured properly

Yesha Vora created AMBARI-21770:
-----------------------------------

             Summary: spark1 shuffle service not configured properly
                 Key: AMBARI-21770
                 URL: https://issues.apache.org/jira/browse/AMBARI-21770
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.5.2
            Reporter: Yesha Vora
            Priority: Blocker


Steps to reproduce:
* Install BI 4.2.0
* Upgrade to Ambari-2.5.2.0
* Upgrade to HDP-2.6

Here, Spark-shuffle service is not configured for Spark1 component. It only configures Spark2-shuffle service
{code:title=current yarn-site.xml after upgrade}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
yarn.nodemanager.aux-services.spark2_shuffle.class = org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes  <EMPTY VALUE>
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath <EMPTY VALUE>{code}

{code:title=expected yarn-site.xml}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)