You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Yesha Vora (JIRA)" <ji...@apache.org> on 2017/08/21 18:10:06 UTC
[jira] [Created] (AMBARI-21770) spark1 shuffle service not
configured properly
Yesha Vora created AMBARI-21770:
-----------------------------------
Summary: spark1 shuffle service not configured properly
Key: AMBARI-21770
URL: https://issues.apache.org/jira/browse/AMBARI-21770
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.5.2
Reporter: Yesha Vora
Priority: Blocker
Steps to reproduce:
* Install BI 4.2.0
* Upgrade to Ambari-2.5.2.0
* Upgrade to HDP-2.6
Here, Spark-shuffle service is not configured for Spark1 component. It only configures Spark2-shuffle service
{code:title=current yarn-site.xml after upgrade}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark2_shuffle
yarn.nodemanager.aux-services.spark2_shuffle.class = org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark2_shuffle.classpath={{stack_root}}/${hdp.version}/spark2/aux/*
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes <EMPTY VALUE>
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath <EMPTY VALUE>{code}
{code:title=expected yarn-site.xml}
yarn.nodemanager.aux-services = mapreduce_shuffle,spark_shuffle
yarn.nodemanager.aux-services.spark_shuffle.class=org.apache.spark.network.yarn.YarnShuffleService
yarn.nodemanager.aux-services.spark_shuffle.classpath={{stack_root}}/${hdp.version}/spark/aux/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath=/usr/hdp/${hdp.version}/spark/hdpLib/*
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes=org.apache.spark.deploy.history.yarn.plugin.SparkATSPlugin{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)