You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Oleh Halenok (JIRA)" <ji...@apache.org> on 2015/09/22 12:13:04 UTC
[jira] [Commented] (SPARK-6284) Support framework authentication
and role in Mesos framework
[ https://issues.apache.org/jira/browse/SPARK-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902336#comment-14902336 ]
Oleh Halenok commented on SPARK-6284:
-------------------------------------
Have a cluster, Mesos 0.22.1, hadoop 2.5.0, 3 masters, 8 slaves, which are 3 - role "fast", 3 - role "*", 2 other roles.
Spark 1.5.0 is deployed to use cluster mode with dispatcher, it looks working, however I can not get it working on the particular nodes with role "fast" configured.
Job submitted as # bin/spark-submit --class org.apache.spark.examples.SparkPi --master master.novalocal:7077 --deploy-mode cluster --conf spark.mesos.role=fast http://www/spark.jar 10
Also, I've tried to configure Spark defaults for submitted jobs and added to /opt/spark/conf/spark-defaults.conf the line "spark.mesos.role fast"
In any case after submitting job is failing with the next error throwing, it looks driver starts with some defaults - cpu, mem, from spark ui:
State: TASK_ERROR, Message: Task uses more resources cpus(*):1; mem(*):1024 than available cpus(fast):8; mem(fast):14863; disk(fast):45243; ports(fast):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-65535], Source: SOURCE_MASTER, Reason: REASON_TASK_INVALID, Time: 1.442868857232848E9
Can you please describe how to configure/start it properly?
> Support framework authentication and role in Mesos framework
> ------------------------------------------------------------
>
> Key: SPARK-6284
> URL: https://issues.apache.org/jira/browse/SPARK-6284
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Assignee: Timothy Chen
> Fix For: 1.5.0
>
>
> Support framework authentication and role in both Coarse grain and fine grain mode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org