You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jia Yu <ji...@gmail.com> on 2016/07/26 23:10:46 UTC
Re: how to use spark.mesos.constraints
Hi,
I am also trying to use the spark.mesos.constraints but it gives me the
same error: job has not be accepted by any resources.
I am doubting that I should start some additional service like
./sbin/start-mesos-shuffle-service.sh. Am I correct?
Thanks,
Jia
On Tue, Dec 1, 2015 at 5:14 PM, rarediel <br...@gettyimages.com>
wrote:
> I am trying to add mesos constraints to my spark-submit command in my
> marathon file I am setting it to spark.mesos.coarse=true.
>
> Here is an example of a constraint I am trying to set.
>
> --conf spark.mesos.constraint=cpus:2
>
> I want to use the constraints to control the amount of executors are
> created
> so I can control the total memory of my spark job.
>
> I've tried many variations of resource constraints, but no matter which
> resource or what number, range, etc. I do I always get the error "Initial
> job has not accepted any resources; check your cluster UI...". My cluster
> has the available resources. Is there any examples I can look at where
> people use resource constraints?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-use-spark-mesos-constraints-tp25541.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: how to use spark.mesos.constraints
Posted by Michael Gummelt <mg...@mesosphere.io>.
If you run your jobs with debug logging on in Mesos, it should print why
the offer is being declined:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala#L301
On Tue, Jul 26, 2016 at 6:38 PM, Rodrick Brown <ro...@orchardplatform.com>
wrote:
> Shuffle service has nothing to do with constraints it is however advised
> to run the mesos-shuffle-service on each of your agent nodes running spark.
>
> Here is the command I use to run a typical spark jobs on my cluster using
> constraints (this is generated but from another script we run but should
> give you a clear idea)
>
> Jobs not being accepted by any resources could mean what you're asking for
> is way larger than the resources you have available.
>
> /usr/bin/timeout 3600 /opt/spark-1.6.1/bin/spark-submit
> --master "mesos://zk://prod-zk-1:2181,prod-zk-2:2181,prod-zk-3:2181/mesos"
> --conf spark.ui.port=40046
> --conf spark.mesos.coarse=true
> --conf spark.sql.broadcastTimeout=3600
> --conf spark.cores.max=5
> --conf spark.mesos.constraints="rack:spark"
> --conf spark.sql.tungsten.enabled=true
> --conf spark.shuffle.service.enabled=true
> --conf spark.dynamicAllocation.enabled=true
> --conf spark.mesos.executor.memoryOverhead=3211
> --class
> com.orchard.dataloader.library.originators.xxxx.LoadAccountDetail_LC
> --total-executor-cores 5
> --driver-memory 5734M
> --executor-memory 8028M
> --jars /data/orchard/etc/config/load-accountdetail-accumulo-prod.jar
> /data/orchard/jars/dataloader-library-assembled.jar 1
>
> Nodes used for my spark jobs are all using the constraint 'rack:spark'
>
> I hope this helps!
>
>
> On Tue, Jul 26, 2016 at 7:10 PM, Jia Yu <ji...@gmail.com> wrote:
>
>> Hi,
>>
>> I am also trying to use the spark.mesos.constraints but it gives me the
>> same error: job has not be accepted by any resources.
>>
>> I am doubting that I should start some additional service like
>> ./sbin/start-mesos-shuffle-service.sh. Am I correct?
>>
>> Thanks,
>> Jia
>>
>> On Tue, Dec 1, 2015 at 5:14 PM, rarediel <br...@gettyimages.com>
>> wrote:
>>
>>> I am trying to add mesos constraints to my spark-submit command in my
>>> marathon file I am setting it to spark.mesos.coarse=true.
>>>
>>> Here is an example of a constraint I am trying to set.
>>>
>>> --conf spark.mesos.constraint=cpus:2
>>>
>>> I want to use the constraints to control the amount of executors are
>>> created
>>> so I can control the total memory of my spark job.
>>>
>>> I've tried many variations of resource constraints, but no matter which
>>> resource or what number, range, etc. I do I always get the error "Initial
>>> job has not accepted any resources; check your cluster UI...". My
>>> cluster
>>> has the available resources. Is there any examples I can look at where
>>> people use resource constraints?
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-use-spark-mesos-constraints-tp25541.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>>
>>
>
>
> --
>
> [image: Orchard Platform] <http://www.orchardplatform.com/>
>
> *Rodrick Brown */ *DevOPs*
>
> 9174456839 / rodrick@orchardplatform.com
>
> Orchard Platform
> 101 5th Avenue, 4th Floor, New York, NY
>
> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
> for the use of the addressee only. If you are not an intended recipient of
> this communication, please delete it immediately and notify the sender by
> return email. Unauthorized reading, dissemination, distribution or copying
> of this communication is prohibited. This communication does not constitute
> an offer to sell or a solicitation of an indication of interest to purchase
> any loan, security or any other financial product or instrument, nor is it
> an offer to sell or a solicitation of an indication of interest to purchase
> any products or services to any persons who are prohibited from receiving
> such information under applicable law. The contents of this communication
> may not be accurate or complete and are subject to change without notice.
> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
> "Orchard") makes no representation regarding the accuracy or completeness
> of the information contained herein. The intended recipient is advised to
> consult its own professional advisors, including those specializing in
> legal, tax and accounting matters. Orchard does not provide legal, tax or
> accounting advice.
>
--
Michael Gummelt
Software Engineer
Mesosphere
Re: how to use spark.mesos.constraints
Posted by Rodrick Brown <ro...@orchardplatform.com>.
Shuffle service has nothing to do with constraints it is however advised to
run the mesos-shuffle-service on each of your agent nodes running spark.
Here is the command I use to run a typical spark jobs on my cluster using
constraints (this is generated but from another script we run but should
give you a clear idea)
Jobs not being accepted by any resources could mean what you're asking for
is way larger than the resources you have available.
/usr/bin/timeout 3600 /opt/spark-1.6.1/bin/spark-submit
--master "mesos://zk://prod-zk-1:2181,prod-zk-2:2181,prod-zk-3:2181/mesos"
--conf spark.ui.port=40046
--conf spark.mesos.coarse=true
--conf spark.sql.broadcastTimeout=3600
--conf spark.cores.max=5
--conf spark.mesos.constraints="rack:spark"
--conf spark.sql.tungsten.enabled=true
--conf spark.shuffle.service.enabled=true
--conf spark.dynamicAllocation.enabled=true
--conf spark.mesos.executor.memoryOverhead=3211
--class
com.orchard.dataloader.library.originators.xxxx.LoadAccountDetail_LC
--total-executor-cores 5
--driver-memory 5734M
--executor-memory 8028M
--jars /data/orchard/etc/config/load-accountdetail-accumulo-prod.jar
/data/orchard/jars/dataloader-library-assembled.jar 1
Nodes used for my spark jobs are all using the constraint 'rack:spark'
I hope this helps!
On Tue, Jul 26, 2016 at 7:10 PM, Jia Yu <ji...@gmail.com> wrote:
> Hi,
>
> I am also trying to use the spark.mesos.constraints but it gives me the
> same error: job has not be accepted by any resources.
>
> I am doubting that I should start some additional service like
> ./sbin/start-mesos-shuffle-service.sh. Am I correct?
>
> Thanks,
> Jia
>
> On Tue, Dec 1, 2015 at 5:14 PM, rarediel <br...@gettyimages.com>
> wrote:
>
>> I am trying to add mesos constraints to my spark-submit command in my
>> marathon file I am setting it to spark.mesos.coarse=true.
>>
>> Here is an example of a constraint I am trying to set.
>>
>> --conf spark.mesos.constraint=cpus:2
>>
>> I want to use the constraints to control the amount of executors are
>> created
>> so I can control the total memory of my spark job.
>>
>> I've tried many variations of resource constraints, but no matter which
>> resource or what number, range, etc. I do I always get the error "Initial
>> job has not accepted any resources; check your cluster UI...". My cluster
>> has the available resources. Is there any examples I can look at where
>> people use resource constraints?
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-use-spark-mesos-constraints-tp25541.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
--
[image: Orchard Platform] <http://www.orchardplatform.com/>
*Rodrick Brown */ *DevOPs*
9174456839 / rodrick@orchardplatform.com
Orchard Platform
101 5th Avenue, 4th Floor, New York, NY
--
*NOTICE TO RECIPIENTS*: This communication is confidential and intended for
the use of the addressee only. If you are not an intended recipient of this
communication, please delete it immediately and notify the sender by return
email. Unauthorized reading, dissemination, distribution or copying of this
communication is prohibited. This communication does not constitute an
offer to sell or a solicitation of an indication of interest to purchase
any loan, security or any other financial product or instrument, nor is it
an offer to sell or a solicitation of an indication of interest to purchase
any products or services to any persons who are prohibited from receiving
such information under applicable law. The contents of this communication
may not be accurate or complete and are subject to change without notice.
As such, Orchard App, Inc. (including its subsidiaries and affiliates,
"Orchard") makes no representation regarding the accuracy or completeness
of the information contained herein. The intended recipient is advised to
consult its own professional advisors, including those specializing in
legal, tax and accounting matters. Orchard does not provide legal, tax or
accounting advice.