You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by JG Perrin <jp...@lumeris.com> on 2017/09/26 13:40:57 UTC

Debugging Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Hi,

I get the infamous:
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

I run the app via Eclipse, connecting:
        SparkSession spark = SparkSession.builder()
                .appName("Converter - Benchmark")
                .master(ConfigurationManager.getMaster())
                .config("spark.cores.max", "4")
                .config("spark.executor.memory", "16g")
                .getOrCreate();


Everything seems ok on the cluster side:
[cid:image001.png@01D336A3.2B215410]


I probably missed something super obvious, but can't find it...

Any help/hint is welcome! - TIA

jg

Re: Debugging Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Posted by Sathishkumar Manimoorthy <mr...@gmail.com>.
@Ayan

It seems to be running on spark standalone. Not mostly on Yarn I guess.

Thanks,
Sathish

On Tue, Sep 26, 2017 at 9:09 PM, ayan guha <gu...@gmail.com> wrote:

> I would check the queue you are submitting job, assuming it is yarn...
>
> On Tue, Sep 26, 2017 at 11:40 PM, JG Perrin <jp...@lumeris.com> wrote:
>
>> Hi,
>>
>>
>>
>> I get the infamous:
>>
>> Initial job has not accepted any resources; check your cluster UI to
>> ensure that workers are registered and have sufficient resources
>>
>>
>>
>> I run the app via Eclipse, connecting:
>>
>>         SparkSession spark = SparkSession.*builder*()
>>
>>                 .appName("Converter - Benchmark")
>>
>>                 .master(ConfigurationManager.*getMaster*())
>>
>>                 .config("spark.cores.max", "4")
>>
>>                 .config("spark.executor.memory", "16g")
>>
>>                 .getOrCreate();
>>
>>
>>
>>
>>
>> Everything seems ok on the cluster side:
>>
>>
>>
>>
>>
>> I probably missed something super obvious, but can’t find it…
>>
>>
>>
>> Any help/hint is welcome! - TIA
>>
>>
>>
>> jg
>>
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Best Regards,
> Ayan Guha
>

RE: Debugging Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Posted by JG Perrin <jp...@lumeris.com>.
not using Yarn, just standalone cluster with 2 nodes here (physical, not even VM). network seems good between the nodes .

From: ayan guha [mailto:guha.ayan@gmail.com]
Sent: Tuesday, September 26, 2017 10:39 AM
To: JG Perrin <jp...@lumeris.com>
Cc: user@spark.apache.org
Subject: Re: Debugging Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

I would check the queue you are submitting job, assuming it is yarn...

On Tue, Sep 26, 2017 at 11:40 PM, JG Perrin <jp...@lumeris.com>> wrote:
Hi,

I get the infamous:
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

I run the app via Eclipse, connecting:
        SparkSession spark = SparkSession.builder()
                .appName("Converter - Benchmark")
                .master(ConfigurationManager.getMaster())
                .config("spark.cores.max", "4")
                .config("spark.executor.memory", "16g")
                .getOrCreate();


Everything seems ok on the cluster side:
[cid:image001.png@01D336B5.FF7933B0]


I probably missed something super obvious, but can’t find it…

Any help/hint is welcome! - TIA

jg






--
Best Regards,
Ayan Guha

Re: Debugging Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Posted by ayan guha <gu...@gmail.com>.
I would check the queue you are submitting job, assuming it is yarn...

On Tue, Sep 26, 2017 at 11:40 PM, JG Perrin <jp...@lumeris.com> wrote:

> Hi,
>
>
>
> I get the infamous:
>
> Initial job has not accepted any resources; check your cluster UI to
> ensure that workers are registered and have sufficient resources
>
>
>
> I run the app via Eclipse, connecting:
>
>         SparkSession spark = SparkSession.*builder*()
>
>                 .appName("Converter - Benchmark")
>
>                 .master(ConfigurationManager.*getMaster*())
>
>                 .config("spark.cores.max", "4")
>
>                 .config("spark.executor.memory", "16g")
>
>                 .getOrCreate();
>
>
>
>
>
> Everything seems ok on the cluster side:
>
>
>
>
>
> I probably missed something super obvious, but can’t find it…
>
>
>
> Any help/hint is welcome! - TIA
>
>
>
> jg
>
>
>
>
>
>
>



-- 
Best Regards,
Ayan Guha