You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@livy.apache.org by kant kodali <ka...@gmail.com> on 2017/12/06 07:34:43 UTC
How to set driverMemory, driverCores, executorMemory using livy?
Hi All,
I do see POST /sessions API where I can pass driverMemory, driverCores,
executorMemory as part of the request body but I am using programmatic API
to submit upload the Jar and submit my job so how do I set values
for driverMemory, driverCores, executorMemory ?
Thanks!
Re: How to set driverMemory, driverCores, executorMemory using livy?
Posted by Saisai Shao <sa...@gmail.com>.
You should write as *Spark configuration* like spark.xxx.
2017-12-06 16:23 GMT+08:00 kant kodali <ka...@gmail.com>:
> Does the below code look right?
>
> new LivyClientBuilder()
> .setURI(new URI(livyUrl))
> .setConf("driverMemory", "4g").setConf("executorMemory", "4g").setConf("driverCores", "4").build();
>
>
> On Wed, Dec 6, 2017 at 12:20 AM, kant kodali <ka...@gmail.com> wrote:
>
>> Thanks much!
>>
>> On Wed, Dec 6, 2017 at 12:16 AM, Saisai Shao <sa...@gmail.com>
>> wrote:
>>
>>> Using this API "public LivyClientBuilder setConf(String key, String
>>> value)" to set Spark configurations you wanted.
>>>
>>> 2017-12-06 15:34 GMT+08:00 kant kodali <ka...@gmail.com>:
>>>
>>>> Hi All,
>>>>
>>>>
>>>> I do see POST /sessions API where I can pass driverMemory, driverCores,
>>>> executorMemory as part of the request body but I am using programmatic API
>>>> to submit upload the Jar and submit my job so how do I set values
>>>> for driverMemory, driverCores, executorMemory ?
>>>>
>>>> Thanks!
>>>>
>>>>
>>>
>>
>
Re: How to set driverMemory, driverCores, executorMemory using livy?
Posted by kant kodali <ka...@gmail.com>.
Does the below code look right?
new LivyClientBuilder()
.setURI(new URI(livyUrl))
.setConf("driverMemory", "4g").setConf("executorMemory",
"4g").setConf("driverCores", "4").build();
On Wed, Dec 6, 2017 at 12:20 AM, kant kodali <ka...@gmail.com> wrote:
> Thanks much!
>
> On Wed, Dec 6, 2017 at 12:16 AM, Saisai Shao <sa...@gmail.com>
> wrote:
>
>> Using this API "public LivyClientBuilder setConf(String key, String
>> value)" to set Spark configurations you wanted.
>>
>> 2017-12-06 15:34 GMT+08:00 kant kodali <ka...@gmail.com>:
>>
>>> Hi All,
>>>
>>>
>>> I do see POST /sessions API where I can pass driverMemory, driverCores,
>>> executorMemory as part of the request body but I am using programmatic API
>>> to submit upload the Jar and submit my job so how do I set values
>>> for driverMemory, driverCores, executorMemory ?
>>>
>>> Thanks!
>>>
>>>
>>
>
Re: How to set driverMemory, driverCores, executorMemory using livy?
Posted by kant kodali <ka...@gmail.com>.
Thanks much!
On Wed, Dec 6, 2017 at 12:16 AM, Saisai Shao <sa...@gmail.com> wrote:
> Using this API "public LivyClientBuilder setConf(String key, String
> value)" to set Spark configurations you wanted.
>
> 2017-12-06 15:34 GMT+08:00 kant kodali <ka...@gmail.com>:
>
>> Hi All,
>>
>>
>> I do see POST /sessions API where I can pass driverMemory, driverCores,
>> executorMemory as part of the request body but I am using programmatic API
>> to submit upload the Jar and submit my job so how do I set values
>> for driverMemory, driverCores, executorMemory ?
>>
>> Thanks!
>>
>>
>
Re: How to set driverMemory, driverCores, executorMemory using livy?
Posted by Saisai Shao <sa...@gmail.com>.
Using this API "public LivyClientBuilder setConf(String key, String
value)" to set Spark configurations you wanted.
2017-12-06 15:34 GMT+08:00 kant kodali <ka...@gmail.com>:
> Hi All,
>
>
> I do see POST /sessions API where I can pass driverMemory, driverCores,
> executorMemory as part of the request body but I am using programmatic API
> to submit upload the Jar and submit my job so how do I set values
> for driverMemory, driverCores, executorMemory ?
>
> Thanks!
>
>