Spark executor memory information

[Naveen Dabas <na...@ymail.com>]

Hi,



I am new to spark and need some guidance on below mentioned points:
1)    I am using spark 1.2,is it possible to see how much memory is being allocated to an executor for web UI. If not how can we figure that out.2)    I am interested in source code of mlib,it is possible to get access to it and spark core implementations.

   

Re: Spark executor memory information

[Akhil Das <ak...@sigmoidanalytics.com>]

1. Yes open up the webui running on 8080 to see the memory/cores allocated
to your workers, and open up the ui running on 4040 and click on the
Executor tab to see the memory allocated for the executor.

2. mllib codes can be found over here
<https://github.com/apache/spark/tree/master/mllib> and spark core from here
<https://github.com/apache/spark/tree/master/core>

Thanks
Best Regards

On Tue, Jul 14, 2015 at 11:49 AM, Naveen Dabas <na...@ymail.com>
wrote:

> Hi,
>
>
>
> I am new to spark and need some guidance on below mentioned points:
>
> 1)    I am using spark 1.2,is it possible to see how much memory is being
> allocated to an executor for web UI. If not how can we figure that out.
> 2)    I am interested in source code of mlib,it is possible to get access
> to it and spark core implementations.
>
>
>