You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Heisenberg Bb <hb...@gmail.com> on 2015/05/13 14:57:01 UTC

Building Spark

I tried to build Spark in my local machine Ubuntu 14.04 ( 4 GB Ram), my
system  is getting hanged (freezed). When I monitered system processes, the
build process is found to consume 85% of my memory. Why does it need lot of
resources. Is there any efficient method to build Spark.

Thanks
Akhil

Re: Building Spark

Posted by Emre Sevinc <em...@gmail.com>.
My 2 cents: If you have Java 8, you don't need any extra settings for
Maven.

--
Emre Sevinç

On Wed, May 13, 2015 at 3:02 PM, Stephen Boesch <ja...@gmail.com> wrote:

> Hi Akhil,   Building with sbt tends to need around 3.5GB whereas maven
> requirements are much lower , around 1.7GB. So try using maven .
>
> For reference I have the following settings and both do compile.  sbt
> would not work with lower values.
>
>
> $echo $SBT_OPTS
> -Xmx3012m -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=512m
> $echo $MAVEN_OPTS
> -Xmx1280m -XX:MaxPermSize=384m
>
> 2015-05-13 5:57 GMT-07:00 Heisenberg Bb <hb...@gmail.com>:
>
>> I tried to build Spark in my local machine Ubuntu 14.04 ( 4 GB Ram), my
>> system  is getting hanged (freezed). When I monitered system processes, the
>> build process is found to consume 85% of my memory. Why does it need lot of
>> resources. Is there any efficient method to build Spark.
>>
>> Thanks
>> Akhil
>>
>
>


-- 
Emre Sevinc

Re: Building Spark

Posted by Stephen Boesch <ja...@gmail.com>.
Hi Akhil,   Building with sbt tends to need around 3.5GB whereas maven
requirements are much lower , around 1.7GB. So try using maven .

For reference I have the following settings and both do compile.  sbt would
not work with lower values.


$echo $SBT_OPTS
-Xmx3012m -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=512m
$echo $MAVEN_OPTS
-Xmx1280m -XX:MaxPermSize=384m

2015-05-13 5:57 GMT-07:00 Heisenberg Bb <hb...@gmail.com>:

> I tried to build Spark in my local machine Ubuntu 14.04 ( 4 GB Ram), my
> system  is getting hanged (freezed). When I monitered system processes, the
> build process is found to consume 85% of my memory. Why does it need lot of
> resources. Is there any efficient method to build Spark.
>
> Thanks
> Akhil
>