You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Vladimir Protsenko <pr...@gmail.com> on 2014/12/23 15:57:41 UTC
Spark Installation Maven PermGen OutOfMemoryException
I am installing Spark 1.2.0 on CentOS 6.6. Just downloaded code from github,
installed maven and trying to compile system:
git clone https://github.com/apache/spark.git
git checkout v1.2.0
mvn -DskipTests clean package
leads to OutOfMemoryException. What amount of memory does it requires?
export MAVEN_OPTS=`-Xmx=1500m -XX:MaxPermSize=512m
-XX:ReservedCodeCacheSize=512m` doesn't help.
Waht is a straight forward way to start using Spark?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Installation-Maven-PermGen-OutOfMemoryException-tp20831.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Spark Installation Maven PermGen OutOfMemoryException
Posted by varun sharma <va...@gmail.com>.
This works for me:
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M
-XX:ReservedCodeCacheSize=512m" && mvn -DskipTests clean package
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Installation-Maven-PermGen-OutOfMemoryException-tp20831p20868.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: Spark Installation Maven PermGen OutOfMemoryException
Posted by Somnath Pandeya <So...@infosys.com>.
I think you should use minimum of 2gb of memory for building it from maven .
-Somnath
-----Original Message-----
From: Vladimir Protsenko [mailto:protsenkovi@gmail.com]
Sent: Tuesday, December 23, 2014 8:28 PM
To: user@spark.apache.org
Subject: Spark Installation Maven PermGen OutOfMemoryException
I am installing Spark 1.2.0 on CentOS 6.6. Just downloaded code from github, installed maven and trying to compile system:
git clone https://github.com/apache/spark.git
git checkout v1.2.0
mvn -DskipTests clean package
leads to OutOfMemoryException. What amount of memory does it requires?
export MAVEN_OPTS=`-Xmx=1500m -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=512m` doesn't help.
Waht is a straight forward way to start using Spark?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Installation-Maven-PermGen-OutOfMemoryException-tp20831.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Spark Installation Maven PermGen OutOfMemoryException
Posted by Vladimir Protsenko <pr...@gmail.com>.
Thanks. Bad mistake.
2014-12-24 14:02 GMT+04:00 Sean Owen <so...@cloudera.com>:
> That command is still wrong. It is -Xmx3g with no =.
> On Dec 24, 2014 9:50 AM, "Vladimir Protsenko" <pr...@gmail.com>
> wrote:
>
>> Java 8 rpm 64bit downloaded from official oracle site solved my problem.
>> And I need not set max heap size, final memory shown at the end of maven
>> build was 81/1943M. I want to learn spark so have no restriction on
>> choosing java version.
>>
>> Guru Medasani, thanks for the tip.
>>
>> I will repeat info, that I wrongly send only to Sean. I have tried export
>> MAVEN_OPTS=`-Xmx=3g -XX:MaxPermSize=1g -XX:ReservedCodeCacheSize=1g` and
>> it doesn't work also.
>>
>> Best Regards,
>> Vladimir Protsenko
>>
>> 2014-12-23 19:45 GMT+04:00 Guru Medasani <gd...@outlook.com>:
>>
>>> Thanks for the clarification Sean.
>>>
>>> Best Regards,
>>> Guru Medasani
>>>
>>>
>>>
>>>
>>> > From: sowen@cloudera.com
>>> > Date: Tue, 23 Dec 2014 15:39:59 +0000
>>> > Subject: Re: Spark Installation Maven PermGen OutOfMemoryException
>>> > To: gdmeda@outlook.com
>>> > CC: protsenkovi@gmail.com; user@spark.apache.org
>>>
>>> >
>>> > The text there is actually unclear. In Java 8, you still need to set
>>> > the max heap size (-Xmx2g). The optional bit is the
>>> > "-XX:MaxPermSize=512M" actually. Java 8 no longer has a separate
>>> > permanent generation.
>>> >
>>> > On Tue, Dec 23, 2014 at 3:32 PM, Guru Medasani <gd...@outlook.com>
>>> wrote:
>>> > > Hi Vladimir,
>>> > >
>>> > > From the link Sean posted, if you use Java 8 there is this following
>>> note.
>>> > >
>>> > > Note: For Java 8 and above this step is not required.
>>> > >
>>> > > So if you have no problems using Java 8, give it a shot.
>>> > >
>>> > > Best Regards,
>>> > > Guru Medasani
>>> > >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> > For additional commands, e-mail: user-help@spark.apache.org
>>> >
>>>
>>
>>
Re: Spark Installation Maven PermGen OutOfMemoryException
Posted by Sean Owen <so...@cloudera.com>.
That command is still wrong. It is -Xmx3g with no =.
On Dec 24, 2014 9:50 AM, "Vladimir Protsenko" <pr...@gmail.com> wrote:
> Java 8 rpm 64bit downloaded from official oracle site solved my problem.
> And I need not set max heap size, final memory shown at the end of maven
> build was 81/1943M. I want to learn spark so have no restriction on
> choosing java version.
>
> Guru Medasani, thanks for the tip.
>
> I will repeat info, that I wrongly send only to Sean. I have tried export
> MAVEN_OPTS=`-Xmx=3g -XX:MaxPermSize=1g -XX:ReservedCodeCacheSize=1g` and
> it doesn't work also.
>
> Best Regards,
> Vladimir Protsenko
>
> 2014-12-23 19:45 GMT+04:00 Guru Medasani <gd...@outlook.com>:
>
>> Thanks for the clarification Sean.
>>
>> Best Regards,
>> Guru Medasani
>>
>>
>>
>>
>> > From: sowen@cloudera.com
>> > Date: Tue, 23 Dec 2014 15:39:59 +0000
>> > Subject: Re: Spark Installation Maven PermGen OutOfMemoryException
>> > To: gdmeda@outlook.com
>> > CC: protsenkovi@gmail.com; user@spark.apache.org
>>
>> >
>> > The text there is actually unclear. In Java 8, you still need to set
>> > the max heap size (-Xmx2g). The optional bit is the
>> > "-XX:MaxPermSize=512M" actually. Java 8 no longer has a separate
>> > permanent generation.
>> >
>> > On Tue, Dec 23, 2014 at 3:32 PM, Guru Medasani <gd...@outlook.com>
>> wrote:
>> > > Hi Vladimir,
>> > >
>> > > From the link Sean posted, if you use Java 8 there is this following
>> note.
>> > >
>> > > Note: For Java 8 and above this step is not required.
>> > >
>> > > So if you have no problems using Java 8, give it a shot.
>> > >
>> > > Best Regards,
>> > > Guru Medasani
>> > >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> > For additional commands, e-mail: user-help@spark.apache.org
>> >
>>
>
>
Re: Spark Installation Maven PermGen OutOfMemoryException
Posted by Vladimir Protsenko <pr...@gmail.com>.
Java 8 rpm 64bit downloaded from official oracle site solved my problem.
And I need not set max heap size, final memory shown at the end of maven
build was 81/1943M. I want to learn spark so have no restriction on
choosing java version.
Guru Medasani, thanks for the tip.
I will repeat info, that I wrongly send only to Sean. I have tried export
MAVEN_OPTS=`-Xmx=3g -XX:MaxPermSize=1g -XX:ReservedCodeCacheSize=1g` and it
doesn't work also.
Best Regards,
Vladimir Protsenko
2014-12-23 19:45 GMT+04:00 Guru Medasani <gd...@outlook.com>:
> Thanks for the clarification Sean.
>
> Best Regards,
> Guru Medasani
>
>
>
>
> > From: sowen@cloudera.com
> > Date: Tue, 23 Dec 2014 15:39:59 +0000
> > Subject: Re: Spark Installation Maven PermGen OutOfMemoryException
> > To: gdmeda@outlook.com
> > CC: protsenkovi@gmail.com; user@spark.apache.org
>
> >
> > The text there is actually unclear. In Java 8, you still need to set
> > the max heap size (-Xmx2g). The optional bit is the
> > "-XX:MaxPermSize=512M" actually. Java 8 no longer has a separate
> > permanent generation.
> >
> > On Tue, Dec 23, 2014 at 3:32 PM, Guru Medasani <gd...@outlook.com>
> wrote:
> > > Hi Vladimir,
> > >
> > > From the link Sean posted, if you use Java 8 there is this following
> note.
> > >
> > > Note: For Java 8 and above this step is not required.
> > >
> > > So if you have no problems using Java 8, give it a shot.
> > >
> > > Best Regards,
> > > Guru Medasani
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> > For additional commands, e-mail: user-help@spark.apache.org
> >
>
RE: Spark Installation Maven PermGen OutOfMemoryException
Posted by Guru Medasani <gd...@outlook.com>.
Thanks for the clarification Sean.
Best Regards,Guru Medasani
> From: sowen@cloudera.com
> Date: Tue, 23 Dec 2014 15:39:59 +0000
> Subject: Re: Spark Installation Maven PermGen OutOfMemoryException
> To: gdmeda@outlook.com
> CC: protsenkovi@gmail.com; user@spark.apache.org
>
> The text there is actually unclear. In Java 8, you still need to set
> the max heap size (-Xmx2g). The optional bit is the
> "-XX:MaxPermSize=512M" actually. Java 8 no longer has a separate
> permanent generation.
>
> On Tue, Dec 23, 2014 at 3:32 PM, Guru Medasani <gd...@outlook.com> wrote:
> > Hi Vladimir,
> >
> > From the link Sean posted, if you use Java 8 there is this following note.
> >
> > Note: For Java 8 and above this step is not required.
> >
> > So if you have no problems using Java 8, give it a shot.
> >
> > Best Regards,
> > Guru Medasani
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
Re: Spark Installation Maven PermGen OutOfMemoryException
Posted by Sean Owen <so...@cloudera.com>.
The text there is actually unclear. In Java 8, you still need to set
the max heap size (-Xmx2g). The optional bit is the
"-XX:MaxPermSize=512M" actually. Java 8 no longer has a separate
permanent generation.
On Tue, Dec 23, 2014 at 3:32 PM, Guru Medasani <gd...@outlook.com> wrote:
> Hi Vladimir,
>
> From the link Sean posted, if you use Java 8 there is this following note.
>
> Note: For Java 8 and above this step is not required.
>
> So if you have no problems using Java 8, give it a shot.
>
> Best Regards,
> Guru Medasani
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: Spark Installation Maven PermGen OutOfMemoryException
Posted by Guru Medasani <gd...@outlook.com>.
Hi Vladimir,
>From the link Sean posted, if you use Java 8 there is this following note.
Note: For Java 8 and above this step is not required.
So if you have no problems using Java 8, give it a shot.
Best Regards,Guru Medasani
> From: sowen@cloudera.com
> Date: Tue, 23 Dec 2014 15:04:42 +0000
> Subject: Re: Spark Installation Maven PermGen OutOfMemoryException
> To: protsenkovi@gmail.com
> CC: user@spark.apache.org
>
> You might try a little more. The official guidance suggests 2GB:
>
> https://spark.apache.org/docs/latest/building-spark.html#setting-up-mavens-memory-usage
>
>
> On Tue, Dec 23, 2014 at 2:57 PM, Vladimir Protsenko
> <pr...@gmail.com> wrote:
> > I am installing Spark 1.2.0 on CentOS 6.6. Just downloaded code from github,
> > installed maven and trying to compile system:
> >
> > git clone https://github.com/apache/spark.git
> > git checkout v1.2.0
> > mvn -DskipTests clean package
> >
> > leads to OutOfMemoryException. What amount of memory does it requires?
> >
> > export MAVEN_OPTS=`-Xmx=1500m -XX:MaxPermSize=512m
> > -XX:ReservedCodeCacheSize=512m` doesn't help.
> >
> > Waht is a straight forward way to start using Spark?
> >
> >
> >
> > --
> > View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Installation-Maven-PermGen-OutOfMemoryException-tp20831.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> > For additional commands, e-mail: user-help@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
Re: Spark Installation Maven PermGen OutOfMemoryException
Posted by Sean Owen <so...@cloudera.com>.
You might try a little more. The official guidance suggests 2GB:
https://spark.apache.org/docs/latest/building-spark.html#setting-up-mavens-memory-usage
On Tue, Dec 23, 2014 at 2:57 PM, Vladimir Protsenko
<pr...@gmail.com> wrote:
> I am installing Spark 1.2.0 on CentOS 6.6. Just downloaded code from github,
> installed maven and trying to compile system:
>
> git clone https://github.com/apache/spark.git
> git checkout v1.2.0
> mvn -DskipTests clean package
>
> leads to OutOfMemoryException. What amount of memory does it requires?
>
> export MAVEN_OPTS=`-Xmx=1500m -XX:MaxPermSize=512m
> -XX:ReservedCodeCacheSize=512m` doesn't help.
>
> Waht is a straight forward way to start using Spark?
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Installation-Maven-PermGen-OutOfMemoryException-tp20831.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org