You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by rohit sarewar <ro...@gmail.com> on 2013/05/15 16:12:46 UTC
java heap space inside my Rhive
Hi
I am running a rhive job on Hadoop cluster on 40 mb data.
It throws an error " java.lang.OutOfMemoryError: Java heap space" as shown
below.
*How do I set java heap space inside my Rhive job ?*
[mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
Loading required package: rJava
Loading required package: methods
Loading required package: Rserve
This is RHive 0.0-7. For overview type ‘?RHive’.
HIVE_HOME=/usr/lib/hive
call rhive.init() because HIVE_HOME is set.
13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
Instead, use fs.defaultFS
13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
Instead, use fs.defaultFS
13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
java.lang.OutOfMemoryError: Java heap space
Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
Execution halted
Thanks
Rohit Sarewar
Re: R for Hadoop
Posted by Amal G Jose <am...@gmail.com>.
Try Rhipe, it is good.
http://amalgjose.wordpress.com/2013/05/05/rhipe-installation/
http://www.datadr.org/
http://amalgjose.wordpress.com/2013/05/05/r-installation-in-linux-platforms/
On Mon, May 20, 2013 at 2:23 PM, sudhakara st <su...@gmail.com>wrote:
> Hi
> You find good start up materiel for RHadoop here,
>
> https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
>
> http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
>
> I am also working on RHadoop, mail me if any difficulty in RHadoop.
>
>
> On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> You can try to search for "Rhadoop" using your favourite search engine.
>>
>> I think you are going to have to put in a bit more effort on your own.
>>
>> Marco
>>
>>
>>
>
>
> --
>
> Regards,
> ...Sudhakara.st
>
>
Re: R for Hadoop
Posted by Amal G Jose <am...@gmail.com>.
Try Rhipe, it is good.
http://amalgjose.wordpress.com/2013/05/05/rhipe-installation/
http://www.datadr.org/
http://amalgjose.wordpress.com/2013/05/05/r-installation-in-linux-platforms/
On Mon, May 20, 2013 at 2:23 PM, sudhakara st <su...@gmail.com>wrote:
> Hi
> You find good start up materiel for RHadoop here,
>
> https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
>
> http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
>
> I am also working on RHadoop, mail me if any difficulty in RHadoop.
>
>
> On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> You can try to search for "Rhadoop" using your favourite search engine.
>>
>> I think you are going to have to put in a bit more effort on your own.
>>
>> Marco
>>
>>
>>
>
>
> --
>
> Regards,
> ...Sudhakara.st
>
>
Re: R for Hadoop
Posted by Amal G Jose <am...@gmail.com>.
Try Rhipe, it is good.
http://amalgjose.wordpress.com/2013/05/05/rhipe-installation/
http://www.datadr.org/
http://amalgjose.wordpress.com/2013/05/05/r-installation-in-linux-platforms/
On Mon, May 20, 2013 at 2:23 PM, sudhakara st <su...@gmail.com>wrote:
> Hi
> You find good start up materiel for RHadoop here,
>
> https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
>
> http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
>
> I am also working on RHadoop, mail me if any difficulty in RHadoop.
>
>
> On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> You can try to search for "Rhadoop" using your favourite search engine.
>>
>> I think you are going to have to put in a bit more effort on your own.
>>
>> Marco
>>
>>
>>
>
>
> --
>
> Regards,
> ...Sudhakara.st
>
>
Re: R for Hadoop
Posted by Amal G Jose <am...@gmail.com>.
Try Rhipe, it is good.
http://amalgjose.wordpress.com/2013/05/05/rhipe-installation/
http://www.datadr.org/
http://amalgjose.wordpress.com/2013/05/05/r-installation-in-linux-platforms/
On Mon, May 20, 2013 at 2:23 PM, sudhakara st <su...@gmail.com>wrote:
> Hi
> You find good start up materiel for RHadoop here,
>
> https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
>
> http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
>
> I am also working on RHadoop, mail me if any difficulty in RHadoop.
>
>
> On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> You can try to search for "Rhadoop" using your favourite search engine.
>>
>> I think you are going to have to put in a bit more effort on your own.
>>
>> Marco
>>
>>
>>
>
>
> --
>
> Regards,
> ...Sudhakara.st
>
>
Re: R for Hadoop
Posted by sudhakara st <su...@gmail.com>.
Hi
You find good start up materiel for RHadoop here,
https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
I am also working on RHadoop, mail me if any difficulty in RHadoop.
On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
> You can try to search for "Rhadoop" using your favourite search engine.
>
> I think you are going to have to put in a bit more effort on your own.
>
> Marco
>
>
>
--
Regards,
...Sudhakara.st
Re: R for Hadoop
Posted by sudhakara st <su...@gmail.com>.
Hi
You find good start up materiel for RHadoop here,
https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
I am also working on RHadoop, mail me if any difficulty in RHadoop.
On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
> You can try to search for "Rhadoop" using your favourite search engine.
>
> I think you are going to have to put in a bit more effort on your own.
>
> Marco
>
>
>
--
Regards,
...Sudhakara.st
Re: R for Hadoop
Posted by sudhakara st <su...@gmail.com>.
Hi
You find good start up materiel for RHadoop here,
https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
I am also working on RHadoop, mail me if any difficulty in RHadoop.
On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
> You can try to search for "Rhadoop" using your favourite search engine.
>
> I think you are going to have to put in a bit more effort on your own.
>
> Marco
>
>
>
--
Regards,
...Sudhakara.st
Re: R for Hadoop
Posted by sudhakara st <su...@gmail.com>.
Hi
You find good start up materiel for RHadoop here,
https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md
http://bighadoop.wordpress.com/2013/02/25/r-and-hadoop-data-analysis-rhadoop/
I am also working on RHadoop, mail me if any difficulty in RHadoop.
On Mon, May 20, 2013 at 12:20 AM, Marco Shaw <ma...@gmail.com> wrote:
> You can try to search for "Rhadoop" using your favourite search engine.
>
> I think you are going to have to put in a bit more effort on your own.
>
> Marco
>
>
>
--
Regards,
...Sudhakara.st
Re: R for Hadoop
Posted by Marco Shaw <ma...@gmail.com>.
You can try to search for "Rhadoop" using your favourite search engine.
I think you are going to have to put in a bit more effort on your own.
Marco
Re: R for Hadoop
Posted by Marco Shaw <ma...@gmail.com>.
You can try to search for "Rhadoop" using your favourite search engine.
I think you are going to have to put in a bit more effort on your own.
Marco
Re: R for Hadoop
Posted by Marco Shaw <ma...@gmail.com>.
You can try to search for "Rhadoop" using your favourite search engine.
I think you are going to have to put in a bit more effort on your own.
Marco
Re: R for Hadoop
Posted by Marco Shaw <ma...@gmail.com>.
You can try to search for "Rhadoop" using your favourite search engine.
I think you are going to have to put in a bit more effort on your own.
Marco
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
Sorry, but I'm not the expert on this. I've just started looking at R
on Hadoop.
David
On 5/19/2013 2:18 PM, Raj Hadoop wrote:
> Thanks David.
>
> Is RHIPE the only package for R on Hadoop availaable and standard now.
> Is this the standard ? Or do you have any other packages ?
>
> Can you please provide a 'Hello World or a 'Word count' code
> equivalent of R on Hadoop using RHIPE. I want to get started soon and
> I will defaintely look att he website you recommedned. But a few
> working scripts from experts like you - gets me jump started on this.
>
> Thanks,
> Raj
>
> ------------------------------------------------------------------------
> *From:* David Ritch <da...@gmail.com>
> *To:* user@hadoop.apache.org
> *Cc:* Raj Hadoop <ha...@yahoo.com>
> *Sent:* Sunday, May 19, 2013 1:21 PM
> *Subject:* Re: R for Hadoop
>
> You can find RHIPE here: http://www.datadr.org/
>
> On 5/19/2013 10:26 AM, Raj Hadoop wrote:
>> Hi,
>>
>> I want to explore R for Hadoop. Where can I get the download ? Any
>> suggested material on the website to explore ? Please advise.
>>
>> Thanks,
>> Raj
>
>
>
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
Sorry, but I'm not the expert on this. I've just started looking at R
on Hadoop.
David
On 5/19/2013 2:18 PM, Raj Hadoop wrote:
> Thanks David.
>
> Is RHIPE the only package for R on Hadoop availaable and standard now.
> Is this the standard ? Or do you have any other packages ?
>
> Can you please provide a 'Hello World or a 'Word count' code
> equivalent of R on Hadoop using RHIPE. I want to get started soon and
> I will defaintely look att he website you recommedned. But a few
> working scripts from experts like you - gets me jump started on this.
>
> Thanks,
> Raj
>
> ------------------------------------------------------------------------
> *From:* David Ritch <da...@gmail.com>
> *To:* user@hadoop.apache.org
> *Cc:* Raj Hadoop <ha...@yahoo.com>
> *Sent:* Sunday, May 19, 2013 1:21 PM
> *Subject:* Re: R for Hadoop
>
> You can find RHIPE here: http://www.datadr.org/
>
> On 5/19/2013 10:26 AM, Raj Hadoop wrote:
>> Hi,
>>
>> I want to explore R for Hadoop. Where can I get the download ? Any
>> suggested material on the website to explore ? Please advise.
>>
>> Thanks,
>> Raj
>
>
>
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
Sorry, but I'm not the expert on this. I've just started looking at R
on Hadoop.
David
On 5/19/2013 2:18 PM, Raj Hadoop wrote:
> Thanks David.
>
> Is RHIPE the only package for R on Hadoop availaable and standard now.
> Is this the standard ? Or do you have any other packages ?
>
> Can you please provide a 'Hello World or a 'Word count' code
> equivalent of R on Hadoop using RHIPE. I want to get started soon and
> I will defaintely look att he website you recommedned. But a few
> working scripts from experts like you - gets me jump started on this.
>
> Thanks,
> Raj
>
> ------------------------------------------------------------------------
> *From:* David Ritch <da...@gmail.com>
> *To:* user@hadoop.apache.org
> *Cc:* Raj Hadoop <ha...@yahoo.com>
> *Sent:* Sunday, May 19, 2013 1:21 PM
> *Subject:* Re: R for Hadoop
>
> You can find RHIPE here: http://www.datadr.org/
>
> On 5/19/2013 10:26 AM, Raj Hadoop wrote:
>> Hi,
>>
>> I want to explore R for Hadoop. Where can I get the download ? Any
>> suggested material on the website to explore ? Please advise.
>>
>> Thanks,
>> Raj
>
>
>
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
Sorry, but I'm not the expert on this. I've just started looking at R
on Hadoop.
David
On 5/19/2013 2:18 PM, Raj Hadoop wrote:
> Thanks David.
>
> Is RHIPE the only package for R on Hadoop availaable and standard now.
> Is this the standard ? Or do you have any other packages ?
>
> Can you please provide a 'Hello World or a 'Word count' code
> equivalent of R on Hadoop using RHIPE. I want to get started soon and
> I will defaintely look att he website you recommedned. But a few
> working scripts from experts like you - gets me jump started on this.
>
> Thanks,
> Raj
>
> ------------------------------------------------------------------------
> *From:* David Ritch <da...@gmail.com>
> *To:* user@hadoop.apache.org
> *Cc:* Raj Hadoop <ha...@yahoo.com>
> *Sent:* Sunday, May 19, 2013 1:21 PM
> *Subject:* Re: R for Hadoop
>
> You can find RHIPE here: http://www.datadr.org/
>
> On 5/19/2013 10:26 AM, Raj Hadoop wrote:
>> Hi,
>>
>> I want to explore R for Hadoop. Where can I get the download ? Any
>> suggested material on the website to explore ? Please advise.
>>
>> Thanks,
>> Raj
>
>
>
Re: R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Thanks David.
Is RHIPE the only package for R on Hadoop availaable and standard now. Is this the standard ? Or do you have any other packages ?
Can you please provide a 'Hello World or a 'Word count' code equivalent of R on Hadoop using RHIPE. I want to get started soon and I will defaintely look att he website you recommedned. But a few working scripts from experts like you - gets me jump started on this.
Thanks,
Raj
________________________________
From: David Ritch <da...@gmail.com>
To: user@hadoop.apache.org
Cc: Raj Hadoop <ha...@yahoo.com>
Sent: Sunday, May 19, 2013 1:21 PM
Subject: Re: R for Hadoop
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
Hi,
>
>
>I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
>
>
>Thanks,
>Raj
>
Re: R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Thanks David.
Is RHIPE the only package for R on Hadoop availaable and standard now. Is this the standard ? Or do you have any other packages ?
Can you please provide a 'Hello World or a 'Word count' code equivalent of R on Hadoop using RHIPE. I want to get started soon and I will defaintely look att he website you recommedned. But a few working scripts from experts like you - gets me jump started on this.
Thanks,
Raj
________________________________
From: David Ritch <da...@gmail.com>
To: user@hadoop.apache.org
Cc: Raj Hadoop <ha...@yahoo.com>
Sent: Sunday, May 19, 2013 1:21 PM
Subject: Re: R for Hadoop
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
Hi,
>
>
>I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
>
>
>Thanks,
>Raj
>
Re: R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Thanks David.
Is RHIPE the only package for R on Hadoop availaable and standard now. Is this the standard ? Or do you have any other packages ?
Can you please provide a 'Hello World or a 'Word count' code equivalent of R on Hadoop using RHIPE. I want to get started soon and I will defaintely look att he website you recommedned. But a few working scripts from experts like you - gets me jump started on this.
Thanks,
Raj
________________________________
From: David Ritch <da...@gmail.com>
To: user@hadoop.apache.org
Cc: Raj Hadoop <ha...@yahoo.com>
Sent: Sunday, May 19, 2013 1:21 PM
Subject: Re: R for Hadoop
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
Hi,
>
>
>I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
>
>
>Thanks,
>Raj
>
Re: R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Thanks David.
Is RHIPE the only package for R on Hadoop availaable and standard now. Is this the standard ? Or do you have any other packages ?
Can you please provide a 'Hello World or a 'Word count' code equivalent of R on Hadoop using RHIPE. I want to get started soon and I will defaintely look att he website you recommedned. But a few working scripts from experts like you - gets me jump started on this.
Thanks,
Raj
________________________________
From: David Ritch <da...@gmail.com>
To: user@hadoop.apache.org
Cc: Raj Hadoop <ha...@yahoo.com>
Sent: Sunday, May 19, 2013 1:21 PM
Subject: Re: R for Hadoop
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
Hi,
>
>
>I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
>
>
>Thanks,
>Raj
>
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
> Hi,
>
> I want to explore R for Hadoop. Where can I get the download ? Any
> suggested material on the website to explore ? Please advise.
>
> Thanks,
> Raj
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
> Hi,
>
> I want to explore R for Hadoop. Where can I get the download ? Any
> suggested material on the website to explore ? Please advise.
>
> Thanks,
> Raj
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
> Hi,
>
> I want to explore R for Hadoop. Where can I get the download ? Any
> suggested material on the website to explore ? Please advise.
>
> Thanks,
> Raj
Re: R for Hadoop
Posted by David Ritch <da...@gmail.com>.
You can find RHIPE here: http://www.datadr.org/
On 5/19/2013 10:26 AM, Raj Hadoop wrote:
> Hi,
>
> I want to explore R for Hadoop. Where can I get the download ? Any
> suggested material on the website to explore ? Please advise.
>
> Thanks,
> Raj
R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Hi,
I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
Thanks,
Raj
________________________________
From: Amal G Jose <am...@gmail.com>
To: user@hadoop.apache.org
Sent: Saturday, May 18, 2013 11:47 PM
Subject: Re: java heap space inside my Rhive
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com> wrote:
Hi
>
>I am running a rhive job on Hadoop cluster on 40 mb data.
>It throws an error " java.lang.OutOfMemoryError: Java heap space" as shown below.
>How do I set java heap space inside my Rhive job ?
>
>
>[mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
>Loading required package: rJava
>Loading required package: methods
>Loading required package: Rserve
>This is RHive 0.0-7. For overview type ‘?RHive’.
>HIVE_HOME=/usr/lib/hive
>call rhive.init() because HIVE_HOME is set.
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
>Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
>Execution halted
>
>Thanks
>Rohit Sarewar
>
R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Hi,
I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
Thanks,
Raj
________________________________
From: Amal G Jose <am...@gmail.com>
To: user@hadoop.apache.org
Sent: Saturday, May 18, 2013 11:47 PM
Subject: Re: java heap space inside my Rhive
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com> wrote:
Hi
>
>I am running a rhive job on Hadoop cluster on 40 mb data.
>It throws an error " java.lang.OutOfMemoryError: Java heap space" as shown below.
>How do I set java heap space inside my Rhive job ?
>
>
>[mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
>Loading required package: rJava
>Loading required package: methods
>Loading required package: Rserve
>This is RHive 0.0-7. For overview type ‘?RHive’.
>HIVE_HOME=/usr/lib/hive
>call rhive.init() because HIVE_HOME is set.
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
>Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
>Execution halted
>
>Thanks
>Rohit Sarewar
>
R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Hi,
I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
Thanks,
Raj
________________________________
From: Amal G Jose <am...@gmail.com>
To: user@hadoop.apache.org
Sent: Saturday, May 18, 2013 11:47 PM
Subject: Re: java heap space inside my Rhive
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com> wrote:
Hi
>
>I am running a rhive job on Hadoop cluster on 40 mb data.
>It throws an error " java.lang.OutOfMemoryError: Java heap space" as shown below.
>How do I set java heap space inside my Rhive job ?
>
>
>[mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
>Loading required package: rJava
>Loading required package: methods
>Loading required package: Rserve
>This is RHive 0.0-7. For overview type ‘?RHive’.
>HIVE_HOME=/usr/lib/hive
>call rhive.init() because HIVE_HOME is set.
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
>Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
>Execution halted
>
>Thanks
>Rohit Sarewar
>
R for Hadoop
Posted by Raj Hadoop <ha...@yahoo.com>.
Hi,
I want to explore R for Hadoop. Where can I get the download ? Any suggested material on the website to explore ? Please advise.
Thanks,
Raj
________________________________
From: Amal G Jose <am...@gmail.com>
To: user@hadoop.apache.org
Sent: Saturday, May 18, 2013 11:47 PM
Subject: Re: java heap space inside my Rhive
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com> wrote:
Hi
>
>I am running a rhive job on Hadoop cluster on 40 mb data.
>It throws an error " java.lang.OutOfMemoryError: Java heap space" as shown below.
>How do I set java heap space inside my Rhive job ?
>
>
>[mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
>Loading required package: rJava
>Loading required package: methods
>Loading required package: Rserve
>This is RHive 0.0-7. For overview type ‘?RHive’.
>HIVE_HOME=/usr/lib/hive
>call rhive.init() because HIVE_HOME is set.
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
>13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
>Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
>Execution halted
>
>Thanks
>Rohit Sarewar
>
Re: java heap space inside my Rhive
Posted by Amal G Jose <am...@gmail.com>.
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com>wrote:
> Hi
>
> I am running a rhive job on Hadoop cluster on 40 mb data.
> It throws an error " java.lang.OutOfMemoryError: Java heap space" as
> shown below.
> *How do I set java heap space inside my Rhive job ?*
>
>
> [mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
> Loading required package: rJava
> Loading required package: methods
> Loading required package: Rserve
> This is RHive 0.0-7. For overview type ‘?RHive’.
> HIVE_HOME=/usr/lib/hive
> call rhive.init() because HIVE_HOME is set.
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
> Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
> Execution halted
>
> Thanks
> Rohit Sarewar
>
Re: java heap space inside my Rhive
Posted by Amal G Jose <am...@gmail.com>.
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com>wrote:
> Hi
>
> I am running a rhive job on Hadoop cluster on 40 mb data.
> It throws an error " java.lang.OutOfMemoryError: Java heap space" as
> shown below.
> *How do I set java heap space inside my Rhive job ?*
>
>
> [mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
> Loading required package: rJava
> Loading required package: methods
> Loading required package: Rserve
> This is RHive 0.0-7. For overview type ‘?RHive’.
> HIVE_HOME=/usr/lib/hive
> call rhive.init() because HIVE_HOME is set.
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
> Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
> Execution halted
>
> Thanks
> Rohit Sarewar
>
Re: java heap space inside my Rhive
Posted by Amal G Jose <am...@gmail.com>.
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com>wrote:
> Hi
>
> I am running a rhive job on Hadoop cluster on 40 mb data.
> It throws an error " java.lang.OutOfMemoryError: Java heap space" as
> shown below.
> *How do I set java heap space inside my Rhive job ?*
>
>
> [mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
> Loading required package: rJava
> Loading required package: methods
> Loading required package: Rserve
> This is RHive 0.0-7. For overview type ‘?RHive’.
> HIVE_HOME=/usr/lib/hive
> call rhive.init() because HIVE_HOME is set.
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
> Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
> Execution halted
>
> Thanks
> Rohit Sarewar
>
Re: java heap space inside my Rhive
Posted by Amal G Jose <am...@gmail.com>.
which version of hadoop are you using?.
I faced a similar exception with Rhipe-0.68 in CDH4.
The reason was because that version of Rhipe doesnt support CDH4.
My problem got resolved with the release of Rhipe-0.73.
So check whether it supports your hadoop version.
On Wed, May 15, 2013 at 7:42 PM, rohit sarewar <ro...@gmail.com>wrote:
> Hi
>
> I am running a rhive job on Hadoop cluster on 40 mb data.
> It throws an error " java.lang.OutOfMemoryError: Java heap space" as
> shown below.
> *How do I set java heap space inside my Rhive job ?*
>
>
> [mycompany@arjun10 Desktop]$ Rscript muFreqBig.R
> Loading required package: rJava
> Loading required package: methods
> Loading required package: Rserve
> This is RHive 0.0-7. For overview type ‘?RHive’.
> HIVE_HOME=/usr/lib/hive
> call rhive.init() because HIVE_HOME is set.
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN conf.Configuration: fs.default.name is deprecated.
> Instead, use fs.defaultFS
> 13/05/15 05:51:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
> java.lang.OutOfMemoryError: Java heap space
> Calls: muFreqBig ... <Anonymous> -> .jrcall -> .jcall -> .jcheck -> .Call
> Execution halted
>
> Thanks
> Rohit Sarewar
>