You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Suneel Marthi (JIRA)" <ji...@apache.org> on 2012/12/18 21:28:15 UTC
[jira] [Comment Edited] (MAHOUT-1129) Mahout Java Heap Out of
Memory in TrainNewsGroup class
[ https://issues.apache.org/jira/browse/MAHOUT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535269#comment-13535269 ]
Suneel Marthi edited comment on MAHOUT-1129 at 12/18/12 8:27 PM:
-----------------------------------------------------------------
Did u try running the example by increasing the JVM memory settings ?
was (Author: smarthi):
Did u try running the example by setting the JVM memory settings ?
> Mahout Java Heap Out of Memory in TrainNewsGroup class
> ------------------------------------------------------
>
> Key: MAHOUT-1129
> URL: https://issues.apache.org/jira/browse/MAHOUT-1129
> Project: Mahout
> Issue Type: Bug
> Components: Classification, Examples
> Affects Versions: 0.7
> Environment: Linux - RHEL 5.6
> Reporter: Huned Lokhandwala
> Priority: Critical
> Labels: test
> Fix For: 0.7
>
>
> On Mahout on Linux RHEL 5.6 (Single node, no cluster), a Java Heap Space Out of Memory Exception occurs when calling the TrainNewsGroup class from the mahout-examples-0.7.0.17-job.jar file. I downloaded the 20news-bydate.tar.gz (from http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz ) to use as classification data and added it in the folder as below, and called the org.apache.mahout.classifier.sgd.TrainNewsGroups on the folder and found the Java Heap Space Out of Memory Exception as show below.
> Output from run:
> > /usr/lib/mahout/bin/mahout org.apache.mahout.classifier.sgd.TrainNewsGroups /artifacts/mahout_sgd_classifier/20news-bydate-train
> Running on hadoop, using /usr/bin/hadoop and HADOOP_CONF_DIR=
> MAHOUT-JOB: /usr/lib/mahout/mahout-examples-0.7.0.17-job.jar
> 12/12/17 20:35:06 WARN driver.MahoutDriver: No org.apache.mahout.classifier.sgd.TrainNewsGroups.props found on classpath, will use command-line arguments only
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at org.apache.mahout.math.DenseMatrix.<init>(DenseMatrix.java:50)
> at org.apache.mahout.classifier.sgd.OnlineLogisticRegression.<init>(OnlineLogisticRegression.java:60)
> at org.apache.mahout.classifier.sgd.CrossFoldLearner.<init>(CrossFoldLearner.java:67)
> at org.apache.mahout.classifier.sgd.CrossFoldLearner.copy(CrossFoldLearner.java:217)
> at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$Wrapper.copy(AdaptiveLogisticRegression.java:399)
> at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$Wrapper.copy(AdaptiveLogisticRegression.java:386)
> at org.apache.mahout.ep.State.copy(State.java:94)
> at org.apache.mahout.ep.State.mutate(State.java:114)
> at org.apache.mahout.ep.EvolutionaryProcess.initializePopulation(EvolutionaryProcess.java:103)
> at org.apache.mahout.ep.EvolutionaryProcess.<init>(EvolutionaryProcess.java:97)
> at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.setupOptimizer(AdaptiveLogisticRegression.java:279)
> at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.setPoolSize(AdaptiveLogisticRegression.java:265)
> at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.<init>(AdaptiveLogisticRegression.java:121)
> at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.<init>(AdaptiveLogisticRegression.java:100)
> at org.apache.mahout.classifier.sgd.TrainNewsGroups.main(TrainNewsGroups.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira