You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2009/05/24 18:10:45 UTC
[jira] Commented: (HADOOP-5906) Stream test TestStreamingExitStatus
fails with Out of Memory
[ https://issues.apache.org/jira/browse/HADOOP-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712569#action_12712569 ]
Steve Loughran commented on HADOOP-5906:
----------------------------------------
stack trace. Presumably we just need to set the JVM parameters up right, but what are the right values?
{code}
[junit] Tests run: 4, Failures: 1, Errors: 0, Time elapsed: 13.164 sec
[junit] Test org.apache.hadoop.streaming.TestStreamingExitStatus FAILED
09/05/24 12:00:11 INFO streaming.PipeMapRed: R/W/S=1/0/0 in:NA [rec/s] out:NA [rec/s]
09/05/24 12:00:11 INFO streaming.PipeMapRed: Records R/W=1/1
09/05/24 12:00:11 INFO streaming.PipeMapRed: MRErrorThread done
09/05/24 12:00:11 WARN mapred.LocalJobRunner: job_local_0002
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:366)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:568)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:360)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:11 INFO streaming.StreamJob: map 0% reduce 0%
09/05/24 12:00:11 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:11 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:11 INFO streaming.StreamJob: killJob...
09/05/24 12:00:11 WARN streaming.StreamJob: -jobconf option is deprecated, please use -D instead.
09/05/24 12:00:11 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
09/05/24 12:00:11 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
09/05/24 12:00:12 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
09/05/24 12:00:12 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:13 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:13 INFO streaming.StreamJob: getLocalDirs(): [/Users/slo/Java/Apache/hadoop-branches/hadoop-5847/build/contrib/streaming/test/mapred/local]
09/05/24 12:00:13 INFO streaming.StreamJob: Running job: job_local_0003
09/05/24 12:00:13 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:13 INFO mapred.MapTask: numReduceTasks: 1
09/05/24 12:00:13 INFO mapred.MapTask: io.sort.mb = 100
09/05/24 12:00:13 WARN mapred.LocalJobRunner: job_local_0003
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:689)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:14 INFO streaming.StreamJob: map 0% reduce 0%
09/05/24 12:00:14 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:14 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:14 INFO streaming.StreamJob: killJob...
09/05/24 12:00:14 WARN streaming.StreamJob: -jobconf option is deprecated, please use -D instead.
09/05/24 12:00:14 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
09/05/24 12:00:14 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
09/05/24 12:00:14 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
09/05/24 12:00:14 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:15 INFO streaming.StreamJob: getLocalDirs(): [/Users/slo/Java/Apache/hadoop-branches/hadoop-5847/build/contrib/streaming/test/mapred/local]
09/05/24 12:00:15 INFO streaming.StreamJob: Running job: job_local_0004
09/05/24 12:00:15 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:15 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:15 INFO mapred.MapTask: numReduceTasks: 1
09/05/24 12:00:15 INFO mapred.MapTask: io.sort.mb = 100
09/05/24 12:00:16 WARN mapred.LocalJobRunner: job_local_0004
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:689)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:16 INFO streaming.StreamJob: map 0% reduce 0%
09/05/24 12:00:16 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:16 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:16 INFO streaming.StreamJob: killJob...
{code}
> Stream test TestStreamingExitStatus fails with Out of Memory
> ------------------------------------------------------------
>
> Key: HADOOP-5906
> URL: https://issues.apache.org/jira/browse/HADOOP-5906
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.21.0
> Environment: OS/X, 64-bit x86 imac, 4GB RAM.
> Reporter: Steve Loughran
> Priority: Minor
>
> contrib/streaming tests are failing a test with an Out of Memory error on an OS/X Mac -same problem does not surface on Linux.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.