You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2008/04/03 13:54:24 UTC
[jira] Updated: (HADOOP-3154) Job successful but dropping records
(when disk full)
[ https://issues.apache.org/jira/browse/HADOOP-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-3154:
--------------------------------
Attachment: 3154.patch
This patch is for trunk. I wonder whether we should fix this in 16 too.
> Job successful but dropping records (when disk full)
> ----------------------------------------------------
>
> Key: HADOOP-3154
> URL: https://issues.apache.org/jira/browse/HADOOP-3154
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.16.1
> Environment: hadoop-0.16.1-H3011-H3033-H3056
> Reporter: Koji Noguchi
> Attachments: 3154.patch
>
>
> I have a mapreduce code that takes an input and just shuffles.
> # of input should be equal to # of output.
> However, when disks of the nodes were filled accidentally, I started to see some records dropping, although jobs themselves were successful.
> {noformat}
> 08/03/30 00:17:04 INFO mapred.JobClient: Job complete: job_200803292134_0001
> 08/03/30 00:17:04 INFO mapred.JobClient: Counters: 11
> 08/03/30 00:17:04 INFO mapred.JobClient: Job Counters
> 08/03/30 00:17:04 INFO mapred.JobClient: Launched map tasks=23
> 08/03/30 00:17:04 INFO mapred.JobClient: Launched reduce tasks=4
> 08/03/30 00:17:04 INFO mapred.JobClient: Map-Reduce Framework
> 08/03/30 00:17:04 INFO mapred.JobClient: Map-Reduce Framework
> 08/03/30 00:17:04 INFO mapred.JobClient: Map input records=6852926
> 08/03/30 00:17:04 INFO mapred.JobClient: Map output records=6852926
> 08/03/30 00:17:04 INFO mapred.JobClient: Map input bytes=18802382982
> 08/03/30 00:17:04 INFO mapred.JobClient: Map output bytes=21278202852
> 08/03/30 00:17:04 INFO mapred.JobClient: Combine input records=0
> 08/03/30 00:17:04 INFO mapred.JobClient: Combine output records=0
> 08/03/30 00:17:04 INFO mapred.JobClient: Reduce input groups=6722633
> 08/03/30 00:17:04 INFO mapred.JobClient: Reduce input records=6839731
> 08/03/30 00:17:04 INFO mapred.JobClient: Reduce output records=6839731
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.