You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/01/15 09:03:59 UTC
[jira] Assigned: (HADOOP-5046) Chukwa log4j appender logs corrupted
data if the system is under high stress
[ https://issues.apache.org/jira/browse/HADOOP-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang reassigned HADOOP-5046:
---------------------------------
Assignee: Eric Yang
> Chukwa log4j appender logs corrupted data if the system is under high stress
> ----------------------------------------------------------------------------
>
> Key: HADOOP-5046
> URL: https://issues.apache.org/jira/browse/HADOOP-5046
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
>
> Data from Iostat indicates that log files did not write properly when system is under high stress.
> 2008-12-29 03:03:48,510 INFO org.apache.hadoop.chukwa.inputtools.plugin.metrics.Exec: Linux 2.6.9-55.ELsmp (example1002) 12/29/08^D
> ^D
> avg-cpu: %user %nice %sys %iowait %idle^D
> 1.19 0.35 0.85 2.63 94.99^D
> ^D
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util^D
> sda 0.13 33.31 3.02 3.53 281.94 311.02 140.97 155.51 90.52 0.56 86.19 2.30 1.51^D
> sdb 4.52 3.45 5.93 1.51 107.12 39.67 53.56 19.83 19.74 0.07 9.98 3.53 2.63^D
> sdc 4.57 22.76 7.12 1.71 395.58 195.76 197.79 97.88 66.93 0.24 27.13 3.90 3.44^D
> sdd 4.52 18.17 6.13 1.65 151.17 158.59 75.58 79.30 39.81 0.45 57.96 3.84 2.98^D
> ^D
> avg-cpu: %user %nice %sys %iowait %idle^D
> 24.83 0.00 0.29 0.07 74.81^D
> ^D
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util^D
> sda 1.04 1.78 41.99 3.64 8706.75 45.10 4353.37 22.55 191.82 0.21 4.50 3.70 16.89^D
> sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2008-12-29 03:08:48,513 INFO org.apache.hadoop.chukwa.inputtools.plugin.metrics.Exec: Linux 2.6.9-55.ELsmp (example1002) 12/29/08^D
> ^D
> The most probable reason is that disk buffer got paged out before it is written to disk. Exec plugin can be configured to always flush on every output. For hadoop logs, this need to be fine tuned.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.