[jira] Updated: (HADOOP-5585) FileSystem statistic counters are too high when JVM reuse is enabled.

["Owen O'Malley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley updated HADOOP-5585:
----------------------------------

    Attachment: h5585.patch

This patch:
  1. Adds a reset method for statistics, so that when the framework switches to a new task, it can clean the counters.
  2. Removes the map from scheme to statistics, since it was only used to iterate through.
  3. Removes a race condition where the map was passed back out of the synchronized method, which could cause ConcurrentModificationException.

> FileSystem statistic counters are too high when JVM reuse is enabled.
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-5585
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5585
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: h5585.patch
>
>
> When JVM reuse is enabled, the FileSystem.Statistics are not cleared between tasks. That means that the second task gets credit for its own reads and writes as well as the first. The third gets credit for all 3 tasks reads and writes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.