[jira] Resolved: (PIG-357) PERFORMANCE: progress reported on every tuple

["Alan Gates (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates resolved PIG-357.
----------------------------

    Resolution: Won't Fix

I made a change in ProgressableReporter to only report progress every 2 minutes.  I then ran pigmix 3 times each with top of trunk build and the change.  The average time for the top of trunk was 3021, and with the patch 3036.  This, together with the earlier experiment of removing all the progress reporting convinces me that this is not something we need to be concerned about.

> PERFORMANCE: progress reported on every tuple
> ---------------------------------------------
>
>                 Key: PIG-357
>                 URL: https://issues.apache.org/jira/browse/PIG-357
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.2.0
>            Reporter: Olga Natkovich
>            Assignee: Alan Gates
>            Priority: Minor
>
> Currently, if the reporter is set, we report progress on every tuple. This could be too expensive and impact performance. In the old code, we used to do it on every 1000th tuple or something like that.
> We might want to go to similar model.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.