[jira] [Updated] (PIG-1883) Pig's progress estimation should account for parallel job executions

["Laukik Chitnis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Laukik Chitnis updated PIG-1883:
--------------------------------

    Attachment: PIG-1883-3.patch

I have reintroduced a test file which was probably deleted when test cases related to old logical plan were ported over. However, the old OperatorPlan is still in use via the MROperPlan.

Also, this org.apache.pig.impl.plan.OperatorPlan is an abstract class (newplan.OperatorPlan is an interface) and already has methods such as size(); so I have retained the getHeight() method in this class.

I have also switched the option that chooses the new progress indication technique to take an optional parameter with which one can have both estimates (this was also useful for comparing performance)


> Pig's progress estimation should account for parallel job executions
> --------------------------------------------------------------------
>
>                 Key: PIG-1883
>                 URL: https://issues.apache.org/jira/browse/PIG-1883
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Laukik Chitnis
>            Assignee: Laukik Chitnis
>         Attachments: PIG-1883-2.patch, PIG-1883-3.patch
>
>
> Currently, Pig's progress estimation is based on the percentage of jobs completed out of the total number of MR jobs. However, since the MR operators are arranged in a DAG (and hence more than 1 job might be submitted for execution in parallel), the progress estimation can be improved by considering the number of jobs in the critical path, instead of just the total number of jobs.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira