[jira] [Created] (GIRAPH-808) Giraph should report progress more accurately when running on Map/Reduce

["Rob Vesse (JIRA)" <ji...@apache.org>]

Rob Vesse created GIRAPH-808:
--------------------------------

             Summary: Giraph should report progress more accurately when running on Map/Reduce
                 Key: GIRAPH-808
                 URL: https://issues.apache.org/jira/browse/GIRAPH-808
             Project: Giraph
          Issue Type: Improvement
    Affects Versions: 1.0.0, 1.1.0
            Reporter: Rob Vesse


The current way that Giraph reports progress when running on Map/Reduce seems rather flawed.  When running a Giraph program the map tasks are launched and after initialisation their progress almost immediately goes to 100% and stays there throughout.  So the only way to monitor progress is to 

I appreciate that there is no way for Giraph to report accurate progress since it does not know in advance how many super steps there will be but it could report progress in a more useful way.

For example:
- First N percent of progress is the input phase, this part could likely be accurately calculated by using standard Hadoop input APIs which Giraph input is built on
- Next N percent of progress is an estimation that trends towards the final value but does not reach it until the computation has halted i.e. (Superstep + 1) / (N - 1) so this will naturally trend towards N.  Once the computation halts then the value becomes N
- Last N percentage of progress is the output phase, again this part could likely be accurately calculated easily since Giraph knows how many items it has to output

What does anybody else think?



--
This message was sent by Atlassian JIRA
(v6.1#6144)