You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Cheolsoo Park <pi...@gmail.com> on 2014/03/17 17:02:32 UTC

Review Request 19298: PIG-3812: Several clean-ups in PigStats and JobStats

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19298/
-----------------------------------------------------------

Review request for pig.


Bugs: PIG-3812
    https://issues.apache.org/jira/browse/PIG-3812


Repository: pig-git


Description
-------

While implementing counter support in Tez mode (PIG-3603), I made several changes to PigStats and JobStats classes. To avoid breaking backward compatibility between 0.13 and onwards, I'd like to commit this patch into trunk prior to 0.13 release.

The changes include-
* Renamed SimpleFetchPigStats to EmptyPigStats. This dummy PigStats can be used not only when the direct fetch mode is enabled but also when no Hadoop jobs are submitted due to any front-end error. So I renamed it to EmptyPigStats.
* Moved getHdfsBytes[Read|Written]() from MRPigStatsUtil to PigStatsUtil since these are not MR specific.
* Moved [MAP|REDUCE]_[IN|OUT]PUT_RECORDS from MRPigStatsUtil to PigStatsUtil since Tez MRInput and MROutput also use them.
* Fixed a typo in JobStats#getAvgREduceTime(): REduce -> Reduce.
* Fixed white spaces.


Diffs
-----

  src/org/apache/pig/PigServer.java 3d50a3e 
  src/org/apache/pig/backend/hadoop/executionengine/fetch/FetchLauncher.java d81f03b 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java b483323 
  src/org/apache/pig/parser/QueryParserDriver.java ec5b2b9 
  src/org/apache/pig/parser/SourceLocation.java b69546f 
  src/org/apache/pig/tools/pigstats/EmptyPigStats.java e69de29 
  src/org/apache/pig/tools/pigstats/InputStats.java 38c8372 
  src/org/apache/pig/tools/pigstats/JobStats.java 4484348 
  src/org/apache/pig/tools/pigstats/OutputStats.java 6a3e3eb 
  src/org/apache/pig/tools/pigstats/PigStats.java 3032728 
  src/org/apache/pig/tools/pigstats/PigStatsUtil.java e690b8d 
  src/org/apache/pig/tools/pigstats/ScriptState.java d58310d 
  src/org/apache/pig/tools/pigstats/SimpleFetchPigStats.java 61dbe3c 
  src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java 115ae1d 
  src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java ed791fd 
  src/org/apache/pig/tools/pigstats/mapreduce/MRScriptState.java 4669c4b 
  test/org/apache/pig/test/TestPigRunner.java 6964c38 

Diff: https://reviews.apache.org/r/19298/diff/


Testing
-------

All unit tests pass.


Thanks,

Cheolsoo Park


Re: Review Request 19298: PIG-3812: Several clean-ups in PigStats and JobStats

Posted by Daniel Dai <da...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19298/#review37494
-----------------------------------------------------------

Ship it!


Ship It!

- Daniel Dai


On March 17, 2014, 4:02 p.m., Cheolsoo Park wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19298/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 4:02 p.m.)
> 
> 
> Review request for pig.
> 
> 
> Bugs: PIG-3812
>     https://issues.apache.org/jira/browse/PIG-3812
> 
> 
> Repository: pig-git
> 
> 
> Description
> -------
> 
> While implementing counter support in Tez mode (PIG-3603), I made several changes to PigStats and JobStats classes. To avoid breaking backward compatibility between 0.13 and onwards, I'd like to commit this patch into trunk prior to 0.13 release.
> 
> The changes include-
> * Renamed SimpleFetchPigStats to EmptyPigStats. This dummy PigStats can be used not only when the direct fetch mode is enabled but also when no Hadoop jobs are submitted due to any front-end error. So I renamed it to EmptyPigStats.
> * Moved getHdfsBytes[Read|Written]() from MRPigStatsUtil to PigStatsUtil since these are not MR specific.
> * Moved [MAP|REDUCE]_[IN|OUT]PUT_RECORDS from MRPigStatsUtil to PigStatsUtil since Tez MRInput and MROutput also use them.
> * Fixed a typo in JobStats#getAvgREduceTime(): REduce -> Reduce.
> * Fixed white spaces.
> 
> 
> Diffs
> -----
> 
>   src/org/apache/pig/PigServer.java 3d50a3e 
>   src/org/apache/pig/backend/hadoop/executionengine/fetch/FetchLauncher.java d81f03b 
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java b483323 
>   src/org/apache/pig/parser/QueryParserDriver.java ec5b2b9 
>   src/org/apache/pig/parser/SourceLocation.java b69546f 
>   src/org/apache/pig/tools/pigstats/EmptyPigStats.java e69de29 
>   src/org/apache/pig/tools/pigstats/InputStats.java 38c8372 
>   src/org/apache/pig/tools/pigstats/JobStats.java 4484348 
>   src/org/apache/pig/tools/pigstats/OutputStats.java 6a3e3eb 
>   src/org/apache/pig/tools/pigstats/PigStats.java 3032728 
>   src/org/apache/pig/tools/pigstats/PigStatsUtil.java e690b8d 
>   src/org/apache/pig/tools/pigstats/ScriptState.java d58310d 
>   src/org/apache/pig/tools/pigstats/SimpleFetchPigStats.java 61dbe3c 
>   src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java 115ae1d 
>   src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java ed791fd 
>   src/org/apache/pig/tools/pigstats/mapreduce/MRScriptState.java 4669c4b 
>   test/org/apache/pig/test/TestPigRunner.java 6964c38 
> 
> Diff: https://reviews.apache.org/r/19298/diff/
> 
> 
> Testing
> -------
> 
> All unit tests pass.
> 
> 
> Thanks,
> 
> Cheolsoo Park
> 
>