You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Cheolsoo Park <pi...@gmail.com> on 2014/03/17 17:02:32 UTC
Review Request 19298: PIG-3812: Several clean-ups in PigStats and JobStats
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19298/
-----------------------------------------------------------
Review request for pig.
Bugs: PIG-3812
https://issues.apache.org/jira/browse/PIG-3812
Repository: pig-git
Description
-------
While implementing counter support in Tez mode (PIG-3603), I made several changes to PigStats and JobStats classes. To avoid breaking backward compatibility between 0.13 and onwards, I'd like to commit this patch into trunk prior to 0.13 release.
The changes include-
* Renamed SimpleFetchPigStats to EmptyPigStats. This dummy PigStats can be used not only when the direct fetch mode is enabled but also when no Hadoop jobs are submitted due to any front-end error. So I renamed it to EmptyPigStats.
* Moved getHdfsBytes[Read|Written]() from MRPigStatsUtil to PigStatsUtil since these are not MR specific.
* Moved [MAP|REDUCE]_[IN|OUT]PUT_RECORDS from MRPigStatsUtil to PigStatsUtil since Tez MRInput and MROutput also use them.
* Fixed a typo in JobStats#getAvgREduceTime(): REduce -> Reduce.
* Fixed white spaces.
Diffs
-----
src/org/apache/pig/PigServer.java 3d50a3e
src/org/apache/pig/backend/hadoop/executionengine/fetch/FetchLauncher.java d81f03b
src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java b483323
src/org/apache/pig/parser/QueryParserDriver.java ec5b2b9
src/org/apache/pig/parser/SourceLocation.java b69546f
src/org/apache/pig/tools/pigstats/EmptyPigStats.java e69de29
src/org/apache/pig/tools/pigstats/InputStats.java 38c8372
src/org/apache/pig/tools/pigstats/JobStats.java 4484348
src/org/apache/pig/tools/pigstats/OutputStats.java 6a3e3eb
src/org/apache/pig/tools/pigstats/PigStats.java 3032728
src/org/apache/pig/tools/pigstats/PigStatsUtil.java e690b8d
src/org/apache/pig/tools/pigstats/ScriptState.java d58310d
src/org/apache/pig/tools/pigstats/SimpleFetchPigStats.java 61dbe3c
src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java 115ae1d
src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java ed791fd
src/org/apache/pig/tools/pigstats/mapreduce/MRScriptState.java 4669c4b
test/org/apache/pig/test/TestPigRunner.java 6964c38
Diff: https://reviews.apache.org/r/19298/diff/
Testing
-------
All unit tests pass.
Thanks,
Cheolsoo Park
Re: Review Request 19298: PIG-3812: Several clean-ups in PigStats and
JobStats
Posted by Daniel Dai <da...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19298/#review37494
-----------------------------------------------------------
Ship it!
Ship It!
- Daniel Dai
On March 17, 2014, 4:02 p.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19298/
> -----------------------------------------------------------
>
> (Updated March 17, 2014, 4:02 p.m.)
>
>
> Review request for pig.
>
>
> Bugs: PIG-3812
> https://issues.apache.org/jira/browse/PIG-3812
>
>
> Repository: pig-git
>
>
> Description
> -------
>
> While implementing counter support in Tez mode (PIG-3603), I made several changes to PigStats and JobStats classes. To avoid breaking backward compatibility between 0.13 and onwards, I'd like to commit this patch into trunk prior to 0.13 release.
>
> The changes include-
> * Renamed SimpleFetchPigStats to EmptyPigStats. This dummy PigStats can be used not only when the direct fetch mode is enabled but also when no Hadoop jobs are submitted due to any front-end error. So I renamed it to EmptyPigStats.
> * Moved getHdfsBytes[Read|Written]() from MRPigStatsUtil to PigStatsUtil since these are not MR specific.
> * Moved [MAP|REDUCE]_[IN|OUT]PUT_RECORDS from MRPigStatsUtil to PigStatsUtil since Tez MRInput and MROutput also use them.
> * Fixed a typo in JobStats#getAvgREduceTime(): REduce -> Reduce.
> * Fixed white spaces.
>
>
> Diffs
> -----
>
> src/org/apache/pig/PigServer.java 3d50a3e
> src/org/apache/pig/backend/hadoop/executionengine/fetch/FetchLauncher.java d81f03b
> src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java b483323
> src/org/apache/pig/parser/QueryParserDriver.java ec5b2b9
> src/org/apache/pig/parser/SourceLocation.java b69546f
> src/org/apache/pig/tools/pigstats/EmptyPigStats.java e69de29
> src/org/apache/pig/tools/pigstats/InputStats.java 38c8372
> src/org/apache/pig/tools/pigstats/JobStats.java 4484348
> src/org/apache/pig/tools/pigstats/OutputStats.java 6a3e3eb
> src/org/apache/pig/tools/pigstats/PigStats.java 3032728
> src/org/apache/pig/tools/pigstats/PigStatsUtil.java e690b8d
> src/org/apache/pig/tools/pigstats/ScriptState.java d58310d
> src/org/apache/pig/tools/pigstats/SimpleFetchPigStats.java 61dbe3c
> src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java 115ae1d
> src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java ed791fd
> src/org/apache/pig/tools/pigstats/mapreduce/MRScriptState.java 4669c4b
> test/org/apache/pig/test/TestPigRunner.java 6964c38
>
> Diff: https://reviews.apache.org/r/19298/diff/
>
>
> Testing
> -------
>
> All unit tests pass.
>
>
> Thanks,
>
> Cheolsoo Park
>
>