Re: Review Request: PIG-2831: MR-Cube implementation (Distributed cubing for holistic measures)

[j....@gmail.com]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6651/
-----------------------------------------------------------

(Updated Oct. 8, 2012, 10:27 p.m.)


Review request for pig and Dmitriy Ryaboy.


Changes
-------

Updated patch with new partition algorithm that distributes the reducer workload more evenly. 

Unit tests: All passed

Pre-commit tests: All passed
ant clean test-commit


Description
-------

This is a review board request for https://issues.apache.org/jira/browse/PIG-2831


This addresses bug PIG-2831.
    https://issues.apache.org/jira/browse/PIG-2831


Diffs (updated)
-----

  src/org/apache/pig/ResourceStatistics.java 97d2c53 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/CubeCompiler.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java 996f9b8 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java 834e932 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceOper.java c4af165 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigOutputCommitter.java 9d096ba 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/RichInterInputFormat.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/RichInterOutputFormat.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/RichInterRecordReader.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/RichInterRecordWriter.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/RichInterStorage.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/plans/MRPrinter.java 157caad 
  src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POCast.java e8c2f2c 
  src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/plans/PhyPlanVisitor.java abb16ff 
  src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POCube.java PRE-CREATION 
  src/org/apache/pig/backend/hadoop/executionengine/util/MapRedUtil.java 0502917 
  src/org/apache/pig/builtin/CubeDimensions.java 5652029 
  src/org/apache/pig/builtin/JsonMetadata.java 28d5f91 
  src/org/apache/pig/builtin/RollupDimensions.java f6c26e4 
  src/org/apache/pig/data/NestedProxyBag.java PRE-CREATION 
  src/org/apache/pig/data/TupleFieldFilter.java PRE-CREATION 
  src/org/apache/pig/impl/builtin/CubeSampleLoader.java PRE-CREATION 
  src/org/apache/pig/impl/builtin/HolisticCube.java PRE-CREATION 
  src/org/apache/pig/impl/builtin/HolisticCubeCompoundKey.java PRE-CREATION 
  src/org/apache/pig/impl/builtin/PartitionLargeGroups.java PRE-CREATION 
  src/org/apache/pig/impl/builtin/PostProcessCube.java PRE-CREATION 
  src/org/apache/pig/impl/util/Utils.java 270cb6a 
  src/org/apache/pig/newplan/logical/optimizer/LogicalPlanPrinter.java b18c047 
  src/org/apache/pig/newplan/logical/relational/LOCube.java b262efb 
  src/org/apache/pig/newplan/logical/relational/LogToPhyTranslationVisitor.java 5f33b07 
  src/org/apache/pig/newplan/logical/rules/ColumnPruneHelper.java 19e7628 
  src/org/apache/pig/parser/LogicalPlanBuilder.java 27f29d7 
  src/org/apache/pig/pen/EquivalenceClasses.java 194f8cb 
  src/org/apache/pig/pen/LineageTrimmingVisitor.java 917073c 
  src/org/apache/pig/pen/util/DisplayExamples.java 265f8f7 
  test/org/apache/pig/impl/builtin/TestHolisticCubeCompundKey.java PRE-CREATION 
  test/org/apache/pig/impl/builtin/TestPostProcessCube.java PRE-CREATION 
  test/org/apache/pig/test/TestCubeOperator.java 65d56a6 
  test/org/apache/pig/test/TestNestedProxyBag.java PRE-CREATION 

Diff: https://reviews.apache.org/r/6651/diff/


Testing
-------

Unit tests: All passed

Pre-commit tests: All passed
ant clean test-commit


Thanks,

Prasanth_J