You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2015/05/13 13:02:33 UTC

Re: Review Request 33581: LENS-198: LENS:198: Allow timed dimensions of cube to be non-partitioned columns as well

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33581/
-----------------------------------------------------------

(Updated May 13, 2015, 4:32 p.m.)


Review request for lens.


Summary (updated)
-----------------

LENS-198: LENS:198: Allow timed dimensions of cube to be non-partitioned columns as well


Bugs: LENS-198
    https://issues.apache.org/jira/browse/LENS-198


Repository: lens


Description
-------

Allow timed dimensions to be a field and not a time partitioning column.


Diffs (updated)
-----

  lens-api/src/main/resources/cube-0.1.xsd 10b1dfa 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 01d76c0 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java 8a6dcba 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 74a26fe 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java f644c16 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java 1837ae9 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRangeList.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java 8a533fc 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java 6646718 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AliasReplacer.java a65bc96 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 47b9fc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java 165525d 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java b7c3ee9 
  lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 82c43a7 
  lens-cube/src/main/java/org/apache/lens/cube/parse/DenormalizationResolver.java f5d2115 
  lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java e200b12 
  lens-cube/src/main/java/org/apache/lens/cube/parse/DimOnlyHQLContext.java 8a23ded 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java c81cf34 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactHQLContext.java 766769f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FieldValidator.java ddabc25 
  lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java a41424a 
  lens-cube/src/main/java/org/apache/lens/cube/parse/MaxCoveringFactResolver.java fcc6114 
  lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactHQLContext.java beaa551 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java e5a6b32 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 612cdf9 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 1a30228 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 3203354 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 37c578b 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestBaseCubeQueries.java b1d7a32 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java c915353 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestDateUtil.java 3df704d 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java 29240b6 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQueryRewrite.java 06de5cd 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestRewriterPlan.java 79d9be3 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeExtractor.java bd51971 
  lens-examples/src/main/java/org/apache/lens/examples/PopulateSampleMetastore.java 7883b15 
  lens-examples/src/main/resources/cube-queries.sql 7a0b542 
  lens-examples/src/main/resources/customer.xml 64024f1 
  lens-examples/src/main/resources/sales-aggr-fact2-local-parts.xml 35a8801 
  lens-examples/src/main/resources/sales-aggr-fact2.xml b535a87 
  lens-examples/src/main/resources/sales-cube.xml f5e2f93 

Diff: https://reviews.apache.org/r/33581/diff/


Testing (updated)
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.197s]
[INFO] Lens .............................................. SUCCESS [3.371s]
[INFO] Lens API .......................................... SUCCESS [26.972s]
[INFO] Lens API for server and extensions ................ SUCCESS [36.260s]
[INFO] Lens Cube ......................................... SUCCESS [4:27.436s]
[INFO] Lens DB storage ................................... SUCCESS [33.380s]
[INFO] Lens Query Library ................................ SUCCESS [29.792s]
[INFO] Lens Hive Driver .................................. SUCCESS [3:31.969s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [33.706s]
[INFO] Lens Server ....................................... SUCCESS [4:52.979s]
[INFO] Lens client ....................................... SUCCESS [41.132s]
[INFO] Lens CLI .......................................... SUCCESS [3:07.546s]
[INFO] Lens Examples ..................................... SUCCESS [9.252s]
[INFO] Lens Distribution ................................. SUCCESS [11.228s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:19.637s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.691s]
[INFO] Lens Regression ................................... SUCCESS [7.449s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:18.026s
[INFO] Finished at: Thu May 07 08:59:21 UTC 2015
[INFO] Final Memory: 113M/1253M
[INFO] ------------------------------------------------------------------------

Newly added Example query also verified by running separately.

Newly added test cases:
1. Prefer fact that has part col for time dim queried
2. Missing partitions on fallback part col should be reported
3. transitive fallback
4. multiple time dims queried, then fallback should be local to the range where clause
5. prefer property in fact. tested in examples
6. fallback to property in cube. tested in test cases
7. Added test cases for time diff parsing


Thanks,

Rajat Khandelwal


Re: Review Request 33581: LENS-198: LENS:198: Allow timed dimensions of cube to be non-partitioned columns as well

Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33581/#review83590
-----------------------------------------------------------

Ship it!


Ship It!

- Amareshwari Sriramadasu


On May 13, 2015, 11:02 a.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33581/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 11:02 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-198
>     https://issues.apache.org/jira/browse/LENS-198
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Allow timed dimensions to be a field and not a time partitioning column.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/resources/cube-0.1.xsd 10b1dfa 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 01d76c0 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java 8a6dcba 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 74a26fe 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java f644c16 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java 1837ae9 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRangeList.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java 8a533fc 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java 6646718 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AliasReplacer.java a65bc96 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 47b9fc1 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java 165525d 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java b7c3ee9 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 82c43a7 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/DenormalizationResolver.java f5d2115 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java e200b12 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/DimOnlyHQLContext.java 8a23ded 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java c81cf34 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactHQLContext.java 766769f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FieldValidator.java ddabc25 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java a41424a 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/MaxCoveringFactResolver.java fcc6114 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactHQLContext.java beaa551 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java e5a6b32 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 612cdf9 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 1a30228 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 3203354 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 37c578b 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestBaseCubeQueries.java b1d7a32 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java c915353 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestDateUtil.java 3df704d 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java 29240b6 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestQueryRewrite.java 06de5cd 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestRewriterPlan.java 79d9be3 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeExtractor.java bd51971 
>   lens-examples/src/main/java/org/apache/lens/examples/PopulateSampleMetastore.java 7883b15 
>   lens-examples/src/main/resources/cube-queries.sql 7a0b542 
>   lens-examples/src/main/resources/customer.xml 64024f1 
>   lens-examples/src/main/resources/sales-aggr-fact2-local-parts.xml 35a8801 
>   lens-examples/src/main/resources/sales-aggr-fact2.xml b535a87 
>   lens-examples/src/main/resources/sales-cube.xml f5e2f93 
> 
> Diff: https://reviews.apache.org/r/33581/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.197s]
> [INFO] Lens .............................................. SUCCESS [3.371s]
> [INFO] Lens API .......................................... SUCCESS [26.972s]
> [INFO] Lens API for server and extensions ................ SUCCESS [36.260s]
> [INFO] Lens Cube ......................................... SUCCESS [4:27.436s]
> [INFO] Lens DB storage ................................... SUCCESS [33.380s]
> [INFO] Lens Query Library ................................ SUCCESS [29.792s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:31.969s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [33.706s]
> [INFO] Lens Server ....................................... SUCCESS [4:52.979s]
> [INFO] Lens client ....................................... SUCCESS [41.132s]
> [INFO] Lens CLI .......................................... SUCCESS [3:07.546s]
> [INFO] Lens Examples ..................................... SUCCESS [9.252s]
> [INFO] Lens Distribution ................................. SUCCESS [11.228s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.637s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.691s]
> [INFO] Lens Regression ................................... SUCCESS [7.449s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:18.026s
> [INFO] Finished at: Thu May 07 08:59:21 UTC 2015
> [INFO] Final Memory: 113M/1253M
> [INFO] ------------------------------------------------------------------------
> 
> Newly added Example query also verified by running separately.
> 
> Newly added test cases:
> 1. Prefer fact that has part col for time dim queried
> 2. Missing partitions on fallback part col should be reported
> 3. transitive fallback
> 4. multiple time dims queried, then fallback should be local to the range where clause
> 5. prefer property in fact. tested in examples
> 6. fallback to property in cube. tested in test cases
> 7. Added test cases for time diff parsing
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>