You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@lens.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2015/02/18 12:04:20 UTC

Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

Review request for lens.


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description
-------

Added data structure and small code usage.


Diffs
-----

  bin-dist-files/LICENSE PRE-CREATION 
  bin-dist-files/LICENSE-CDDL-v1.0 PRE-CREATION 
  bin-dist-files/LICENSE-CDDLv1.0-GPLv2_withCPE PRE-CREATION 
  bin-dist-files/LICENSE-CDDLv1.1-GPLv2_withCPE PRE-CREATION 
  bin-dist-files/LICENSE-EDL PRE-CREATION 
  bin-dist-files/LICENSE-EPL PRE-CREATION 
  bin-dist-files/LICENSE-GPLv3 PRE-CREATION 
  bin-dist-files/LICENSE-LGPL PRE-CREATION 
  bin-dist-files/NOTICE PRE-CREATION 
  bin-dist-files/README PRE-CREATION 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensFactCommands.java 0330272d04703f19442e1a497e6d9b9543657559 
  lens-cli/src/test/java/org/apache/lens/cli/TestLensFactCommands.java 4764ac3122e0746ebfa0c25c04e6d71b328caae3 
  lens-client-dist/LICENSE.txt 261eeb9e9f8b2b4b0d119366dda99c6fd7d35c64 
  lens-client-dist/pom.xml cb177a1b130a6196857fad31ddf66ac0970d94ec 
  lens-client-dist/src/site/apt/index.apt b45f4f8a6aca1a019523398681493b05ea1e062f 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java 2458620f0de1ebce1c0f383e19cb7de63875c3da 
  lens-client/src/main/java/org/apache/lens/client/LensMetadataClient.java a4c5a4d1afb663b8467e2d7be25aad4d6e096ef9 
  lens-client/src/test/java/org/apache/lens/client/TestLensClient.java 1563104d637192b19f9fb0a54dc8829db53af0a0 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java c920afbccabd5ea414330173b7a2faa6c3c38800 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 418ebe4d8494aa21bfdf50c177529d359cc3175c 
  lens-dist/pom.xml f1a0233c6d01fa6fa7d4f8ad3689f3f50c9a2934 
  lens-dist/src/deb/control/client-control/control 51d3c9f63ed5a4393ce5962a4c62f2a75429bc9d 
  lens-dist/src/deb/control/server-control/control b8f733345731f051b55fe1ba273cfb9eafaa954c 
  lens-dist/src/deb/control/server-control/postinst 0cd9d57b9ef45e028a19b8e9df2ad84b111c9368 
  lens-dist/src/main/assembly/bin-dist.xml 4ec2d5581a3b8a5c62ea1430ad28d0d1e8d9f0d7 
  lens-dist/src/main/assembly/server-dist.xml 50122c4b93cce1e7627388e36f2f9f215fb07e69 
  lens-driver-jdbc/pom.xml 9d70849d2dc67eb4d2e6eb525001058f70254eb1 
  lens-examples/src/main/java/org/apache/lens/examples/SampleQueries.java 66a577489aa68fd5e6ea151923d3ba2d2eee39d4 
  lens-ml-lib/pom.xml e256ab7f7c7650d5809942250c5064f181380714 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java b4cef60d1ea7f718b0495c6eaf9037a4cd8d3d49 
  lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java 88d48691d5665f64c9d225fcbb8e3084eb548597 
  lens-server/pom.xml f119ef1b6a7a227ee7b0bedc90b05461778d6ed1 
  lens-server/src/main/java/org/apache/lens/server/LensServer.java 5f420050fc038d2402fdbb0169b044c925ea181f 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java dd2cff2c6f22b22058918abf2351091de25dfafb 
  lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java 61e2d22a71108c08e0775e2ee5eaf8639bde9318 
  lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 22a9440c02f8a5ba152a0d70d42b7150ef592a1f 
  lens-server/src/main/java/org/apache/lens/server/ui/SessionUIResource.java c32edb4df8e9c9b572dcb45d4c304a2c1d269ffa 
  lens-server/src/test/java/org/apache/lens/server/TestLensServer.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java ec2656fffaf72e19206e41a18d1ae43687d71044 
  lens-server/src/test/java/org/apache/lens/server/session/TestSessionResource.java 78ffc87959813e38d9c120517030bff415606562 
  pom.xml ba861897214a66b3ade54306f54b174ed06e2e59 
  src/site/apt/admin/config.apt c5f3b2fccfa1993b49ef0dfd0444eab0df82e9bb 
  src/site/apt/developer/contribute.apt 5eb5ff5a3660abf577517185fd57a2e70c88506d 
  src/site/apt/lenshome/install-and-run.apt 98c1c6647de736772f1cc2e269b426ef73d25515 
  src/site/apt/releases/incompatibility.apt 8a8293a2a499ee684da2524e003de3ce62a99ff5 
  src/site/apt/releases/release-note.apt 8a8293a2a499ee684da2524e003de3ce62a99ff5 
  src/site/site.xml a134ff2ec4c5264b1093b99e2d6b38f02c461853 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java, line 1199
> > <https://reviews.apache.org/r/31154/diff/9/?file=885920#file885920line1199>
> >
> >     Can we enable asserts now? It should be easy to construct expected queries now, since all partitions are getting added.

Will need to modify the assertion code again after LENS-282. So let's delegete this issue uptill LENS-282


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75476
-----------------------------------------------------------


On March 9, 2015, 5:36 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 9, 2015, 5:36 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 30
> > <https://reviews.apache.org/r/31154/diff/9/?file=885904#file885904line30>
> >
> >     Right now the non of the classes in org.apache.lens.cube.metadata depend on org.apache.lens.cube.parse. Its only the other way. Should FactPartition be moved to metadata?

Moved.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 287
> > <https://reviews.apache.org/r/31154/diff/9/?file=885904#file885904line287>
> >
> >     Do we need similar method for drop as well?

defined.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75476
-----------------------------------------------------------


On March 9, 2015, 5:03 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 9, 2015, 5:03 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java, line 31
> > <https://reviews.apache.org/r/31154/diff/9/?file=885902#file885902line31>
> >
> >     Isn't the type of key String here?

No. `get` always takes `Object`.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 111
> > <https://reviews.apache.org/r/31154/diff/9/?file=885904#file885904line111>
> >
> >     Why are we not populating the cache if not already populated?
> >     
> >     It might give wrong values because cache is populated for one of the storage and not for other storage.

We are populating. `get(a,b,c,c)` calls `get(a,b).get(c).get(d)` and `get(a,b)` takes care of auto-populating


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 951
> > <https://reviews.apache.org/r/31154/diff/9/?file=885904#file885904line951>
> >
> >     Can we check usage of this method?

only in `TestCubeMetastoreClient`


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java, line 54
> > <https://reviews.apache.org/r/31154/diff/9/?file=885910#file885910line54>
> >
> >     What is the significance of return value here? Is it saying true when updated and false othewise?

Yes. Just like standara java Data structures, add/delete/... return boolean indicating whether the operation succeeded.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java, line 78
> > <https://reviews.apache.org/r/31154/diff/9/?file=885910#file885910line78>
> >
> >     Why is result value boolean & here?

Wanting to return true only when adding of all the partitions succeeded.


> On March 6, 2015, 4:33 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java, line 40
> > <https://reviews.apache.org/r/31154/diff/9/?file=885913#file885913line40>
> >
> >     I hope this is not the default timeline right now. Added only for exploration purpose. If so, can you add a comment?

Yes. In test cases, one of the tables is using this. Rest all are using the default, i.e. `EndsAndHolesPartitionTimeline`.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75476
-----------------------------------------------------------


On March 5, 2015, 9:06 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 9:06 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-client/src/main/java/org/apache/lens/client/LensClientSingletonWrapper.java 05964e1021910038f46a2ca141d1bf56ee2f4e03 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75476
-----------------------------------------------------------



lens-client/src/main/java/org/apache/lens/client/LensClientSingletonWrapper.java
<https://reviews.apache.org/r/31154/#comment122539>

    Can you remove system.out statements? I think they will be handled in LENS-391



lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java
<https://reviews.apache.org/r/31154/#comment122540>

    Isn't the type of key String here?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java
<https://reviews.apache.org/r/31154/#comment122541>

    type of key - String?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java
<https://reviews.apache.org/r/31154/#comment122542>

    type of key - String?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122543>

    Right now the non of the classes in org.apache.lens.cube.metadata depend on org.apache.lens.cube.parse. Its only the other way. Should FactPartition be moved to metadata?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122544>

    Why are we not populating the cache if not already populated?
    
    It might give wrong values because cache is populated for one of the storage and not for other storage.



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122545>

    can the third param be only partCol, instead of latestPartCol?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122546>

    Lets name the variable partCols instead of cols.
    
    Can we move variables timeParts and partCols outside the loop? 
    
    They are used not just in this for loop, but in other blocks of this method as well. They can be fetched once and used in all blocks.



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122547>

    could not understand the description. Is it updating the PartitionTimeline for all time partition columns of added partition?
    
    If so, please update description and the method name as well. Should method name be updatePartitionTimelineForAddPartition



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122548>

    Do we need similar method for drop as well?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122549>

    Shall we call the method partitionTimeExists?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122551>

    Can we check usage of this method?



lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java
<https://reviews.apache.org/r/31154/#comment122552>

    Please include unit tests for all the helper methods, trucation and date format use cases.



lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122553>

    what does excluding edges mean? Can you include example here?



lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122554>

    What is the significance of return value here? Is it saying true when updated and false othewise?



lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122556>

    Why is result value boolean & here?



lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122555>

    Again, whats the significance of return value here?



lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122557>

    I hope this is not the default timeline right now. Added only for exploration purpose. If so, can you add a comment?



lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122558>

    Please go ahead and remove commented code



lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java
<https://reviews.apache.org/r/31154/#comment122559>

    Can we use constant strings for properties here?



lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java
<https://reviews.apache.org/r/31154/#comment122560>

    Can we enable asserts now? It should be easy to construct expected queries now, since all partitions are getting added.


- Amareshwari Sriramadasu


On March 5, 2015, 3:36 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 3:36 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-client/src/main/java/org/apache/lens/client/LensClientSingletonWrapper.java 05964e1021910038f46a2ca141d1bf56ee2f4e03 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 9, 2015, 6:53 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java f48acf2efe074541f1317edc1375e9b3e42d95e5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing (updated)
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.180s]
[INFO] Lens .............................................. SUCCESS [1.755s]
[INFO] Lens API .......................................... SUCCESS [5.827s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.234s]
[INFO] Lens Cube ......................................... SUCCESS [2:15.346s]
[INFO] Lens DB storage ................................... SUCCESS [10.320s]
[INFO] Lens Query Library ................................ SUCCESS [4.555s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:36.663s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [25.845s]
[INFO] Lens Server ....................................... SUCCESS [4:31.369s]
[INFO] Lens client ....................................... SUCCESS [22.091s]
[INFO] Lens CLI .......................................... SUCCESS [1:49.309s]
[INFO] Lens Examples ..................................... SUCCESS [0.836s]
[INFO] Lens Distribution ................................. SUCCESS [9.830s]
[INFO] Lens ML Lib ....................................... SUCCESS [42.300s]
[INFO] Lens Regression ................................... SUCCESS [0.475s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:25.908s
[INFO] Finished at: Mon Mar 09 12:41:37 UTC 2015
[INFO] Final Memory: 108M/1268M
[INFO] ------------------------------------------------------------------------

For timeline tests, modified one existing test case to insert and delete the following partitions in the order specified:

In the test case, a fact table is created with three time parts. many partitions are added and all are deleted one by one. The order of addition and deletion is given in the following table: 

0 means time value of the partition column is `now`
x means time value of the partition column is `now + x hours`
-x means time value of the partition column is `now - x hours`

Each add/drop is done in two storages, one storage is using EndsAndHoles timeline, the other is using StoreAll Timeline. After each add/drop, I verify both kinds of timelines for correctness. The expected value is provided manually. Also verifying a foreach loop on both timelines gives same collection of partitions in the same order. 

Delete order is in no way related to add order, was randomly picked. 

At the start, first,latest should be null and holes should be empty. Same thing at the end when all the partitions are dropped. 

|-------------------------------------------
| dt | it | et | add order | delete order
|----|----|----|-----------|-------------
| 0  | 0  | 0  |     1     |      7
| 0  | -  | 1  |     2     |      3
| 0  | -  | 0  |     3     |      5  
| 0  | 1  | -1 |     4     |      4 
| 1  | -1 | -2 |     5     |      1
| -2 | -1 | -2 |     6     |      6
| -5 | -5 | -5 |     7     |      2


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 9, 2015, 6:47 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java f48acf2efe074541f1317edc1375e9b3e42d95e5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing (updated)
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.180s]
[INFO] Lens .............................................. SUCCESS [1.755s]
[INFO] Lens API .......................................... SUCCESS [5.827s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.234s]
[INFO] Lens Cube ......................................... SUCCESS [2:15.346s]
[INFO] Lens DB storage ................................... SUCCESS [10.320s]
[INFO] Lens Query Library ................................ SUCCESS [4.555s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:36.663s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [25.845s]
[INFO] Lens Server ....................................... SUCCESS [4:31.369s]
[INFO] Lens client ....................................... SUCCESS [22.091s]
[INFO] Lens CLI .......................................... SUCCESS [1:49.309s]
[INFO] Lens Examples ..................................... SUCCESS [0.836s]
[INFO] Lens Distribution ................................. SUCCESS [9.830s]
[INFO] Lens ML Lib ....................................... SUCCESS [42.300s]
[INFO] Lens Regression ................................... SUCCESS [0.475s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:25.908s
[INFO] Finished at: Mon Mar 09 12:41:37 UTC 2015
[INFO] Final Memory: 108M/1268M
[INFO] ------------------------------------------------------------------------

For timeline tests, modified one existing test case to insert and delete the following partitions in the order specified:


|-------------------------------------------
| dt | it | et | add order | delete order
|----|----|----|-----------|-------------
| 0  | 0  | 0  |     1     |      7
| 0  | -  | 1  |     2     |      3
| 0  | -  | 0  |     3     |      5  
| 0  | 1  | -1 |     4     |      4 
| 1  | -1 | -2 |     5     |      1
| -2 | -1 | -2 |     6     |      6
| -5 | -5 | -5 |     7     |      2


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 9, 2015, 6:20 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 630
> > <https://reviews.apache.org/r/31154/diff/12/?file=889166#file889166line630>
> >
> >     Shall we update latest partition also until LENS-390? This will allow us to rollback the changes easily, if required.
> >     
> >     What say? How big will be the changes?

Not needed. The only use case of `partition = 'latest'` was the `getLatest` call and that is no longer using it.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75693
-----------------------------------------------------------


On March 9, 2015, 6:53 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 9, 2015, 6:53 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java f48acf2efe074541f1317edc1375e9b3e42d95e5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.180s]
> [INFO] Lens .............................................. SUCCESS [1.755s]
> [INFO] Lens API .......................................... SUCCESS [5.827s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.234s]
> [INFO] Lens Cube ......................................... SUCCESS [2:15.346s]
> [INFO] Lens DB storage ................................... SUCCESS [10.320s]
> [INFO] Lens Query Library ................................ SUCCESS [4.555s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:36.663s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [25.845s]
> [INFO] Lens Server ....................................... SUCCESS [4:31.369s]
> [INFO] Lens client ....................................... SUCCESS [22.091s]
> [INFO] Lens CLI .......................................... SUCCESS [1:49.309s]
> [INFO] Lens Examples ..................................... SUCCESS [0.836s]
> [INFO] Lens Distribution ................................. SUCCESS [9.830s]
> [INFO] Lens ML Lib ....................................... SUCCESS [42.300s]
> [INFO] Lens Regression ................................... SUCCESS [0.475s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:25.908s
> [INFO] Finished at: Mon Mar 09 12:41:37 UTC 2015
> [INFO] Final Memory: 108M/1268M
> [INFO] ------------------------------------------------------------------------
> 
> For timeline tests, modified one existing test case to insert and delete the following partitions in the order specified:
> 
> In the test case, a fact table is created with three time parts. many partitions are added and all are deleted one by one. The order of addition and deletion is given in the following table: 
> 
> 0 means time value of the partition column is `now`
> x means time value of the partition column is `now + x hours`
> -x means time value of the partition column is `now - x hours`
> 
> Each add/drop is done in two storages, one storage is using EndsAndHoles timeline, the other is using StoreAll Timeline. After each add/drop, I verify both kinds of timelines for correctness. The expected value is provided manually. Also verifying a foreach loop on both timelines gives same collection of partitions in the same order. 
> 
> Delete order is in no way related to add order, was randomly picked. 
> 
> At the start, first,latest should be null and holes should be empty. Same thing at the end when all the partitions are dropped. 
> 
> |-------------------------------------------
> | dt | it | et | add order | delete order
> |----|----|----|-----------|-------------
> | 0  | 0  | 0  |     1     |      7
> | 0  | -  | 1  |     2     |      3
> | 0  | -  | 0  |     3     |      5  
> | 0  | 1  | -1 |     4     |      4 
> | 1  | -1 | -2 |     5     |      1
> | -2 | -1 | -2 |     6     |      6
> | -5 | -5 | -5 |     7     |      2
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75693
-----------------------------------------------------------


Overall changes look fine.


lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122913>

    Assumption seems fine.



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122914>

    Shall we update latest partition also until LENS-390? This will allow us to rollback the changes easily, if required.
    
    What say? How big will be the changes?


- Amareshwari Sriramadasu


On March 9, 2015, 12:30 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 9, 2015, 12:30 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java f48acf2efe074541f1317edc1375e9b3e42d95e5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> For timeline tests, modified one existing test case to insert and delete the following partitions in the order specified:
> 
> 
> |-------------------------------------------
> | dt | it | et | add order | delete order
> |----|----|----|-----------|-------------
> | 0  | 0  | 0  |     1     |      7
> | 0  | -  | 1  |     2     |      3
> | 0  | -  | 0  |     3     |      5  
> | 0  | 1  | -1 |     4     |      4 
> | 1  | -1 | -2 |     5     |      1
> | -2 | -1 | -2 |     6     |      6
> | -5 | -5 | -5 |     7     |      2
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 9, 2015, 6 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java f48acf2efe074541f1317edc1375e9b3e42d95e5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing (updated)
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


For timeline tests, modified one existing test case to insert and delete the following partitions in the order specified:


|-------------------------------------------
| dt | it | et | add order | delete order
|----|----|----|-----------|-------------
| 0  | 0  | 0  |     1     |      7
| 0  | -  | 1  |     2     |      3
| 0  | -  | 0  |     3     |      5  
| 0  | 1  | -1 |     4     |      4 
| 1  | -1 | -2 |     5     |      1
| -2 | -1 | -2 |     6     |      6
| -5 | -5 | -5 |     7     |      2


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 9, 2015, 5:37 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 9, 2015, 5:36 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 9, 2015, 5:03 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java 14ea806638ac82fa3aece4219d5bcb834e6e6e54 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java 722230b444093e369238293ef67033b63ebbafbf 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java 14feb2479ddecaebe94ef5aa4b303be2ade4ee4c 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java 27c3b89624d036f05e7ac7a97951a29767559cc1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java c6e275f2755a5bace0aeac6338036339a088cc9f 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java 911bcef0475e9db8326bf105ff19f3f10415eb67 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 418fb085aa39c85cd502954f3f8b52cdd3aa87e8 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 5, 2015, 9:06 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

finished with changes.


Diffs (updated)
-----

  lens-client/src/main/java/org/apache/lens/client/LensClientSingletonWrapper.java 05964e1021910038f46a2ca141d1bf56ee2f4e03 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 5, 2015, 5:34 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description (updated)
-------

finished with changes.


Diffs
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 5, 2015, 5:33 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 

cube tests are all passing, remaining things:


1. PartitionInfo API, name change and some util methods addition
2. Some newly added methods and code belongs in other classes, will move them appropriately.
3. Server tests,(TestMetastoreResource)
4. javadoc
5. will see if some more abstraction can be made, redundancies removed.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing (updated)
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
[INFO] Lens .............................................. SUCCESS [3.121s]
[INFO] Lens API .......................................... SUCCESS [7.252s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
[INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
[INFO] Lens DB storage ................................... SUCCESS [9.340s]
[INFO] Lens Query Library ................................ SUCCESS [5.294s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
[INFO] Lens Server ....................................... SUCCESS [4:36.485s]
[INFO] Lens client ....................................... SUCCESS [21.727s]
[INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
[INFO] Lens Examples ..................................... SUCCESS [0.785s]
[INFO] Lens Distribution ................................. SUCCESS [9.733s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
[INFO] Lens Regression ................................... SUCCESS [0.487s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:54.728s
[INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
[INFO] Final Memory: 108M/1102M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 5, 2015, 4:15 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 

cube tests are all passing, remaining things:


1. PartitionInfo API, name change and some util methods addition
2. Some newly added methods and code belongs in other classes, will move them appropriately.
3. Server tests,(TestMetastoreResource)
4. javadoc
5. will see if some more abstraction can be made, redundancies removed.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java ed906fee11fa4b68854b76deb0919b181f0e277a 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
[INFO] Lens .............................................. SUCCESS [2.144s]
[INFO] Lens API .......................................... SUCCESS [6.331s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
[INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
[INFO] Lens DB storage ................................... SUCCESS [9.680s]
[INFO] Lens Query Library ................................ SUCCESS [4.783s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
[INFO] Lens Server ....................................... SUCCESS [4:35.297s]
[INFO] Lens client ....................................... SUCCESS [20.677s]
[INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
[INFO] Lens Examples ..................................... SUCCESS [0.873s]
[INFO] Lens Distribution ................................. SUCCESS [9.986s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
[INFO] Lens Regression ................................... SUCCESS [0.434s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:35.639s
[INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
[INFO] Final Memory: 110M/1305M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 5, 2015, 12:24 p.m.)


Review request for lens.


Bugs: LENS-281
    https://issues.apache.org/jira/browse/LENS-281


Repository: lens


Description
-------

Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 

cube tests are all passing, remaining things:


1. PartitionInfo API, name change and some util methods addition
2. Some newly added methods and code belongs in other classes, will move them appropriately.
3. Server tests,(TestMetastoreResource)
4. javadoc
5. will see if some more abstraction can be made, redundancies removed.


Diffs
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
[INFO] Lens .............................................. SUCCESS [2.144s]
[INFO] Lens API .......................................... SUCCESS [6.331s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
[INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
[INFO] Lens DB storage ................................... SUCCESS [9.680s]
[INFO] Lens Query Library ................................ SUCCESS [4.783s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
[INFO] Lens Server ....................................... SUCCESS [4:35.297s]
[INFO] Lens client ....................................... SUCCESS [20.677s]
[INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
[INFO] Lens Examples ..................................... SUCCESS [0.873s]
[INFO] Lens Distribution ................................. SUCCESS [9.986s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
[INFO] Lens Regression ................................... SUCCESS [0.434s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:35.639s
[INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
[INFO] Final Memory: 110M/1305M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 5, 2015, 12:22 p.m.)


Review request for lens.


Summary (updated)
-----------------

LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description
-------

Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 

cube tests are all passing, remaining things:


1. PartitionInfo API, name change and some util methods addition
2. Some newly added methods and code belongs in other classes, will move them appropriately.
3. Server tests,(TestMetastoreResource)
4. javadoc
5. will see if some more abstraction can be made, redundancies removed.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
[INFO] Lens .............................................. SUCCESS [2.144s]
[INFO] Lens API .......................................... SUCCESS [6.331s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
[INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
[INFO] Lens DB storage ................................... SUCCESS [9.680s]
[INFO] Lens Query Library ................................ SUCCESS [4.783s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
[INFO] Lens Server ....................................... SUCCESS [4:35.297s]
[INFO] Lens client ....................................... SUCCESS [20.677s]
[INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
[INFO] Lens Examples ..................................... SUCCESS [0.873s]
[INFO] Lens Distribution ................................. SUCCESS [9.986s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
[INFO] Lens Regression ................................... SUCCESS [0.434s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:35.639s
[INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
[INFO] Final Memory: 110M/1305M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Jaideep dhok <ja...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75163
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java
<https://reviews.apache.org/r/31154/#comment122131>

    Might need to override other methods of HashMap as well, otherwise implementation is not consistent.
    For ex - containsKey, putAll, constructor which takes map etc.
    
    Also, HashMap allows null keys, here it will throw NPE.



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122132>

    Do you mean max here?



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122134>

    


- Jaideep dhok


On March 3, 2015, 8:34 a.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 8:34 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Cube.java d9af602a87cb9d005b0de024750a4d4937888e27 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
>   lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java a0abb0afe3798218f5f4c06d422b5fd36ec628d3 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 4, 2015, 5:08 p.m., Jaideep dhok wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java, line 106
> > <https://reviews.apache.org/r/31154/diff/5/?file=882819#file882819line106>
> >
> >     Why are we removing this partition from holes here? shouldn't we be modifying holes only in add or remove methods?

This function is only used in drop. And that too when the partition to be dropped is first/latest. If edge partition is dropped, we have to find the next edge partition and that would be the first non-hole partition between the dropped edge partition and the other edge partition. We have to remove the partitions encountered during this calculation to ensure the timeline is in consistent state i.e. all holes should be between the edge partitions.


> On March 4, 2015, 5:08 p.m., Jaideep dhok wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java, line 112
> > <https://reviews.apache.org/r/31154/diff/5/?file=882819#file882819line112>
> >
> >     Are we compacting holes here?

Yes. Renamed such methods.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75167
-----------------------------------------------------------


On March 5, 2015, 12:24 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 12:24 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 4, 2015, 5:08 p.m., Jaideep dhok wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java, line 19
> > <https://reviews.apache.org/r/31154/diff/5/?file=882819#file882819line19>
> >
> >     +1 on this class, but let's add unit tests around this.

taken care inside CubeTestSetup.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75167
-----------------------------------------------------------


On March 5, 2015, 5:34 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 5:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Jaideep dhok <ja...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75167
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122141>

    Use stringutils.split since we are using stringutis.join while storing.



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122143>

    +1 on this class, but let's add unit tests around this.



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122137>

    Why are we removing this partition from holes here? shouldn't we be modifying holes only in add or remove methods?



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java
<https://reviews.apache.org/r/31154/#comment122138>

    Are we compacting holes here?


- Jaideep dhok


On March 3, 2015, 8:34 a.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 8:34 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Cube.java d9af602a87cb9d005b0de024750a4d4937888e27 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
>   lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java a0abb0afe3798218f5f4c06d422b5fd36ec628d3 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Jaideep dhok <ja...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75169
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java
<https://reviews.apache.org/r/31154/#comment122145>

    What are we doing here?


- Jaideep dhok


On March 3, 2015, 8:34 a.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 8:34 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Cube.java d9af602a87cb9d005b0de024750a4d4937888e27 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
>   lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java a0abb0afe3798218f5f4c06d422b5fd36ec628d3 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 4, 2015, 12:59 p.m., Jaideep dhok wrote:
> > Please briefly explain the high level design of this change in the JIRA. I am not able to understand from the diff itself, especially since the diff is a bit large. Also, I see many new classes created without any comments about their purpose or how they are supposed to hold partition info.

https://issues.apache.org/jira/browse/LENS-281


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75157
-----------------------------------------------------------


On March 5, 2015, 12:24 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 12:24 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Jaideep dhok <ja...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75157
-----------------------------------------------------------


Please briefly explain the high level design of this change in the JIRA. I am not able to understand from the diff itself, especially since the diff is a bit large. Also, I see many new classes created without any comments about their purpose or how they are supposed to hold partition info.


lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment122120>

    Can you explain reason behind all these checks by a comment? I think we are doing level checks on the tree, but it would make it easier to understand for everyone.


- Jaideep dhok


On March 3, 2015, 8:34 a.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 8:34 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Cube.java d9af602a87cb9d005b0de024750a4d4937888e27 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
>   lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java a0abb0afe3798218f5f4c06d422b5fd36ec628d3 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On March 5, 2015, 11:09 a.m., Jaideep dhok wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java, line 169
> > <https://reviews.apache.org/r/31154/diff/5/?file=882826#file882826line169>
> >
> >     Needs non zero check on increment.

Will throw a runtime Arithmetic Exception. I don't think a checked exception is needed here.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75296
-----------------------------------------------------------


On March 5, 2015, 12:24 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 12:24 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Jaideep dhok <ja...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review75296
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java
<https://reviews.apache.org/r/31154/#comment122279>

    All the static iterable methods need Javadoc.



lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java
<https://reviews.apache.org/r/31154/#comment122275>

    Needs non zero check on increment.



lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java
<https://reviews.apache.org/r/31154/#comment122276>

    This is updating the argument numIters, not the field. It should be this.numIters = 0;



lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java
<https://reviews.apache.org/r/31154/#comment122277>

    Move to default constructor instead of instance initializer block.



lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java
<https://reviews.apache.org/r/31154/#comment122278>

    Throw UnsupportedOperationException instead.


- Jaideep dhok


On March 3, 2015, 8:34 a.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 8:34 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Cube.java d9af602a87cb9d005b0de024750a4d4937888e27 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
>   lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java a0abb0afe3798218f5f4c06d422b5fd36ec628d3 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
> [INFO] Lens .............................................. SUCCESS [2.144s]
> [INFO] Lens API .......................................... SUCCESS [6.331s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
> [INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
> [INFO] Lens DB storage ................................... SUCCESS [9.680s]
> [INFO] Lens Query Library ................................ SUCCESS [4.783s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
> [INFO] Lens Server ....................................... SUCCESS [4:35.297s]
> [INFO] Lens client ....................................... SUCCESS [20.677s]
> [INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
> [INFO] Lens Examples ..................................... SUCCESS [0.873s]
> [INFO] Lens Distribution ................................. SUCCESS [9.986s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
> [INFO] Lens Regression ................................... SUCCESS [0.434s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:35.639s
> [INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
> [INFO] Final Memory: 110M/1305M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 3, 2015, 2:04 p.m.)


Review request for lens.


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description
-------

Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 

cube tests are all passing, remaining things:


1. PartitionInfo API, name change and some util methods addition
2. Some newly added methods and code belongs in other classes, will move them appropriately.
3. Server tests,(TestMetastoreResource)
4. javadoc
5. will see if some more abstraction can be made, redundancies removed.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveHashMap.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Cube.java d9af602a87cb9d005b0de024750a4d4937888e27 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionTimeline.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
  lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
  lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
  lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java a0abb0afe3798218f5f4c06d422b5fd36ec628d3 

Diff: https://reviews.apache.org/r/31154/diff/


Testing (updated)
-------

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.445s]
[INFO] Lens .............................................. SUCCESS [2.144s]
[INFO] Lens API .......................................... SUCCESS [6.331s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.346s]
[INFO] Lens Cube ......................................... SUCCESS [2:19.585s]
[INFO] Lens DB storage ................................... SUCCESS [9.680s]
[INFO] Lens Query Library ................................ SUCCESS [4.783s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:35.169s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [26.350s]
[INFO] Lens Server ....................................... SUCCESS [4:35.297s]
[INFO] Lens client ....................................... SUCCESS [20.677s]
[INFO] Lens CLI .......................................... SUCCESS [1:51.237s]
[INFO] Lens Examples ..................................... SUCCESS [0.873s]
[INFO] Lens Distribution ................................. SUCCESS [9.986s]
[INFO] Lens ML Lib ....................................... SUCCESS [43.241s]
[INFO] Lens Regression ................................... SUCCESS [0.434s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:35.639s
[INFO] Finished at: Tue Mar 03 08:12:37 UTC 2015
[INFO] Final Memory: 110M/1305M
[INFO] ------------------------------------------------------------------------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Jaideep dhok <ja...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review74910
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java
<https://reviews.apache.org/r/31154/#comment121737>

    Can you justify why we need to extend HashMap?
    Could we achive the same thing by composing the HashMap inside PartitionInfo object?


- Jaideep dhok


On March 1, 2015, 2:23 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 1, 2015, 2:23 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 1, 2015, 7:53 p.m.)


Review request for lens.


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description (updated)
-------

Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 

cube tests are all passing, remaining things:


1. PartitionInfo API, name change and some util methods addition
2. Some newly added methods and code belongs in other classes, will move them appropriately.
3. Server tests,(TestMetastoreResource)
4. javadoc
5. will see if some more abstraction can be made, redundancies removed.


Diffs
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated March 1, 2015, 7:49 p.m.)


Review request for lens.


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description
-------

Added data structure and small code usage.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
  lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
  lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
  lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On Feb. 19, 2015, 2:59 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java, line 157
> > <https://reviews.apache.org/r/31154/diff/3/?file=867834#file867834line157>
> >
> >     use getTimePartSpec. We cannot assume all are time part columns

The code moved around a lot. Can you re-check if the issue is still there?


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review73109
-----------------------------------------------------------


On March 5, 2015, 5:34 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 5:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review73109
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java
<https://reviews.apache.org/r/31154/#comment119324>

    Can we add more javadoc? Not able to understand what is each method doing and why are they required



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java
<https://reviews.apache.org/r/31154/#comment119322>

    Can we different method name than addPartition ? Its confusing



lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java
<https://reviews.apache.org/r/31154/#comment119320>

    use getTimePartSpec. We cannot assume all are time part columns


- Amareshwari Sriramadasu


On Feb. 18, 2015, 3:38 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated Feb. 18, 2015, 3:38 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Added data structure and small code usage.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java c920afbccabd5ea414330173b7a2faa6c3c38800 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On Feb. 24, 2015, 12:50 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 366
> > <https://reviews.apache.org/r/31154/diff/3/?file=867831#file867831line366>
> >
> >     We should remove symlinked latest partitions as well when we move to table partitions from symlinks.
> >     
> >     Otherwise, those would be holding old values and cause confusion.

We need to add an integration test in which we create partition with older version of lens (which creates `latest` partition) and do another partition operation with a newer version of lens(which should delete the `latest` partition and keep the info in table properties.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review73529
-----------------------------------------------------------


On March 1, 2015, 7:53 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 1, 2015, 7:53 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Posting the changes that show the logic change. Not complete yet, it's just so that anyone who wants to look at the new logic can see. 
> 
> cube tests are all passing, remaining things:
> 
> 
> 1. PartitionInfo API, name change and some util methods addition
> 2. Some newly added methods and code belongs in other classes, will move them appropriately.
> 3. Server tests,(TestMetastoreResource)
> 4. javadoc
> 5. will see if some more abstraction can be made, redundancies removed.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-281: Create Partition Timeline and keep it cached and stored as storage table properties

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On Feb. 24, 2015, 12:50 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 366
> > <https://reviews.apache.org/r/31154/diff/3/?file=867831#file867831line366>
> >
> >     We should remove symlinked latest partitions as well when we move to table partitions from symlinks.
> >     
> >     Otherwise, those would be holding old values and cause confusion.
> 
> Rajat Khandelwal wrote:
>     We need to add an integration test in which we create partition with older version of lens (which creates `latest` partition) and do another partition operation with a newer version of lens(which should delete the `latest` partition and keep the info in table properties.

I believe this Should be done separately.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review73529
-----------------------------------------------------------


On March 5, 2015, 5:34 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 5:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-281
>     https://issues.apache.org/jira/browse/LENS-281
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> finished with changes.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 7e6fe7e93a07ddb6ae4369b78e7856dc2b0a65b5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 240516d6911ecdab6a07ef77b194c013aba42f4e 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java c4b3c3a4d469f5cc609db67437b00855d907c51d 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java PRE-CREATION 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java b1a85adfb229fa0f6a708a047a7629b2faa28e30 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java a60041bf3df0fa03091674b602a78a2323f88d55 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 7eaf597bac879430dff086914865d34102003f0f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java 550df6c99b7ffe7372d319b8c4b101cdd5a4f830 
>   lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 96ba240404d1f21fbcac572ced0a4e3c911a5731 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 609b7f3a2750a0272f649d4cce3181a0588013e9 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 3c960622c4bc7d95bcbd9c66f3f64a8c7eff2cd2 
>   lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java 4047e0c8b2098929cd6f1d320f7e52db40a280aa 
>   lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 9f4595f96354eacabd758053b57183026de116c0 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.118s]
> [INFO] Lens .............................................. SUCCESS [3.121s]
> [INFO] Lens API .......................................... SUCCESS [7.252s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.777s]
> [INFO] Lens Cube ......................................... SUCCESS [2:25.228s]
> [INFO] Lens DB storage ................................... SUCCESS [9.340s]
> [INFO] Lens Query Library ................................ SUCCESS [5.294s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:33.994s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [27.057s]
> [INFO] Lens Server ....................................... SUCCESS [4:36.485s]
> [INFO] Lens client ....................................... SUCCESS [21.727s]
> [INFO] Lens CLI .......................................... SUCCESS [1:58.127s]
> [INFO] Lens Examples ..................................... SUCCESS [0.785s]
> [INFO] Lens Distribution ................................. SUCCESS [9.733s]
> [INFO] Lens ML Lib ....................................... SUCCESS [43.829s]
> [INFO] Lens Regression ................................... SUCCESS [0.487s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:54.728s
> [INFO] Finished at: Thu Mar 05 11:01:59 UTC 2015
> [INFO] Final Memory: 108M/1102M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review73529
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/31154/#comment119926>

    We should remove symlinked latest partitions as well when we move to table partitions from symlinks.
    
    Otherwise, those would be holding old values and cause confusion.


- Amareshwari Sriramadasu


On Feb. 18, 2015, 3:38 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated Feb. 18, 2015, 3:38 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Added data structure and small code usage.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java c920afbccabd5ea414330173b7a2faa6c3c38800 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/#review73091
-----------------------------------------------------------



lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java
<https://reviews.apache.org/r/31154/#comment119293>

    Can be removed?


- Amareshwari Sriramadasu


On Feb. 18, 2015, 3:38 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31154/
> -----------------------------------------------------------
> 
> (Updated Feb. 18, 2015, 3:38 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-280
>     https://issues.apache.org/jira/browse/LENS-280
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Added data structure and small code usage.
> 
> 
> Diffs
> -----
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java c920afbccabd5ea414330173b7a2faa6c3c38800 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/31154/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated Feb. 18, 2015, 9:08 p.m.)


Review request for lens.


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description
-------

Added data structure and small code usage.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java c920afbccabd5ea414330173b7a2faa6c3c38800 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------


Thanks,

Rajat Khandelwal

Re: Review Request 31154: LENS-280: Reduce number of metastore lookups during cube query rewriting

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31154/
-----------------------------------------------------------

(Updated Feb. 18, 2015, 4:35 p.m.)


Review request for lens.


Bugs: LENS-280
    https://issues.apache.org/jira/browse/LENS-280


Repository: lens


Description
-------

Added data structure and small code usage.


Diffs (updated)
-----

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java b3ed6efd102ed800519ebb9b5d15f81a2a32d5d5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java c920afbccabd5ea414330173b7a2faa6c3c38800 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 979de6d7f1d43f0fb5621e3287f3a6eef3dcdaef 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java c21c72a5ceb0271134423b8a9caf928e524cf150 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/PartitionInfo.java PRE-CREATION 

Diff: https://reviews.apache.org/r/31154/diff/


Testing
-------


Thanks,

Rajat Khandelwal