You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2015/03/30 12:02:50 UTC
Review Request 32610: LENS-462: Handle the case when a table property
becomes large for a timeline
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
Review request for lens.
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
Diff: https://reviews.apache.org/r/32610/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
> On April 2, 2015, 7:46 p.m., Himanshu Gahlaut wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java, line 123
> > <https://reviews.apache.org/r/32610/diff/3/?file=911977#file911977line123>
> >
> > It seems like this can also be modularized into a separate test method with a better name which clearly explains what is being tested.
> > expectedExceptions attribute of @Test annotation can help in avoiding the introduction of getLensExceptionInPartitionRangeCreation method here as well.
We can name it testTimeRangeCreationWithDifferentUpdatePeriod
- Himanshu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78705
-----------------------------------------------------------
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78705
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
<https://reviews.apache.org/r/32610/#comment127662>
It seems like this can also be modularized into a separate test method with a better name which clearly explains what is being tested.
expectedExceptions attribute of @Test annotation can help in avoiding the introduction of getLensExceptionInPartitionRangeCreation method here as well.
- Himanshu Gahlaut
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On April 3, 2015, 12:58 a.m., Himanshu Gahlaut wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java, line 102
> > <https://reviews.apache.org/r/32610/diff/3/?file=911977#file911977line102>
> >
> > I guess what we are trying to test on this line is that begin of time range should be less than end. It can make test cases more modular if this is a separate test case as described below. Also instead of introducing getLensExceptionInPartitionRangeCreation method, TestNg's expectedExceptions attribute in @Test annotation can be used in this particular use case.
> >
> > @Test(expectedExceptions = LensException.class)
> > public void testTimeRangeBeginShouldBeLessThanEnd() {
> > // try to create a time range where begin is more than end
> > }
This line is inside a for loop. Essencially testing that for each update period exception should come. Testng's annotation is good but will have to write a function for each update period. I didn't want test for just one update period and discard all others since they all are instances of an Enum and can be quite different. If an update period is added in future, then it should also follow the restrictions laid down by these test cases.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78699
-----------------------------------------------------------
On April 1, 2015, 12:26 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 12:26 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
> On April 2, 2015, 7:28 p.m., Himanshu Gahlaut wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java, line 102
> > <https://reviews.apache.org/r/32610/diff/3/?file=911977#file911977line102>
> >
> > I guess what we are trying to test on this line is that begin of time range should be less than end. It can make test cases more modular if this is a separate test case as described below. Also instead of introducing getLensExceptionInPartitionRangeCreation method, TestNg's expectedExceptions attribute in @Test annotation can be used in this particular use case.
> >
> > @Test(expectedExceptions = LensException.class)
> > public void testTimeRangeBeginShouldBeLessThanEnd() {
> > // try to create a time range where begin is more than end
> > }
>
> Rajat Khandelwal wrote:
> This line is inside a for loop. Essencially testing that for each update period exception should come. Testng's annotation is good but will have to write a function for each update period. I didn't want test for just one update period and discard all others since they all are instances of an Enum and can be quite different. If an update period is added in future, then it should also follow the restrictions laid down by these test cases.
public boolean before(TimePartition when) {
return this.date.before(when.date);
}
As per code before method in TimePartition is independent of update period. It is only dependent on Date object of TimePartition.
To test that TimePartition begin should be less that TimePartition end in TimeRange data model constraint, we don't need to test for all update periods. I might be missing something here, please feel free to point the same.
- Himanshu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78699
-----------------------------------------------------------
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78699
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
<https://reviews.apache.org/r/32610/#comment127650>
I guess what we are trying to test on this line is that begin of time range should be less than end. It can make test cases more modular if this is a separate test case as described below. Also instead of introducing getLensExceptionInPartitionRangeCreation method, TestNg's expectedExceptions attribute in @Test annotation can be used in this particular use case.
@Test(expectedExceptions = LensException.class)
public void testTimeRangeBeginShouldBeLessThanEnd() {
// try to create a time range where begin is more than end
}
- Himanshu Gahlaut
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78702
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
<https://reviews.apache.org/r/32610/#comment127652>
Please consider using static imports for asserts.
- Himanshu Gahlaut
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On April 3, 2015, 1:33 a.m., Himanshu Gahlaut wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java, line 99
> > <https://reviews.apache.org/r/32610/diff/3/?file=911977#file911977line99>
> >
> > Please elaborate on some use cases which will not be caught by testing only for one update period.
> > Not able to figure out the need of testing for all update periods. Please help in understanding the same.
Size of range was different for quarterly and wouldn't have been caught if not for this loop. That was because quarterly and monthly have the same calenderField.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78713
-----------------------------------------------------------
On April 1, 2015, 12:26 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 12:26 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
> On April 2, 2015, 8:03 p.m., Himanshu Gahlaut wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java, line 99
> > <https://reviews.apache.org/r/32610/diff/3/?file=911977#file911977line99>
> >
> > Please elaborate on some use cases which will not be caught by testing only for one update period.
> > Not able to figure out the need of testing for all update periods. Please help in understanding the same.
>
> Rajat Khandelwal wrote:
> Size of range was different for quarterly and wouldn't have been caught if not for this loop. That was because quarterly and monthly have the same calenderField.
Thats hinting to have a separate and modular testTimeRangeSize method and test it for all UpdatePeriods, instead of testing all use cases for all updatePeriods. Monolithic test cases have lesser scope of running in parallel and are also harder to understand.
- Himanshu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78713
-----------------------------------------------------------
On April 6, 2015, 9:15 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 6, 2015, 9:15 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
> Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
> [INFO] Lens .............................................. SUCCESS [1.808s]
> [INFO] Lens API .......................................... SUCCESS [5.496s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
> [INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
> [INFO] Lens DB storage ................................... SUCCESS [10.480s]
> [INFO] Lens Query Library ................................ SUCCESS [4.543s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
> [INFO] Lens Server ....................................... SUCCESS [4:32.022s]
> [INFO] Lens client ....................................... SUCCESS [20.184s]
> [INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
> [INFO] Lens Examples ..................................... SUCCESS [0.946s]
> [INFO] Lens Distribution ................................. SUCCESS [10.040s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
> [INFO] Lens Regression ................................... SUCCESS [0.691s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:39.305s
> [INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
> [INFO] Final Memory: 108M/972M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78713
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
<https://reviews.apache.org/r/32610/#comment127668>
Please elaborate on some use cases which will not be caught by testing only for one update period.
Not able to figure out the need of testing for all update periods. Please help in understanding the same.
- Himanshu Gahlaut
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78647
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/32610/#comment127564>
Can we print timeline on loading at info level? This would help to know the holes if any or what are the partitions available when the server comes up. very much useful in debugging. We should make sure it is not printed on every get() but only the first time.
- Amareshwari Sriramadasu
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78701
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
<https://reviews.apache.org/r/32610/#comment127651>
Please consider adding blank lines in testTimeRange method to make it more readable.
- Himanshu Gahlaut
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78708
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
<https://reviews.apache.org/r/32610/#comment127664>
Please consider static import for TimePartition.TimePartitionRange.
It is getting repeated.
- Himanshu Gahlaut
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On April 1, 2015, 8:38 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java, line 415
> > <https://reviews.apache.org/r/32610/diff/3/?file=911971#file911971line415>
> >
> > We should return null, as it will be handled by caller as expected from props.get(key) than RunTimeException
>
> Rajat Khandelwal wrote:
> Essencially I'm trying to catch the NPE early.
In timeline code itself, null is something that is always handled, because developers except a proerty not to be set. RuntimeException is something unexpected and usually not handled by developers. Since this method is returning a value of a property, it should return null, is not present and not throw RuntimeException
String rangesStr = MetastoreUtil.getNamedStringValue(properties, "ranges");
if (!Strings.isNullOrEmpty(rangesStr)) {
> On April 1, 2015, 8:38 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 217
> > <https://reviews.apache.org/r/32610/diff/3/?file=911970#file911970line217>
> >
> > Can you pass storageTable in params directly?
>
> Rajat Khandelwal wrote:
> We need both `storageTablename` and `storageTable`. Since the two paramaters are completely dependent(and deducible) on the other two parameters, it doesn't make sense to pass them separately.
Ok with it as getTable returns cached table. If no caching, this would again hit metastore for getting the table.
> On April 1, 2015, 8:38 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 196
> > <https://reviews.apache.org/r/32610/diff/3/?file=911970#file911970line196>
> >
> > Write now, do we know of any exceptions/errors that can cause this? This code block should be something we should not reach
>
> Rajat Khandelwal wrote:
> Any exception here should only be RuntimeException. Timeline was supposed to be set correctly but it somehow didn't manage to get set, or was set incompletely. In that case instead of saying that I can't load timeline, let's load from another source since it's already available.
Sure. Fine with it, deployment needs to be aware of this because getting all partitions would require more memory at runtime.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78489
-----------------------------------------------------------
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On April 1, 2015, 2:08 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 196
> > <https://reviews.apache.org/r/32610/diff/3/?file=911970#file911970line196>
> >
> > Write now, do we know of any exceptions/errors that can cause this? This code block should be something we should not reach
Any exception here should only be RuntimeException. Timeline was supposed to be set correctly but it somehow didn't manage to get set, or was set incompletely. In that case instead of saying that I can't load timeline, let's load from another source since it's already available.
> On April 1, 2015, 2:08 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 217
> > <https://reviews.apache.org/r/32610/diff/3/?file=911970#file911970line217>
> >
> > Can you pass storageTable in params directly?
We need both `storageTablename` and `storageTable`. Since the two paramaters are completely dependent(and deducible) on the other two parameters, it doesn't make sense to pass them separately.
> On April 1, 2015, 2:08 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java, line 415
> > <https://reviews.apache.org/r/32610/diff/3/?file=911971#file911971line415>
> >
> > We should return null, as it will be handled by caller as expected from props.get(key) than RunTimeException
Essencially I'm trying to catch the NPE early.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78489
-----------------------------------------------------------
On April 1, 2015, 12:26 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 12:26 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78489
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/32610/#comment127303>
+1 for separate methods. Improves readability
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/32610/#comment127304>
Write now, do we know of any exceptions/errors that can cause this? This code block should be something we should not reach
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/32610/#comment127310>
Can you pass storageTable in params directly?
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java
<https://reviews.apache.org/r/32610/#comment127311>
We should return null, as it will be handled by caller as expected from props.get(key) than RunTimeException
- Amareshwari Sriramadasu
On April 1, 2015, 6:56 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:56 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review79127
-----------------------------------------------------------
Ship it!
Ship It!
- Amareshwari Sriramadasu
On April 6, 2015, 1:04 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 6, 2015, 1:04 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
> Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java PRE-CREATION
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 6e63483
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
> [INFO] Lens .............................................. SUCCESS [1.808s]
> [INFO] Lens API .......................................... SUCCESS [5.496s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
> [INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
> [INFO] Lens DB storage ................................... SUCCESS [10.480s]
> [INFO] Lens Query Library ................................ SUCCESS [4.543s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
> [INFO] Lens Server ....................................... SUCCESS [4:32.022s]
> [INFO] Lens client ....................................... SUCCESS [20.184s]
> [INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
> [INFO] Lens Examples ..................................... SUCCESS [0.946s]
> [INFO] Lens Distribution ................................. SUCCESS [10.040s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
> [INFO] Lens Regression ................................... SUCCESS [0.691s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:39.305s
> [INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
> [INFO] Final Memory: 108M/972M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On April 7, 2015, 11:24 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java, line 195
> > <https://reviews.apache.org/r/32610/diff/5/?file=916358#file916358line195>
> >
> > How does this work if change timeline from endsholes to say ranges?
> >
> > Can we do such changes right now? If not, we can do a follow up issue.
This can be done in current setup. Set presence property to false for all tables and set one more property indicating which class to use for timeline. That'll load timeline again from all partitions. However, we do need a way for migrating from one timeline to another timeline. This is why Timeline implementations are required to implement Iterable.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review79123
-----------------------------------------------------------
On April 6, 2015, 6:34 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 6, 2015, 6:34 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
> Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java PRE-CREATION
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 6e63483
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
> [INFO] Lens .............................................. SUCCESS [1.808s]
> [INFO] Lens API .......................................... SUCCESS [5.496s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
> [INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
> [INFO] Lens DB storage ................................... SUCCESS [10.480s]
> [INFO] Lens Query Library ................................ SUCCESS [4.543s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
> [INFO] Lens Server ....................................... SUCCESS [4:32.022s]
> [INFO] Lens client ....................................... SUCCESS [20.184s]
> [INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
> [INFO] Lens Examples ..................................... SUCCESS [0.946s]
> [INFO] Lens Distribution ................................. SUCCESS [10.040s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
> [INFO] Lens Regression ................................... SUCCESS [0.691s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:39.305s
> [INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
> [INFO] Final Memory: 108M/972M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review79123
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/32610/#comment128240>
How does this work if change timeline from endsholes to say ranges?
Can we do such changes right now? If not, we can do a follow up issue.
- Amareshwari Sriramadasu
On April 6, 2015, 1:04 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 6, 2015, 1:04 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
> Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java PRE-CREATION
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 6e63483
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
> [INFO] Lens .............................................. SUCCESS [1.808s]
> [INFO] Lens API .......................................... SUCCESS [5.496s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
> [INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
> [INFO] Lens DB storage ................................... SUCCESS [10.480s]
> [INFO] Lens Query Library ................................ SUCCESS [4.543s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
> [INFO] Lens Server ....................................... SUCCESS [4:32.022s]
> [INFO] Lens client ....................................... SUCCESS [20.184s]
> [INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
> [INFO] Lens Examples ..................................... SUCCESS [0.946s]
> [INFO] Lens Distribution ................................. SUCCESS [10.040s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
> [INFO] Lens Regression ................................... SUCCESS [0.691s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:39.305s
> [INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
> [INFO] Final Memory: 108M/972M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated April 6, 2015, 6:34 p.m.)
Review request for lens.
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java PRE-CREATION
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 6e63483
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
Diff: https://reviews.apache.org/r/32610/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
[INFO] Lens .............................................. SUCCESS [1.808s]
[INFO] Lens API .......................................... SUCCESS [5.496s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
[INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
[INFO] Lens DB storage ................................... SUCCESS [10.480s]
[INFO] Lens Query Library ................................ SUCCESS [4.543s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
[INFO] Lens Server ....................................... SUCCESS [4:32.022s]
[INFO] Lens client ....................................... SUCCESS [20.184s]
[INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
[INFO] Lens Examples ..................................... SUCCESS [0.946s]
[INFO] Lens Distribution ................................. SUCCESS [10.040s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
[INFO] Lens Regression ................................... SUCCESS [0.691s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:39.305s
[INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
[INFO] Final Memory: 108M/972M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On April 6, 2015, 3:52 p.m., Himanshu Gahlaut wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java, line 156
> > <https://reviews.apache.org/r/32610/diff/4/?file=916326#file916326line156>
> >
> > Nested classes are created when enclosing instances needs them and they don't have an existence without enclosing instance.
> >
> > Current Data Model:
> >
> > public class TimePartition {
> >
> >
> > public static class TimePartitionRange {
> >
> > private TimePartition begin;
> > private TimePartition end;
> > }
> > }
> >
> > TimePartition class does not need TimePartitionRange. TimePartitionRange needs TimePartition. This data model doesn't seem to be correct. Here is an analogy:
> >
> > public class Room {
> >
> > public static class House {
> > private List<Room> rooms;
> > }
> > }
> >
> > House is composed of rooms but making it a static nested class of Room doesn't appear to be correct.
> >
> > As House doesn't look nice as nested class of Room, similarly TimePartitionRange doesn't look nice as a nested class of TimePartition. Makes data model harder to understand.
Addressed. Moved to separate class.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78906
-----------------------------------------------------------
On April 6, 2015, 2:45 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 6, 2015, 2:45 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
> Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
> [INFO] Lens .............................................. SUCCESS [1.808s]
> [INFO] Lens API .......................................... SUCCESS [5.496s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
> [INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
> [INFO] Lens DB storage ................................... SUCCESS [10.480s]
> [INFO] Lens Query Library ................................ SUCCESS [4.543s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
> [INFO] Lens Server ....................................... SUCCESS [4:32.022s]
> [INFO] Lens client ....................................... SUCCESS [20.184s]
> [INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
> [INFO] Lens Examples ..................................... SUCCESS [0.946s]
> [INFO] Lens Distribution ................................. SUCCESS [10.040s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
> [INFO] Lens Regression ................................... SUCCESS [0.691s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:39.305s
> [INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
> [INFO] Final Memory: 108M/972M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Himanshu Gahlaut <ga...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78906
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java
<https://reviews.apache.org/r/32610/#comment128007>
Nested classes are created when enclosing instances needs them and they don't have an existence without enclosing instance.
Current Data Model:
public class TimePartition {
public static class TimePartitionRange {
private TimePartition begin;
private TimePartition end;
}
}
TimePartition class does not need TimePartitionRange. TimePartitionRange needs TimePartition. This data model doesn't seem to be correct. Here is an analogy:
public class Room {
public static class House {
private List<Room> rooms;
}
}
House is composed of rooms but making it a static nested class of Room doesn't appear to be correct.
As House doesn't look nice as nested class of Room, similarly TimePartitionRange doesn't look nice as a nested class of TimePartition. Makes data model harder to understand.
- Himanshu Gahlaut
On April 6, 2015, 9:15 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 6, 2015, 9:15 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
> Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
> [INFO] Lens .............................................. SUCCESS [1.808s]
> [INFO] Lens API .......................................... SUCCESS [5.496s]
> [INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
> [INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
> [INFO] Lens DB storage ................................... SUCCESS [10.480s]
> [INFO] Lens Query Library ................................ SUCCESS [4.543s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
> [INFO] Lens Server ....................................... SUCCESS [4:32.022s]
> [INFO] Lens client ....................................... SUCCESS [20.184s]
> [INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
> [INFO] Lens Examples ..................................... SUCCESS [0.946s]
> [INFO] Lens Distribution ................................. SUCCESS [10.040s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
> [INFO] Lens Regression ................................... SUCCESS [0.691s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:39.305s
> [INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
> [INFO] Final Memory: 108M/972M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated April 6, 2015, 2:45 p.m.)
Review request for lens.
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
Diff: https://reviews.apache.org/r/32610/diff/
Testing (updated)
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.282s]
[INFO] Lens .............................................. SUCCESS [1.808s]
[INFO] Lens API .......................................... SUCCESS [5.496s]
[INFO] Lens API for server and extensions ................ SUCCESS [7.267s]
[INFO] Lens Cube ......................................... SUCCESS [2:23.812s]
[INFO] Lens DB storage ................................... SUCCESS [10.480s]
[INFO] Lens Query Library ................................ SUCCESS [4.543s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:32.578s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [18.730s]
[INFO] Lens Server ....................................... SUCCESS [4:32.022s]
[INFO] Lens client ....................................... SUCCESS [20.184s]
[INFO] Lens CLI .......................................... SUCCESS [2:49.930s]
[INFO] Lens Examples ..................................... SUCCESS [0.946s]
[INFO] Lens Distribution ................................. SUCCESS [10.040s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:52.042s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [5.537s]
[INFO] Lens Regression ................................... SUCCESS [0.691s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:39.305s
[INFO] Finished at: Mon Apr 06 09:09:34 UTC 2015
[INFO] Final Memory: 108M/972M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated April 6, 2015, 2:23 p.m.)
Review request for lens.
Repository: lens
Description (updated)
-------
handle storing of large collection objects in timeline data structures
Addressed all review comments. Again hitting some problem with git. Equivalent diff at: https://github.com/apache/incubator-lens/pull/7
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
Diff: https://reviews.apache.org/r/32610/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
[INFO] Lens .............................................. SUCCESS [1.896s]
[INFO] Lens API .......................................... SUCCESS [6.175s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
[INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
[INFO] Lens DB storage ................................... SUCCESS [10.638s]
[INFO] Lens Query Library ................................ SUCCESS [5.112s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
[INFO] Lens Server ....................................... SUCCESS [6:09.928s]
[INFO] Lens client ....................................... SUCCESS [20.684s]
[INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
[INFO] Lens Examples ..................................... SUCCESS [0.941s]
[INFO] Lens Distribution ................................. SUCCESS [9.567s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
[INFO] Lens Regression ................................... SUCCESS [0.525s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:55.802s
[INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
[INFO] Final Memory: 96M/1058M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated April 6, 2015, 2:22 p.m.)
Review request for lens.
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a1
lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e44
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e3
lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5
Diff: https://reviews.apache.org/r/32610/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
[INFO] Lens .............................................. SUCCESS [1.896s]
[INFO] Lens API .......................................... SUCCESS [6.175s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
[INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
[INFO] Lens DB storage ................................... SUCCESS [10.638s]
[INFO] Lens Query Library ................................ SUCCESS [5.112s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
[INFO] Lens Server ....................................... SUCCESS [6:09.928s]
[INFO] Lens client ....................................... SUCCESS [20.684s]
[INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
[INFO] Lens Examples ..................................... SUCCESS [0.941s]
[INFO] Lens Distribution ................................. SUCCESS [9.567s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
[INFO] Lens Regression ................................... SUCCESS [0.525s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:55.802s
[INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
[INFO] Final Memory: 96M/1058M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78657
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
<https://reviews.apache.org/r/32610/#comment127577>
They are already getting printed. Inside both loading functions, there are logs.
In load by all partitions it has:
log.info("after commit batch additions, timeline is: " + this);
in load by properties, it has:
log.info("initialized to " + toProperties());
- Rajat Khandelwal
On April 1, 2015, 12:26 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 12:26 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated April 1, 2015, 12:26 p.m.)
Review request for lens.
Changes
-------
merged with apache master
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java b2ae9b5e8076f1eb22e700bb9e3680346733861a
Diff: https://reviews.apache.org/r/32610/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
[INFO] Lens .............................................. SUCCESS [1.896s]
[INFO] Lens API .......................................... SUCCESS [6.175s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
[INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
[INFO] Lens DB storage ................................... SUCCESS [10.638s]
[INFO] Lens Query Library ................................ SUCCESS [5.112s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
[INFO] Lens Server ....................................... SUCCESS [6:09.928s]
[INFO] Lens client ....................................... SUCCESS [20.684s]
[INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
[INFO] Lens Examples ..................................... SUCCESS [0.941s]
[INFO] Lens Distribution ................................. SUCCESS [9.567s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
[INFO] Lens Regression ................................... SUCCESS [0.525s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:55.802s
[INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
[INFO] Final Memory: 96M/1058M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78363
-----------------------------------------------------------
lens-client/src/test/java/org/apache/lens/client/TestLensClient.java
<https://reviews.apache.org/r/32610/#comment126926>
Seems there are merge issues in changes put up. Can you clean them up?
- Amareshwari Sriramadasu
On March 31, 2015, 6:45 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated March 31, 2015, 6:45 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-client/src/test/java/org/apache/lens/client/TestLensClient.java 81a536ed8ac3f7ea550794cf188db6ea9d599c7d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
> lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 7d082127e83e90062bca704f8caa42c915f55a27
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 218dc53145398f1fc584efa6160c49fb6b8f2d0c
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b16c34665c9cceeb40cbd0d633a5f8d2802de381
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 121b56bf2b513dbaf27562061a404b205c55a74b
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 550a4680d9c2784660dbfe053da2f454f1b9bef7
> lens-driver-jdbc/testdata/DatabaseJarSerde.java 03caff31916072b0e6944d1bd13161dca5cef878
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 6799e0c7a0f487280a581b9ba034f70a1c814418
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java ea2da14f51dc0eb4f0e06e73ae7752f1fe5cb769
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 8c970821015b04e4adaff7d2949520ea5a0c4c82
> lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 20856a09610d76ca8072851e00125fc50aa4343c
> lens-server/src/test/java/org/apache/lens/server/LensTestUtil.java e44816372e889bdc7ce2ac34a43048e8af85181e
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 6306b51cdf2607c9762aec522d663019b202760f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java e3e3d4baf43f67f96687b47b3273046e24720254
> lens-server/testdata/DatabaseJarSerde.java 03caff31916072b0e6944d1bd13161dca5cef878
> lens-server/testdata/serde.jar ec86e49a0be7cb9872756a4313ae81bd3cb5e543
> lens-server/testdata/test.jar 1644d8cada37749f6a8c3a2a6c26b752ea7bac0f
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated March 31, 2015, 12:15 p.m.)
Review request for lens.
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Diffs (updated)
-----
lens-client/src/test/java/org/apache/lens/client/TestLensClient.java 81a536ed8ac3f7ea550794cf188db6ea9d599c7d
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 1835d2f5f604548126c60592869187c29f0f4392
lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 0be8e5f2031dad8f175bbd00d3cbfc73695d31db
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 1094e4413c1355655de00b04f8a64e507e2270ee
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java 4c98d84e2c55bedd3226c887b6052a5b5cf77145
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 7d082127e83e90062bca704f8caa42c915f55a27
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 218dc53145398f1fc584efa6160c49fb6b8f2d0c
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b16c34665c9cceeb40cbd0d633a5f8d2802de381
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 121b56bf2b513dbaf27562061a404b205c55a74b
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 550a4680d9c2784660dbfe053da2f454f1b9bef7
lens-driver-jdbc/testdata/DatabaseJarSerde.java 03caff31916072b0e6944d1bd13161dca5cef878
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 6799e0c7a0f487280a581b9ba034f70a1c814418
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java ea2da14f51dc0eb4f0e06e73ae7752f1fe5cb769
lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 8c970821015b04e4adaff7d2949520ea5a0c4c82
lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 20856a09610d76ca8072851e00125fc50aa4343c
lens-server/src/test/java/org/apache/lens/server/LensTestUtil.java e44816372e889bdc7ce2ac34a43048e8af85181e
lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 6306b51cdf2607c9762aec522d663019b202760f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java e3e3d4baf43f67f96687b47b3273046e24720254
lens-server/testdata/DatabaseJarSerde.java 03caff31916072b0e6944d1bd13161dca5cef878
lens-server/testdata/serde.jar ec86e49a0be7cb9872756a4313ae81bd3cb5e543
lens-server/testdata/test.jar 1644d8cada37749f6a8c3a2a6c26b752ea7bac0f
Diff: https://reviews.apache.org/r/32610/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
[INFO] Lens .............................................. SUCCESS [1.896s]
[INFO] Lens API .......................................... SUCCESS [6.175s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
[INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
[INFO] Lens DB storage ................................... SUCCESS [10.638s]
[INFO] Lens Query Library ................................ SUCCESS [5.112s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
[INFO] Lens Server ....................................... SUCCESS [6:09.928s]
[INFO] Lens client ....................................... SUCCESS [20.684s]
[INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
[INFO] Lens Examples ..................................... SUCCESS [0.941s]
[INFO] Lens Distribution ................................. SUCCESS [9.567s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
[INFO] Lens Regression ................................... SUCCESS [0.525s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:55.802s
[INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
[INFO] Final Memory: 96M/1058M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/
-----------------------------------------------------------
(Updated March 30, 2015, 5:49 p.m.)
Review request for lens.
Repository: lens
Description
-------
handle storing of large collection objects in timeline data structures
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
Diff: https://reviews.apache.org/r/32610/diff/
Testing (updated)
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
[INFO] Lens .............................................. SUCCESS [1.896s]
[INFO] Lens API .......................................... SUCCESS [6.175s]
[INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
[INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
[INFO] Lens DB storage ................................... SUCCESS [10.638s]
[INFO] Lens Query Library ................................ SUCCESS [5.112s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
[INFO] Lens Server ....................................... SUCCESS [6:09.928s]
[INFO] Lens client ....................................... SUCCESS [20.684s]
[INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
[INFO] Lens Examples ..................................... SUCCESS [0.941s]
[INFO] Lens Distribution ................................. SUCCESS [9.567s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
[INFO] Lens Regression ................................... SUCCESS [0.525s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:55.802s
[INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
[INFO] Final Memory: 96M/1058M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On March 30, 2015, 4:05 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java, line 120
> > <https://reviews.apache.org/r/32610/diff/1/?file=908991#file908991line120>
> >
> > We need to read from older properties as well. Or mark this as incompatible to reset timeline.present property to override timeline again.
In that case, this will throw NPE while trying to parse `*.size` property from table. I think a slight change in design in which you piggyback on loading timeline from all partitions if loading from properties fails.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78194
-----------------------------------------------------------
On March 30, 2015, 5:49 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated March 30, 2015, 5:49 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On March 30, 2015, 10:35 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java, line 120
> > <https://reviews.apache.org/r/32610/diff/1/?file=908991#file908991line120>
> >
> > We need to read from older properties as well. Or mark this as incompatible to reset timeline.present property to override timeline again.
>
> Rajat Khandelwal wrote:
> In that case, this will throw NPE while trying to parse `*.size` property from table. I think a slight change in design in which you piggyback on loading timeline from all partitions if loading from properties fails.
I would say that will be too much overhead to pull all partitions. We can check if named strings are present for holes, if not, get the value from direct property.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78194
-----------------------------------------------------------
On March 30, 2015, 12:19 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated March 30, 2015, 12:19 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On March 30, 2015, 4:05 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java, line 120
> > <https://reviews.apache.org/r/32610/diff/1/?file=908991#file908991line120>
> >
> > We need to read from older properties as well. Or mark this as incompatible to reset timeline.present property to override timeline again.
>
> Rajat Khandelwal wrote:
> In that case, this will throw NPE while trying to parse `*.size` property from table. I think a slight change in design in which you piggyback on loading timeline from all partitions if loading from properties fails.
>
> Amareshwari Sriramadasu wrote:
> I would say that will be too much overhead to pull all partitions. We can check if named strings are present for holes, if not, get the value from direct property.
Handled in code of get named string. It would have thrown NPE anyways. Avoiding NPE this way.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78194
-----------------------------------------------------------
On March 30, 2015, 5:49 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated March 30, 2015, 5:49 p.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.151s]
> [INFO] Lens .............................................. SUCCESS [1.896s]
> [INFO] Lens API .......................................... SUCCESS [6.175s]
> [INFO] Lens API for server and extensions ................ SUCCESS [6.715s]
> [INFO] Lens Cube ......................................... SUCCESS [2:12.362s]
> [INFO] Lens DB storage ................................... SUCCESS [10.638s]
> [INFO] Lens Query Library ................................ SUCCESS [5.112s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:31.556s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [17.345s]
> [INFO] Lens Server ....................................... SUCCESS [6:09.928s]
> [INFO] Lens client ....................................... SUCCESS [20.684s]
> [INFO] Lens CLI .......................................... SUCCESS [2:34.762s]
> [INFO] Lens Examples ..................................... SUCCESS [0.941s]
> [INFO] Lens Distribution ................................. SUCCESS [9.567s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:01.797s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.750s]
> [INFO] Lens Regression ................................... SUCCESS [0.525s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 15:55.802s
> [INFO] Finished at: Mon Mar 30 11:44:12 UTC 2015
> [INFO] Final Memory: 96M/1058M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 32610: LENS-462: Handle the case when a table
property becomes large for a timeline
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32610/#review78194
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java
<https://reviews.apache.org/r/32610/#comment126705>
We need to read from older properties as well. Or mark this as incompatible to reset timeline.present property to override timeline again.
- Amareshwari Sriramadasu
On March 30, 2015, 10:02 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32610/
> -----------------------------------------------------------
>
> (Updated March 30, 2015, 10:02 a.m.)
>
>
> Review request for lens.
>
>
> Repository: lens
>
>
> Description
> -------
>
> handle storing of large collection objects in timeline data structures
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java b948467e61618695cba5b6fcfdc084d8d26c0c4d
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java 79e8a62629b6466f42152678c4267e40f003a5a3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java fb2d0a86d773825cd2a8a635c451e9945b122ce8
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java d6ee0a17aae8d0952fb685997120dbfc778d2530
> lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java 50b75e38f9fcd890e96787399d10434cac6f6045
>
> Diff: https://reviews.apache.org/r/32610/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>