You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Sushil Mohanty <su...@gmail.com> on 2016/05/02 12:24:08 UTC
Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
Review request for lens.
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
[INFO] Lens .............................................. SUCCESS [3.469s]
[INFO] Lens API .......................................... SUCCESS [20.412s]
[INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
[INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
[INFO] Lens DB storage ................................... SUCCESS [16.546s]
[INFO] Lens Query Library ................................ SUCCESS [14.961s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
[INFO] Lens Server ....................................... SUCCESS [16:37.087s]
[INFO] Lens client ....................................... SUCCESS [1:39.534s]
[INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
[INFO] Lens Examples ..................................... SUCCESS [8.573s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
[INFO] Lens Distribution ................................. SUCCESS [9.838s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
[INFO] Lens Regression ................................... SUCCESS [9.944s]
[INFO] Lens UI ........................................... SUCCESS [43.727s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34:31.633s
[INFO] Finished at: Mon May 02 10:19:12 UTC 2016
[INFO] Final Memory: 183M/2727M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132014
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 1359)
<https://reviews.apache.org/r/46890/#comment196055>
Checking is dimFilter through getString() will costly. We need to iterate through ASTNode and check alias.
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 1413)
<https://reviews.apache.org/r/46890/#comment196056>
If we are already doing this we can do the same for finding whether a filter dimfilter or not.
- Amareshwari Sriramadasu
On May 3, 2016, 5:10 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 3, 2016, 5:10 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
> [INFO] Lens .............................................. SUCCESS [3.469s]
> [INFO] Lens API .......................................... SUCCESS [20.412s]
> [INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
> [INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
> [INFO] Lens DB storage ................................... SUCCESS [16.546s]
> [INFO] Lens Query Library ................................ SUCCESS [14.961s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
> [INFO] Lens Server ....................................... SUCCESS [16:37.087s]
> [INFO] Lens client ....................................... SUCCESS [1:39.534s]
> [INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
> [INFO] Lens Examples ..................................... SUCCESS [8.573s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
> [INFO] Lens Distribution ................................. SUCCESS [9.838s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
> [INFO] Lens Regression ................................... SUCCESS [9.944s]
> [INFO] Lens UI ........................................... SUCCESS [43.727s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 34:31.633s
> [INFO] Finished at: Mon May 02 10:19:12 UTC 2016
> [INFO] Final Memory: 183M/2727M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
> On May 6, 2016, 1:41 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java, lines 177-178
> > <https://reviews.apache.org/r/46890/diff/2/?file=1369621#file1369621line177>
> >
> > We can keep enums for these.
This is not required anymore, did some refactoring after review.
- Sushil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132023
-----------------------------------------------------------
On May 3, 2016, 5:10 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 3, 2016, 5:10 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
> [INFO] Lens .............................................. SUCCESS [3.469s]
> [INFO] Lens API .......................................... SUCCESS [20.412s]
> [INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
> [INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
> [INFO] Lens DB storage ................................... SUCCESS [16.546s]
> [INFO] Lens Query Library ................................ SUCCESS [14.961s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
> [INFO] Lens Server ....................................... SUCCESS [16:37.087s]
> [INFO] Lens client ....................................... SUCCESS [1:39.534s]
> [INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
> [INFO] Lens Examples ..................................... SUCCESS [8.573s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
> [INFO] Lens Distribution ................................. SUCCESS [9.838s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
> [INFO] Lens Regression ................................... SUCCESS [9.944s]
> [INFO] Lens UI ........................................... SUCCESS [43.727s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 34:31.633s
> [INFO] Finished at: Mon May 02 10:19:12 UTC 2016
> [INFO] Final Memory: 183M/2727M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132023
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (lines 177 - 178)
<https://reviews.apache.org/r/46890/#comment196065>
We can keep enums for these.
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java (lines 29 - 33)
<https://reviews.apache.org/r/46890/#comment196066>
Let's change other method names too, for consistency.
- Rajat Khandelwal
On May 3, 2016, 10:40 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 3, 2016, 10:40 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
> [INFO] Lens .............................................. SUCCESS [3.469s]
> [INFO] Lens API .......................................... SUCCESS [20.412s]
> [INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
> [INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
> [INFO] Lens DB storage ................................... SUCCESS [16.546s]
> [INFO] Lens Query Library ................................ SUCCESS [14.961s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
> [INFO] Lens Server ....................................... SUCCESS [16:37.087s]
> [INFO] Lens client ....................................... SUCCESS [1:39.534s]
> [INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
> [INFO] Lens Examples ..................................... SUCCESS [8.573s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
> [INFO] Lens Distribution ................................. SUCCESS [9.838s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
> [INFO] Lens Regression ................................... SUCCESS [9.944s]
> [INFO] Lens UI ........................................... SUCCESS [43.727s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 34:31.633s
> [INFO] Finished at: Mon May 02 10:19:12 UTC 2016
> [INFO] Final Memory: 183M/2727M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132645
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 997)
<https://reviews.apache.org/r/46890/#comment196917>
Should it be alias instead of cfact.getBaseTable().getName()?
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 1319)
<https://reviews.apache.org/r/46890/#comment196916>
Shouldn't you pass cubeAlias?
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 1320)
<https://reviews.apache.org/r/46890/#comment196918>
Will allFilters be empty ever? I dont think so. Shall we remove the check?
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (lines 1345 - 1351)
<https://reviews.apache.org/r/46890/#comment196920>
Code looks same as in the above block. Should be moved to a method?
- Amareshwari Sriramadasu
On May 11, 2016, 9:58 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 11, 2016, 9:58 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
> [INFO] Lens .............................................. SUCCESS [3.003s]
> [INFO] Lens API .......................................... SUCCESS [21.845s]
> [INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
> [INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
> [INFO] Lens DB storage ................................... SUCCESS [14.147s]
> [INFO] Lens Query Library ................................ SUCCESS [11.250s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
> [INFO] Lens Server ....................................... SUCCESS [14:10.700s]
> [INFO] Lens client ....................................... SUCCESS [1:25.019s]
> [INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
> [INFO] Lens Examples ..................................... SUCCESS [8.918s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
> [INFO] Lens Distribution ................................. SUCCESS [8.732s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
> [INFO] Lens Regression ................................... SUCCESS [10.038s]
> [INFO] Lens UI ........................................... SUCCESS [44.771s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 31:23.276s
> [INFO] Finished at: Mon May 09 15:04:12 UTC 2016
> [INFO] Final Memory: 163M/2144M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review133062
-----------------------------------------------------------
Ship it!
Ship It!
- Amareshwari Sriramadasu
On May 13, 2016, 9:19 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 13, 2016, 9:19 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java cd7a02c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
> [INFO] Lens .............................................. SUCCESS [3.003s]
> [INFO] Lens API .......................................... SUCCESS [21.845s]
> [INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
> [INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
> [INFO] Lens DB storage ................................... SUCCESS [14.147s]
> [INFO] Lens Query Library ................................ SUCCESS [11.250s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
> [INFO] Lens Server ....................................... SUCCESS [14:10.700s]
> [INFO] Lens client ....................................... SUCCESS [1:25.019s]
> [INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
> [INFO] Lens Examples ..................................... SUCCESS [8.918s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
> [INFO] Lens Distribution ................................. SUCCESS [8.732s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
> [INFO] Lens Regression ................................... SUCCESS [10.038s]
> [INFO] Lens UI ........................................... SUCCESS [44.771s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 31:23.276s
> [INFO] Finished at: Mon May 09 15:04:12 UTC 2016
> [INFO] Final Memory: 163M/2144M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
(Updated May 13, 2016, 9:19 a.m.)
Review request for lens.
Changes
-------
- Addressed review comment.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Tests run: 396, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,014.919 sec - in TestSuite
Results :
Tests run: 396, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java cd7a02c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
[INFO] Lens .............................................. SUCCESS [3.003s]
[INFO] Lens API .......................................... SUCCESS [21.845s]
[INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
[INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
[INFO] Lens DB storage ................................... SUCCESS [14.147s]
[INFO] Lens Query Library ................................ SUCCESS [11.250s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
[INFO] Lens Server ....................................... SUCCESS [14:10.700s]
[INFO] Lens client ....................................... SUCCESS [1:25.019s]
[INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
[INFO] Lens Examples ..................................... SUCCESS [8.918s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
[INFO] Lens Distribution ................................. SUCCESS [8.732s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
[INFO] Lens Regression ................................... SUCCESS [10.038s]
[INFO] Lens UI ........................................... SUCCESS [44.771s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:23.276s
[INFO] Finished at: Mon May 09 15:04:12 UTC 2016
[INFO] Final Memory: 163M/2144M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
(Updated May 13, 2016, 8:18 a.m.)
Review request for lens.
Changes
-------
- Addressed review comments
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Tests run: 384, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,101.837 sec - in TestSuite
Results :
Tests run: 384, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java cd7a02c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
[INFO] Lens .............................................. SUCCESS [3.003s]
[INFO] Lens API .......................................... SUCCESS [21.845s]
[INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
[INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
[INFO] Lens DB storage ................................... SUCCESS [14.147s]
[INFO] Lens Query Library ................................ SUCCESS [11.250s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
[INFO] Lens Server ....................................... SUCCESS [14:10.700s]
[INFO] Lens client ....................................... SUCCESS [1:25.019s]
[INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
[INFO] Lens Examples ..................................... SUCCESS [8.918s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
[INFO] Lens Distribution ................................. SUCCESS [8.732s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
[INFO] Lens Regression ................................... SUCCESS [10.038s]
[INFO] Lens UI ........................................... SUCCESS [44.771s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:23.276s
[INFO] Finished at: Mon May 09 15:04:12 UTC 2016
[INFO] Final Memory: 163M/2144M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On May 12, 2016, 4:51 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java, line 766
> > <https://reviews.apache.org/r/46890/diff/5/?file=1379641#file1379641line766>
> >
> > subquery should be 'select id from countrydim cubecountry where ..."
> >
> > Seems actual table is missed and only alias is put in from clause.
> >
> > Also, is dimtable has snapshot dumps, we need to add latest time filter additionally.
Reopening this as the from clause should actually contain the storage table
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132814
-----------------------------------------------------------
On May 13, 2016, 8:18 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 13, 2016, 8:18 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java cd7a02c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
> [INFO] Lens .............................................. SUCCESS [3.003s]
> [INFO] Lens API .......................................... SUCCESS [21.845s]
> [INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
> [INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
> [INFO] Lens DB storage ................................... SUCCESS [14.147s]
> [INFO] Lens Query Library ................................ SUCCESS [11.250s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
> [INFO] Lens Server ....................................... SUCCESS [14:10.700s]
> [INFO] Lens client ....................................... SUCCESS [1:25.019s]
> [INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
> [INFO] Lens Examples ..................................... SUCCESS [8.918s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
> [INFO] Lens Distribution ................................. SUCCESS [8.732s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
> [INFO] Lens Regression ................................... SUCCESS [10.038s]
> [INFO] Lens UI ........................................... SUCCESS [44.771s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 31:23.276s
> [INFO] Finished at: Mon May 09 15:04:12 UTC 2016
> [INFO] Final Memory: 163M/2144M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132814
-----------------------------------------------------------
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java (line 766)
<https://reviews.apache.org/r/46890/#comment197067>
subquery should be 'select id from countrydim cubecountry where ..."
Seems actual table is missed and only alias is put in from clause.
Also, is dimtable has snapshot dumps, we need to add latest time filter additionally.
- Amareshwari Sriramadasu
On May 11, 2016, 1:57 p.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 11, 2016, 1:57 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
> [INFO] Lens .............................................. SUCCESS [3.003s]
> [INFO] Lens API .......................................... SUCCESS [21.845s]
> [INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
> [INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
> [INFO] Lens DB storage ................................... SUCCESS [14.147s]
> [INFO] Lens Query Library ................................ SUCCESS [11.250s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
> [INFO] Lens Server ....................................... SUCCESS [14:10.700s]
> [INFO] Lens client ....................................... SUCCESS [1:25.019s]
> [INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
> [INFO] Lens Examples ..................................... SUCCESS [8.918s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
> [INFO] Lens Distribution ................................. SUCCESS [8.732s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
> [INFO] Lens Regression ................................... SUCCESS [10.038s]
> [INFO] Lens UI ........................................... SUCCESS [44.771s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 31:23.276s
> [INFO] Finished at: Mon May 09 15:04:12 UTC 2016
> [INFO] Final Memory: 163M/2144M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
(Updated May 11, 2016, 1:57 p.m.)
Review request for lens.
Changes
-------
- Addressed all review comments.
- Added test to validate cube alias is written properly in filter query.
T E S T S
Running TestSuite
Tests run: 384, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,376.862 sec - in TestSuite
Results :
Tests run: 384, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
[INFO] Lens .............................................. SUCCESS [3.003s]
[INFO] Lens API .......................................... SUCCESS [21.845s]
[INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
[INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
[INFO] Lens DB storage ................................... SUCCESS [14.147s]
[INFO] Lens Query Library ................................ SUCCESS [11.250s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
[INFO] Lens Server ....................................... SUCCESS [14:10.700s]
[INFO] Lens client ....................................... SUCCESS [1:25.019s]
[INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
[INFO] Lens Examples ..................................... SUCCESS [8.918s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
[INFO] Lens Distribution ................................. SUCCESS [8.732s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
[INFO] Lens Regression ................................... SUCCESS [10.038s]
[INFO] Lens UI ........................................... SUCCESS [44.771s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:23.276s
[INFO] Finished at: Mon May 09 15:04:12 UTC 2016
[INFO] Final Memory: 163M/2144M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
(Updated May 11, 2016, 9:58 a.m.)
Review request for lens.
Changes
-------
- Renamed method names in QueryAST to have consistent names.
- Fixed test cases wrt. latest master.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Tests run: 384, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,274.862 sec - in TestSuite
Results :
Tests run: 384, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 9674f73
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
[INFO] Lens .............................................. SUCCESS [3.003s]
[INFO] Lens API .......................................... SUCCESS [21.845s]
[INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
[INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
[INFO] Lens DB storage ................................... SUCCESS [14.147s]
[INFO] Lens Query Library ................................ SUCCESS [11.250s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
[INFO] Lens Server ....................................... SUCCESS [14:10.700s]
[INFO] Lens client ....................................... SUCCESS [1:25.019s]
[INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
[INFO] Lens Examples ..................................... SUCCESS [8.918s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
[INFO] Lens Distribution ................................. SUCCESS [8.732s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
[INFO] Lens Regression ................................... SUCCESS [10.038s]
[INFO] Lens UI ........................................... SUCCESS [44.771s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:23.276s
[INFO] Finished at: Mon May 09 15:04:12 UTC 2016
[INFO] Final Memory: 163M/2144M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
(Updated May 9, 2016, 5:01 p.m.)
Review request for lens.
Changes
-------
- Addressed review comments.
- Enhanced test cases.
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 6f016f2
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java f4ef5ca
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing (updated)
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.453s]
[INFO] Lens .............................................. SUCCESS [3.003s]
[INFO] Lens API .......................................... SUCCESS [21.845s]
[INFO] Lens API for server and extensions ................ SUCCESS [15.399s]
[INFO] Lens Cube ......................................... SUCCESS [6:54.821s]
[INFO] Lens DB storage ................................... SUCCESS [14.147s]
[INFO] Lens Query Library ................................ SUCCESS [11.250s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.637s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.131s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [12.729s]
[INFO] Lens Server ....................................... SUCCESS [14:10.700s]
[INFO] Lens client ....................................... SUCCESS [1:25.019s]
[INFO] Lens CLI .......................................... SUCCESS [2:13.299s]
[INFO] Lens Examples ..................................... SUCCESS [8.918s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.709s]
[INFO] Lens Distribution ................................. SUCCESS [8.732s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:10.374s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.440s]
[INFO] Lens Regression ................................... SUCCESS [10.038s]
[INFO] Lens UI ........................................... SUCCESS [44.771s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:23.276s
[INFO] Finished at: Mon May 09 15:04:12 UTC 2016
[INFO] Final Memory: 163M/2144M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review132012
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (lines 1007 - 1016)
<https://reviews.apache.org/r/46890/#comment196052>
Code in both if and else look the same, except where clause construction. We can pass the if condition checks to where clause construction and construct accordingly and avoid if and else.
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (lines 1026 - 1033)
<https://reviews.apache.org/r/46890/#comment196053>
Instead of filterNodeToTypeMap and reconstruction of filters this way - use ArrayList of filter strings and simply populate string for each filter node.
- Amareshwari Sriramadasu
On May 3, 2016, 5:10 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 3, 2016, 5:10 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
> [INFO] Lens .............................................. SUCCESS [3.469s]
> [INFO] Lens API .......................................... SUCCESS [20.412s]
> [INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
> [INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
> [INFO] Lens DB storage ................................... SUCCESS [16.546s]
> [INFO] Lens Query Library ................................ SUCCESS [14.961s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
> [INFO] Lens Server ....................................... SUCCESS [16:37.087s]
> [INFO] Lens client ....................................... SUCCESS [1:39.534s]
> [INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
> [INFO] Lens Examples ..................................... SUCCESS [8.573s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
> [INFO] Lens Distribution ................................. SUCCESS [9.838s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
> [INFO] Lens Regression ................................... SUCCESS [9.944s]
> [INFO] Lens UI ........................................... SUCCESS [43.727s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 34:31.633s
> [INFO] Finished at: Mon May 02 10:19:12 UTC 2016
> [INFO] Final Memory: 183M/2727M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Sushil Mohanty <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/
-----------------------------------------------------------
(Updated May 3, 2016, 5:10 a.m.)
Review request for lens.
Changes
-------
Addressed review comments.
Bugs: LENS-1028
https://issues.apache.org/jira/browse/LENS-1028
Repository: lens
Description
-------
Changes include dim filters to rewrite as fact filers for star joins.
e.g.
where dim.name in ('x', 'y')
becomes
where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
src/site/apt/user/olap-query-conf.apt 6372fb1
Diff: https://reviews.apache.org/r/46890/diff/
Testing
-------
Added unit tests. mvn clean install works fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
[INFO] Lens .............................................. SUCCESS [3.469s]
[INFO] Lens API .......................................... SUCCESS [20.412s]
[INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
[INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
[INFO] Lens DB storage ................................... SUCCESS [16.546s]
[INFO] Lens Query Library ................................ SUCCESS [14.961s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
[INFO] Lens Server ....................................... SUCCESS [16:37.087s]
[INFO] Lens client ....................................... SUCCESS [1:39.534s]
[INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
[INFO] Lens Examples ..................................... SUCCESS [8.573s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
[INFO] Lens Distribution ................................. SUCCESS [9.838s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
[INFO] Lens Regression ................................... SUCCESS [9.944s]
[INFO] Lens UI ........................................... SUCCESS [43.727s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34:31.633s
[INFO] Finished at: Mon May 02 10:19:12 UTC 2016
[INFO] Final Memory: 183M/2727M
[INFO] ------------------------------------------------------------------------
Thanks,
Sushil Mohanty
Re: Review Request 46890: LENS-1028 : Convert dimension filter to fact
filters for perfomace improvement on outer join queries
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46890/#review131285
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 994)
<https://reviews.apache.org/r/46890/#comment195206>
if shouldReplaceDimFilterWithFactFilter is true and join resolution disabled, seems we are not setting where string.
lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java (line 66)
<https://reviews.apache.org/r/46890/#comment195207>
Is passing AST still required? We should not be doing as AST and where string are not in sync any more.
- Amareshwari Sriramadasu
On May 2, 2016, 10:24 a.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46890/
> -----------------------------------------------------------
>
> (Updated May 2, 2016, 10:24 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-1028
> https://issues.apache.org/jira/browse/LENS-1028
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include dim filters to rewrite as fact filers for star joins.
> e.g.
> where dim.name in ('x', 'y')
> becomes
> where fact.dimid in (select dim.id from dim where dim.name in ('x','y'))
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 3f724b6
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java a57292c
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java c9b1475
> lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultQueryAST.java 0697e78
> lens-cube/src/main/java/org/apache/lens/cube/parse/DimHQLContext.java 7c14be7
> lens-cube/src/main/java/org/apache/lens/cube/parse/MultiFactHQLContext.java 4d6ce9e
> lens-cube/src/main/java/org/apache/lens/cube/parse/QueryAST.java f064dcb
> lens-cube/src/main/java/org/apache/lens/cube/parse/join/AutoJoinContext.java 8b24f70
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java a223449
> src/site/apt/user/olap-query-conf.apt 6372fb1
>
> Diff: https://reviews.apache.org/r/46890/diff/
>
>
> Testing
> -------
>
> Added unit tests. mvn clean install works fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.657s]
> [INFO] Lens .............................................. SUCCESS [3.469s]
> [INFO] Lens API .......................................... SUCCESS [20.412s]
> [INFO] Lens API for server and extensions ................ SUCCESS [16.309s]
> [INFO] Lens Cube ......................................... SUCCESS [7:01.036s]
> [INFO] Lens DB storage ................................... SUCCESS [16.546s]
> [INFO] Lens Query Library ................................ SUCCESS [14.961s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:24.649s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [29.037s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [13.653s]
> [INFO] Lens Server ....................................... SUCCESS [16:37.087s]
> [INFO] Lens client ....................................... SUCCESS [1:39.534s]
> [INFO] Lens CLI .......................................... SUCCESS [2:28.860s]
> [INFO] Lens Examples ..................................... SUCCESS [8.573s]
> [INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.867s]
> [INFO] Lens Distribution ................................. SUCCESS [9.838s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:09.035s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.741s]
> [INFO] Lens Regression ................................... SUCCESS [9.944s]
> [INFO] Lens UI ........................................... SUCCESS [43.727s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 34:31.633s
> [INFO] Finished at: Mon May 02 10:19:12 UTC 2016
> [INFO] Final Memory: 183M/2727M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Sushil Mohanty
>
>