You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Sushil Mohanty <su...@gmail.com> on 2015/10/19 16:07:59 UTC
Review Request 39439: LENS-835 : Optimizing fact subquery by removing
non aggregated expression columns from group by clause
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/
-----------------------------------------------------------
Review request for lens.
Bugs: LENS-835
https://issues.apache.org/jira/browse/LENS-835
Repository: lens
Description
-------
Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
Diffs
-----
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302
Diff: https://reviews.apache.org/r/39439/diff/
Testing
-------
Updated unit tests. mvn clean install is running fine.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
[INFO] Lens .............................................. SUCCESS [54.867s]
[INFO] Lens API .......................................... SUCCESS [1:06.700s]
[INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
[INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
[INFO] Lens DB storage ................................... SUCCESS [36.905s]
[INFO] Lens Query Library ................................ SUCCESS [31.101s]
[INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
[INFO] Lens Server ....................................... SUCCESS [21:58.491s]
[INFO] Lens client ....................................... SUCCESS [1:08.860s]
[INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
[INFO] Lens Examples ..................................... SUCCESS [24.718s]
[INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
[INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
[INFO] Lens Regression ................................... SUCCESS [36.204s]
[INFO] Lens UI ........................................... SUCCESS [43.419s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:17:53.867s
[INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
[INFO] Final Memory: 181M/1820M
Thanks,
Sushil Mohanty
Re: Review Request 39439: LENS-835 : Optimizing fact subquery by
removing non aggregated expression columns from group by clause
Posted by Sushil Mohanty <su...@gmail.com>.
> On Oct. 21, 2015, 9:44 a.m., Rajat Khandelwal wrote:
> > lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java, lines 635-641
> > <https://reviews.apache.org/r/39439/diff/1/?file=1101088#file1101088line635>
> >
> > What's the change here? Seems only re-structuring?
There was an extra column in group by "sales_fact___fact.dollars_sold" which is part of expression not supposed to be in group by.
- Sushil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/#review103380
-----------------------------------------------------------
On Oct. 19, 2015, 2:07 p.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39439/
> -----------------------------------------------------------
>
> (Updated Oct. 19, 2015, 2:07 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-835
> https://issues.apache.org/jira/browse/LENS-835
>
>
> Repository: lens
>
>
> Description
> -------
>
> Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
> Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
>
>
> Diffs
> -----
>
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302
>
> Diff: https://reviews.apache.org/r/39439/diff/
>
>
> Testing
> -------
>
> Updated unit tests. mvn clean install is running fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
> [INFO] Lens .............................................. SUCCESS [54.867s]
> [INFO] Lens API .......................................... SUCCESS [1:06.700s]
> [INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
> [INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
> [INFO] Lens DB storage ................................... SUCCESS [36.905s]
> [INFO] Lens Query Library ................................ SUCCESS [31.101s]
> [INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
> [INFO] Lens Server ....................................... SUCCESS [21:58.491s]
> [INFO] Lens client ....................................... SUCCESS [1:08.860s]
> [INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
> [INFO] Lens Examples ..................................... SUCCESS [24.718s]
> [INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
> [INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
> [INFO] Lens Regression ................................... SUCCESS [36.204s]
> [INFO] Lens UI ........................................... SUCCESS [43.419s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:17:53.867s
> [INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
> [INFO] Final Memory: 181M/1820M
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 39439: LENS-835 : Optimizing fact subquery by
removing non aggregated expression columns from group by clause
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/#review103380
-----------------------------------------------------------
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java (lines 633 - 639)
<https://reviews.apache.org/r/39439/#comment161399>
What's the change here? Seems only re-structuring?
- Rajat Khandelwal
On Oct. 19, 2015, 7:37 p.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39439/
> -----------------------------------------------------------
>
> (Updated Oct. 19, 2015, 7:37 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-835
> https://issues.apache.org/jira/browse/LENS-835
>
>
> Repository: lens
>
>
> Description
> -------
>
> Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
> Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
>
>
> Diffs
> -----
>
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302
>
> Diff: https://reviews.apache.org/r/39439/diff/
>
>
> Testing
> -------
>
> Updated unit tests. mvn clean install is running fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
> [INFO] Lens .............................................. SUCCESS [54.867s]
> [INFO] Lens API .......................................... SUCCESS [1:06.700s]
> [INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
> [INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
> [INFO] Lens DB storage ................................... SUCCESS [36.905s]
> [INFO] Lens Query Library ................................ SUCCESS [31.101s]
> [INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
> [INFO] Lens Server ....................................... SUCCESS [21:58.491s]
> [INFO] Lens client ....................................... SUCCESS [1:08.860s]
> [INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
> [INFO] Lens Examples ..................................... SUCCESS [24.718s]
> [INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
> [INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
> [INFO] Lens Regression ................................... SUCCESS [36.204s]
> [INFO] Lens UI ........................................... SUCCESS [43.419s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:17:53.867s
> [INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
> [INFO] Final Memory: 181M/1820M
>
>
> Thanks,
>
> Sushil Mohanty
>
>
Re: Review Request 39439: LENS-835 : Optimizing fact subquery by
removing non aggregated expression columns from group by clause
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/#review103915
-----------------------------------------------------------
Ship it!
Ship It!
- Amareshwari Sriramadasu
On Oct. 19, 2015, 2:07 p.m., Sushil Mohanty wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39439/
> -----------------------------------------------------------
>
> (Updated Oct. 19, 2015, 2:07 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-835
> https://issues.apache.org/jira/browse/LENS-835
>
>
> Repository: lens
>
>
> Description
> -------
>
> Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
> Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
>
>
> Diffs
> -----
>
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302
>
> Diff: https://reviews.apache.org/r/39439/diff/
>
>
> Testing
> -------
>
> Updated unit tests. mvn clean install is running fine.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
> [INFO] Lens .............................................. SUCCESS [54.867s]
> [INFO] Lens API .......................................... SUCCESS [1:06.700s]
> [INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
> [INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
> [INFO] Lens DB storage ................................... SUCCESS [36.905s]
> [INFO] Lens Query Library ................................ SUCCESS [31.101s]
> [INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
> [INFO] Lens Server ....................................... SUCCESS [21:58.491s]
> [INFO] Lens client ....................................... SUCCESS [1:08.860s]
> [INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
> [INFO] Lens Examples ..................................... SUCCESS [24.718s]
> [INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
> [INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
> [INFO] Lens Regression ................................... SUCCESS [36.204s]
> [INFO] Lens UI ........................................... SUCCESS [43.419s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:17:53.867s
> [INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
> [INFO] Final Memory: 181M/1820M
>
>
> Thanks,
>
> Sushil Mohanty
>
>