You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Sushil Mohanty <su...@gmail.com> on 2015/10/19 16:07:59 UTC

Review Request 39439: LENS-835 : Optimizing fact subquery by removing non aggregated expression columns from group by clause

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/
-----------------------------------------------------------

Review request for lens.


Bugs: LENS-835
    https://issues.apache.org/jira/browse/LENS-835


Repository: lens


Description
-------

Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)


Diffs
-----

  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2 
  lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302 

Diff: https://reviews.apache.org/r/39439/diff/


Testing
-------

Updated unit tests. mvn clean install is running fine.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
[INFO] Lens .............................................. SUCCESS [54.867s]
[INFO] Lens API .......................................... SUCCESS [1:06.700s]
[INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
[INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
[INFO] Lens DB storage ................................... SUCCESS [36.905s]
[INFO] Lens Query Library ................................ SUCCESS [31.101s]
[INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
[INFO] Lens Server ....................................... SUCCESS [21:58.491s]
[INFO] Lens client ....................................... SUCCESS [1:08.860s]
[INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
[INFO] Lens Examples ..................................... SUCCESS [24.718s]
[INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
[INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
[INFO] Lens Regression ................................... SUCCESS [36.204s]
[INFO] Lens UI ........................................... SUCCESS [43.419s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:17:53.867s
[INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
[INFO] Final Memory: 181M/1820M


Thanks,

Sushil Mohanty


Re: Review Request 39439: LENS-835 : Optimizing fact subquery by removing non aggregated expression columns from group by clause

Posted by Sushil Mohanty <su...@gmail.com>.

> On Oct. 21, 2015, 9:44 a.m., Rajat Khandelwal wrote:
> > lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java, lines 635-641
> > <https://reviews.apache.org/r/39439/diff/1/?file=1101088#file1101088line635>
> >
> >     What's the change here? Seems only re-structuring?

There was an extra column in group by "sales_fact___fact.dollars_sold" which is part of expression not supposed to be in group by.


- Sushil


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/#review103380
-----------------------------------------------------------


On Oct. 19, 2015, 2:07 p.m., Sushil Mohanty wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39439/
> -----------------------------------------------------------
> 
> (Updated Oct. 19, 2015, 2:07 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-835
>     https://issues.apache.org/jira/browse/LENS-835
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
> Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
> 
> 
> Diffs
> -----
> 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2 
>   lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302 
> 
> Diff: https://reviews.apache.org/r/39439/diff/
> 
> 
> Testing
> -------
> 
> Updated unit tests. mvn clean install is running fine.
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
> [INFO] Lens .............................................. SUCCESS [54.867s]
> [INFO] Lens API .......................................... SUCCESS [1:06.700s]
> [INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
> [INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
> [INFO] Lens DB storage ................................... SUCCESS [36.905s]
> [INFO] Lens Query Library ................................ SUCCESS [31.101s]
> [INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
> [INFO] Lens Server ....................................... SUCCESS [21:58.491s]
> [INFO] Lens client ....................................... SUCCESS [1:08.860s]
> [INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
> [INFO] Lens Examples ..................................... SUCCESS [24.718s]
> [INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
> [INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
> [INFO] Lens Regression ................................... SUCCESS [36.204s]
> [INFO] Lens UI ........................................... SUCCESS [43.419s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:17:53.867s
> [INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
> [INFO] Final Memory: 181M/1820M
> 
> 
> Thanks,
> 
> Sushil Mohanty
> 
>


Re: Review Request 39439: LENS-835 : Optimizing fact subquery by removing non aggregated expression columns from group by clause

Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/#review103380
-----------------------------------------------------------



lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java (lines 633 - 639)
<https://reviews.apache.org/r/39439/#comment161399>

    What's the change here? Seems only re-structuring?


- Rajat Khandelwal


On Oct. 19, 2015, 7:37 p.m., Sushil Mohanty wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39439/
> -----------------------------------------------------------
> 
> (Updated Oct. 19, 2015, 7:37 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-835
>     https://issues.apache.org/jira/browse/LENS-835
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
> Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
> 
> 
> Diffs
> -----
> 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2 
>   lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302 
> 
> Diff: https://reviews.apache.org/r/39439/diff/
> 
> 
> Testing
> -------
> 
> Updated unit tests. mvn clean install is running fine.
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
> [INFO] Lens .............................................. SUCCESS [54.867s]
> [INFO] Lens API .......................................... SUCCESS [1:06.700s]
> [INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
> [INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
> [INFO] Lens DB storage ................................... SUCCESS [36.905s]
> [INFO] Lens Query Library ................................ SUCCESS [31.101s]
> [INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
> [INFO] Lens Server ....................................... SUCCESS [21:58.491s]
> [INFO] Lens client ....................................... SUCCESS [1:08.860s]
> [INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
> [INFO] Lens Examples ..................................... SUCCESS [24.718s]
> [INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
> [INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
> [INFO] Lens Regression ................................... SUCCESS [36.204s]
> [INFO] Lens UI ........................................... SUCCESS [43.419s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:17:53.867s
> [INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
> [INFO] Final Memory: 181M/1820M
> 
> 
> Thanks,
> 
> Sushil Mohanty
> 
>


Re: Review Request 39439: LENS-835 : Optimizing fact subquery by removing non aggregated expression columns from group by clause

Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39439/#review103915
-----------------------------------------------------------

Ship it!


Ship It!

- Amareshwari Sriramadasu


On Oct. 19, 2015, 2:07 p.m., Sushil Mohanty wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39439/
> -----------------------------------------------------------
> 
> (Updated Oct. 19, 2015, 2:07 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-835
>     https://issues.apache.org/jira/browse/LENS-835
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Currently the fact keys used in SQL are part of group by clause by default. If single row functions e.g case, having fact keys part of it and used in an aggregate expression such keys need to be removed from fact subquery group by clause.
> Eg. sum(case when fact.key1 = '123' then msr1 when fact.key2 = '345' then msr2 end)
> 
> 
> Diffs
> -----
> 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 2d81ff2 
>   lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 77ab302 
> 
> Diff: https://reviews.apache.org/r/39439/diff/
> 
> 
> Testing
> -------
> 
> Updated unit tests. mvn clean install is running fine.
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [36.932s]
> [INFO] Lens .............................................. SUCCESS [54.867s]
> [INFO] Lens API .......................................... SUCCESS [1:06.700s]
> [INFO] Lens API for server and extensions ................ SUCCESS [50.545s]
> [INFO] Lens Cube ......................................... SUCCESS [7:46.586s]
> [INFO] Lens DB storage ................................... SUCCESS [36.905s]
> [INFO] Lens Query Library ................................ SUCCESS [31.101s]
> [INFO] Lens Hive Driver .................................. SUCCESS [4:36.710s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [1:01.830s]
> [INFO] Lens Elastic Search Driver ........................ SUCCESS [35.007s]
> [INFO] Lens Server ....................................... SUCCESS [21:58.491s]
> [INFO] Lens client ....................................... SUCCESS [1:08.860s]
> [INFO] Lens CLI .......................................... SUCCESS [5:50.915s]
> [INFO] Lens Examples ..................................... SUCCESS [24.718s]
> [INFO] Lens Distribution ................................. SUCCESS [3:59.018s]
> [INFO] Lens ML Lib ....................................... SUCCESS [24:14.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [19.281s]
> [INFO] Lens Regression ................................... SUCCESS [36.204s]
> [INFO] Lens UI ........................................... SUCCESS [43.419s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:17:53.867s
> [INFO] Finished at: Mon Oct 19 17:20:57 IST 2015
> [INFO] Final Memory: 181M/1820M
> 
> 
> Thanks,
> 
> Sushil Mohanty
> 
>