You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Rajitha R <ra...@gmail.com> on 2017/11/08 12:43:08 UTC
Review Request 63665: Implementation for Jdbc driver cost
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
Review request for lens and Puneet Gupta.
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeVirtualFactTable.java 8b55985
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageCandidate.java fd07bf2
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/1/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/#review191037
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java
Lines 42 (patched)
<https://reviews.apache.org/r/63665/#comment268626>
can RangeBasedQueryCostTypeDecider class take string as argument in constructor instead of QueryCostTypeRangeConf
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java
Lines 38 (patched)
<https://reviews.apache.org/r/63665/#comment268631>
Do we need to persist query cost at class level ?
- Puneet Gupta
On Nov. 15, 2017, 7:38 a.m., Rajitha R wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63665/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 7:38 a.m.)
>
>
> Review request for lens and Puneet Gupta.
>
>
> Bugs: LENS-1483
> https://issues.apache.org/jira/browse/LENS-1483
>
>
> Repository: lens
>
>
> Description
> -------
>
> Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
>
>
> Diffs
> -----
>
> lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
> lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
> lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
> lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
> src/site/apt/admin/hivedriver-config.apt a9f020d
> src/site/apt/admin/jdbcdriver-config.apt eb56927
>
>
> Diff: https://reviews.apache.org/r/63665/diff/11/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajitha R
>
>
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/#review191056
-----------------------------------------------------------
Ship it!
Ship It!
- Puneet Gupta
On Nov. 15, 2017, 10:32 a.m., Rajitha R wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63665/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 10:32 a.m.)
>
>
> Review request for lens and Puneet Gupta.
>
>
> Bugs: LENS-1483
> https://issues.apache.org/jira/browse/LENS-1483
>
>
> Repository: lens
>
>
> Description
> -------
>
> Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
>
>
> Diffs
> -----
>
> lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
> lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
> lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
> lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/MockQueryCostCalculator.java fb4085e
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
> src/site/apt/admin/hivedriver-config.apt a9f020d
> src/site/apt/admin/jdbcdriver-config.apt eb56927
>
>
> Diff: https://reviews.apache.org/r/63665/diff/15/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajitha R
>
>
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/#review191044
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java
Lines 43 (patched)
<https://reviews.apache.org/r/63665/#comment268632>
Can we push this to constructor?
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java
Line 343 (original), 349 (patched)
<https://reviews.apache.org/r/63665/#comment268633>
Can we have a init() method that can do this and have a default constructor ?
lens-driver-hive/src/main/resources/hivedriver-default.xml
Lines 171 (patched)
<https://reviews.apache.org/r/63665/#comment268639>
update description.
lens-driver-hive/src/main/resources/hivedriver-default.xml
Lines 176 (patched)
<https://reviews.apache.org/r/63665/#comment268634>
can we have it same as default in code "LOW,0.0,HIGH"
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java
Lines 395 (patched)
<https://reviews.apache.org/r/63665/#comment268635>
can we have common code in AbstractDriver ?
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
Lines 282 (patched)
<https://reviews.apache.org/r/63665/#comment268637>
update description.
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
Lines 288 (patched)
<https://reviews.apache.org/r/63665/#comment268638>
update description
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
Lines 293 (patched)
<https://reviews.apache.org/r/63665/#comment268636>
can we have it same as default in code
- Puneet Gupta
On Nov. 15, 2017, 10:32 a.m., Rajitha R wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63665/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 10:32 a.m.)
>
>
> Review request for lens and Puneet Gupta.
>
>
> Bugs: LENS-1483
> https://issues.apache.org/jira/browse/LENS-1483
>
>
> Repository: lens
>
>
> Description
> -------
>
> Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
>
>
> Diffs
> -----
>
> lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
> lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
> lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
> lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
> src/site/apt/admin/hivedriver-config.apt a9f020d
> src/site/apt/admin/jdbcdriver-config.apt eb56927
>
>
> Diff: https://reviews.apache.org/r/63665/diff/13/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajitha R
>
>
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Rajitha R <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
(Updated Nov. 15, 2017, 10:32 a.m.)
Review request for lens and Puneet Gupta.
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/hivedriver-config.apt a9f020d
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/13/
Changes: https://reviews.apache.org/r/63665/diff/12-13/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Rajitha R <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
(Updated Nov. 15, 2017, 10:32 a.m.)
Review request for lens and Puneet Gupta.
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/hivedriver-config.apt a9f020d
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/12/
Changes: https://reviews.apache.org/r/63665/diff/11-12/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Rajitha R <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
(Updated Nov. 15, 2017, 7:38 a.m.)
Review request for lens and Puneet Gupta.
Changes
-------
Querycost comparator changes
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/hivedriver-config.apt a9f020d
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/11/
Changes: https://reviews.apache.org/r/63665/diff/10-11/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Rajitha R <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
(Updated Nov. 14, 2017, 12:36 p.m.)
Review request for lens and Puneet Gupta.
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 1261409
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeRangeConf.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/RangeBasedQueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/MockQueryCostCalculator.java fb4085e
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/7/
Changes: https://reviews.apache.org/r/63665/diff/6-7/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/#review190922
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java
Lines 36 (patched)
<https://reviews.apache.org/r/63665/#comment268491>
Since this is a static cost and cost type per driver, We should try not to evaluate it each time from conf.
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java
Lines 54 (patched)
<https://reviews.apache.org/r/63665/#comment268490>
Shoudl be call CostRangeQueryTypeDecider as
RangeBasedQueryCostTypeDecider
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java
Lines 99 (patched)
<https://reviews.apache.org/r/63665/#comment268492>
Shoul we have default as "LOW,0.0,HIGH" ?
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java
Lines 143 (patched)
<https://reviews.apache.org/r/63665/#comment268493>
Who should create the instance of QueryCostTypeDecider . Driver or QueryCostCalculator ?
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
Lines 293 (patched)
<https://reviews.apache.org/r/63665/#comment268494>
Should we have default as LOW,0.0,HIGH ?
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostRangeQueryTypeDecider.java
Lines 37 (patched)
<https://reviews.apache.org/r/63665/#comment268495>
The constructor can take the config string "LOW,0.0,HIGH" and costToQueryTypeRangeMap can be internal to the implementation of this class.
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostToQueryTypeRangeConf.java
Lines 28 (patched)
<https://reviews.apache.org/r/63665/#comment268496>
Should we rename this class also?
QueryCostTypeRangeConf
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java
Lines 29 (patched)
<https://reviews.apache.org/r/63665/#comment268497>
update comments
- Puneet Gupta
On Nov. 14, 2017, 6:25 a.m., Rajitha R wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63665/
> -----------------------------------------------------------
>
> (Updated Nov. 14, 2017, 6:25 a.m.)
>
>
> Review request for lens and Puneet Gupta.
>
>
> Bugs: LENS-1483
> https://issues.apache.org/jira/browse/LENS-1483
>
>
> Repository: lens
>
>
> Description
> -------
>
> Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
>
>
> Diffs
> -----
>
> lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
> lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
> lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
> lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostRangeQueryTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostToQueryTypeRangeConf.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTOBuilder.java 81f1fd1
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/MockQueryCostCalculator.java fb4085e
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
> src/site/apt/admin/jdbcdriver-config.apt eb56927
>
>
> Diff: https://reviews.apache.org/r/63665/diff/5/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajitha R
>
>
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Rajitha R <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
(Updated Nov. 14, 2017, 6:25 a.m.)
Review request for lens and Puneet Gupta.
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostRangeQueryTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostToQueryTypeRangeConf.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTOBuilder.java 81f1fd1
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/MockQueryCostCalculator.java fb4085e
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/4/
Changes: https://reviews.apache.org/r/63665/diff/3-4/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Rajitha R <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/
-----------------------------------------------------------
(Updated Nov. 14, 2017, 6:09 a.m.)
Review request for lens and Puneet Gupta.
Bugs: LENS-1483
https://issues.apache.org/jira/browse/LENS-1483
Repository: lens
Description
-------
Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 3c157ee
lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
lens-cube/src/test/java/org/apache/lens/cube/query/cost/TestFactPartitionBasedQueryCostCalculator.java b5e73fd
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0b4db9a
lens-driver-hive/src/main/resources/hivedriver-default.xml a13d3b0
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryPlan.java 2036dce
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostRangeQueryTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/CostToQueryTypeRangeConf.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java eba8f0d
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java 9d7320a
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostCalculator.java 8c71c00
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCostTOBuilder.java 81f1fd1
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryTypeDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/priority/RangeConf.java 43cc53c
lens-server-api/src/test/java/org/apache/lens/server/api/driver/hooks/QueryCostBasedQueryHookTest.java 80cddf3
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/MockQueryCostCalculator.java fb4085e
lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
src/site/apt/admin/jdbcdriver-config.apt eb56927
Diff: https://reviews.apache.org/r/63665/diff/3/
Changes: https://reviews.apache.org/r/63665/diff/2-3/
Testing
-------
Thanks,
Rajitha R
Re: Review Request 63665: Implementation for Jdbc driver cost
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63665/#review190674
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeVirtualFactTable.java
Lines 196 (patched)
<https://reviews.apache.org/r/63665/#comment268246>
This is unrelated to current JIRA .
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageCandidate.java
Line 322 (original), 322 (patched)
<https://reviews.apache.org/r/63665/#comment268247>
Unrelated to JIRA LENS-1483. Please post in a separate request
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java
Lines 389 (patched)
<https://reviews.apache.org/r/63665/#comment268249>
This code seems common for Hive and JDBC driver. Is it possible to move this to AbstarctDriver and use a common property ?
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java
Lines 48 (patched)
<https://reviews.apache.org/r/63665/#comment268250>
is it possible to make < 0.0, 0.0, >0.0 configureable ?
lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java
Lines 60 (patched)
<https://reviews.apache.org/r/63665/#comment268251>
Can we have all common methods in a base abstract class AbstractQueryCost for StaticQueryCost and FactPartitionBasedQueryCost to extend it ?
- Puneet Gupta
On Nov. 8, 2017, 12:43 p.m., Rajitha R wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63665/
> -----------------------------------------------------------
>
> (Updated Nov. 8, 2017, 12:43 p.m.)
>
>
> Review request for lens and Puneet Gupta.
>
>
> Bugs: LENS-1483
> https://issues.apache.org/jira/browse/LENS-1483
>
>
> Repository: lens
>
>
> Description
> -------
>
> Jdbc driver cost has been hardcoded to zero currently in Lens code. This makes it difficult to add more jdbc drivers which have different costs. Hence there is a need for having a driver level implementation for calculating cost.
>
>
> Diffs
> -----
>
> lens-api/src/main/java/org/apache/lens/api/query/QueryCostType.java 37eda42
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeVirtualFactTable.java 8b55985
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageCandidate.java fd07bf2
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/StaticCostCalculator.java PRE-CREATION
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 528f857
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 51abc96
> lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 5545fac
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 446e759
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 7fd487c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/StaticQueryCost.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java e047c91
> src/site/apt/admin/jdbcdriver-config.apt eb56927
>
>
> Diff: https://reviews.apache.org/r/63665/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajitha R
>
>