You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Amareshwari Sriramadasu <am...@apache.org> on 2015/02/05 11:38:28 UTC
Review Request 30659: LENS-272 : Fixes missing partitions error
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30659/
-----------------------------------------------------------
Review request for lens, Himanshu Gahlaut, Jaideep dhok, Rajat Khandelwal, and Srikanth Sundarrajan.
Bugs: LENS-272
https://issues.apache.org/jira/browse/LENS-272
Repository: lens
Description
-------
Changes include :
* makes MORE_WEIGHT and MORE_PARTITIONS with least precendence among errors.
* Adds the first missing partition when lens.cube.query.add.nonexisting.partitions is set to false. This makes sure that the NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE error comes only when update period is not available for the range queried
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java 166c7fa
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 64b835e
lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java e1469c1
Diff: https://reviews.apache.org/r/30659/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.817s]
[INFO] Lens .............................................. SUCCESS [2.151s]
[INFO] Lens API .......................................... SUCCESS [8.772s]
[INFO] Lens API for server and extensions ................ SUCCESS [5.723s]
[INFO] Lens Cube ......................................... SUCCESS [6:25.288s]
[INFO] Lens DB storage ................................... SUCCESS [11.695s]
[INFO] Lens Query Library ................................ SUCCESS [6.051s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:40.497s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [24.385s]
[INFO] Lens Server ....................................... SUCCESS [4:21.197s]
[INFO] Lens client ....................................... SUCCESS [19.738s]
[INFO] Lens CLI .......................................... SUCCESS [1:48.086s]
[INFO] Lens Examples ..................................... SUCCESS [0.852s]
[INFO] Lens Distribution ................................. SUCCESS [3.503s]
[INFO] Lens Client Distribution .......................... SUCCESS [5.661s]
[INFO] Lens ML Lib ....................................... SUCCESS [41.839s]
[INFO] Lens Regression ................................... SUCCESS [0.820s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17:28.935s
[INFO] Finished at: Thu Feb 05 10:27:33 UTC 2015
[INFO] Final Memory: 99M/1272M
[INFO] ------------------------------------------------------------------------
Here are the errors being thrown now :
lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00:00', '2014-06-25-00:05')
Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"No fact update periods for given range","details":{"fact2,fact1,rawfact":[{"cause":"NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE"}]}}
lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25"]}]}}
lens-shell>set lens.cube.query.add.nonexisting.partitions=true
Set param succeeded
lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25, 2014-06-26-02, 2014-06-26]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25","2014-06-26-02","2014-06-26"]}]}}
lens-shell>
lens-shell>set lens.cube.query.pick.lightest.fact.first=true
Set param succeeded
lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
lens-shell>set lens.cube.query.add.nonexisting.partitions=false
Set param succeeded
lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
Thanks,
Amareshwari Sriramadasu
Re: Review Request 30659: LENS-272 : Fixes missing partitions error
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30659/#review71423
-----------------------------------------------------------
Ship it!
Ship It!
- Suma Shivaprasad
On Feb. 5, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30659/
> -----------------------------------------------------------
>
> (Updated Feb. 5, 2015, 10:38 a.m.)
>
>
> Review request for lens, Himanshu Gahlaut, Jaideep dhok, Rajat Khandelwal, and Srikanth Sundarrajan.
>
>
> Bugs: LENS-272
> https://issues.apache.org/jira/browse/LENS-272
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include :
>
> * makes MORE_WEIGHT and MORE_PARTITIONS with least precendence among errors.
> * Adds the first missing partition when lens.cube.query.add.nonexisting.partitions is set to false. This makes sure that the NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE error comes only when update period is not available for the range queried
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java 166c7fa
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 64b835e
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java e1469c1
>
> Diff: https://reviews.apache.org/r/30659/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.817s]
> [INFO] Lens .............................................. SUCCESS [2.151s]
> [INFO] Lens API .......................................... SUCCESS [8.772s]
> [INFO] Lens API for server and extensions ................ SUCCESS [5.723s]
> [INFO] Lens Cube ......................................... SUCCESS [6:25.288s]
> [INFO] Lens DB storage ................................... SUCCESS [11.695s]
> [INFO] Lens Query Library ................................ SUCCESS [6.051s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:40.497s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [24.385s]
> [INFO] Lens Server ....................................... SUCCESS [4:21.197s]
> [INFO] Lens client ....................................... SUCCESS [19.738s]
> [INFO] Lens CLI .......................................... SUCCESS [1:48.086s]
> [INFO] Lens Examples ..................................... SUCCESS [0.852s]
> [INFO] Lens Distribution ................................. SUCCESS [3.503s]
> [INFO] Lens Client Distribution .......................... SUCCESS [5.661s]
> [INFO] Lens ML Lib ....................................... SUCCESS [41.839s]
> [INFO] Lens Regression ................................... SUCCESS [0.820s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 17:28.935s
> [INFO] Finished at: Thu Feb 05 10:27:33 UTC 2015
> [INFO] Final Memory: 99M/1272M
> [INFO] ------------------------------------------------------------------------
>
> Here are the errors being thrown now :
>
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00:00', '2014-06-25-00:05')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"No fact update periods for given range","details":{"fact2,fact1,rawfact":[{"cause":"NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE"}]}}
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25"]}]}}
> lens-shell>set lens.cube.query.add.nonexisting.partitions=true
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25, 2014-06-26-02, 2014-06-26]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25","2014-06-26-02","2014-06-26"]}]}}
> lens-shell>
> lens-shell>set lens.cube.query.pick.lightest.fact.first=true
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
> lens-shell>set lens.cube.query.add.nonexisting.partitions=false
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
>
>
> Thanks,
>
> Amareshwari Sriramadasu
>
>
Re: Review Request 30659: LENS-272 : Fixes missing partitions error
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On Feb. 6, 2015, 12:51 p.m., Suma Shivaprasad wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java, line 369
> > <https://reviews.apache.org/r/30659/diff/1/?file=850390#file850390line369>
> >
> > where are we taking care of adding only first missing partition according to this comment. Couldnt find the code path for that
Changes from line #520-522 do that.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30659/#review71416
-----------------------------------------------------------
On Feb. 5, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30659/
> -----------------------------------------------------------
>
> (Updated Feb. 5, 2015, 10:38 a.m.)
>
>
> Review request for lens, Himanshu Gahlaut, Jaideep dhok, Rajat Khandelwal, and Srikanth Sundarrajan.
>
>
> Bugs: LENS-272
> https://issues.apache.org/jira/browse/LENS-272
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include :
>
> * makes MORE_WEIGHT and MORE_PARTITIONS with least precendence among errors.
> * Adds the first missing partition when lens.cube.query.add.nonexisting.partitions is set to false. This makes sure that the NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE error comes only when update period is not available for the range queried
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java 166c7fa
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 64b835e
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java e1469c1
>
> Diff: https://reviews.apache.org/r/30659/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.817s]
> [INFO] Lens .............................................. SUCCESS [2.151s]
> [INFO] Lens API .......................................... SUCCESS [8.772s]
> [INFO] Lens API for server and extensions ................ SUCCESS [5.723s]
> [INFO] Lens Cube ......................................... SUCCESS [6:25.288s]
> [INFO] Lens DB storage ................................... SUCCESS [11.695s]
> [INFO] Lens Query Library ................................ SUCCESS [6.051s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:40.497s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [24.385s]
> [INFO] Lens Server ....................................... SUCCESS [4:21.197s]
> [INFO] Lens client ....................................... SUCCESS [19.738s]
> [INFO] Lens CLI .......................................... SUCCESS [1:48.086s]
> [INFO] Lens Examples ..................................... SUCCESS [0.852s]
> [INFO] Lens Distribution ................................. SUCCESS [3.503s]
> [INFO] Lens Client Distribution .......................... SUCCESS [5.661s]
> [INFO] Lens ML Lib ....................................... SUCCESS [41.839s]
> [INFO] Lens Regression ................................... SUCCESS [0.820s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 17:28.935s
> [INFO] Finished at: Thu Feb 05 10:27:33 UTC 2015
> [INFO] Final Memory: 99M/1272M
> [INFO] ------------------------------------------------------------------------
>
> Here are the errors being thrown now :
>
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00:00', '2014-06-25-00:05')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"No fact update periods for given range","details":{"fact2,fact1,rawfact":[{"cause":"NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE"}]}}
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25"]}]}}
> lens-shell>set lens.cube.query.add.nonexisting.partitions=true
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25, 2014-06-26-02, 2014-06-26]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25","2014-06-26-02","2014-06-26"]}]}}
> lens-shell>
> lens-shell>set lens.cube.query.pick.lightest.fact.first=true
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
> lens-shell>set lens.cube.query.add.nonexisting.partitions=false
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
>
>
> Thanks,
>
> Amareshwari Sriramadasu
>
>
Re: Review Request 30659: LENS-272 : Fixes missing partitions error
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30659/#review71416
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java
<https://reviews.apache.org/r/30659/#comment117173>
where are we taking care of adding only first missing partition according to this comment. Couldnt find the code path for that
- Suma Shivaprasad
On Feb. 5, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30659/
> -----------------------------------------------------------
>
> (Updated Feb. 5, 2015, 10:38 a.m.)
>
>
> Review request for lens, Himanshu Gahlaut, Jaideep dhok, Rajat Khandelwal, and Srikanth Sundarrajan.
>
>
> Bugs: LENS-272
> https://issues.apache.org/jira/browse/LENS-272
>
>
> Repository: lens
>
>
> Description
> -------
>
> Changes include :
>
> * makes MORE_WEIGHT and MORE_PARTITIONS with least precendence among errors.
> * Adds the first missing partition when lens.cube.query.add.nonexisting.partitions is set to false. This makes sure that the NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE error comes only when update period is not available for the range queried
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java 166c7fa
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 64b835e
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java e1469c1
>
> Diff: https://reviews.apache.org/r/30659/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.817s]
> [INFO] Lens .............................................. SUCCESS [2.151s]
> [INFO] Lens API .......................................... SUCCESS [8.772s]
> [INFO] Lens API for server and extensions ................ SUCCESS [5.723s]
> [INFO] Lens Cube ......................................... SUCCESS [6:25.288s]
> [INFO] Lens DB storage ................................... SUCCESS [11.695s]
> [INFO] Lens Query Library ................................ SUCCESS [6.051s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:40.497s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [24.385s]
> [INFO] Lens Server ....................................... SUCCESS [4:21.197s]
> [INFO] Lens client ....................................... SUCCESS [19.738s]
> [INFO] Lens CLI .......................................... SUCCESS [1:48.086s]
> [INFO] Lens Examples ..................................... SUCCESS [0.852s]
> [INFO] Lens Distribution ................................. SUCCESS [3.503s]
> [INFO] Lens Client Distribution .......................... SUCCESS [5.661s]
> [INFO] Lens ML Lib ....................................... SUCCESS [41.839s]
> [INFO] Lens Regression ................................... SUCCESS [0.820s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 17:28.935s
> [INFO] Finished at: Thu Feb 05 10:27:33 UTC 2015
> [INFO] Final Memory: 99M/1272M
> [INFO] ------------------------------------------------------------------------
>
> Here are the errors being thrown now :
>
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00:00', '2014-06-25-00:05')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"No fact update periods for given range","details":{"fact2,fact1,rawfact":[{"cause":"NO_FACT_UPDATE_PERIODS_FOR_GIVEN_RANGE"}]}}
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25"]}]}}
> lens-shell>set lens.cube.query.add.nonexisting.partitions=true
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26], [2014-06-25-00, 2014-06-25, 2014-06-26-02, 2014-06-26]]","details":{"fact2,fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}],"rawfact":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-25-00","2014-06-25","2014-06-26-02","2014-06-26"]}]}}
> lens-shell>
> lens-shell>set lens.cube.query.pick.lightest.fact.first=true
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
> lens-shell>set lens.cube.query.add.nonexisting.partitions=false
> Set param succeeded
> lens-shell>query explain cube select measure2 from sample_cube where time_range_in(dt, '2014-06-25-00', '2014-06-27-00')
> Explain FAILED:No driver accepted the query, because Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}} Driver :org.apache.lens.driver.hive.HiveDriver Cause :No candidate fact table available to answer the query, because {"brief":"Missing partitions for the cube table: [[2014-06-26-01, 2014-06-26]]","details":{"fact2,rawfact":[{"cause":"MORE_WEIGHT"}],"fact1":[{"cause":"MISSING_PARTITIONS","missingPartitions":["2014-06-26-01","2014-06-26"]}]}}
>
>
> Thanks,
>
> Amareshwari Sriramadasu
>
>