You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Amareshwari Sriramadasu <am...@apache.org> on 2015/07/28 13:25:38 UTC
Review Request 36884: LENS-426 : Review logging in estimate path
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36884/
-----------------------------------------------------------
Review request for lens.
Bugs: LENS-426
https://issues.apache.org/jira/browse/LENS-426
Repository: lens
Description
-------
Have gone through all the code flow of estimate. Changes include :
- Changing some logs to debug; put isDebugEnabled check for doing costly computation on params
- Have moved to parameterized logging wherever it is not
Diffs
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 65c240a
lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java 895cb06
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 5fb2441
lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 820d77d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 1316257
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 52edffc
Diff: https://reviews.apache.org/r/36884/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.069s]
[INFO] Lens .............................................. SUCCESS [3.084s]
[INFO] Lens API .......................................... SUCCESS [21.467s]
[INFO] Lens API for server and extensions ................ SUCCESS [19.744s]
[INFO] Lens Cube ......................................... SUCCESS [6:02.161s]
[INFO] Lens DB storage ................................... SUCCESS [18.850s]
[INFO] Lens Query Library ................................ SUCCESS [15.269s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:45.689s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [35.300s]
[INFO] Lens Server ....................................... SUCCESS [5:22.277s]
[INFO] Lens client ....................................... SUCCESS [33.811s]
[INFO] Lens CLI .......................................... SUCCESS [2:35.257s]
[INFO] Lens Examples ..................................... SUCCESS [8.561s]
[INFO] Lens Distribution ................................. SUCCESS [8.048s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:21.468s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.727s]
[INFO] Lens Regression ................................... SUCCESS [10.513s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:06.336s
[INFO] Finished at: Tue Jul 28 11:17:44 UTC 2015
[INFO] Final Memory: 175M/1186M
[INFO] ------------------------------------------------------------------------
Thanks,
Amareshwari Sriramadasu
Re: Review Request 36884: LENS-426 : Review logging in estimate path
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On July 28, 2015, 12:04 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java, line 444
> > <https://reviews.apache.org/r/36884/diff/2/?file=1023523#file1023523line444>
> >
> > I think this can be omitted. http://stackoverflow.com/a/29561054
>> log.debug("Adding optional dimension:{} optDim:{} {} isRef:{}", dim , optDim,
>> (cubeCol == null ? "" : " for column:" + cubeCol), isRef);
Have put the check because the second line is doing some computation and string concat.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36884/#review93276
-----------------------------------------------------------
On July 28, 2015, 11:32 a.m., Amareshwari Sriramadasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36884/
> -----------------------------------------------------------
>
> (Updated July 28, 2015, 11:32 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-426
> https://issues.apache.org/jira/browse/LENS-426
>
>
> Repository: lens
>
>
> Description
> -------
>
> Have gone through all the code flow of estimate. Changes include :
>
> - Changing some logs to debug; put isDebugEnabled check for doing costly computation on params
> - Have moved to parameterized logging wherever it is not
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 65c240a
> lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java 895cb06
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 5fb2441
> lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 820d77d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 1316257
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 52edffc
>
> Diff: https://reviews.apache.org/r/36884/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.069s]
> [INFO] Lens .............................................. SUCCESS [3.084s]
> [INFO] Lens API .......................................... SUCCESS [21.467s]
> [INFO] Lens API for server and extensions ................ SUCCESS [19.744s]
> [INFO] Lens Cube ......................................... SUCCESS [6:02.161s]
> [INFO] Lens DB storage ................................... SUCCESS [18.850s]
> [INFO] Lens Query Library ................................ SUCCESS [15.269s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:45.689s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [35.300s]
> [INFO] Lens Server ....................................... SUCCESS [5:22.277s]
> [INFO] Lens client ....................................... SUCCESS [33.811s]
> [INFO] Lens CLI .......................................... SUCCESS [2:35.257s]
> [INFO] Lens Examples ..................................... SUCCESS [8.561s]
> [INFO] Lens Distribution ................................. SUCCESS [8.048s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:21.468s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.727s]
> [INFO] Lens Regression ................................... SUCCESS [10.513s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:06.336s
> [INFO] Finished at: Tue Jul 28 11:17:44 UTC 2015
> [INFO] Final Memory: 175M/1186M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Amareshwari Sriramadasu
>
>
Re: Review Request 36884: LENS-426 : Review logging in estimate path
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36884/#review93276
-----------------------------------------------------------
Ship it!
Seems okay.
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java (line 444)
<https://reviews.apache.org/r/36884/#comment147653>
I think this can be omitted. http://stackoverflow.com/a/29561054
- Rajat Khandelwal
On July 28, 2015, 5:02 p.m., Amareshwari Sriramadasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36884/
> -----------------------------------------------------------
>
> (Updated July 28, 2015, 5:02 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-426
> https://issues.apache.org/jira/browse/LENS-426
>
>
> Repository: lens
>
>
> Description
> -------
>
> Have gone through all the code flow of estimate. Changes include :
>
> - Changing some logs to debug; put isDebugEnabled check for doing costly computation on params
> - Have moved to parameterized logging wherever it is not
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 65c240a
> lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java 895cb06
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 5fb2441
> lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 820d77d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 1316257
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 52edffc
>
> Diff: https://reviews.apache.org/r/36884/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.069s]
> [INFO] Lens .............................................. SUCCESS [3.084s]
> [INFO] Lens API .......................................... SUCCESS [21.467s]
> [INFO] Lens API for server and extensions ................ SUCCESS [19.744s]
> [INFO] Lens Cube ......................................... SUCCESS [6:02.161s]
> [INFO] Lens DB storage ................................... SUCCESS [18.850s]
> [INFO] Lens Query Library ................................ SUCCESS [15.269s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:45.689s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [35.300s]
> [INFO] Lens Server ....................................... SUCCESS [5:22.277s]
> [INFO] Lens client ....................................... SUCCESS [33.811s]
> [INFO] Lens CLI .......................................... SUCCESS [2:35.257s]
> [INFO] Lens Examples ..................................... SUCCESS [8.561s]
> [INFO] Lens Distribution ................................. SUCCESS [8.048s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:21.468s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.727s]
> [INFO] Lens Regression ................................... SUCCESS [10.513s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:06.336s
> [INFO] Finished at: Tue Jul 28 11:17:44 UTC 2015
> [INFO] Final Memory: 175M/1186M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Amareshwari Sriramadasu
>
>
Re: Review Request 36884: LENS-426 : Review logging in estimate path
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36884/
-----------------------------------------------------------
(Updated July 28, 2015, 11:32 a.m.)
Review request for lens.
Bugs: LENS-426
https://issues.apache.org/jira/browse/LENS-426
Repository: lens
Description
-------
Have gone through all the code flow of estimate. Changes include :
- Changing some logs to debug; put isDebugEnabled check for doing costly computation on params
- Have moved to parameterized logging wherever it is not
Diffs (updated)
-----
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 65c240a
lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java 895cb06
lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 5fb2441
lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 820d77d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 1316257
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 52edffc
Diff: https://reviews.apache.org/r/36884/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.069s]
[INFO] Lens .............................................. SUCCESS [3.084s]
[INFO] Lens API .......................................... SUCCESS [21.467s]
[INFO] Lens API for server and extensions ................ SUCCESS [19.744s]
[INFO] Lens Cube ......................................... SUCCESS [6:02.161s]
[INFO] Lens DB storage ................................... SUCCESS [18.850s]
[INFO] Lens Query Library ................................ SUCCESS [15.269s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:45.689s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [35.300s]
[INFO] Lens Server ....................................... SUCCESS [5:22.277s]
[INFO] Lens client ....................................... SUCCESS [33.811s]
[INFO] Lens CLI .......................................... SUCCESS [2:35.257s]
[INFO] Lens Examples ..................................... SUCCESS [8.561s]
[INFO] Lens Distribution ................................. SUCCESS [8.048s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:21.468s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.727s]
[INFO] Lens Regression ................................... SUCCESS [10.513s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:06.336s
[INFO] Finished at: Tue Jul 28 11:17:44 UTC 2015
[INFO] Final Memory: 175M/1186M
[INFO] ------------------------------------------------------------------------
Thanks,
Amareshwari Sriramadasu
Re: Review Request 36884: LENS-426 : Review logging in estimate path
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36884/#review93273
-----------------------------------------------------------
Here is the log estimate on example query :
28 Jul 2015 16:47:37,797 [b6b7a2bf-c4eb-418c-8377-7e73c86621b2] [Grizzly-worker(8)] INFO org.apache.lens.server.query.QueryExecutionServiceImpl - ExecuteAsync: session:<?xml version="1.0" encoding="UTF-8" standalone="yes"?><lensSessionHandle><publicId>5dba21c3-2a0e-4d51-9e55-a22e29f854b8</publicId><secretId>120a77c2-483a-429d-933c-ef3fead506bb</secretId></lensSessionHandle> query: cube select measure2 from sample_cube where time_range_in(dt, '2014-06-24-23', '2014-06-25-00')
28 Jul 2015 16:47:37,799 [b6b7a2bf-c4eb-418c-8377-7e73c86621b2] [Grizzly-worker(8)] INFO org.apache.lens.server.query.QueryExecutionServiceImpl - Returning handle 53deaaa2-31a6-4f0e-a46f-47f2f00af15e
28 Jul 2015 16:47:37,799 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [QuerySubmitter] INFO org.apache.lens.server.query.QueryExecutionServiceImpl - Processing query:cube select measure2 from sample_cube where time_range_in(dt, '2014-06-24-23', '2014-06-25-00')
28 Jul 2015 16:47:37,829 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [QuerySubmitter] INFO hive.ql.parse.ParseDriver - Parsing command: cube select measure2 from sample_cube where time_range_in(dt, '2014-06-24-23', '2014-06-25-00')
28 Jul 2015 16:47:37,831 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [QuerySubmitter] INFO hive.ql.parse.ParseDriver - Parse Completed
28 Jul 2015 16:47:37,833 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO hive.ql.parse.ParseDriver - Parsing command: cube select measure2 from sample_cube where time_range_in(dt, '2014-06-24-23', '2014-06-25-00')
28 Jul 2015 16:47:37,833 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO hive.ql.parse.ParseDriver - Parsing command: cube select measure2 from sample_cube where time_range_in(dt, '2014-06-24-23', '2014-06-25-00')
28 Jul 2015 16:47:37,835 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO hive.ql.parse.ParseDriver - Parse Completed
28 Jul 2015 16:47:37,835 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO hive.ql.parse.ParseDriver - Parse Completed
28 Jul 2015 16:47:37,836 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.CandidateTableResolver - Populated candidate facts: [fact2, rawfact, fact1]
28 Jul 2015 16:47:37,836 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.CandidateTableResolver - Measure covering fact sets :[[fact2], [rawfact], [fact1]]
28 Jul 2015 16:47:37,837 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.JoinResolver - No dimension tables to resolve and no join chains present!
28 Jul 2015 16:47:37,836 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.CandidateTableResolver - Populated candidate facts: [fact2, rawfact, fact1]
28 Jul 2015 16:47:37,837 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.CandidateTableResolver - Measure covering fact sets :[[fact2], [rawfact], [fact1]]
28 Jul 2015 16:47:37,838 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.JoinResolver - No dimension tables to resolve and no join chains present!
28 Jul 2015 16:47:37,839 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for fact2 from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,839 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for fact2 from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Tue Jun 24 23:00:00 IST 2014
28 Jul 2015 16:47:37,839 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.StorageTableResolver - Skipping storage: local as it is not supported
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for fact2 from fromDate:Wed Jun 25 00:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.StorageTableResolver - Not considering fact table:fact2 as it does not have any storage tables
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.StorageTableResolver - Skipping storage: local as it is not supported
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.StorageTableResolver - Not considering fact table:rawfact as it does not have any storage tables
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.StorageTableResolver - Skipping storage: local as it is not supported
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - Resolved partitions for fact fact2: [dt = '2014-06-24-23'] storageTables:[local_fact2]
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.StorageTableResolver - Not considering fact table:fact1 as it does not have any storage tables
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for rawfact from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.CubeQueryContext - Not considering fact table set:[fact2] as they have non candidate tables and facts missing because of NO_CANDIDATE_STORAGES
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.CubeQueryContext - Not considering fact table set:[rawfact] as they have non candidate tables and facts missing because of NO_CANDIDATE_STORAGES
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] INFO org.apache.lens.cube.parse.CubeQueryContext - Not considering fact table set:[fact1] as they have non candidate tables and facts missing because of NO_CANDIDATE_STORAGES
28 Jul 2015 16:47:37,840 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - Partition:dt = '2014-06-24-23' does not exist in any storage table
28 Jul 2015 16:47:37,841 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for rawfact from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,841 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - No max interval for range: Tue Jun 24 23:00:00 IST 2014 to Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,842 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for rawfact from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Tue Jun 24 23:00:00 IST 2014
28 Jul 2015 16:47:37,842 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for rawfact from fromDate:Wed Jun 25 00:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,843 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - No partitions for range:dt [2014-06-24-23:00:00,000 to 2014-06-25-00:00:00,000). fallback range: null
28 Jul 2015 16:47:37,844 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - No partitions for fallback range:dt [2014-06-24-23:00:00,000 to 2014-06-25-00:00:00,000)
28 Jul 2015 16:47:37,846 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - Not considering fact table:rawfact as it could not find partition for given ranges: [dt [2014-06-24-23:00:00,000 to 2014-06-25-00:00:00,000)]
28 Jul 2015 16:47:37,846 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for fact1 from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 201428 Jul 2015 16:47:37,846 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for fact1 from fromDate:Tue Jun 24 23:00:00 IST 2014 toDate:Tue Jun 24 23:00:00 IST 2014
28 Jul 2015 16:47:37,847 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - getPartitions for fact1 from fromDate:Wed Jun 25 00:00:00 IST 2014 toDate:Wed Jun 25 00:00:00 IST 2014
28 Jul 2015 16:47:37,847 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.StorageTableResolver - Resolved partitions for fact fact1: [dt = '2014-06-24-23'] storageTables:[local_fact1]
28 Jul 2015 16:47:37,847 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.CubeQueryContext - Not considering fact table set:[rawfact] as they have non candidate tables and facts missing because of NO_CANDIDATE_STORAGES
28 Jul 2015 16:47:37,858 [f3b437fb-30fe-4fc1-b265-c5649b80b92b] [Grizzly-worker(1)] INFO org.apache.lens.server.AuthenticationFilter - Request from user: null , path=/queryapi/queries/53deaaa2-31a6-4f0e-a46f-47f2f00af15e
28 Jul 2015 16:47:37,859 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.LightestFactResolver - Not considering facts:[fact2] from candidate fact tables as it has more fact weight:200.0 minimum:100.0
28 Jul 2015 16:47:37,859 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.CubeQueryContext - Not considering fact table:fact2 as CandidateTablePruneCause(cause=MORE_WEIGHT, storageCauses=null, missingPartitions=null, missingUpdatePeriods=null, missingColumns=null, missingExpressions=null, joinColumns=null, columnsMissingDefaultAggregate=null, unsupportedTimeDims=null, maxTimeCovered=null, invalidRanges=null)
28 Jul 2015 16:47:37,859 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.CubeQueryContext - Available candidate facts:[[fact1]], picking up [fact1] for querying
28 Jul 2015 16:47:37,859 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO hive.ql.parse.ParseDriver - Parsing expression: (sample_cube.dt = '2014-06-24-23')
28 Jul 2015 16:47:37,860 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO hive.ql.parse.ParseDriver - Parse Completed
28 Jul 2015 16:47:37,860 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.cube.parse.CubeQueryContext - Picked Fact:[fact1] dimsToQuery: {}{}
28 Jul 2015 16:47:37,860 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.server.rewrite.RewriteUtil - Final rewritten query for driver: org.apache.lens.driver.hive.HiveDriver@78c76c3a is: SELECT sum(( sample_cube . measure2 )) FROM local_fact1 sample_cube WHERE (((( sample_cube . dt ) = '2014-06-24-23' )))
28 Jul 2015 16:47:37,861 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-3] INFO org.apache.lens.driver.hive.HiveDriver - Estimate: SELECT sum(( sample_cube . measure2 )) FROM local_fact1 sample_cube WHERE (((( sample_cube . dt ) = '2014-06-24-23' )))
28 Jul 2015 16:47:37,862 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] WARN org.apache.lens.server.rewrite.RewriteUtil - Driver : org.apache.lens.driver.jdbc.JDBCDriver@3f807a01 Skipped for the query rewriting due to
org.apache.hadoop.hive.ql.parse.SemanticException: No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}}
at org.apache.lens.cube.parse.CubeQueryContext.pickCandidateFactToQuery(CubeQueryContext.java:841)
at org.apache.lens.cube.parse.CubeQueryContext.toHQL(CubeQueryContext.java:852)
at org.apache.lens.server.rewrite.RewriteUtil$DriverRewriterRunnable.run(RewriteUtil.java:327)
at org.apache.lens.server.query.QueryExecutionServiceImpl$RewriteEstimateRunnable.run(QueryExecutionServiceImpl.java:1348)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
28 Jul 2015 16:47:37,862 [53deaaa2-31a6-4f0e-a46f-47f2f00af15e] [estimate-4] ERROR org.apache.lens.server.query.QueryExecutionServiceImpl - Estimate skipped since rewrite failed for driver org.apache.lens.driver.jdbc.JDBCDriver@3f807a01 cause: Driver :org.apache.lens.driver.jdbc.JDBCDriver Cause :No candidate fact table available to answer the query, because {"brief":"No candidate storages for any table","details":{"fact2,fact1,rawfact":[{"cause":"NO_CANDIDATE_STORAGES","storageCauses":{"local":{"cause":"UNSUPPORTED"}}}]}}
- Amareshwari Sriramadasu
On July 28, 2015, 11:25 a.m., Amareshwari Sriramadasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36884/
> -----------------------------------------------------------
>
> (Updated July 28, 2015, 11:25 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-426
> https://issues.apache.org/jira/browse/LENS-426
>
>
> Repository: lens
>
>
> Description
> -------
>
> Have gone through all the code flow of estimate. Changes include :
>
> - Changing some logs to debug; put isDebugEnabled check for doing costly computation on params
> - Have moved to parameterized logging wherever it is not
>
>
> Diffs
> -----
>
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 65c240a
> lens-cube/src/main/java/org/apache/lens/cube/parse/ExpressionResolver.java 895cb06
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 5fb2441
> lens-cube/src/main/java/org/apache/lens/cube/parse/TimerangeResolver.java 820d77d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 1316257
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 52edffc
>
> Diff: https://reviews.apache.org/r/36884/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.069s]
> [INFO] Lens .............................................. SUCCESS [3.084s]
> [INFO] Lens API .......................................... SUCCESS [21.467s]
> [INFO] Lens API for server and extensions ................ SUCCESS [19.744s]
> [INFO] Lens Cube ......................................... SUCCESS [6:02.161s]
> [INFO] Lens DB storage ................................... SUCCESS [18.850s]
> [INFO] Lens Query Library ................................ SUCCESS [15.269s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:45.689s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [35.300s]
> [INFO] Lens Server ....................................... SUCCESS [5:22.277s]
> [INFO] Lens client ....................................... SUCCESS [33.811s]
> [INFO] Lens CLI .......................................... SUCCESS [2:35.257s]
> [INFO] Lens Examples ..................................... SUCCESS [8.561s]
> [INFO] Lens Distribution ................................. SUCCESS [8.048s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:21.468s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.727s]
> [INFO] Lens Regression ................................... SUCCESS [10.513s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:06.336s
> [INFO] Finished at: Tue Jul 28 11:17:44 UTC 2015
> [INFO] Final Memory: 175M/1186M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Amareshwari Sriramadasu
>
>