You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@lens.apache.org by Himanshu Gahlaut <ga...@gmail.com> on 2015/07/10 14:15:12 UTC

Review Request 36393: LENS-619: Applying Query Launching Constraints before allowing a query to be launched

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description
-------

LENS-619: Applying Query Constraints before allowing a query to be launched

Lens will always accept a new query from a user and put it in a scheduling queue for processing.

Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.

When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.

At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.

New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.

Waiting Queries will be persisted across server restarts.

Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than  a cumulative query cost ceiling for launching a new query. 

Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.

Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the selected driver are less than max concurrent queries allowed on the driver. 

Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.


Diffs
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91452
-----------------------------------------------------------



lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java (line 93)
<https://reviews.apache.org/r/36393/#comment144838>

    What is the transition to WAITING? Is it New -> waiting? If so, can we put it in the same order?
    
    I definitely think it is not between RUNNING to EXECUTED.


- Amareshwari Sriramadasu


On July 13, 2015, 9:48 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 9:48 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Lens will always accept a new query from a user and put it in a scheduling queue for processing.
> Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.
> 
> When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
> At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.
> 
> New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
> Waiting Queries will be persisted across server restarts.
> Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
> Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.
> 
> Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
> selected driver are less than max concurrent queries allowed on the driver.
> 
> Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 13, 2015, 10:08 a.m., Rajat Khandelwal wrote:
> > lens-api/src/main/java/org/apache/lens/api/Addable.java, line 32
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008960#file1008960line32>
> >
> >     would QueryCollection be a more suitable name?
> >     
> >     
> >     `new QueryCollection()` makes more sense than `new Queries()`. Since you're creating single object of a type.
> 
> Himanshu Gahlaut wrote:
>     I have thought about this as follows:
>     
>     added 1 query to waiting queries
>     added 1 query to waiting query collection
>     
>     Both are almost equivalent in english language and both are easy to understand.
>     
>     Used the first form because it allowed to have shorter variable names.
>     
>     waitingQueries.add(q1);
>     
>     instead of
>     
>     waitingQueryCollection.add(q1);
>     
>     Is this logic looking good to use queries throughout instead of query collection ?
> 
> Rajat Khandelwal wrote:
>     Even for java standard collections, we generally name objects like this:
>     
>     ```
>     Collection<Object> objects
>     List<String> strings
>     ```
>     etc.

EstimatedImmutableQueries would become EstimatedImmutableQueryCollection. That would be a lengthy name. I'm in to change Queries to QueryCollection, if we can have a shorter name for EstimatedImmutableQueryCollection.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91442
-----------------------------------------------------------


On July 13, 2015, 2:58 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 2:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Lens will always accept a new query from a user and put it in a scheduling queue for processing.
> Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.
> 
> When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
> At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.
> 
> New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
> Waiting Queries will be persisted across server restarts.
> Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
> Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.
> 
> Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
> selected driver are less than max concurrent queries allowed on the driver.
> 
> Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Rajat Khandelwal <ra...@gmail.com>.


> On July 13, 2015, 3:38 p.m., Rajat Khandelwal wrote:
> > lens-api/src/main/java/org/apache/lens/api/Addable.java, line 32
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008960#file1008960line32>
> >
> >     would QueryCollection be a more suitable name?
> >     
> >     
> >     `new QueryCollection()` makes more sense than `new Queries()`. Since you're creating single object of a type.
> 
> Himanshu Gahlaut wrote:
>     I have thought about this as follows:
>     
>     added 1 query to waiting queries
>     added 1 query to waiting query collection
>     
>     Both are almost equivalent in english language and both are easy to understand.
>     
>     Used the first form because it allowed to have shorter variable names.
>     
>     waitingQueries.add(q1);
>     
>     instead of
>     
>     waitingQueryCollection.add(q1);
>     
>     Is this logic looking good to use queries throughout instead of query collection ?

Even for java standard collections, we generally name objects like this:

```
Collection<Object> objects
List<String> strings
```
etc.


- Rajat


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91442
-----------------------------------------------------------


On July 13, 2015, 3:18 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 3:18 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Lens will always accept a new query from a user and put it in a scheduling queue for processing.
> Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.
> 
> When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
> At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.
> 
> New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
> Waiting Queries will be persisted across server restarts.
> Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
> Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.
> 
> Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
> selected driver are less than max concurrent queries allowed on the driver.
> 
> Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 13, 2015, 10:08 a.m., Rajat Khandelwal wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java, line 38
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008983#file1008983line38>
> >
> >     ```
> >     if(x)
> >      return true;
> >     else
> >      return false;
> >     ```
> >     
> >     can be replaced with:
> >     
> >     ```
> >     return x;
> >     ```

Addressed.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91442
-----------------------------------------------------------


On July 13, 2015, 7:13 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 7:13 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Modified
> 
> 
> LENS-619: Updated test cases
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Fixed test cases
> 
> 
> LENS-619: Fixed test cases and added comments
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Made poll timeout configurable
> 
> 
> LENS-619: Added Unit Test cases
> 
> 
> LENS-619: Added Debug Logs for production issues debugging
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 13, 2015, 10:08 a.m., Rajat Khandelwal wrote:
> > lens-api/src/main/java/org/apache/lens/api/Addable.java, line 32
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008960#file1008960line32>
> >
> >     would QueryCollection be a more suitable name?
> >     
> >     
> >     `new QueryCollection()` makes more sense than `new Queries()`. Since you're creating single object of a type.

I have thought about this as follows:

added 1 query to waiting queries
added 1 query to waiting query collection

Both are almost equivalent in english language and both are easy to understand.

Used the first form because it allowed to have shorter variable names.

waitingQueries.add(q1);

instead of

waitingQueryCollection.add(q1);

Is this logic looking good to use queries throughout instead of query collection ?


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91442
-----------------------------------------------------------


On July 13, 2015, 9:48 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 9:48 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Lens will always accept a new query from a user and put it in a scheduling queue for processing.
> Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.
> 
> When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
> At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.
> 
> New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
> Waiting Queries will be persisted across server restarts.
> Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
> Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.
> 
> Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
> selected driver are less than max concurrent queries allowed on the driver.
> 
> Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91442
-----------------------------------------------------------



lens-api/src/main/java/org/apache/lens/api/Addable.java (line 30)
<https://reviews.apache.org/r/36393/#comment144830>

    would QueryCollection be a more suitable name?
    
    `new QueryCollection()` makes more sense than `new Queries()`. Since you're creating single object of a type.



lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java (line 904)
<https://reviews.apache.org/r/36393/#comment144831>

    



lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java (line 38)
<https://reviews.apache.org/r/36393/#comment144834>

    ```
    if(x)
     return true;
    else
     return false;
    ```
    
    can be replaced with:
    
    ```
    return x;
    ```


- Rajat Khandelwal


On July 13, 2015, 3:18 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 3:18 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Lens will always accept a new query from a user and put it in a scheduling queue for processing.
> Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.
> 
> When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
> At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.
> 
> New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
> Waiting Queries will be persisted across server restarts.
> Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
> Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.
> 
> Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
> selected driver are less than max concurrent queries allowed on the driver.
> 
> Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.


> On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 96
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012488#file1012488line96>
> >
> >     Can the config name include launching?
> >     
> >     Can you document the new configs added in hivedriver-default.xml and generate apt files?
> >     
> >     For me, giving comma separated factories - does not look intuitive than to give implementations. But we can hold on that, until i see more code down.
> 
> Himanshu Gahlaut wrote:
>     Yes will include that. Have been missed in recent renaming.

Can you take care of name change and documenting the configs? I dont see them in latest patch uploaded. Reopening.


- Amareshwari


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91731
-----------------------------------------------------------


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 96
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012488#file1012488line96>
> >
> >     Can the config name include launching?
> >     
> >     Can you document the new configs added in hivedriver-default.xml and generate apt files?
> >     
> >     For me, giving comma separated factories - does not look intuitive than to give implementations. But we can hold on that, until i see more code down.
> 
> Himanshu Gahlaut wrote:
>     Yes will include that. Have been missed in recent renaming.
> 
> Amareshwari Sriramadasu wrote:
>     Can you take care of name change and documenting the configs? I dont see them in latest patch uploaded. Reopening.

I haven't uploaded any new diff for past two days. The last patch uploaded is 2 days and 1 hour old.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91731
-----------------------------------------------------------


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.


> On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 786
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012522#file1012522line786>
> >
> >     Should process next waiting query happen after following data structure updates?
> >     
> >     Also, can you check if updateFinishedQuery(ctx) is done with ctx  locked. If so, processing next waiting query should not require this lock. Might have to after the lock is given out.
> 
> Himanshu Gahlaut wrote:
>     While processing waiting queries, ctx is given as an input to waiting queries selector and selection policies. While Selection policies are reading information from ctx, other threads should not do any change of state of information. Hence ctx is locked during waiting queries selection.

ctx here is finsihed query. Why should finished query be locked for next waiting query selection? On the finished mostly no changes are done. With locking put on by the caller - the calls like cancel can take more time because same thread will be choosing the next query to launch. The user api threads should not be used for launching next query.


> On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java, line 350
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012502#file1012502line350>
> >
> >     Why are removing synchronized here?
> >     
> >     We want to make sure setStatus is not done by multiple threads through synchronizing on context
> 
> Himanshu Gahlaut wrote:
>     Is there any thread which is doing setStatus outside of synchronized(ctx) block ? If Yes, then we can keep it.

Keeping the method synchronized removes the double check of callers locking the object. If ctx is already locked, synchrozied here will be no-op.


> On July 15, 2015, 12:18 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java, line 123
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012498#file1012498line123>
> >
> >     Is this still emitted after removing WAITING state?
> 
> Himanshu Gahlaut wrote:
>     Yes. It wil help in looking at the graph of usage of waiting queries store.

I see it is implemented as guage on waiting queries. Seems fine.


- Amareshwari


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91731
-----------------------------------------------------------


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91731
-----------------------------------------------------------



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 96)
<https://reviews.apache.org/r/36393/#comment145321>

    Can the config name include launching?
    
    Can you document the new configs added in hivedriver-default.xml and generate apt files?
    
    For me, giving comma separated factories - does not look intuitive than to give implementations. But we can hold on that, until i see more code down.



lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml (line 224)
<https://reviews.apache.org/r/36393/#comment145322>

    This should just come from max pool size incase of JDBCDriver. These two values cannot be different.



lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java (line 123)
<https://reviews.apache.org/r/36393/#comment145323>

    Is this still emitted after removing WAITING state?



lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java (lines 91 - 97)
<https://reviews.apache.org/r/36393/#comment145324>

    Any reason for doing this change - materializing in context than reading from driverContext ?



lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (line 350)
<https://reviews.apache.org/r/36393/#comment145330>

    Why are removing synchronized here?
    
    We want to make sure setStatus is not done by multiple threads through synchronizing on context



lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (lines 432 - 437)
<https://reviews.apache.org/r/36393/#comment145325>

    Why are these required after driver selection?



lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java (line 30)
<https://reviews.apache.org/r/36393/#comment145326>

    What do we mean by estimate is not run yet?
    
    Does it enforce querycost is available with query? If so, can we put those details?



lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java (line 119)
<https://reviews.apache.org/r/36393/#comment145327>

    This change here has to be published as incompatible and provide an alter command for existing tables in production - as part of release.



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 638)
<https://reviews.apache.org/r/36393/#comment145329>

    I see that doing validation early is failing early wrt invalid trasitions. But how are we making sure the state is fine by this call ?
    
    The calls
    addSessionResourcesToDriver(query); query.getSelectedDriver().executeAsync(query);
    
    can be expensive.



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (lines 691 - 692)
<https://reviews.apache.org/r/36393/#comment145328>

    StatusPoller works on a copy for iterating it can move finished queries out of launchedQueries - and it should not result in ConcurrentModificationException for iteration.
    
    Not sure how it will work after this change put.



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 770)
<https://reviews.apache.org/r/36393/#comment145332>

    Should process next waiting query happen after following data structure updates?
    
    Also, can you check if updateFinishedQuery(ctx) is done with ctx  locked. If so, processing next waiting query should not require this lock. Might have to after the lock is given out.



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 854)
<https://reviews.apache.org/r/36393/#comment145333>

    The event time sent here is the when was query queued - which is submission time. not current time of event raising.


- Amareshwari Sriramadasu


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 2570
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012522#file1012522line2570>
> >
> >     Should we put the method in interface too?

https://reviews.apache.org/r/36393/diff/16?page=2#20

Already there in interface. Seems @Override is missing here.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92044
-----------------------------------------------------------


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml, line 210
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012491#file1012491line210>
> >
> >     similar changes in hive driver default. And generate apt files after that.

Duplicate comment. Apt file generation is already commented in some earlier comment.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java, line 51
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012497#file1012497line51>
> >
> >     We are assuming costs to be non null. Let's document that in the estimate api of driver.

Seems unrelated in context of this ticket.

MinQueryCostSelector has below line of code which ensures only non null values are passed to compareTo function. It is not dependent on Driver's estimate API returning non null.

final Collection<LensDriver> drivers = ctx.getDriverContext().getDriversWithValidQueryCost();


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java, line 50
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012502#file1012502line50>
> >
> >     it will include all members. Please just verify once whether all members is what we want in tostring. maybe some of them can be omitted.

I guess I added this for debugging. Have forgotten to remove it.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java, line 119
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012521#file1012521line119>
> >
> >     Is this launch time or submission time? We might need two fields.

This is launchtime. Submission time is already present as submissiontime.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java, line 98
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012523#file1012523line98>
> >
> >     `addAll` should consume `Collection<? extends QueryContext> queries`. All the way till the interfaces.  http://stackoverflow.com/questions/2723397/java-generics-what-is-pecs

There is no requirement to hold subtypes of QueryContext class.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java, line 46
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012524#file1012524line46>
> >
> >     Can probably take Collection here as well.

This is enforced as Set. After relaxing it, code would have to worry about duplicates.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java, line 73
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012526#file1012526line73>
> >
> >     selected driver selection policy. Does this mean Query Selection policy of the selected driver?

Yes.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java, line 50
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012527#file1012527line50>
> >
> >     `Collection<? extends QueryContext>`

Same as above. Restrict enforced to take a Set and no requirement to hold and worry about sub types in logic.


> On July 17, 2015, 9:55 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java, line 64
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012527#file1012527line64>
> >
> >     Collection

Duplicate comment


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92044
-----------------------------------------------------------


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92044
-----------------------------------------------------------



lens-api/src/main/resources/lens-errors.conf (line 40)
<https://reviews.apache.org/r/36393/#comment145932>

    Remove whitespace.



lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml (line 210)
<https://reviews.apache.org/r/36393/#comment145934>

    similar changes in hive driver default. And generate apt files after that.



lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java (line 50)
<https://reviews.apache.org/r/36393/#comment145937>

    We are assuming costs to be non null. Let's document that in the estimate api of driver.



lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 
<https://reviews.apache.org/r/36393/#comment145938>

    +1



lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (line 50)
<https://reviews.apache.org/r/36393/#comment145940>

    it will include all members. Please just verify once whether all members is what we want in tostring. maybe some of them can be omitted.



lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java (line 26)
<https://reviews.apache.org/r/36393/#comment145943>

    The class name is a verb, as opposed to noun, which is making it hard to comprehend.



lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java (line 29)
<https://reviews.apache.org/r/36393/#comment145944>

    Class name again verb. Less confusing than SelectDriverSpecificWaitingQueries though



lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java (line 28)
<https://reviews.apache.org/r/36393/#comment145945>

    Same as above.



lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java (line 33)
<https://reviews.apache.org/r/36393/#comment145948>

    name.



lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java (line 33)
<https://reviews.apache.org/r/36393/#comment145949>

    name.



lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java (line 119)
<https://reviews.apache.org/r/36393/#comment145951>

    Is this launch time or submission time? We might need two fields.



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 2549)
<https://reviews.apache.org/r/36393/#comment145953>

    Should we put the method in interface too?



lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java (line 98)
<https://reviews.apache.org/r/36393/#comment145955>

    `addAll` should consume `Collection<? extends QueryContext> queries`. All the way till the interfaces.  http://stackoverflow.com/questions/2723397/java-generics-what-is-pecs



lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java (line 117)
<https://reviews.apache.org/r/36393/#comment145956>

    same thing as `addAll`



lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java (line 46)
<https://reviews.apache.org/r/36393/#comment145957>

    Can probably take Collection here as well.



lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java (line 77)
<https://reviews.apache.org/r/36393/#comment145958>

    `@Override`?



lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java (line 69)
<https://reviews.apache.org/r/36393/#comment145959>

    +1



lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java (line 73)
<https://reviews.apache.org/r/36393/#comment145960>

    selected driver selection policy. Does this mean Query Selection policy of the selected driver?



lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java (line 50)
<https://reviews.apache.org/r/36393/#comment145961>

    `Collection<? extends QueryContext>`



lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java (line 64)
<https://reviews.apache.org/r/36393/#comment145962>

    Collection



lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java (line 27)
<https://reviews.apache.org/r/36393/#comment145963>

    name should be noun.



lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java (line 27)
<https://reviews.apache.org/r/36393/#comment145964>

    name.



lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java (line 70)
<https://reviews.apache.org/r/36393/#comment145966>

    Can probably use `Sets.union` here. since the if-else block is just avoiding unnecessary copying.



lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java (line 30)
<https://reviews.apache.org/r/36393/#comment145967>

    name noun.



lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java (line 30)
<https://reviews.apache.org/r/36393/#comment145968>

    class name should be noun.



lens-server/src/main/resources/lensserver-default.xml (line 719)
<https://reviews.apache.org/r/36393/#comment145970>

    Can we change the name to something that implies that this param is only useful when one particular factory <insert-factory-name> is one of the constraint factories.



lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java (line 34)
<https://reviews.apache.org/r/36393/#comment145971>

    class name.



lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java (line 34)
<https://reviews.apache.org/r/36393/#comment145972>

    class name.


- Rajat Khandelwal


On July 15, 2015, 2:25 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 2:25 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 17, 2015, 10:10 a.m., Amareshwari Sriramadasu wrote:
> > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java, line 98
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012490#file1012490line98>
> >
> >     Change name to query.launching.constraint. ?

Duplicate comment.


> On July 17, 2015, 10:10 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java, line 903
> > <https://reviews.apache.org/r/36393/diff/16/?file=1012493#file1012493line903>
> >
> >     Name change to query.launching.constraint. ?

Duplicate comment.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92057
-----------------------------------------------------------


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92057
-----------------------------------------------------------



lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java (line 98)
<https://reviews.apache.org/r/36393/#comment145974>

    Change name to query.launching.constraint. ?



lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml (line 213)
<https://reviews.apache.org/r/36393/#comment145977>

    Can you update documentation about underlying interfaces to implement?



lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java (line 903)
<https://reviews.apache.org/r/36393/#comment145975>

    Name change to query.launching.constraint. ?



lens-server/src/test/resources/log4j.properties (line 21)
<https://reviews.apache.org/r/36393/#comment145979>

    INFO logging makes failed tests debugging better. We have logs and STDOUT redirected to files, Is INFO logging still a concern?


- Amareshwari Sriramadasu


On July 15, 2015, 8:55 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 15, 2015, 8:55 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching Constraints before allowing a query to be launched
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 17, 2015, 12:47 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml, line 229
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014161#file1014161line229>
> >
> >     Shall we mention value should be same as max pool size ?

Will mention it.


> On July 17, 2015, 12:47 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/resources/hivedriver-default.xml, line 160
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014158#file1014158line160>
> >
> >     Shall we mention Factory should create which interface's implementation?
> >     
> >     Applies to all the configs added.

will mention it.


> On July 17, 2015, 12:47 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java, line 208
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014171#file1014171line208>
> >
> >     We may not have selectedDriver set, if query is failed to be launched. Is that taken care?

Yes it is taken care of. Waiting query selection only happens when the method removing a query from launched queries returns true. A query which is removed from launched queries always has selected driver set.


> On July 17, 2015, 12:47 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 276
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014192#file1014192line276>
> >
> >     Is this call starting thread also? Right now, we are starting all threads in start()

No it doesn't start a thread.


> On July 17, 2015, 12:47 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 2781
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014192#file1014192line2781>
> >
> >     Can we add a metric for error?

Currently I don't think there will be use cases for adding any monitoring or alerts on this OR there is a need of visualizing trends in errors in waiting queries processing. If there is an issue in waiting queries processing, then a grep on logs will be good enough to count and see the cause of errors as a one time operation.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92073
-----------------------------------------------------------


On July 17, 2015, 12:31 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 12:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> Testing not done.
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 17, 2015, 12:47 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/resources/lensserver-default.xml, line 731
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014209#file1014209line731>
> >
> >     Whats rationale for setting the value to be 90.0 ?
> >     
> >     Can the default value be max enough for not changing the current default behavior of running queries?

Will change the default value such that it translates to no user limits.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92073
-----------------------------------------------------------


On July 17, 2015, 2:34 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 2:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> Testing not done.
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92073
-----------------------------------------------------------



lens-driver-hive/src/main/resources/hivedriver-default.xml (line 160)
<https://reviews.apache.org/r/36393/#comment145998>

    Shall we mention Factory should create which interface's implementation?
    
    Applies to all the configs added.



lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml (line 229)
<https://reviews.apache.org/r/36393/#comment145999>

    Shall we mention value should be same as max pool size ?



lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java (line 208)
<https://reviews.apache.org/r/36393/#comment146000>

    We may not have selectedDriver set, if query is failed to be launched. Is that taken care?



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 276)
<https://reviews.apache.org/r/36393/#comment146004>

    Is this call starting thread also? Right now, we are starting all threads in start()



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 2760)
<https://reviews.apache.org/r/36393/#comment146003>

    Can we add a metric for error?



lens-server/src/main/resources/lensserver-default.xml (line 731)
<https://reviews.apache.org/r/36393/#comment146005>

    Whats rationale for setting the value to be 90.0 ?
    
    Can the default value be max enough for not changing the current default behavior of running queries?


- Amareshwari Sriramadasu


On July 17, 2015, 12:31 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 12:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> Testing not done.
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92158
-----------------------------------------------------------


Changes look fine to me. I mainly looked at changes to existing files and overall flow. 

Can you update testing done with the patch?

- Amareshwari Sriramadasu


On July 17, 2015, 3:32 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 3:32 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactoryTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> Testing not done.
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 17, 2015, 3:32 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description
-------

LENS-619: Applying Query Launching constraints before launching a new query


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactoryTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
  src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
  src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
  src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
  src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------

Testing not done.


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 17, 2015, 2:34 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description
-------

LENS-619: Applying Query Launching constraints before launching a new query


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
  src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
  src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
  src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
  src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------

Testing not done.


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 17, 2015, 12:39 p.m., Rajat Khandelwal wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java, line 224
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014154#file1014154line224>
> >
> >     Should we make other methods as well? like QueryStatus?

Not required for now.


> On July 17, 2015, 12:39 p.m., Rajat Khandelwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, lines 781-790
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014157#file1014157line781>
> >
> >     Can probably be replaced by just getters

getQueryConstraints  can be replaced


> On July 17, 2015, 12:39 p.m., Rajat Khandelwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, lines 781-790
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014157#file1014157line781>
> >
> >

getQueryConstraints  can be replaced


> On July 17, 2015, 12:39 p.m., Rajat Khandelwal wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java, line 31
> > <https://reviews.apache.org/r/36393/diff/17/?file=1014180#file1014180line31>
> >
> >     Can we change the config name to indicate it's specific to this class of constraints
> >     
> >     e.g. `driver.launching.constraint.max.concurrent.queries`

will rename to driver.max.concurrent.launched.queries


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92071
-----------------------------------------------------------


On July 17, 2015, 12:31 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 12:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> Testing not done.
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Rajat Khandelwal <ra...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92071
-----------------------------------------------------------



lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java (line 224)
<https://reviews.apache.org/r/36393/#comment145994>

    Should we make other methods as well? like QueryStatus?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (lines 781 - 790)
<https://reviews.apache.org/r/36393/#comment145995>

    Can probably be replaced by just getters



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (lines 781 - 790)
<https://reviews.apache.org/r/36393/#comment145996>

    



lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java (line 1116)
<https://reviews.apache.org/r/36393/#comment145997>

    null? What about `this.selectionPolicies`?



lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java (line 38)
<https://reviews.apache.org/r/36393/#comment146001>

    signature change is required?



lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java (line 31)
<https://reviews.apache.org/r/36393/#comment146002>

    Can we change the config name to indicate it's specific to this class of constraints
    
    e.g. `driver.launching.constraint.max.concurrent.queries`


- Rajat Khandelwal


On July 17, 2015, 6:01 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 6:01 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> Testing not done.
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 17, 2015, 12:31 p.m.)


Review request for lens.


Changes
-------

Apt files.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description
-------

LENS-619: Applying Query Launching constraints before launching a new query


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
  src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
  src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
  src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
  src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 

Diff: https://reviews.apache.org/r/36393/diff/


Testing (updated)
-------

Testing not done.


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review92067
-----------------------------------------------------------


will be pushing apt files after testing.

- Himanshu Gahlaut


On July 17, 2015, 11:07 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 17, 2015, 11:07 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Applying Query Launching constraints before launching a new query
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
>   src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
>   src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
>   src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
>   src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> All Test cases passed:
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
> [INFO] Lens .............................................. SUCCESS [4.091s]
> [INFO] Lens API .......................................... SUCCESS [29.290s]
> [INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
> [INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
> [INFO] Lens DB storage ................................... SUCCESS [26.042s]
> [INFO] Lens Query Library ................................ SUCCESS [21.532s]
> [INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
> [INFO] Lens Server ....................................... SUCCESS [23:41.881s]
> [INFO] Lens client ....................................... SUCCESS [1:04.296s]
> [INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
> [INFO] Lens Examples ..................................... SUCCESS [15.041s]
> [INFO] Lens Distribution ................................. SUCCESS [21.181s]
> [INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
> [INFO] Lens Regression ................................... SUCCESS [17.566s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 45:57.904s
> [INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
> [INFO] Final Memory: 120M/324M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Launching constraints before launching a new query

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 17, 2015, 11:07 a.m.)


Review request for lens.


Changes
-------

Addressed review comments. Testing not done.


Summary (updated)
-----------------

LENS-619: Applying Query Launching constraints before launching a new query


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Applying Query Launching constraints before launching a new query


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-hive/src/main/resources/hivedriver-default.xml 85a3f89deba44a9ca5ef4ffd41074b1405d846b6 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e048a78a08b29045f86e69f0180ab04f50705315 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java eab42d095ced23b9fd5b1fb854043d9668307883 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 1334d9a96d90f17788367b123451d9c3dcb70824 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 35112367b1b40309c04e6748f7ae69ceac959a28 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml 2b540b828a976378bdfbd238e8427a22941fb7e8 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
  src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 
  src/site/apt/admin/hivedriver-config.apt b668e160f4fd1ce05179bb50c7dc6264ea6101e2 
  src/site/apt/admin/jdbcdriver-config.apt b3d25f6751dd030a4ae818845a4a504ca41d7895 
  src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------

All Test cases passed:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
[INFO] Lens .............................................. SUCCESS [4.091s]
[INFO] Lens API .......................................... SUCCESS [29.290s]
[INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
[INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
[INFO] Lens DB storage ................................... SUCCESS [26.042s]
[INFO] Lens Query Library ................................ SUCCESS [21.532s]
[INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
[INFO] Lens Server ....................................... SUCCESS [23:41.881s]
[INFO] Lens client ....................................... SUCCESS [1:04.296s]
[INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
[INFO] Lens Examples ..................................... SUCCESS [15.041s]
[INFO] Lens Distribution ................................. SUCCESS [21.181s]
[INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
[INFO] Lens Regression ................................... SUCCESS [17.566s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 45:57.904s
[INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
[INFO] Final Memory: 120M/324M
[INFO] ------------------------------------------------------------------------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 15, 2015, 8:55 a.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Applying Query Launching Constraints before allowing a query to be launched


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
  src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d 

Diff: https://reviews.apache.org/r/36393/diff/


Testing (updated)
-------

All Test cases passed:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.514s]
[INFO] Lens .............................................. SUCCESS [4.091s]
[INFO] Lens API .......................................... SUCCESS [29.290s]
[INFO] Lens API for server and extensions ................ SUCCESS [30.638s]
[INFO] Lens Cube ......................................... SUCCESS [5:35.451s]
[INFO] Lens DB storage ................................... SUCCESS [26.042s]
[INFO] Lens Query Library ................................ SUCCESS [21.532s]
[INFO] Lens Hive Driver .................................. SUCCESS [3:37.060s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [49.002s]
[INFO] Lens Server ....................................... SUCCESS [23:41.881s]
[INFO] Lens client ....................................... SUCCESS [1:04.296s]
[INFO] Lens CLI .......................................... SUCCESS [4:34.347s]
[INFO] Lens Examples ..................................... SUCCESS [15.041s]
[INFO] Lens Distribution ................................. SUCCESS [21.181s]
[INFO] Lens ML Lib ....................................... SUCCESS [3:33.233s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [12.803s]
[INFO] Lens Regression ................................... SUCCESS [17.566s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 45:57.904s
[INFO] Finished at: Wed Jul 15 14:09:04 IST 2015
[INFO] Final Memory: 120M/324M
[INFO] ------------------------------------------------------------------------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 15, 2015, 7:43 a.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


LENS-619: Added Unit Test cases


LENS-619: Added Debug Logs for production issues debugging


LENS-619: Fixed some test cases


LENS-619: Addressing comments


LENS-619: Removed unused import


LENS-619: Removed WAITING state


LENS-619: Fixed a breaking test case


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java a778a1ff0b166a6e465bdc711133061c45968b33 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 14, 2015, 12:34 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


LENS-619: Added Unit Test cases


LENS-619: Added Debug Logs for production issues debugging


LENS-619: Fixed some test cases


LENS-619: Addressing comments


LENS-619: Removed unused import


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java 06343c2f3ba0d127cae6a7dd1ca8422e3c2fbc40 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 14, 2015, 12:21 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


LENS-619: Added Unit Test cases


LENS-619: Added Debug Logs for production issues debugging


LENS-619: Fixed some test cases


LENS-619: Addressing comments


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java 06343c2f3ba0d127cae6a7dd1ca8422e3c2fbc40 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 14, 2015, 10:59 a.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


LENS-619: Added Unit Test cases


LENS-619: Added Debug Logs for production issues debugging


LENS-619: Fixed some test cases


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java 06343c2f3ba0d127cae6a7dd1ca8422e3c2fbc40 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91603
-----------------------------------------------------------



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 50)
<https://reviews.apache.org/r/36393/#comment145092>

    Shall the package be 
    org.apache.lens.server.api.query.policy.selection?
    
    org.apache.lens.server.api.query.wait - did not seem intuitive to me.



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 96)
<https://reviews.apache.org/r/36393/#comment145093>

    Is it loading factories or factory?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 98)
<https://reviews.apache.org/r/36393/#comment145095>

    Again is it factories or factory?
    
    Are there any factories for waiting query selection policy?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 141)
<https://reviews.apache.org/r/36393/#comment145096>

    Whay is QueryConstraint here is it a Launch Constraint ?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 142)
<https://reviews.apache.org/r/36393/#comment145097>

    Should it be ImmutableSet<SelectionPolicy> instead of ImmutableSet<WaitingQueriesSelectionPolicy>  ?


- Amareshwari Sriramadasu


On July 13, 2015, 7:13 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 7:13 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Modified
> 
> 
> LENS-619: Updated test cases
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Fixed test cases
> 
> 
> LENS-619: Fixed test cases and added comments
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Made poll timeout configurable
> 
> 
> LENS-619: Added Unit Test cases
> 
> 
> LENS-619: Added Debug Logs for production issues debugging
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 13, 2015, 7:13 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


LENS-619: Added Unit Test cases


LENS-619: Added Debug Logs for production issues debugging


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 13, 2015, 6:35 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


LENS-619: Added Unit Test cases


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 13, 2015, 2:58 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

Lens will always accept a new query from a user and put it in a scheduling queue for processing.
Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.

When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.

New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
Waiting Queries will be persisted across server restarts.
Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.

Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
selected driver are less than max concurrent queries allowed on the driver.

Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.


Diffs
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 13, 2015, 2:57 p.m.)


Review request for lens.


Changes
-------

Lens will always accept a new query from a user and put it in a scheduling queue for processing.
Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.

When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.

New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
Waiting Queries will be persisted across server restarts.
Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.

Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
selected driver are less than max concurrent queries allowed on the driver.

Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


LENS-619: Made poll timeout configurable


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.


> On July 13, 2015, 2:51 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, lines 225-229
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008962#file1008962line225>
> >
> >     Seems QUEUED -> WAITING and WAITING->QUEUED are both allowed transitions.
> >     
> >     Can we put when does those state changes happen?
> >     
> >     Also, can we update the doc src/site/apt/user/index.apt wrt this change?
> 
> Amareshwari Sriramadasu wrote:
>     Is state change from QUEUED->WAITING->QUEUED required?. Can we just live with QUEUED?

Currently WAITING state is being used to restore waiting queries in correct collection during a server restart. If we drop this state then during server restart all waiting queries will be added to queued queries priority queue. That is equivalent to making all waiting queries eligible for scheduling after server restart. This can be expensive if there are lot of queries. However all queries are only becoming eligible for scheduling together only once during server restart, so it should be tolerable. In light of this use case of WAITING state, let me know if anyone predicts any other issues. Currently I don't see any issues and if there are no issues foreseen until end of this review request, then will drop WAITING state.


- Himanshu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91469
-----------------------------------------------------------


On July 14, 2015, 12:34 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 14, 2015, 12:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Modified
> 
> 
> LENS-619: Updated test cases
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Fixed test cases
> 
> 
> LENS-619: Fixed test cases and added comments
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Made poll timeout configurable
> 
> 
> LENS-619: Added Unit Test cases
> 
> 
> LENS-619: Added Debug Logs for production issues debugging
> 
> 
> LENS-619: Fixed some test cases
> 
> 
> LENS-619: Addressing comments
> 
> 
> LENS-619: Removed unused import
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java 06343c2f3ba0d127cae6a7dd1ca8422e3c2fbc40 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.


> On July 13, 2015, 2:51 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, lines 225-229
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008962#file1008962line225>
> >
> >     Seems QUEUED -> WAITING and WAITING->QUEUED are both allowed transitions.
> >     
> >     Can we put when does those state changes happen?
> >     
> >     Also, can we update the doc src/site/apt/user/index.apt wrt this change?
> 
> Amareshwari Sriramadasu wrote:
>     Is state change from QUEUED->WAITING->QUEUED required?. Can we just live with QUEUED?
> 
> Himanshu Gahlaut wrote:
>     Currently WAITING state is being used to restore waiting queries in correct collection during a server restart. If we drop this state then during server restart all waiting queries will be added to queued queries priority queue. That is equivalent to making all waiting queries eligible for scheduling after server restart. This can be expensive if there are lot of queries. However all queries are only becoming eligible for scheduling together only once during server restart, so it should be tolerable. In light of this use case of WAITING state, let me know if anyone predicts any other issues. Currently I don't see any issues and if there are no issues foreseen until end of this review request, then will drop WAITING state.

I would say restoring waiting queries upon restore as the same collection can be droppped. I agree with "However all queries are only becoming eligible for scheduling together only once during server restart, so it should be tolerable.". Also after restart, the launching constraints can be changed which should change the waiting queries collection. So, it makes sense to build the collection afresh after restart with newer constraints.


- Amareshwari


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91469
-----------------------------------------------------------


On July 14, 2015, 12:34 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 14, 2015, 12:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Modified
> 
> 
> LENS-619: Updated test cases
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Fixed test cases
> 
> 
> LENS-619: Fixed test cases and added comments
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Made poll timeout configurable
> 
> 
> LENS-619: Added Unit Test cases
> 
> 
> LENS-619: Added Debug Logs for production issues debugging
> 
> 
> LENS-619: Fixed some test cases
> 
> 
> LENS-619: Addressing comments
> 
> 
> LENS-619: Removed unused import
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java 06343c2f3ba0d127cae6a7dd1ca8422e3c2fbc40 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.


> On July 13, 2015, 2:51 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, lines 225-229
> > <https://reviews.apache.org/r/36393/diff/8/?file=1008962#file1008962line225>
> >
> >     Seems QUEUED -> WAITING and WAITING->QUEUED are both allowed transitions.
> >     
> >     Can we put when does those state changes happen?
> >     
> >     Also, can we update the doc src/site/apt/user/index.apt wrt this change?

Is state change from QUEUED->WAITING->QUEUED required?. Can we just live with QUEUED?


- Amareshwari


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91469
-----------------------------------------------------------


On July 14, 2015, 12:34 p.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 14, 2015, 12:34 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> LENS-619: Modified
> 
> 
> LENS-619: Updated test cases
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Fixed test cases
> 
> 
> LENS-619: Fixed test cases and added comments
> 
> 
> LENS-619: More updates
> 
> 
> LENS-619: Made poll timeout configurable
> 
> 
> LENS-619: Added Unit Test cases
> 
> 
> LENS-619: Added Debug Logs for production issues debugging
> 
> 
> LENS-619: Fixed some test cases
> 
> 
> LENS-619: Addressing comments
> 
> 
> LENS-619: Removed unused import
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java 06343c2f3ba0d127cae6a7dd1ca8422e3c2fbc40 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-api/src/main/resources/lens-errors.conf 6130ad78c2871f78ff9bc11cca5702a538639bbf 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverSelector.java dea0dfb50657b60fc71a9cb263266dec1dfbdf68 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Amareshwari Sriramadasu <am...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/#review91469
-----------------------------------------------------------



lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java (lines 225 - 229)
<https://reviews.apache.org/r/36393/#comment144847>

    Seems QUEUED -> WAITING and WAITING->QUEUED are both allowed transitions.
    
    Can we put when does those state changes happen?
    
    Also, can we update the doc src/site/apt/user/index.apt wrt this change?


- Amareshwari Sriramadasu


On July 13, 2015, 9:48 a.m., Himanshu Gahlaut wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36393/
> -----------------------------------------------------------
> 
> (Updated July 13, 2015, 9:48 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-619
>     https://issues.apache.org/jira/browse/LENS-619
> 
> 
> Repository: lens
> 
> 
> Description
> -------
> 
> Lens will always accept a new query from a user and put it in a scheduling queue for processing.
> Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.
> 
> When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
> At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.
> 
> New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
> Waiting Queries will be persisted across server restarts.
> Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
> Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.
> 
> Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
> selected driver are less than max concurrent queries allowed on the driver.
> 
> Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.
> 
> 
> Diffs
> -----
> 
>   lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
>   lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
>   lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
>   lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
>   lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
>   lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
>   lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
>   lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
>   lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
>   lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
>   lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
>   lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
>   lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
>   lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 
> 
> Diff: https://reviews.apache.org/r/36393/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Himanshu Gahlaut
> 
>

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 13, 2015, 9:48 a.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

Lens will always accept a new query from a user and put it in a scheduling queue for processing.
Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.

When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.
At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.

New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.
Waiting Queries will be persisted across server restarts.
Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than a cumulative query cost ceiling for launching a new query.
Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.

Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the
selected driver are less than max concurrent queries allowed on the driver.

Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.


Diffs
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 11, 2015, 9:06 a.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


LENS-619: More updates


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java c3fe8eb142569c0b254f551c3cbd18502b6eac99 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java 0cccaef5abf391179b04bca0c0bbe799429f9065 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 10:03 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


LENS-619: Fixed test cases and added comments


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 8:29 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


LENS-619: Fixed test cases


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java f594133e8a75603e5bb0c913d62ea189248cfa05 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java f7305fb288167cda088e263a62c77f264d779127 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 3 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


LENS-619: More updates


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 2:12 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


LENS-619: More updates


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 1:39 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


LENS-619: Updated test cases


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 1:26 p.m.)


Review request for lens.


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description (updated)
-------

LENS-619: Modified


Diffs (updated)
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut

Re: Review Request 36393: LENS-619: Applying Query Constraints before allowing a query to be launched

Posted by Himanshu Gahlaut <ga...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36393/
-----------------------------------------------------------

(Updated July 10, 2015, 12:15 p.m.)


Review request for lens.


Summary (updated)
-----------------

LENS-619: Applying Query Constraints before allowing a query to be launched


Bugs: LENS-619
    https://issues.apache.org/jira/browse/LENS-619


Repository: lens


Description
-------

LENS-619: Applying Query Constraints before allowing a query to be launched

Lens will always accept a new query from a user and put it in a scheduling queue for processing.

Next candidate query picked up from scheduling queue for processing will be launched only if all query constraints evaluated on candidate query and launched queries allows the candidate query to be launched, otherwise the candidate query will be added to waiting queries.

When any launched query is finished, a waiting query selector will select waiting queries for rescheduling using a list of waiting queries selection policy. Every waiting query selection policy will return a list of eligible waiting queries to be rescheduled. Query Selector will calculate intersection of multiple list of eligible waiting queries and add the result of intersection to scheduling queue for reprocessing.

At initialization, Query Constraints and Waiting Query Selection Policies will be configured using configuration values.

New Query Constraints and Waiting Query Selection Policies can be added at runtime, without rebuilding and deploying lens module. Drivers should be allowed to add more Query Constraints and Waiting Queries Selection policies.

Waiting Queries will be persisted across server restarts.

Query Constraint 1: Allow a candidate query to be launched, only if, cumulative query cost of all current queries launched by the user, who submitted candidate query, is less than  a cumulative query cost ceiling for launching a new query. 

Waiting Query Selection Policy 1: Select all waiting queries of the user whose query has finished.

Query Constraint 2: Allow a candidate query to be launched, only if, current concurrent queries launched on the selected driver are less than max concurrent queries allowed on the driver. 

Waiting Query Selection Policy 2: Select all waiting queries of the driver whose query has finished.


Diffs
-----

  lens-api/src/main/java/org/apache/lens/api/Addable.java 5a556918afcdb8967deee84472cd7863aeafc4fc 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 539f0b0b0804257fae6b7ec0af8f2f715175d597 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java e87ed23429d3172db86c17e4ec5e37e2e903374b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 0f2b7401b4057b3a93aa5a13066972bc1975142e 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 5abd040c46cc6a3b81c89ef9ccdfc70dc09c2fae 
  lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml 99f6f205dfe6f1c16277ac6624106436df2b2b8f 
  lens-server-api/pom.xml 26ac76373f3c5c56d144d7f9baec43819c6efce7 
  lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java ad27d78e1839fa60df8f2429901b77221be71653 
  lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java b56c67ce6909a11b1cdec9f52427683f0b11dd41 
  lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java 71128b7d35e91fadf38a9c3926ce46d42369ead4 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java 6e7ab5dc1c78407a24e389c001ca3100563e3795 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAddedToWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d72727e7832d02ed2ea90379cd29c7a8e509a8f8 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java 5d7163f112d7c7115ebf8f012ba397bfee63adbc 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryConstraint.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java 7fa5cdb55959a44b52a29884fd629c2479fdfb96 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java be7232134b59f7126edba569358389a9c26a5c74 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueries.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/query/wait/WaitingQueriesSelectionPolicy.java PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java f9262994debe0b943123d13c58a22c10b2015e80 
  lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java b52b08ad78a4c74c61376fb3c6bc6284f38b0de9 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/RestrictConcurrentDriverQueriesTest.java PRE-CREATION 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 
  lens-server-api/src/test/java/org/apache/lens/server/api/query/wait/SelectDriverSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/pom.xml b85292cb21e6cd4681a1f6b2b7ee1367cd2dc60a 
  lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java 39f9f5216acf6e1be7cca6a0851152a78ba66176 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 190435036c5cf8f009d7fbd536ef1743b70c22d9 
  lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 8a322027905edef9403c45fb13cb985e7e4a2f59 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/EstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryConstraintsChecker.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUser.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/constraint/RestrictTotalQueryCostPerUserFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueries.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesFactory.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/query/wait/WaitingQueriesSelector.java PRE-CREATION 
  lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java 715347fab82b5d9d4ad673c45e72900b9a95fc81 
  lens-server/src/main/resources/lensserver-default.xml da00c0d2d94ef982646ad10f142165abe9125d3a 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueriesTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryConstraintsCheckerTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/IntersectingWaitingQueriesSelectorTest.java PRE-CREATION 
  lens-server/src/test/java/org/apache/lens/server/query/wait/SelectUserSpecificWaitingQueriesTest.java PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml 3bcbf0e3e44d986a24c00cad84121be72596c64d 
  lens-server/src/test/resources/log4j.properties e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b 

Diff: https://reviews.apache.org/r/36393/diff/


Testing
-------


Thanks,

Himanshu Gahlaut