[jira] [Commented] (LENS-619) Applying Query Running Constraint before allowing a query to run

["Jaideep Dhok (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LENS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598879#comment-14598879 ] 

Jaideep Dhok commented on LENS-619:
-----------------------------------

Hi [~himanshu.gahlaut] Can you put up a design proposal? Also, we need to break this into subtasks.

I agree with Amareshwari that policy should be pluggable.

Regarding the proposal -
# Is the 'wait queue' persisted across restarts? Is it bounded?
# All N queries for a user may have different resource requirements. Rather than limiting the 'number' of queries we should limit based on the resources consumed by a user across different drivers. One option is to use cumulative query cost of running queries as a way to limit queries of a user. Since query cost is supposed to give a hint about resource usage of the query, it can be used here.
# Is it N queries across all drivers? or N per driver

> Applying Query Running Constraint before allowing a query to run
> ----------------------------------------------------------------
>
>                 Key: LENS-619
>                 URL: https://issues.apache.org/jira/browse/LENS-619
>             Project: Apache Lens
>          Issue Type: New Feature
>            Reporter: Himanshu Gahlaut
>            Assignee: Himanshu Gahlaut
>
> [June 23. 2015]: 
> This is the refined model after knowledge crunching:
> Lens will always accept a new query from a user and put it in a priority queue for processing.
> Next candidate query picked up from priority queue for processing will be run only if a query running constraint evaluated on running queries and candidate query allows the candidate query to be run, otherwise the candidate query will be moved to waiting queries list.
> When any running query is finished, result of evaluation of query constraint might change, waiting queries might become eligible to be run, hence waiting queries will be moved back to priority queue to be re-processed.
> [Initial thought]: Fair Usage Policy for using Lens
> After a user has used the lens system beyond fair usage policy, he will get lens features in degraded mode. 
> One of the forms of fair usage policy could be that at any point in time only N queries can be present in RUNNING state for a user. 
> If N queries of a user are in RUNNING state, then a new query submitted by same user will stay in QUEUED state until one of the N queries in RUNNING state have moved to SUCCESSFUL, FAILED, OR CANCELLED state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)