Limiting number of parallel queries per user

[deansg <de...@gmail.com>]

Hello, 
My team is trying to write a SearchComponent that will limit the amount of
queries a certain user can run in parallel at any given moment. We want to
do this to avoid a certain user from slowing Solr down to much.

 In the search component, we can identify the user sending the request, and
we keep a static map containing data on how many queries the user is
currently running. In the prepare method we increment the entry in the map,
and assert that it is not too big. In finishStage we decrement it.

However, when i tried testing my component by running several queries in
parallel from a single user, it seemed like Solr somehow runs the queries
serially and not in parallel. Every time my component's code is executed it
believes the user isnt running any other queries as finish Stage had been
called on the previous query before prepare is being called on the new
query. What do you think? Is writing such a component a bad idea? Am I
missing something?



--
View this message in context: http://lucene.472066.n3.nabble.com/Limiting-number-of-parallel-queries-per-user-tp4240566.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Limiting number of parallel queries per user

[deansg <de...@gmail.com>]

Just an update: my problem turned out to be that in the search-component, I
decremented the entry for the user running a query in the first call to
finishStage, and didn't realize that most of the query processing and time
occurs only in later stages. 
Because the entry was decremented so quickly, the logs made it seem like
Solr is running the request serially, when it was just that I was sending
queries more slowly than my SearchComponent was processing them.
The search component now works and does limit the amount of queries a user
runs in parallel (which is a benefit in our specific case).



--
View this message in context: http://lucene.472066.n3.nabble.com/Limiting-number-of-parallel-queries-per-user-tp4240566p4240851.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Limiting number of parallel queries per user

[Erick Erickson <er...@gmail.com>]

This will be hard to do in SolrCloud assuming that the entire cluster
is fronted by a load balancer _or_ you're using CloudSolrClient
(CloudSolrServer in 4x) because the node that gets the highest-level
is distributed against all the Solr nodes.

I'd probably go more simply (assuming the above is not a problem).
I'd create a first-component that checks whether to let the query
continue. If so, increment your counter.

Then a last-component to decrement the counter for that user.

As far as the serial processing is concerned, sounds more like
your client is somehow serializing that. Solr should be handling
lots of queries simultaneously.

Best,
Erick

On Tue, Nov 17, 2015 at 6:51 AM, deansg <de...@gmail.com> wrote:
> Hello,
> My team is trying to write a SearchComponent that will limit the amount of
> queries a certain user can run in parallel at any given moment. We want to
> do this to avoid a certain user from slowing Solr down to much.
>
>  In the search component, we can identify the user sending the request, and
> we keep a static map containing data on how many queries the user is
> currently running. In the prepare method we increment the entry in the map,
> and assert that it is not too big. In finishStage we decrement it.
>
> However, when i tried testing my component by running several queries in
> parallel from a single user, it seemed like Solr somehow runs the queries
> serially and not in parallel. Every time my component's code is executed it
> believes the user isnt running any other queries as finish Stage had been
> called on the previous query before prepare is being called on the new
> query. What do you think? Is writing such a component a bad idea? Am I
> missing something?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Limiting-number-of-parallel-queries-per-user-tp4240566.html
> Sent from the Solr - User mailing list archive at Nabble.com.