[jira] [Comment Edited] (SOLR-13578) Implement a generic Resource Manager for monitoring and controlling limited resources

["David Wayne Smiley (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881574#comment-16881574 ] 

David Wayne Smiley edited comment on SOLR-13578 at 9/26/19 8:01 PM:
--------------------------------------------------------------------

This looks useful.  For example I believe [~mkhl]] was looking at BackupRepository impls that might want a shared thread pool, or something like that.  I've also worked on customer specific stuff involving shared connection pools to other systems (e.g. some DB or whatever).


was (Author: dsmiley):
This looks useful.  For example I believe [~mkhludnev] was looking at BackupRepository impls that might want a shared thread pool, or something like that.  I've also worked on customer specific stuff involving shared connection pools to other systems (e.g. some DB or whatever).

> Implement a generic Resource Manager for monitoring and controlling limited resources
> -------------------------------------------------------------------------------------
>
>                 Key: SOLR-13578
>                 URL: https://issues.apache.org/jira/browse/SOLR-13578
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Andrzej Bialecki
>            Assignee: Andrzej Bialecki
>            Priority: Major
>
> Many common resources such as CPUs, threads, file descriptors, heap, etc. are shared between multiple SolrCore-s within a CoreContainer.
> Most of these resources can already be monitored for usage using metrics. However, in most cases Solr doesn't have any control mechanism to actually do something about excessive use (or extreme under-utilization) of a resource by any particular SolrCore or CoreContainer. Furthermore, even when a control mechanism exists it's usually available only as a static configuration parameter (eg. max cache size) and changing it requires at least a core reload, or restarting the JVM.
> This issue is especially important for multi-tenant applications where the admin cannot assume voluntary co-operation of users and needs more fine-grained tools to prevent DOS attacks, either accidental or purposeful.
> This is an umbrella issue that proposes the following:
>  * adding a generic ResourceManager component to Solr, which would run at a CoreContainer level and would be able to monitor and enforce both global limits and a "fair" division of resources among competing SolrCore-s.
>  * extending key existing components so that their resource consumption aspects can be dynamically controlled.
>  * adding a number of management plugins that implement specific strategies for managing eg. the cache sizes according to the specified "fairness" and global limits.
>  * the API should allow for implementation of this control loop both in Solr and as an outside mechanism.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org