You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Kay Kay (JIRA)" <ji...@apache.org> on 2008/12/16 18:30:44 UTC
[jira] Created: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
---------------------------------------------------------------------------------------------------------------------------------------
Key: SOLR-922
URL: https://issues.apache.org/jira/browse/SOLR-922
Project: Solr
Issue Type: Improvement
Components: clients - java
Environment: Tomcat 6, JRE 6
Reporter: Kay Kay
Priority: Minor
For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
$ find . -name *.java | xargs grep -nr 'start()' | grep "}"
./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated SOLR-922:
-------------------------
Attachment: SOLR-922.patch
ExecutorService in CoreContainer - reused by other modules to launch threads.
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
> Attachments: SOLR-922.patch
>
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658215#action_12658215 ]
Kay Kay commented on SOLR-922:
------------------------------
The value add of Solr by providing a single point of ExecutorService is to essentially have a single place of ExecutorPolicy / performance tuning for thread dump / out of resource issue.
* SolrCore: I am a bit reluctant on this one especially with the design of multi SolrCore because that would mean managing the ExecutorService member when it is shut down (close ). This becomes all the more pronounced when SolrCores are re-registered for the same name. I am bit reluctant to go via this path.
* ServletFilter / Outside the web-app : This would be the ideal place for this - but ideally if it were a non-Singleton pattern - it would be great. This needs more investigation though.
* CoreContainer : Given that CoreContainer is more or less a singleton instance (except when used by non-HTTP filter - when another instance seems to be created ) - this can serve the purpose for most of the cases. (at least 4 of the 6 mentioned to begin with ).
This becomes all the more pronounced especially on DataImporter.java , when a new thread is started for a delta-import / full-import commands. For one of our cases - we send frequent delta imports and creating thread objects and launching them seems to be taking a big hit.
Attached here is a profiling program - that shows a ~80% improvement in the launch of concurrent units , using ExecutorService ( FixedThreadPool implementation ) as compared to the raw Thread(). start() .
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
> Attachments: SOLR-922.patch
>
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657279#action_12657279 ]
Noble Paul commented on SOLR-922:
---------------------------------
hi Kay
I am in agreement with the general goal of having a central ExecutorService . But the risk is that we will never know that which consumer is eating up the threads and some other consumers will be starved for threads. These things become hard to debug.
leave aside DIH because it almost entirely needs its threads and it uses it 100% . And leave aside the classes in common.util
Probably Snapshooter/Snappuller can use that
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657432#action_12657432 ]
Kay Kay commented on SOLR-922:
------------------------------
| But the risk is that we will never know that which consumer is eating up the threads and some other consumers will be starved for threads. These things become hard to debug.
Sure. Suppose - we have multiple plugins to the Solr system and consumers are starved for threads, (assuming a uniform ExecutorService) , we can load the number of threads in the thread pool configurable from a property in the Solr System or change the policy of creation ( FixedSize / CachedPools etc. ) at one point and experiment with the same.
On the other hand - having local ExecutorServices for modules would mean optimizing the problem locally in that module / creating multiple static thread pools , adding to the global memory usage quickly when in reality they cannot be used interchangeably ( A local ExecutorService in DIH - would probably not be accessible to SnapPuller, say). Having a central control over the resources might be more beneficial in this case.
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657670#action_12657670 ]
Noble Paul commented on SOLR-922:
---------------------------------
As I have already mentioned , I am supportive of the idea of having a common pool of threads shared across various services
But, the problem is that means defining an extra API and may be having an extra configuration in say solrconfig.xml
Let us leave aside the contribs (I mean DIH) .In the core there are a few common usecases .
You can go ahead and propose an API and let us see how to use it best
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated SOLR-922:
-------------------------
Attachment: ExecServiceTest.java
Profiling program comparing ExecutorService against Threads when launching.
Approximately ~80% gain in ExecutorService to launch Thread objects (put them into the queue).
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
> Attachments: ExecServiceTest.java, SOLR-922.patch
>
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657679#action_12657679 ]
Noble Paul commented on SOLR-922:
---------------------------------
Kay. Go ahead and give a patch anyway
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657162#action_12657162 ]
Kay Kay commented on SOLR-922:
------------------------------
Ok - I believe having a SolrCore specific Executor/ ExecutorService might work well fine as I believe plugins that are SolrCoreAware can still make use of the same.
The motivation is to prevent multiple Executor policies (/static objects especially ) locally at each place where we need to launch threads or worse - not using them at all .
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657677#action_12657677 ]
Kay Kay commented on SOLR-922:
------------------------------
| But, the problem is that means defining an extra API and may be having an extra configuration in say solrconfig.xml
Selecting the number of threads in the thread pools / specifying the queuing policy for an even distribution of tasks , are some of the various customizations that could be implemented for the ExecutorService implementation. Since DIH and other parts of the solr application server live in the same jvm process - I am not sure why we would like to exclude the same since that would mean multiple resource managers ( thread pools ) in the same process - that would be hard to change / modify if we encounter an OutOfMemoryError (very much possible when there are multiple ExecutorService impls locally ) etc.
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-922) Solr WebApp wide Executor for better
efficient management of threads , separating the logic in the thread from
the launch of the same.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657141#action_12657141 ]
Hoss Man commented on SOLR-922:
-------------------------------
I can understand why there might be value add in having each SolrCore maintain an Executor (or more likely and ExecutorService) which the SolrCore is responsible for starting up and shutting down, but It's not clear to me what value add Solr would provide by creating a *global* Executor for custom plugins to use -- It seems like a very generic (not Solr specific) ServletFilter could accomplish the same goal in webapps, or just a simple Static singleton in non webapp code.
> Solr WebApp wide Executor for better efficient management of threads , separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
>
> For a different jira - when we were discussing bringing in parallelism through threads and using Executors - encountered a case of using a webapp wide Executor for reusing thread pools for better use of thread resources , instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can use this Executor API to retrieve the executor and just submit the Runnable / Callable impls to get the job done while getting the benefits of a thread pool . This might be necessary as we continue to write plugins to the core architecture and centralizing the threadpools might make it easy to control / prevent global Executor objects across the codebase / recreating them locally ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377: }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368: }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134: }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112: }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.