You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Alex Parvulescu (Created) (JIRA)" <ji...@apache.org> on 2011/11/14 15:38:51 UTC
[jira] [Created] (JCR-3147) AbstractIndex should use the repository
executor for indexing tasks
AbstractIndex should use the repository executor for indexing tasks
-------------------------------------------------------------------
Key: JCR-3147
URL: https://issues.apache.org/jira/browse/JCR-3147
Project: Jackrabbit Content Repository
Issue Type: Improvement
Components: jackrabbit-core
Reporter: Alex Parvulescu
Priority: Minor
The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (JCR-3147) AbstractIndex should use
the repository executor for indexing tasks
Posted by "Alex Parvulescu (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149657#comment-13149657 ]
Alex Parvulescu edited comment on JCR-3147 at 11/15/11 2:21 PM:
----------------------------------------------------------------
This is only a soft link, the patch is pretty much standalone.
was (Author: alex.parvulescu):
This is only a soft link, the patch is pretty much standalone.
The required work after JCR-3146 is in, is to mark the tasks submitted by the AbstractIndex as low priority, so they don't clog up the global executor.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Alex Parvulescu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150517#comment-13150517 ]
Alex Parvulescu commented on JCR-3147:
--------------------------------------
I have updated the issue's description and removed the references to low priority tasks.
Tasks submitted by the AbstractIndex will have normal priority, which means they will be executed asap.
Plus the global executor now benefits from a few dedicated threads that can pickup the work load immediately (the low priority tasks occupy up to 75% of the threads OOTB).
Marcel, do you think the current tests cover this indexing issue, or would you like to investigate some more before applying the patch?
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150533#comment-13150533 ]
Marcel Reutegger commented on JCR-3147:
---------------------------------------
> Marcel, do you think the current tests cover this indexing issue
Probably not. To me it is not clear how the system will behave when the pool is busy with index merges and then additional tasks like these in the AbstractIndex are added. I guess the save would be blocked for a while until a thread is available again. This is a slightly different behaviour than before.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3147) AbstractIndex should use the repository
executor for indexing tasks
Posted by "Alex Parvulescu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Parvulescu updated JCR-3147:
---------------------------------
Description: The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor. (was: The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.)
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3147) AbstractIndex should use the repository
executor for indexing tasks
Posted by "Alex Parvulescu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Parvulescu updated JCR-3147:
---------------------------------
Attachment: JCR-3147.patch
attached proposed patch
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Jukka Zitting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150485#comment-13150485 ]
Jukka Zitting commented on JCR-3147:
------------------------------------
> parallelize a number of tasks given the number of available cores
That's also what the repository thread pool does. In addition it supports scheduling of tasks for execution in the future it can (and should) also be used for immediate execution of parallel tasks.
Having just a single thread pool for the repository simplifies things (even after the added concern of keeping some threads available for high-priority tasks), which is why IMHO we should do this.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150507#comment-13150507 ]
Marcel Reutegger commented on JCR-3147:
---------------------------------------
> should we reconsider the patch now?
The indexing tasks in this issue should *not* be low priority. I'm OK with a solution that uses the repository thread pool, but uses the current thread if the pool is busy.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150505#comment-13150505 ]
Marcel Reutegger commented on JCR-3147:
---------------------------------------
> can (and should) also be used for immediate execution of parallel tasks
I agree that it should but currently it can't, because all tasks are treated equally.
I think an important feature that is missing currently, is the ability to specify a 'queue policy'. A client should be able to specify if it is OK to queue the task and execute it later when all the threads in the pool are busy. This would be similar to the RejectedExecutionHandler but would already kick in when the the pool is busy.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150430#comment-13150430 ]
Marcel Reutegger commented on JCR-3147:
---------------------------------------
The usage of DynamicPooledExecutor in AbstractIndex is a bit special because it is not meant to execute tasks at some point in the future, but just parallelize a number of tasks given the number of available cores. It is important that the tasks are scheduled and executed rather immediately. Otherwise the current transaction will be delayed unnecessarily.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Alex Parvulescu (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Parvulescu resolved JCR-3147.
----------------------------------
Resolution: Won't Fix
cool, now I understand. I'll close this issue.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Alex Parvulescu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150501#comment-13150501 ]
Alex Parvulescu commented on JCR-3147:
--------------------------------------
thanks for joining Jukka :)
I'm with Jukka on this one. Having a global executor will simplify things and I think the work on JCR-3146 should address the priority tasks concern.
should we reconsider the patch now?
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
> It should also mark the tasks as 'low priotiry' as soon as JCR-3146 gets into the trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3147) AbstractIndex should use the
repository executor for indexing tasks
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150534#comment-13150534 ]
Marcel Reutegger commented on JCR-3147:
---------------------------------------
It would also be good to understand what the behaviour is when you re-index a workspace that contains a lot of files that trigger text extractions.
> AbstractIndex should use the repository executor for indexing tasks
> -------------------------------------------------------------------
>
> Key: JCR-3147
> URL: https://issues.apache.org/jira/browse/JCR-3147
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
> Attachments: JCR-3147.patch
>
>
> The AbstractIndex still uses its own executor for indexing tasks, it should switch to the global repository executor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira