You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Sami Siren (JIRA)" <ji...@apache.org> on 2012/06/05 14:11:23 UTC
[jira] [Created] (SOLR-3511) Refactor overseer to use a distributed
"work"queue
Sami Siren created SOLR-3511:
--------------------------------
Summary: Refactor overseer to use a distributed "work"queue
Key: SOLR-3511
URL: https://issues.apache.org/jira/browse/SOLR-3511
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Reporter: Sami Siren
Assignee: Sami Siren
By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3511) Refactor overseer to use a
distributed "work"queue
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289384#comment-13289384 ]
Sami Siren commented on SOLR-3511:
----------------------------------
bq. but should we leverage this for collection creation as well or will that be a separate work queue
I think that the same queue could be used easily. The objects that are stored in queue (serialized in json) are currently specific to core state management and leader announcements but it should be easy to make that part + the actual processing commands in overseer more general.
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Attachments: SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3511) Refactor overseer to use a distributed
"work"queue
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sami Siren updated SOLR-3511:
-----------------------------
Attachment: SOLR-3511.patch
New patch with cleanups and minor improvements
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Attachments: SOLR-3511.patch, SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3511) Refactor overseer to use a distributed
"work"queue
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sami Siren updated SOLR-3511:
-----------------------------
Fix Version/s: 4.0
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Fix For: 4.0
>
> Attachments: SOLR-3511.patch, SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3511) Refactor overseer to use a
distributed "work"queue
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292417#comment-13292417 ]
Mark Miller commented on SOLR-3511:
-----------------------------------
This is nice Sami - great work. I've been going over it and working on integrating a first pass at collection creation as well.
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Fix For: 4.0
>
> Attachments: SOLR-3511.patch, SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3511) Refactor overseer to use a distributed
"work"queue
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sami Siren updated SOLR-3511:
-----------------------------
Attachment: SOLR-3511.patch
Work in progress, uses DistributedQueue from zk recipes. All tests pass, lot of cleaning up to do.
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Attachments: SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3511) Refactor overseer to use a
distributed "work"queue
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289380#comment-13289380 ]
Mark Miller commented on SOLR-3511:
-----------------------------------
I have not had a chance to look at this yet - but should we leverage this for collection creation as well or will that be a separate work queue?
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Attachments: SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-3511) Refactor overseer to use a
distributed "work"queue
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sami Siren resolved SOLR-3511.
------------------------------
Resolution: Fixed
Committed to 4.x too
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Fix For: 4.0
>
> Attachments: SOLR-3511.patch, SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3511) Refactor overseer to use a
distributed "work"queue
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291556#comment-13291556 ]
Sami Siren commented on SOLR-3511:
----------------------------------
Committed to trunk, I'll let it bake there for a while before backporting to 4.x
> Refactor overseer to use a distributed "work"queue
> --------------------------------------------------
>
> Key: SOLR-3511
> URL: https://issues.apache.org/jira/browse/SOLR-3511
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Sami Siren
> Assignee: Sami Siren
> Fix For: 4.0
>
> Attachments: SOLR-3511.patch, SOLR-3511.patch
>
>
> By using a queue overseer becomes a watch free, a lot simpler and probably less buggy too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org