You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Karl Wright (Created) (JIRA)" <ji...@apache.org> on 2011/11/04 15:07:01 UTC
[jira] [Created] (CONNECTORS-285) JobManager getStatus() command
always performs slow queries, even when they may not be needed
JobManager getStatus() command always performs slow queries, even when they may not be needed
---------------------------------------------------------------------------------------------
Key: CONNECTORS-285
URL: https://issues.apache.org/jira/browse/CONNECTORS-285
Project: ManifoldCF
Issue Type: Improvement
Components: Framework crawler agent
Affects Versions: ManifoldCF 0.3
Reporter: Karl Wright
Assignee: Karl Wright
Fix For: ManifoldCF 0.4
The JobManager.getStatus() method, and the associated API operation, always obtain the counts of documents processed or in progress. These queries are slow and don't scale because they require sequential scan of the jobqueue. Sometimes all that is needed is just the status of the job. We should have a way of getting that that is performant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-285) JobManager getStatus() command
always performs slow queries, even when they may not be needed
Posted by "Karl Wright (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-285.
------------------------------------
Resolution: Fixed
r1198541.
> JobManager getStatus() command always performs slow queries, even when they may not be needed
> ---------------------------------------------------------------------------------------------
>
> Key: CONNECTORS-285
> URL: https://issues.apache.org/jira/browse/CONNECTORS-285
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework crawler agent
> Affects Versions: ManifoldCF 0.3
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.4
>
>
> The JobManager.getStatus() method, and the associated API operation, always obtain the counts of documents processed or in progress. These queries are slow and don't scale because they require sequential scan of the jobqueue. Sometimes all that is needed is just the status of the job. We should have a way of getting that that is performant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-285) JobManager getStatus() command
always performs slow queries, even when they may not be needed
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144210#comment-13144210 ]
Karl Wright commented on CONNECTORS-285:
----------------------------------------
r1197696 for the actual change to the underlying JobManager functionality, and all tests that use JobManager directly. Still need to add to the API.
> JobManager getStatus() command always performs slow queries, even when they may not be needed
> ---------------------------------------------------------------------------------------------
>
> Key: CONNECTORS-285
> URL: https://issues.apache.org/jira/browse/CONNECTORS-285
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework crawler agent
> Affects Versions: ManifoldCF 0.3
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.4
>
>
> The JobManager.getStatus() method, and the associated API operation, always obtain the counts of documents processed or in progress. These queries are slow and don't scale because they require sequential scan of the jobqueue. Sometimes all that is needed is just the status of the job. We should have a way of getting that that is performant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira