You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Karl Wright (Created) (JIRA)" <ji...@apache.org> on 2012/04/05 19:40:23 UTC
[jira] [Created] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has severe performance problems
ManifoldCF running with Derby 10.8.1.1 has severe performance problems
----------------------------------------------------------------------
Key: CONNECTORS-453
URL: https://issues.apache.org/jira/browse/CONNECTORS-453
Project: ManifoldCF
Issue Type: Bug
Components: Framework core
Affects Versions: ManifoldCF 0.5
Reporter: Karl Wright
Assignee: Karl Wright
Fix For: ManifoldCF 0.6
Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has problems pausing and aborting jobs
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273872#comment-13273872 ]
Karl Wright commented on CONNECTORS-453:
----------------------------------------
r1337457 (release branch)
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -----------------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Priority: Critical
> Fix For: ManifoldCF 0.5.1, ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has problems pausing and aborting jobs
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-453.
------------------------------------
Resolution: Fixed
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -----------------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Priority: Critical
> Fix For: ManifoldCF 0.5.1, ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has problems pausing and aborting jobs
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright reopened CONNECTORS-453:
------------------------------------
Reopening for inclusion in 0.5.1
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -----------------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Priority: Critical
> Fix For: ManifoldCF 0.5.1, ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has severe performance problems
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263148#comment-13263148 ]
Karl Wright commented on CONNECTORS-453:
----------------------------------------
r1331102
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has severe performance problems
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263086#comment-13263086 ]
Karl Wright commented on CONNECTORS-453:
----------------------------------------
I see stalls only at the very beginning of a crawl. Long crawls with lots of documents don't appear to stall, however. Still trying to figure out if this is an actual problem or something more innocuous.
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has problems pausing and aborting jobs
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright updated CONNECTORS-453:
-----------------------------------
Priority: Critical (was: Major)
Fix Version/s: ManifoldCF 0.5.1
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -----------------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Priority: Critical
> Fix For: ManifoldCF 0.5.1, ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has severe performance problems
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263092#comment-13263092 ]
Karl Wright commented on CONNECTORS-453:
----------------------------------------
Here's another example:
Error!
A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, JOBS, (1,7) Waiting XID : {157800, X} , APP, UPDATE jobs SET status=? WHERE id=? Granted XID : {157521, S} , {157653, S} Lock : ROW, JOBQUEUE, (503,86) Waiting XID : {157653, S} , APP, SELECT t0.id,t0.jobid,t0.dochash,t0.docid,t0.status,t0.failtime,t0.failcount,t0.priorityset FROM jobqueue t0 WHERE t0.status IN (?,?) AND t0.checkaction=? AND t0.checktime<=? AND EXISTS(SELECT 'x' FROM jobs t1 WHERE t1.status IN (?,?) AND t1.id=t0.jobid AND t1.priority=?) AND NOT EXISTS(SELECT 'x' FROM jobqueue t2 WHERE t2.dochash=t0.dochash AND t2.status IN (?,?,?,?,?,?) AND t2.jobid!=t0.jobid) AND NOT EXISTS(SELECT 'x' FROM prereqevents t3,events t4 WHERE t0.id=t3.owner AND t3.eventname=t4.name) ORDER BY t0.docpriority ASC,t0.status ASC,t0.checkaction ASC,t0.checktime ASC FETCH NEXT 120 ROWS ONLY Granted XID : {157557, X} Lock : ROW, JOBS, (1,7) Waiting XID : {157557, S} , APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype) VALUES (?,?,?,?,?,?) . The selected victim is XID : 157800.
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has problems pausing and aborting jobs
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright updated CONNECTORS-453:
-----------------------------------
Summary: ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs (was: ManifoldCF running with Derby 10.8.1.1 has severe performance problems)
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -----------------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has severe performance problems
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-453.
------------------------------------
Resolution: Fixed
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-453) ManifoldCF running with Derby
10.8.1.1 has severe performance problems
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263090#comment-13263090 ]
Karl Wright commented on CONNECTORS-453:
----------------------------------------
Clicking pause during the job run yields the following to be displayed in the UI:
A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, JOBS, (1,7) Waiting XID : {147028, X} , APP, UPDATE jobs SET status=? WHERE id=? Granted XID : {146703, S} , {146941, S} Lock : ROW, JOBQUEUE, (481,10) Waiting XID : {146941, S} , APP, SELECT jobid,CAST(COUNT(dochash) AS bigint) AS doccount FROM jobqueue t1 WHERE EXISTS(SELECT 'x' FROM jobs t0 WHERE t0.id=t1.jobid AND id=?) GROUP BY jobid Granted XID : {146612, X} Lock : ROW, HOPCOUNT, (1734,27) Waiting XID : {146612, S} , APP, SELECT parentidhash,linktype,distance FROM hopcount WHERE jobid=? AND parentidhash IN (?,?,?,?,?,?,?,?,?,?) AND linktype=? Granted XID : {146666, X} Lock : ROW, JOBS, (1,7) Waiting XID : {146666, S} , APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype) VALUES (?,?,?,?,?,?) . The selected victim is XID : 147028.
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.6
>
>
> Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.
> In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira