You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Karl Wright (Created) (JIRA)" <ji...@apache.org> on 2011/12/21 01:19:30 UTC
[jira] [Created] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Make ManifoldCF fully run on MySQL
----------------------------------
Key: CONNECTORS-334
URL: https://issues.apache.org/jira/browse/CONNECTORS-334
Project: ManifoldCF
Issue Type: Improvement
Components: Framework core
Affects Versions: ManifoldCF 0.5
Reporter: Karl Wright
Assignee: Karl Wright
Fix For: ManifoldCF 0.5
Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174811#comment-13174811 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1222233 to add more abstraction for MySQL report returns.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174752#comment-13174752 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
Checked in a number of changes across the board. Most notable is that the MySQL JDBC driver's maxCharLength field is completely bogus and returns values that are essentially random. We *were* using this field generally to insure that returned values did not exceed their schema length, but I'm going to have to discontinue use (or conditionalize it) in order to support MySQL.
Once all that was done, the tests are still failing because the crawl still hangs. Reason is unclear. The basic problem now is that when a document is in the "pending" state and the job is "active" we still don't get any processing activity. There are no exceptions - just no documents. The stuffer query must therefore be acting differently on MySQL than on every other database.
This has been the hardest database port we've yet attempted for ManifoldCF. MySQL is incredibly idiosyncratic and appears to do many things in a manner wholly its own. It may therefore be quite some time before this all works.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174125#comment-13174125 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221745 to remove performLocks() abstraction, since it has been subsumed by the appropriate unique constraints and deadlock retries anyway.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173821#comment-13173821 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221587 for a missed part of the merge.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173805#comment-13173805 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221585 for tests.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174776#comment-13174776 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
Tests now pass.
Max activity and max bandwidth reports fail, though, with exceptions such as these:
2011-12-22 07:27:55.318:WARN::Nested in javax.servlet.ServletException: org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database exception: Exception doing query: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DOUBLE PRECISION) / CAST((t1.endtime-t1.starttime) AS DOUBLE PRECISION) AS activ' at line 1:org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database exception: Exception doing query: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DOUBLE PRECISION) / CAST((t1.endtime-t1.starttime) AS DOUBLE PRECISION) AS activ' at line 1
Obviously doesn't like the cast. Will have to abstract that as well.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173713#comment-13173713 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221547 for maven pom.xml changes.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173822#comment-13173822 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
Tests fail; it looks like this might be a case-sensitivity issue. By default MySQL apparently installs case-insensitive??
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174771#comment-13174771 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1222195 for column mapping.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173829#comment-13173829 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221594 in an attempt to find a case-sensitive collation for utf-8. But doesn't work because status characters don't read correctly.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174013#comment-13174013 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221678 for a bunch of fixes; now tests fail in a different way.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173803#comment-13173803 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221585 for tests.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173729#comment-13173729 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221549 for documentation changes.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173686#comment-13173686 ]
Karl Wright commented on CONNECTORS-334:
----------------------------------------
r1221538 for the basic mods to the ant build and the database drivers/interfaces.
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-334) Make ManifoldCF fully run on
MySQL
Posted by "Karl Wright (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-334.
------------------------------------
Resolution: Fixed
> Make ManifoldCF fully run on MySQL
> ----------------------------------
>
> Key: CONNECTORS-334
> URL: https://issues.apache.org/jira/browse/CONNECTORS-334
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.5
> Reporter: Karl Wright
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.5
>
>
> Make ManifoldCF run properly on top of MySQL. The ticket should be viewed as a combination of the MySQL contributions from CONNECTORS-314, plus tests and other changes needed to make MySQL be a viable option for crawling.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira