You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Emmanuel Keller (JIRA)" <ji...@apache.org> on 2011/07/20 17:52:57 UTC
[jira] [Created] (CONNECTORS-224) OpenSearchServer connector
OpenSearchServer connector
--------------------------
Key: CONNECTORS-224
URL: https://issues.apache.org/jira/browse/CONNECTORS-224
Project: ManifoldCF
Issue Type: New Feature
Affects Versions: ManifoldCF 0.3
Reporter: Emmanuel Keller
Provide an output connector for OpenSearchServer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081013#comment-13081013 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
If this artifact works under the Quick-start, it implies that the html resources are properly getting found, since the quick-start relies on a local lib directory as specified in properties.xml. So I think we're all set there.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082223#comment-13082223 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
Thank you for testing. Well, that's why I called it "alpha". I summarize the points:
- JDK 1.5 compliant
- Add javascript checks in the parameter interface
- Implement the check method
- Memory consumption (in real life, not just in a unit test).
I work on this.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084220#comment-13084220 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
It doesn't build on 1.5 still:
compile-connector:
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\build.xml:10:
warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last;
set to false for repeatable builds
[javac] Compiling 9 source files to C:\wip\mcf\CONNECTORS-224\connectors\ope
nsearchserver\build\connector\classes
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:59: cannot find symbol
[javac] symbol : constructor IOException(org.apache.manifoldcf.core.interfa
ces.ManifoldCFException)
[javac] location: class java.io.IOException
[javac] throw new IOException(e);
[javac] ^
[javac] 1 error
This is easily corrected; I'll check in a fix.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Description: Provide an output connector for [OpenSearchServer|http://www.open-search-server.com]. (was: Provide an output connector for OpenSearchServer.)
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082030#comment-13082030 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
The Parameters tab also appears to have no Javascript correctness checking whatsoever. I can specify a blank URL and a blank index name and it still allows me to save it. This too should be addressed.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082029#comment-13082029 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Trying out the connector, it appears that the connection check() method is not doing anything, as I get back "Connection working" even with completely bogus parameters:
View Output Connection Status
Name: test Description:
Connection type: OpenSearchServer Max connections: 10
Server Location (URL): http://localhost:8080/
Index name: index
User name: foo
API Key: bar
Connection status: Connection working
This should be fixed, obviously.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright updated CONNECTORS-224:
-----------------------------------
Component/s: OpenSearchServer connector
Assignee: Karl Wright
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Comment: was deleted
(was: Here is a first version of the OpenSearchServer connector. The connector is not yet finished. This first draft contains the skeleton and the interface part. I thought it was interesting to publish it to show how we have separated the HTML content from the JAVA code. The HTML code is stored in separated files. We are using String replacement to provide dynamic content.)
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Attachment: oss-mfc-alpha2.patch
Sorry, here is the missing file.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084116#comment-13084116 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Checked in the latest patch, and I'll try to evaluate it at first available opportunity.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086267#comment-13086267 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Committed to trunk. r1158655.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch, oss-mfc-rc1.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068454#comment-13068454 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
And btw, thank you for your useful book, it is my current bedside book!
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082034#comment-13082034 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
When I try to crawl using the server and the connection is bogus, the crawl proceeds a while and does not seem to notice that there is no output server. Very shortly thereafter, the whole system runs out of memory:
>>>>>>
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133)
at java.lang.StringCoding.decode(StringCoding.java:173)
at java.lang.StringCoding.decode(StringCoding.java:185)
at java.lang.String.<init>(String.java:570)
at java.lang.String.<init>(String.java:593)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Index.getXML(OpenSearchServerIndex.java:58)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Index.<init>(OpenSearchServerIndex.java:28)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Connector.addOrReplaceDocument(OpenSearchServerConnector.java:153)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.ad
dOrReplaceDocument(IncrementalIngester.java:1499)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.pe
rformIngestion(IncrementalIngester.java:469)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
cumentIngest(IncrementalIngester.java:364)
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
estDocument(WorkerThread.java:1587)
at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.pro
cessDocuments(FileConnector.java:283)
at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.proc
essDocuments(BaseRepositoryConnector.java:423)
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
va:564)
<<<<<<
I haven't looked at the code, but clearly the connector is trying to queue up documents in memory for some reason. If that's what's going on, that's never going to fly because there cannot possibly be proper feedback of error conditions etc.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084101#comment-13084101 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
checkMimeTypeIndexable and checkFileIndexable will only be called from certain connectors, which are in a position to prefilter content. Other connectors will not be able to use these.
For example, the Web connector calls checkMimeTypeIndexable and the JCIFS connector calls checkFileIndexable. Hope this helps.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080999#comment-13080999 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Updated the branch with the second patch.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080998#comment-13080998 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
Thank you for the quick integration. I switched to the new svn branch.
About the resource file, at this time I only checked it with the embedded web server (Jetty). I will make additional tests using Tomcat. The files are directly loaded by our code using getClass().getResourceAsStream().
Because the connector is visible in the webapp, the containing resources should also be available. I suppose it is wise to locate the files in the same package/jar.
Finally, we don't rely on the JSP interpretation.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081010#comment-13081010 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Great - patch 3 committed (and now it builds).
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082021#comment-13082021 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
I checked in and tested Maven build files in the branch.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CONNECTORS-224) OpenSearchServer
connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080867#comment-13080867 ]
Karl Wright edited comment on CONNECTORS-224 at 8/8/11 9:47 AM:
----------------------------------------------------------------
As far as the use of html files, the real question is how resources interact with class loaders in Java. ManifoldCF originally had its HTML code generated by JSP files, but had to change this architecture. The reason ManifoldCF pulled its HTML code out of JSP files was because JSPs don't work well with class loaders. Connectors must be self-contained in that loading the connector class from whatever library directory is specified in the properties.xml file is adequate to locate all resources for the connector. Can you clarify whether or not you think this will work for your html resource files?
was (Author: kwright@metacarta.com):
As far as the use of html files, the real question is how resources interact with class loaders in Java. The reason ManifoldCF pulled its HTML code out of JSP files was because JSPs don't work well with class loaders. Connectors must be self-contained in that loading the connector class from whatever library directory is specified in the properties.xml file is adequate to locate all resources for the connector. Can you clarify whether or not you think this will work for your html resource files?
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-224.
------------------------------------
Resolution: Fixed
Fix Version/s: ManifoldCF 0.3
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Fix For: ManifoldCF 0.3
>
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch, oss-mfc-rc1.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Status: Open (was: Patch Available)
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081009#comment-13081009 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
No issue, the code provided is stand alone. There is no dependency with an OpenSearchServer original library.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068452#comment-13068452 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Looking forward to seeing it!
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for OpenSearchServer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Attachment: oss-mfc-rc1.patch
(1) The purpose of this synchronizer is to be sure we have only one API call to an OSS instance for one index. This prevents memory leaks. I prefer this approach than creating an additional thread with a submission queue.
(2) The inputStream is no more closed in the rc1 patch.
(3) The correct formatting has been applied.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch, oss-mfc-rc1.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CONNECTORS-224) OpenSearchServer
connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082034#comment-13082034 ]
Karl Wright edited comment on CONNECTORS-224 at 8/10/11 1:02 AM:
-----------------------------------------------------------------
When I try to crawl using the connector and the connection parameters are bogus, the crawl proceeds a while and does not seem to notice that there is no actual output server present. Very shortly thereafter, after a few hundred files, the whole system runs out of memory:
>>>>>>
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133)
at java.lang.StringCoding.decode(StringCoding.java:173)
at java.lang.StringCoding.decode(StringCoding.java:185)
at java.lang.String.<init>(String.java:570)
at java.lang.String.<init>(String.java:593)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Index.getXML(OpenSearchServerIndex.java:58)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Index.<init>(OpenSearchServerIndex.java:28)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Connector.addOrReplaceDocument(OpenSearchServerConnector.java:153)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.ad
dOrReplaceDocument(IncrementalIngester.java:1499)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.pe
rformIngestion(IncrementalIngester.java:469)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
cumentIngest(IncrementalIngester.java:364)
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
estDocument(WorkerThread.java:1587)
at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.pro
cessDocuments(FileConnector.java:283)
at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.proc
essDocuments(BaseRepositoryConnector.java:423)
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
va:564)
<<<<<<
I haven't looked at the code, but clearly the connector is trying to queue up documents in memory for some reason. If that's what's going on, that's never going to fly because there cannot possibly be proper feedback of error conditions etc.
was (Author: kwright@metacarta.com):
When I try to crawl using the server and the connection is bogus, the crawl proceeds a while and does not seem to notice that there is no output server. Very shortly thereafter, the whole system runs out of memory:
>>>>>>
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:133)
at java.lang.StringCoding.decode(StringCoding.java:173)
at java.lang.StringCoding.decode(StringCoding.java:185)
at java.lang.String.<init>(String.java:570)
at java.lang.String.<init>(String.java:593)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Index.getXML(OpenSearchServerIndex.java:58)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Index.<init>(OpenSearchServerIndex.java:28)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServer
Connector.addOrReplaceDocument(OpenSearchServerConnector.java:153)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.ad
dOrReplaceDocument(IncrementalIngester.java:1499)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.pe
rformIngestion(IncrementalIngester.java:469)
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
cumentIngest(IncrementalIngester.java:364)
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
estDocument(WorkerThread.java:1587)
at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.pro
cessDocuments(FileConnector.java:283)
at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.proc
essDocuments(BaseRepositoryConnector.java:423)
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
va:564)
<<<<<<
I haven't looked at the code, but clearly the connector is trying to queue up documents in memory for some reason. If that's what's going on, that's never going to fly because there cannot possibly be proper feedback of error conditions etc.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080861#comment-13080861 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
I've created a branch for further development of this connector until it is ready to commit to trunk. The url for the branch is:
https://svn.apache.org/repos/asf/incubator/lcf/branches/CONNECTORS-224.
For subsequent patches, please submit diffs against this branch. Thanks!
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081015#comment-13081015 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
I agree! I confirm it works with the built-in example (cd dist/example; java -jar start.jar)
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Attachment: oss-mfc-alpha.patch
First alpha version of the output connector for OpenSearchServer.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Attachment: oss-mfc-beta.patch
This new version fixes the previous points (Java 5, missing javascript checks, implementation of the check method, memory leak).
We also added a specification tab panel to handle mime types and extension filter, and a limit to the maximum size of documents.
Finally, one issue remains: The following methods are never called: checkMimetypeIndexable and checkFileIndexable. Until now we were not able to understand why. Thanks in advance for your help.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081054#comment-13081054 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
bq. I can provide some JUnit tests that will rely on an already running OpenSearchServer instance.
That's fine, as long as you leave some notes somewhere as to how to set up the instance, and you don't tie those tests into the generic build.xml "test" target.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081018#comment-13081018 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
I'm going to run some basic hand tests on this tonight. Before it can be merged, we'll need to hook it into the maven build also. So you'll need to provide a pom.xml. If you aren't maven-savvy, I can give it a try myself, or ask Piergiorgio if he wants to have a look at it.
Have you given any thought as to how best to test this connector? Is there any hope of having the tests bring up a standalone OpenSearchServer instance stand-alone, which exists only for the life of the test before it is shut down?
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081029#comment-13081029 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
There is no light version of OpenSearchServer. To bring up a standalone OpenSearchServer instance we will need to embed its war file (60 MB).
I can provide some JUnit tests that will rely on an already running OpenSearchServer instance.
And yes, I am not (yet) maven-savvy. Thank you for your help.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068450#comment-13068450 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
We are working on the current trunk. We should be able to provide a first working version of this connector by the end of this week.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for OpenSearchServer.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084111#comment-13084111 ]
Emmanuel Keller commented on CONNECTORS-224:
--------------------------------------------
It helps, thank you. So I will check with these connectors.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084224#comment-13084224 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Another problem that occurs in this connector with no real connection is that notification never completes. Tons of exceptions:
ERROR 2011-08-12 12:15:00,637 (Job notification thread) - java.net.ConnectException: Connection refused: connect
org.apache.manifoldcf.core.interfaces.ManifoldCFException: java.net.ConnectException: Connection refused: connect
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnection.call(OpenSearchServerConnection.java:102)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction.<init>(OpenSearchServerAction.java:19)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector.noteJobComplete(OpenSearchServerConnector.java:328)
at org.apache.manifoldcf.crawler.system.JobNotificationThread.run(JobNotificationThread.java:115)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:249)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(Unknown Source)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(Unknown Source)
at org.apache.commons.httpclient.HttpConnection.open(Unknown Source)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Unknown Source)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(Unknown Source)
at org.apache.commons.httpclient.HttpClient.executeMethod(Unknown Source)
at org.apache.commons.httpclient.HttpClient.executeMethod(Unknown Source)
at org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnection.call(OpenSearchServerConnection.java:93)
... 3 more
This is actually a framework problem, I believe; the job is aborted but notification is attempted anyhow. But it is supposed to give up not try indefinitely.
I'll create a new ticket for that issue.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081000#comment-13081000 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
When I build using "ant build", this is what I get:
compile-connector:
[mkdir] Created dir: /data/mcf/CONNECTORS-224/connectors/opensearchserver/build/connector/classes
[javac] Compiling 5 source files to /data/mcf/CONNECTORS-224/connectors/opensearchserver/build/connector/classes
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:17: package org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction does not exist
[javac] import org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction.CommandEnum;
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:181: cannot find symbol
[javac] symbol : class OpenSearchServerAction
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] OpenSearchServerAction oo = new OpenSearchServerAction(
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:181: cannot find symbol
[javac] symbol : class OpenSearchServerAction
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] OpenSearchServerAction oo = new OpenSearchServerAction(
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:182: cannot find symbol
[javac] symbol : variable CommandEnum
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] CommandEnum.optimize, getParameters(null));
[javac] ^
[javac] 4 errors
BUILD FAILED
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080867#comment-13080867 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
As far as the use of html files, the real question is how resources interact with class loaders in Java. The reason ManifoldCF pulled its HTML code out of JSP files was because JSPs don't work well with class loaders. Connectors must be self-contained in that loading the connector class from whatever library directory is specified in the properties.xml file is adequate to locate all resources for the connector. Can you clarify whether or not you think this will work for your html resource files?
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082027#comment-13082027 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
I just discovered that this connector won't build under jdk 1.5, which is a problem since ManifoldCF has not yet moved to a minimum Java version of 1.6. These are the errors I get:
>>>>>>
compile-connector:
[mkdir] Created dir: C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\b
uild\connector\classes
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\build.xml:10:
warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last;
set to false for repeatable builds
[javac] Compiling 6 source files to C:\wip\mcf\CONNECTORS-224\connectors\ope
nsearchserver\build\connector\classes
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:8: package javax.xml.stream does not exist
[javac] import javax.xml.stream.XMLOutputFactory;
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:9: package javax.xml.stream does not exist
[javac] import javax.xml.stream.XMLStreamException;
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:10: package javax.xml.stream does not exist
[javac] import javax.xml.stream.XMLStreamWriter;
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:42: cannot find symbol
[javac] symbol : class XMLStreamException
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver
.OpenSearchServerIndex
[javac] throws XMLStreamException, IOException {
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:34: cannot find symbol
[javac] symbol : class XMLStreamException
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver
.OpenSearchServerIndex
[javac] } catch (XMLStreamException e) {
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:44: cannot find symbol
[javac] symbol : class XMLOutputFactory
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver
.OpenSearchServerIndex
[javac] XMLOutputFactory factory = XMLOutputFactory.newInstance(
);
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:44: cannot find symbol
[javac] symbol : variable XMLOutputFactory
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver
.OpenSearchServerIndex
[javac] XMLOutputFactory factory = XMLOutputFactory.newInstance(
);
[javac] ^
[javac] C:\wip\mcf\CONNECTORS-224\connectors\opensearchserver\connector\src\
main\java\org\apache\manifoldcf\agents\output\opensearchserver\OpenSearchServerI
ndex.java:46: cannot find symbol
[javac] symbol : class XMLStreamWriter
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver
.OpenSearchServerIndex
[javac] XMLStreamWriter writer = factory.createXMLStreamWriter(s
w);
[javac] ^
[javac] 8 errors
BUILD FAILED
<<<<<<
The XML parser that is used currently throughout ManifoldCF, and is thus available, is a modified version of xerces - you will see it under the "lib" directory.
This does NOT affect maven, because maven is at minimum a 1.6 build already. It will affect the ability, however, to include the opensearchserver connector in the shipping packages. So it seems to me that you have three choices here: (1) wait until we officially adopt Java 1.6 as the minimum standard java version before merging this into trunk; (2) modify your code so that it does not use the 1.6 features; or (3) change the conditional compilation in the ant build so that it won't even try to compile the connector under JDK 1.5. Please let me know what you prefer to do.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Status: Patch Available (was: Open)
Here is a first version of the OpenSearchServer connector. The connector is not yet finished. This first draft contains the skeleton and the interface part. I thought it was interesting to publish it to show how we have separated the HTML content from the JAVA code. The HTML code is stored in separated files. We are using String replacement to provide dynamic content.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084252#comment-13084252 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Ok, I've tested some more.
This is a lot better than before. The issues previously described seem to have been fixed. Now we have a new set of issues to consider. I'm listing these below.
(1) In addOrReplaceDocument(), the following code is present:
>>>>>>
OpenSearchServerConfig config = getConfigParameters(null);
Integer count = addInstance(config);
synchronized (count) {
<<<<<<
Can you explain what the purpose of this synchronizer is? It looks to me like you might be effectively managing your own connection pool here, which is both redundant and would prevent ManifoldCF end users from controlling the size of that pool. Am I correct?
(2) In addOrReplaceDocument(), you close the input stream. You should not do that. The caller closes the stream.
(3) Formatting. Apache guidelines set indenting to 2 spaces, with no tabs. ManifoldCF adheres to this convention. It would be great if you could reformat accordingly.
Thanks again!
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-224) OpenSearchServer connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086245#comment-13086245 ]
Karl Wright commented on CONNECTORS-224:
----------------------------------------
Thanks for the update - I've merged it into the branch.
At this point I think we're just about ready to merge the branch into trunk. I agree with the decision to use a synchronizer to prevent leaks, although I'd love it if there was more of a description of the problem in comments in the code. But that can wait until after this connector hits trunk.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-alpha2.patch, oss-mfc-beta.patch, oss-mfc-dev.patch, oss-mfc-rc1.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CONNECTORS-224) OpenSearchServer
connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081000#comment-13081000 ]
Karl Wright edited comment on CONNECTORS-224 at 8/8/11 3:37 PM:
----------------------------------------------------------------
When I build using "ant build", this is what I get:
compile-connector:
[mkdir] Created dir: /data/mcf/CONNECTORS-224/connectors/opensearchserver/build/connector/classes
[javac] Compiling 5 source files to /data/mcf/CONNECTORS-224/connectors/opensearchserver/build/connector/classes
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:17: package org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction does not exist
[javac] import org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction.CommandEnum;
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:181: cannot find symbol
[javac] symbol : class OpenSearchServerAction
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] OpenSearchServerAction oo = new OpenSearchServerAction(
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:181: cannot find symbol
[javac] symbol : class OpenSearchServerAction
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] OpenSearchServerAction oo = new OpenSearchServerAction(
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:182: cannot find symbol
[javac] symbol : variable CommandEnum
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] CommandEnum.optimize, getParameters(null));
[javac] ^
[javac] 4 errors
BUILD FAILED
Clearly we are missing some jars for Open Search Server. Before I can advise you how to proceed, I need to know exactly how Open Search Server is licensed. If it is an Apache, Eclipse, or BSD license, you can send me the necessary jars and I'll commit them under the lib directory. In that case we will also need any details of the license text that must be included in NOTICE.txt and DEPENDENCIES.txt. If the license is something else, you will need to create a subdirectory of your connectors/opensearchservice directory that contains a readme.txt file explaining how to obtain the necessary jars, and you will need to modify the main build.xml to perform a check as to whether those jars are in place before attempting to build this connector.
was (Author: kwright@metacarta.com):
When I build using "ant build", this is what I get:
compile-connector:
[mkdir] Created dir: /data/mcf/CONNECTORS-224/connectors/opensearchserver/build/connector/classes
[javac] Compiling 5 source files to /data/mcf/CONNECTORS-224/connectors/opensearchserver/build/connector/classes
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:17: package org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction does not exist
[javac] import org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerAction.CommandEnum;
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:181: cannot find symbol
[javac] symbol : class OpenSearchServerAction
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] OpenSearchServerAction oo = new OpenSearchServerAction(
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:181: cannot find symbol
[javac] symbol : class OpenSearchServerAction
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] OpenSearchServerAction oo = new OpenSearchServerAction(
[javac] ^
[javac] /data/mcf/CONNECTORS-224/connectors/opensearchserver/connector/src/main/java/org/apache/manifoldcf/agents/output/opensearchserver/OpenSearchServerConnector.java:182: cannot find symbol
[javac] symbol : variable CommandEnum
[javac] location: class org.apache.manifoldcf.agents.output.opensearchserver.OpenSearchServerConnector
[javac] CommandEnum.optimize, getParameters(null));
[javac] ^
[javac] 4 errors
BUILD FAILED
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Components: OpenSearchServer connector
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Assignee: Karl Wright
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-alpha.patch, oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-224) OpenSearchServer connector
Posted by "Emmanuel Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Emmanuel Keller updated CONNECTORS-224:
---------------------------------------
Attachment: oss-mfc-dev.patch
Here is a first version of the OpenSearchServer connector. The connector is not yet finished. This first draft contains the skeleton and the interface part. I thought it was interesting to publish it to show how we have separated the HTML content from the JAVA code. The HTML code is stored in separated files. We are using String replacement to provide dynamic content.
> OpenSearchServer connector
> --------------------------
>
> Key: CONNECTORS-224
> URL: https://issues.apache.org/jira/browse/CONNECTORS-224
> Project: ManifoldCF
> Issue Type: New Feature
> Affects Versions: ManifoldCF 0.3
> Reporter: Emmanuel Keller
> Labels: OpenSearchServer, connector, outputconnector
> Attachments: oss-mfc-dev.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Provide an output connector for [OpenSearchServer|http://www.open-search-server.com].
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira