You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Dennis Kubes (JIRA)" <ji...@apache.org> on 2007/12/21 18:10:43 UTC
[jira] Created: (NUTCH-594) Serve Nutch search results in XML and
JSON
Serve Nutch search results in XML and JSON
------------------------------------------
Key: NUTCH-594
URL: https://issues.apache.org/jira/browse/NUTCH-594
Project: Nutch
Issue Type: New Feature
Environment: all
Reporter: Dennis Kubes
Assignee: Dennis Kubes
Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (NUTCH-594) Serve Nutch search results
in multiple formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659825#action_12659825 ]
musepwizard edited comment on NUTCH-594 at 12/30/08 6:56 AM:
--------------------------------------------------------------
JSON-LIb and EZMorph are both under Apache. There is an optional Xom library dependency for JSON-Lib which is not included, that is under LGPL, but everything else is Apache.
http://json-lib.sourceforge.net/license.html
http://ezmorph.sourceforge.net/license.html
I put comments about these in the plugin.xml file for response-json. Is there anything else I need to do?
was (Author: musepwizard):
JSON-LIb and EZMorph are both under Apache. There is an optional Xom library dependency for JSON-Lib which is not included, that is under LGPL, but everything is Apache.
http://json-lib.sourceforge.net/license.html
http://ezmorph.sourceforge.net/license.html
I put comments about these in the plugin.xml file for response-json. Is there anything else I need to do?
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: commons-collections-3.2.1.jar
commons collections
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660257#action_12660257 ]
Dennis Kubes commented on NUTCH-594:
------------------------------------
I thought you would like that. Especially since it was just a minor nit :)
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch, NUTCH-594-5-20081231.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: NUTCH-594-3-20081229.patch
A completely reworked framework with extension point for serving search results in different format. Included are plugins for serving results in XML and JSON format. XML is the default. Uses JSON-Lib to convert the results into JSON format.
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: ezmorph-1.0.6.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595380#action_12595380 ]
Dennis Kubes commented on NUTCH-594:
------------------------------------
An example of this in action can be found at: http://search.isc.swlabs.org///nutchsearch?query=jav&hitsPerSite=2&lang=en&hitsPerPage=10&type=json
The available parameters are query for typed in query, hitsPerSite for deduping content, lang for the users language, hitsPerPage for the number of hits to return and type as either xml or json. The type defaults to xml. As you can see from above this is very similar to the opensearch servlet currently in nutch. The servlet url is configured in the web.xml as normal for servlets. In this case it defaults to nutchsearch.
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: NUTCH-594-1-20071221.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "wojtek kolodziejczyk (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595297#action_12595297 ]
wojtek kolodziejczyk commented on NUTCH-594:
--------------------------------------------
Could you please, write some short instruction how to configure Nutch to serve results in XML or JSON using this path?
It would be very useful for newbies like me ;)
thanks in advance
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: NUTCH-594-1-20071221.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: ezmorph-1.0.6.jar
ezmorph jar required for framework
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: ezmorph-1.0.6.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes closed NUTCH-594.
------------------------------
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch, NUTCH-594-5-20081231.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: NUTCH-594-3-20081229.patch
Fixed some things. Added the ability to set mime output type using the plugin.xml file. That way people can have application/json or text.json or text/plain, however they want for their application.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Summary: Serve Nutch search results in multiple formats including XML and JSON (was: Serve Nutch search results in XML and JSON)
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559265#action_12559265 ]
Andrzej Bialecki commented on NUTCH-594:
-----------------------------------------
I like the concept of this patch that there could be multiple (configurable) formatters, and the implementation looks ok to me. One minor issue: in some scenarios it's not required to return summaries, so perhaps the formatters should allow for summaries to be null?
Regarding the configuration through web.xml - on one hand this allows you to run several instances of the servlet with different formatters, on the other hand Nutch users expect the configuration to be managed through nutch-site.xml ...
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: NUTCH-594-1-20071221.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: json-lib-2.2.2-jdk15.jar
json lib jar
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660229#action_12660229 ]
Andrzej Bialecki commented on NUTCH-594:
-----------------------------------------
http://www.joelonsoftware.com/uibook/chapters/fog0000000059.html
:P
But seriously, +1 .
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch, NUTCH-594-5-20081231.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: (was: NUTCH-594-3-20081229.patch)
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: commons-beanutils-1.8.0.jar
commons beanutils
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660069#action_12660069 ]
Andrzej Bialecki commented on NUTCH-594:
-----------------------------------------
Issues to be addressed before committing:
* many public classes lack javadoc - this needs to be added, especially since you are adding a new extension point. Please add the class-level, public and protected methods javadoc.
* SimpleDateFormat is not thread-safe (I wish it were!), so this makes *ResponseWriter-s also not thread-safe. I believe this should be fixed, because servlets can be invoked from multiple threads.
* new configuration properties should be documented in nutch-default.xml and they should be given sensible defaults.
* the patch adds SearchServlet to web.xml, so it should ensure also that at least one ResponseWriter plugin is turned on in nutch-default.xml.
* minor nits: I'm not sure that we really need pretty-printing in XMLResponseWriter - indenting takes space (-> bandwidth). There are 2 spurious spaces in plugin/build.xml diffs.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: NUTCH-594-4-20081230.patch
Final patch. Adds the ability to stop summaries from being returned and to only return a given set of fields by name.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in XML and
JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: NUTCH-594-1-20071221.patch
Here is a basic patch (no docs or unit tests) which serve the results in both XML and JSON formats. All fields are returned along with encoded summaries. This patch supplies an interface for creating new formats and configuring through the web.xml file.
> Serve Nutch search results in XML and JSON
> ------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: NUTCH-594-1-20071221.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659830#action_12659830 ]
Andrzej Bialecki commented on NUTCH-594:
-----------------------------------------
Excellent. Re: the text of license - IMHO if both libs are ASL we can skip the license.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-594:
-------------------------------
Attachment: NUTCH-594-5-20081231.patch
New patch addresses current issues.
>> Issues to be addressed before committing:
>> * many public classes lack javadoc - this needs to be added, especially since you are adding a new extension point. Please add the class-level, public and protected methods javadoc.
Added javadoc.
>> * SimpleDateFormat is not thread-safe (I wish it were!), so this makes *ResponseWriter-s also not thread-safe. I believe this should be fixed, because servlets can be invoked from multiple threads.
Wow. I didn't know that. I changed it to be local scoped variable.
>> * new configuration properties should be documented in nutch-default.xml and they should be given sensible defaults.
Done.
>> * the patch adds SearchServlet to web.xml, so it should ensure also that at least one ResponseWriter plugin is turned on in nutch-default.xml.
Done. I turned on both xml and json response writers by default.
>> * minor nits: I'm not sure that we really need pretty-printing in XMLResponseWriter - indenting takes space (-> bandwidth). There are 2 spurious spaces in plugin/build.xml diffs.
I went half way with you on this. I think it is good for development, debugging, but yes better without it for production. I put in a configuration variable to allow either pretty printing or not.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch, NUTCH-594-5-20081231.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12663226#action_12663226 ]
Hudson commented on NUTCH-594:
------------------------------
Integrated in Nutch-trunk #691 (See [http://hudson.zones.apache.org/hudson/job/Nutch-trunk/691/])
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch, NUTCH-594-5-20081231.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659795#action_12659795 ]
Andrzej Bialecki commented on NUTCH-594:
-----------------------------------------
For third-party jars please include their LICENSE statements. Hmm, isn't the JSON library under LGPL?
Other than that, the patch looks great, +1.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659982#action_12659982 ]
Dennis Kubes commented on NUTCH-594:
------------------------------------
I know this patch has only been out for a day or so, but if nobody has any problems with this one, I am going to commit it later tonight or tomorrow morning.
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-594) Serve Nutch search results in
multiple formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659825#action_12659825 ]
Dennis Kubes commented on NUTCH-594:
------------------------------------
JSON-LIb and EZMorph are both under Apache. There is an optional Xom library dependency for JSON-Lib which is not included, that is under LGPL, but everything is Apache.
http://json-lib.sourceforge.net/license.html
http://ezmorph.sourceforge.net/license.html
I put comments about these in the plugin.xml file for response-json. Is there anything else I need to do?
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (NUTCH-594) Serve Nutch search results in multiple
formats including XML and JSON
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes resolved NUTCH-594.
--------------------------------
Resolution: Fixed
committed with revision 730845
> Serve Nutch search results in multiple formats including XML and JSON
> ---------------------------------------------------------------------
>
> Key: NUTCH-594
> URL: https://issues.apache.org/jira/browse/NUTCH-594
> Project: Nutch
> Issue Type: New Feature
> Environment: all
> Reporter: Dennis Kubes
> Assignee: Dennis Kubes
> Attachments: commons-beanutils-1.8.0.jar, commons-collections-3.2.1.jar, ezmorph-1.0.6.jar, json-lib-2.2.2-jdk15.jar, NUTCH-594-1-20071221.patch, NUTCH-594-3-20081229.patch, NUTCH-594-4-20081230.patch, NUTCH-594-5-20081231.patch
>
>
> Allow search results to be served in XML, JSON, and other configurable formats. Right now there is an OpenSearch servlet that returns returns in RSS. I would like something that has more flexibility in terms of the XML being served and also supports other formats such as JSON or plain text.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.