You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Jayant Kumar Gandhi (JIRA)" <ji...@apache.org> on 2006/11/12 08:36:38 UTC
[jira] Commented: (NUTCH-185) XMLParser is configurable xml parser
plugin.
[ http://issues.apache.org/jira/browse/NUTCH-185?page=comments#action_12449103 ]
Jayant Kumar Gandhi commented on NUTCH-185:
-------------------------------------------
Rida has made an ultimate plugin here. Must have for all people who need to use custom plugin to allow indexing/searching of various fields.
I have successfully got this plugin to work on both 0.7.2 and 0.8.1 version of Nutch.
I have a problem similar to Philippe Eugene above. I have a structure of xml similar to
<tags>
<tag>tag1</tag>
<tag>tag2</tag>
<tag>tag3</tag>
<tags>
If I have the configuration as
<field name="tag" xpath="//Tags/Tag" type="Keyword" boost="1.3" />
The plugin just insert one value for 'tag' and combines all the values and makes a single entry for 'tag'. This makes the content unsearchable by tags are we must give all tags as the whole thing "tag1 tag2 tag3" becomes a keyword.
Instead if I have the configuration as:
<field name="tag" xpath="//Tags/Tag[1]" type="Keyword" boost="1.3" />
<field name="tag" xpath="//Tags/Tag[2]" type="Keyword" boost="1.3" />
<field name="tag" xpath="//Tags/Tag[3]" type="Keyword" boost="1.3" />
...
...
I get the same problem as Philippe, of only having the last value in the index => tag = tag3 will be stored in the index.
I.'m not sure that the problem is on the plugin or is my XPath incorrect. I dont know java much, I am still trying to debug the code to find the cause and solution.
> XMLParser is configurable xml parser plugin.
> --------------------------------------------
>
> Key: NUTCH-185
> URL: http://issues.apache.org/jira/browse/NUTCH-185
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher, indexer
> Affects Versions: 0.7.2, 0.8.1, 0.8
> Environment: OS Independent
> Reporter: Rida Benjelloun
> Attachments: parse-xml.patch, parse-xml.zip, parse-xml.zip
>
>
> Xml parser is configurable plugin. It use XPath and namespaces to do the mapping between the XML elements and Lucene fields.
> Informations :
> 1- Copy "xmlparser-conf.xml" to the nutch/conf dir
> 2- To index your custom XML file, you have to modify the "xmlparser-conf.xml".
> This parser uses namespaces and XPATH to parse XML content
> The config file do the mapping between the XML noeds (using XPATH) and lucene field.
> Example : <field name="dctitle" xpath="//dc:title" type="Text" boost="1.4" />
> 3- The xmlIndexerProperties encapsulate a set of fields associated to a namespace.
> If the namespace is found in the xml document, the fields represented by the namespace will be indexed.
> Example :
> <xmlIndexerProperties type="filePerDocument" namespace=" http://purl.org/dc/elements/1.1/">
> <field name="dctitle" xpath="//dc:title" type="Text" boost=" 1.4" />
> <field name="dccreator" xpath="//dc:creator" type="keyword" boost=" 1.0" />
> </xmlIndexerProperties>
> 4- It is possible to define a default namespace that will be applied when the parser
> didn't find any namespace in the document or when the namespace found in the xml document doesn't match with the namespace defined in the xmlIndexerProperties.
> Example :
> <xmlIndexerProperties type="filePerDocument" namespace="default">
> <field name="xmlcontent" xpath="//*" type="Unstored" boost="1.0" />
> </xmlIndexerProperties>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira