You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Sebastian Nagel (JIRA)" <ji...@apache.org> on 2015/12/06 22:23:11 UTC
[jira] [Resolved] (NUTCH-2172) index-more: document format of
contenttype-mapping.txt
[ https://issues.apache.org/jira/browse/NUTCH-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Nagel resolved NUTCH-2172.
------------------------------------
Resolution: Fixed
Committed to trunk, r1718223.
Thanks, [~nicola.tonellotto]! Although this solution is not that you've suggested, hopefully it will help users to write the content type mappings in the right format.
If a non-comment line does not follow the format the hadoop.log will show now a warning, e.g.:
{noformat}
2015-12-06 22:06:05,169 INFO more.MoreIndexingFilter - Reading content type mappings from file contenttype-mapping.txt
2015-12-06 22:06:05,174 WARN more.MoreIndexingFilter - Wrong format of line: only spaces no tabs
2015-12-06 22:06:05,174 WARN more.MoreIndexingFilter - Expected format: <target type> <tab> <type1> [<tab> <type2> ...]
{noformat}
Btw., multiple tabs as separators are no problem as the original format allows multiple content types to be mapped to one target type.
> index-more: document format of contenttype-mapping.txt
> ------------------------------------------------------
>
> Key: NUTCH-2172
> URL: https://issues.apache.org/jira/browse/NUTCH-2172
> Project: Nutch
> Issue Type: Improvement
> Components: indexer, metadata
> Affects Versions: 1.10
> Environment: Macosx, Java 8
> Reporter: Nicola Tonellotto
> Assignee: Sebastian Nagel
> Priority: Minor
> Labels: easyfix, newbie
> Fix For: 1.12
>
> Attachments: NUTCH-2172-1.patch, NUTCH-2172-2.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> The index-more plugin uses the conf/contenttype-mapping.txt file to build up the mimeMap hash table (in the readConfiguration() method).
> The line splitting is performed around "\t", so it silently skip lines separated by simple spaces or more than one tab (see line 325).
> Changing the single-char string "\t" with the regex "\\s+" should do the magic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)