[jira] [Commented] (NUTCH-585) [PARSE-HTML plugin] Block certain parts of HTML code from being indexed

["Lara Cardozo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533377#comment-14533377 ] 

Lara Cardozo commented on NUTCH-585:
------------------------------------

Hi Bojan Tomic,

Thanks for your contribution. I have tested your plugin and it worked well. But, the creation of a new field (filtered_content) in MySQL haven't worked, the field became NULL on MySQL. I have tried to change the "gora-mysql-mapping.xml" and add the new field, but when i used the inject command, i have got "org.apache.gora.util.GoraException: java.io.IOException: java.lang.NullPointerException". Do I have to do an extra configuration? How can I make it work out?

Thank you.


> [PARSE-HTML plugin] Block certain parts of HTML code from being indexed
> -----------------------------------------------------------------------
>
>                 Key: NUTCH-585
>                 URL: https://issues.apache.org/jira/browse/NUTCH-585
>             Project: Nutch
>          Issue Type: Improvement
>    Affects Versions: 0.9.0
>         Environment: All operating systems
>            Reporter: Andrea Spinelli
>            Assignee: Markus Jelsma
>            Priority: Minor
>             Fix For: 1.11
>
>         Attachments: blacklist_whitelist_plugin.patch, nutch-585-excludeNodes.patch, nutch-585-jostens-excludeDIVs.patch
>
>
> We are using nutch to index our own web sites; we would like not to index certain parts of our pages, because we know they are not relevant (for instance, there are several links to change the background color) and generate spurious matches.
> We have modified the plugin so that it ignores HTML code between certain HTML comments, like
> <!-- START-IGNORE -->
> ... ignored part ...
> <!-- STOP-IGNORE -->
> We feel this might be useful to someone else, maybe factorizing the comment strings as constants in the configuration files (say parser.html.ignore.start and parser.html.ignore.stop in nutch-site.xml).
> We are almost ready to contribute our code snippet.  Looking forward for any expression of  interest - or for an explanation why waht we are doing is plain wrong!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)