[jira] Commented: (NUTCH-51) Removing a plugin after fetch but before indexing causes errors

["Piotr Kosiorowski (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/NUTCH-51?page=comments#action_63652 ]
     
Piotr Kosiorowski commented on NUTCH-51:
----------------------------------------

Looking at initialization log I cannot find any kind of IndexingFilter plugin included. index-basic plugin is not included. Probably you should look at your nutch config file to enable index-basic plugin. Look at plugin.includes or plugin.excludes properties.
I hope it will help.
Regards,
Piotr

> Removing a plugin after fetch but before indexing causes errors
> ---------------------------------------------------------------
>
>          Key: NUTCH-51
>          URL: http://issues.apache.org/jira/browse/NUTCH-51
>      Project: Nutch
>         Type: Bug
>   Components: indexer
>  Environment: Linux/Jdk 1.4.2
>     Reporter: byron miller

>
> I created a segment of 4 million pages with a normal fetch. I created an index on this segment and it took over 30 hours to complete - when it did i had errors running a search that the nutchbean couldn't find the title value.
> I did some researching and it looks like i should have ran a dedup or something else.
> So to debug i tried to re-index without some of the plugins enabled, and that is when i get this error:
> root@elect [/home2/mozdex/nutch]# bin/nutch segread -fix segments/20050420222001/
> run with heapsize 1500
> -Xmx1500m
> 050424 180125 parsing file:/home2/mozdex/nutch/conf/nutch-default.xml
> 050424 180126 parsing file:/home2/mozdex/nutch/conf/nutch-site.xml
> 050424 180126 No FS indicated, using default:local
> root@elect [/home2/mozdex/nutch]# bin/nutch index segments/20050420222001 -dir /home/work
> run with heapsize 1500
> -Xmx1500m
> 050424 180152 parsing file:/home2/mozdex/nutch/conf/nutch-default.xml
> 050424 180152 parsing file:/home2/mozdex/nutch/conf/nutch-site.xml
> 050424 180152 No FS indicated, using default:local
> 050424 180152 indexing segment: segments/20050420222001
> 050424 180152 * Opening segment 20050420222001
> 050424 180153 * Indexing segment 20050420222001
> 050424 180153 Plugins: looking in: /home2/mozdex/nutch/build/plugins
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/protocol-http/plugin.xml
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/parse-html/plugin.xml
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/parse-text/plugin.xml
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/parse-pdf/plugin.xml
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/parse-msword
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/parse-ext
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/index-basic
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/query-basic/plugin.xml
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/query-more
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/query-site/plugin.xml
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/query-url/plugin.xml
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/urlfilter-regex/plugin.xml
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/urlfilter-prefix
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/creativecommons
> 050424 180154 not including: /home2/mozdex/nutch/build/plugins/language-identifier
> 050424 180154 parsing: /home2/mozdex/nutch/build/plugins/clustering-carrot2/plugin.xml
> 050424 180155 parsing: /home2/mozdex/nutch/build/plugins/ontology/plugin.xml
> Exception in thread "main" java.lang.ExceptionInInitializerError
>         at org.apache.nutch.indexer.IndexSegment.indexPages(IndexSegment.java:145)
>         at org.apache.nutch.indexer.IndexSegment.main(IndexSegment.java:254)
> Caused by: java.lang.RuntimeException: org.apache.nutch.indexer.IndexingFilter not found.
>         at org.apache.nutch.indexer.IndexingFilters.<clinit>(IndexingFilters.java:36)
>         ... 2 more
> It seems when you generate a fetchlist with the plugin you can't process the index without the plugin.
> If this is true, a friendly error message would be nice and perhaps a utility to "fix" the segment so it can be processed without the plugin.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira