You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Nathan Gass (JIRA)" <ji...@apache.org> on 2012/11/07 18:25:14 UTC
[jira] [Comment Edited] (NUTCH-956) solrindex issues
[ https://issues.apache.org/jira/browse/NUTCH-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492521#comment-13492521 ]
Nathan Gass edited comment on NUTCH-956 at 11/7/12 5:23 PM:
------------------------------------------------------------
I could reproduce the NPE with nutch 2.1 (I at least believe it is the same):
java.lang.NullPointerException
at org.apache.nutch.indexer.more.MoreIndexingFilter.addType(MoreIndexingFilter.java:200)
at org.apache.nutch.indexer.more.MoreIndexingFilter.filter(MoreIndexingFilter.java:75)
at org.apache.nutch.indexer.IndexingFilters.filter(IndexingFilters.java:107)
at org.apache.nutch.indexer.IndexUtil.index(IndexUtil.java:64)
at org.apache.nutch.indexer.IndexerJob$IndexerMapper.map(IndexerJob.java:103)
at org.apache.nutch.indexer.IndexerJob$IndexerMapper.map(IndexerJob.java:61)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
This only happens when moreIndexingFilter.indexMimeTypeParts is set. Looking at the code, the following fix seems to be more consistent with the code around it, as it actually splits the *indexed* mimeType instead of a different, possibly null, string:
{noformat}
Index: src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java
===================================================================
--- src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java (revision 1406665)
+++ src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java (working copy)
@@ -197,7 +197,7 @@
// Check if we need to split the content type in sub parts
if (conf.getBoolean("moreIndexingFilter.indexMimeTypeParts", true)) {
- String[] parts = getParts(contentType.toString());
+ String[] parts = getParts(mimeType);
for(String part: parts) {
doc.add("type", part);
{noformat}
was (Author: xabbu42):
I could reproduce the NPE with nutch 2.1 (I at least believe it is the same):
java.lang.NullPointerException
at org.apache.nutch.indexer.more.MoreIndexingFilter.addType(MoreIndexingFilter.java:200)
at org.apache.nutch.indexer.more.MoreIndexingFilter.filter(MoreIndexingFilter.java:75)
at org.apache.nutch.indexer.IndexingFilters.filter(IndexingFilters.java:107)
at org.apache.nutch.indexer.IndexUtil.index(IndexUtil.java:64)
at org.apache.nutch.indexer.IndexerJob$IndexerMapper.map(IndexerJob.java:103)
at org.apache.nutch.indexer.IndexerJob$IndexerMapper.map(IndexerJob.java:61)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
This only happens when moreIndexingFilter.indexMimeTypeParts is set. Looking at the code, the following fix seems to be more consistent with the code around it, as it actually splits the *indexed* mimeType instead of a different, possibly null, string:
Index: src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java
===================================================================
--- src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java (revision 1406665)
+++ src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java (working copy)
@@ -197,7 +197,7 @@
// Check if we need to split the content type in sub parts
if (conf.getBoolean("moreIndexingFilter.indexMimeTypeParts", true)) {
- String[] parts = getParts(contentType.toString());
+ String[] parts = getParts(mimeType);
for(String part: parts) {
doc.add("type", part);
> solrindex issues
> ----------------
>
> Key: NUTCH-956
> URL: https://issues.apache.org/jira/browse/NUTCH-956
> Project: Nutch
> Issue Type: Bug
> Components: indexer
> Affects Versions: nutchgora
> Reporter: Alexis
> Fix For: 2.2
>
> Attachments: solr.patch, solr.patch2
>
>
> I ran into a few caveats with solrindex command trying to index documents.
> Please refer to http://techvineyard.blogspot.com/2010/12/build-nutch-20.html#solrindex that describes my tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira