You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Philippe (JIRA)" <ji...@apache.org> on 2012/09/06 11:19:07 UTC
[jira] [Created] (NUTCH-1466) SolrDeleteDuplicates:
java.io.IOException: Job failed!
Philippe created NUTCH-1466:
-------------------------------
Summary: SolrDeleteDuplicates: java.io.IOException: Job failed!
Key: NUTCH-1466
URL: https://issues.apache.org/jira/browse/NUTCH-1466
Project: Nutch
Issue Type: Bug
Components: indexer
Affects Versions: 1.5.1
Environment: Ubuntu
Reporter: Philippe
Hi there,
i'm facing a problem witch just comes up by a specific site. When i try to crawl "http://www.stimme.de/sport/", the crawl process works fine. When it comes to the SolrDeleteDuplicates part it throws the following error.
ERROR:
SolrIndexer: starting at 2012-09-06 09:40:37
SolrIndexer: deleting gone documents: false
SolrIndexer: URL filtering: false
SolrIndexer: URL normalizing: false
java.io.IOException: Job failed!
SolrDeleteDuplicates: starting at 2012-09-06 09:41:20
SolrDeleteDuplicates: Solr url: http://192.168.1.144:8983/solr/
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:373)
at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:353)
at org.apache.nutch.crawl.Crawl.run(Crawl.java:193)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:63)
Since i read this thread about the similar problem http://lucene.472066.n3.nabble.com/SolrIndex-java-io-IOException-Job-failed-td3585509.html i rebuild with solr-solj-3.5 but still the same error.
Any ideas how to fix that?
I'm using solr-3.4 as Solr-Server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1466) SolrDeleteDuplicates:
java.io.IOException: Job failed!
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449561#comment-13449561 ]
Lewis John McGibbney commented on NUTCH-1466:
---------------------------------------------
Thanks for reporting this issue but please go to our mailing list for such instances. This doesn't look like a bug and therefore shouldn't be marked as such. I am going to close this as not an issue.
> SolrDeleteDuplicates: java.io.IOException: Job failed!
> ------------------------------------------------------
>
> Key: NUTCH-1466
> URL: https://issues.apache.org/jira/browse/NUTCH-1466
> Project: Nutch
> Issue Type: Bug
> Components: indexer
> Affects Versions: 1.5.1
> Environment: Ubuntu
> Reporter: Philippe
>
> Hi there,
> i'm facing a problem witch just comes up by a specific site. When i try to crawl "http://www.stimme.de/sport/", the crawl process works fine. When it comes to the SolrDeleteDuplicates part it throws the following error.
> ERROR:
> SolrIndexer: starting at 2012-09-06 09:40:37
> SolrIndexer: deleting gone documents: false
> SolrIndexer: URL filtering: false
> SolrIndexer: URL normalizing: false
> java.io.IOException: Job failed!
> SolrDeleteDuplicates: starting at 2012-09-06 09:41:20
> SolrDeleteDuplicates: Solr url: http://192.168.1.144:8983/solr/
> Exception in thread "main" java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
> at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:373)
> at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:353)
> at org.apache.nutch.crawl.Crawl.run(Crawl.java:193)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.crawl.Crawl.main(Crawl.java:63)
> Since i read this thread about the similar problem http://lucene.472066.n3.nabble.com/SolrIndex-java-io-IOException-Job-failed-td3585509.html i rebuild with solr-solj-3.5 but still the same error.
> Any ideas how to fix that?
> I'm using solr-3.4 as Solr-Server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (NUTCH-1466) SolrDeleteDuplicates:
java.io.IOException: Job failed!
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney resolved NUTCH-1466.
-----------------------------------------
Resolution: Not A Problem
as per my previous comment
> SolrDeleteDuplicates: java.io.IOException: Job failed!
> ------------------------------------------------------
>
> Key: NUTCH-1466
> URL: https://issues.apache.org/jira/browse/NUTCH-1466
> Project: Nutch
> Issue Type: Bug
> Components: indexer
> Affects Versions: 1.5.1
> Environment: Ubuntu
> Reporter: Philippe
>
> Hi there,
> i'm facing a problem witch just comes up by a specific site. When i try to crawl "http://www.stimme.de/sport/", the crawl process works fine. When it comes to the SolrDeleteDuplicates part it throws the following error.
> ERROR:
> SolrIndexer: starting at 2012-09-06 09:40:37
> SolrIndexer: deleting gone documents: false
> SolrIndexer: URL filtering: false
> SolrIndexer: URL normalizing: false
> java.io.IOException: Job failed!
> SolrDeleteDuplicates: starting at 2012-09-06 09:41:20
> SolrDeleteDuplicates: Solr url: http://192.168.1.144:8983/solr/
> Exception in thread "main" java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
> at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:373)
> at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:353)
> at org.apache.nutch.crawl.Crawl.run(Crawl.java:193)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.crawl.Crawl.main(Crawl.java:63)
> Since i read this thread about the similar problem http://lucene.472066.n3.nabble.com/SolrIndex-java-io-IOException-Job-failed-td3585509.html i rebuild with solr-solj-3.5 but still the same error.
> Any ideas how to fix that?
> I'm using solr-3.4 as Solr-Server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1466) SolrDeleteDuplicates:
java.io.IOException: Job failed!
Posted by "Philippe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449611#comment-13449611 ]
Philippe commented on NUTCH-1466:
---------------------------------
Hi there,
i was looking for a solution in different mailing lists for a while now. Nutch simply doesn't index this specific page. A crawl without the solr parameter works fine. Just by indexing to solr (and this is a nutch component) it throws this error. There might be something wrong with the SolrDeleteDuplicates routine. Because of this exceptions nutch doesn't index any of the crawled pages (all over crawl time >30 min.).
Since this is a regular page to crawl i thought this is suppose to be a bug, because all other pages are doing well.
Any suggestions?
> SolrDeleteDuplicates: java.io.IOException: Job failed!
> ------------------------------------------------------
>
> Key: NUTCH-1466
> URL: https://issues.apache.org/jira/browse/NUTCH-1466
> Project: Nutch
> Issue Type: Bug
> Components: indexer
> Affects Versions: 1.5.1
> Environment: Ubuntu
> Reporter: Philippe
>
> Hi there,
> i'm facing a problem witch just comes up by a specific site. When i try to crawl "http://www.stimme.de/sport/", the crawl process works fine. When it comes to the SolrDeleteDuplicates part it throws the following error.
> ERROR:
> SolrIndexer: starting at 2012-09-06 09:40:37
> SolrIndexer: deleting gone documents: false
> SolrIndexer: URL filtering: false
> SolrIndexer: URL normalizing: false
> java.io.IOException: Job failed!
> SolrDeleteDuplicates: starting at 2012-09-06 09:41:20
> SolrDeleteDuplicates: Solr url: http://192.168.1.144:8983/solr/
> Exception in thread "main" java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
> at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:373)
> at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:353)
> at org.apache.nutch.crawl.Crawl.run(Crawl.java:193)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.crawl.Crawl.main(Crawl.java:63)
> Since i read this thread about the similar problem http://lucene.472066.n3.nabble.com/SolrIndex-java-io-IOException-Job-failed-td3585509.html i rebuild with solr-solj-3.5 but still the same error.
> Any ideas how to fix that?
> I'm using solr-3.4 as Solr-Server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira