You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Hiran Chaudhuri (Jira)" <ji...@apache.org> on 2024/10/11 21:38:00 UTC
[jira] [Created] (NUTCH-3075) tld plugin makes injector crash
Hiran Chaudhuri created NUTCH-3075:
--------------------------------------
Summary: tld plugin makes injector crash
Key: NUTCH-3075
URL: https://issues.apache.org/jira/browse/NUTCH-3075
Project: Nutch
Issue Type: Bug
Components: injector
Affects Versions: 1.21
Environment: * Ubuntu 22 LTS
* openjdk version "21.0.4" 2024-07-16 LTS
Reporter: Hiran Chaudhuri
I cloned the current master branch (commit id d6f55b8ea6f5809cef5a31239e5760be23742c00) which nicely compiles to apache-nutch-1.21-SNAPSHOT.job
Even after I added my own protocol-imap implementation. Crawling works to some degree - I am heavily experimenting with IMAP and the data I receive in Solr. Looking at the [IndexStructure|https://cwiki.apache.org/confluence/display/NUTCH/IndexStructure] I hoped to get better information by adding all the mentioned plugins.
Thus I reconfigured nutch-site.xml, especially the `plugin.includes` property to include them all. As soon as `tld` is contained, upon seeding my CrawlDb the injector dies with
{{2024-10-11 23:27:51,519 INFO org.apache.nutch.crawl.Injector [main] Injecting seed URL file file:/home/hiran/NetBeansProjects/nutch/urls/seed.txt}}
{{2024-10-11 23:27:52,778 ERROR org.apache.nutch.crawl.Injector [main] Injector job did not succeed, job id: job_local1500911141_0001, job status: FAILED, reason: NA}}
{{2024-10-11 23:27:52,779 ERROR org.apache.nutch.crawl.Injector [main] Injector: java.lang.RuntimeException: Injector job did not succeed, job id: job_local1500911141_0001, job status: FAILED, reason: NA}}
{{ at org.apache.nutch.crawl.Injector.inject(Injector.java:446)}}
{{ at org.apache.nutch.crawl.Injector.run(Injector.java:574)}}
{{ at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)}}
{{ at org.apache.nutch.crawl.Injector.main(Injector.java:538)}}
The behaviour can be cured by simply removing `tld` from the property.
* Could there be some better error message?
* Why does the tld plugin crash the injector phase at all?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)