You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Gabriele Kahlout (JIRA)" <ji...@apache.org> on 2011/06/20 19:58:47 UTC
[jira] [Created] (NUTCH-1009) Incorrect path when using readdb
Incorrect path when using readdb
--------------------------------
Key: NUTCH-1009
URL: https://issues.apache.org/jira/browse/NUTCH-1009
Project: Nutch
Issue Type: Bug
Affects Versions: 1.3
Reporter: Gabriele Kahlout
Priority: Minor
I've not looked into the code but the path to the data folder is in a part-00000 directory. This seems to occur for unfetched urls only.
{code}
$ $NUTCH_HOME/runtime/deploy/bin/nutch readdb gabriele/crawl/crawldb -url http://www.winonapost.com/stock/functions/VDG_Pub/detail.php?choice=42376&home_page=1&archives=
[1] 7193
[2] 7194
gkahlout@loocia-c1:~/MemoRead/gabriele$ Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://loocia-c1/user/gkahlout/gabriele/crawl/crawldb/current/crawldb-merge-1110533277/data
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:457)
at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:676)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
at org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:302)
at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:284)
at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:273)
at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:260)
at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:253)
at org.apache.hadoop.mapred.MapFileOutputFormat.getReaders(MapFileOutputFormat.java:93)
at org.apache.nutch.crawl.CrawlDbReader.openReaders(CrawlDbReader.java:81)
at org.apache.nutch.crawl.CrawlDbReader.get(CrawlDbReader.java:379)
at org.apache.nutch.crawl.CrawlDbReader.readUrl(CrawlDbReader.java:386)
at org.apache.nutch.crawl.CrawlDbReader.main(CrawlDbReader.java:511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (NUTCH-1009) Incorrect path when using readdb
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma closed NUTCH-1009.
--------------------------------
Resolution: Invalid
This
{code}
[1] 7193
[2] 7194
{code}
is a tell-tale sign that you're not encapsulating an argument containing two ampersands between double quotes.
> Incorrect path when using readdb
> --------------------------------
>
> Key: NUTCH-1009
> URL: https://issues.apache.org/jira/browse/NUTCH-1009
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 1.3
> Reporter: Gabriele Kahlout
> Priority: Minor
>
> I've not looked into the code but the path to the data folder is in a part-00000 directory. This seems to occur for unfetched urls only.
> {code}
> $ $NUTCH_HOME/runtime/deploy/bin/nutch readdb gabriele/crawl/crawldb -url http://www.winonapost.com/stock/functions/VDG_Pub/detail.php?choice=42376&home_page=1&archives=
> [1] 7193
> [2] 7194
> gkahlout@loocia-c1:~/MemoRead/gabriele$ Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://loocia-c1/user/gkahlout/gabriele/crawl/crawldb/current/crawldb-merge-1110533277/data
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:457)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:676)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
> at org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:302)
> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:284)
> at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:273)
> at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:260)
> at org.apache.hadoop.io.MapFile$Reader.<init>(MapFile.java:253)
> at org.apache.hadoop.mapred.MapFileOutputFormat.getReaders(MapFileOutputFormat.java:93)
> at org.apache.nutch.crawl.CrawlDbReader.openReaders(CrawlDbReader.java:81)
> at org.apache.nutch.crawl.CrawlDbReader.get(CrawlDbReader.java:379)
> at org.apache.nutch.crawl.CrawlDbReader.readUrl(CrawlDbReader.java:386)
> at org.apache.nutch.crawl.CrawlDbReader.main(CrawlDbReader.java:511)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira