You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by zud <pr...@gmail.com> on 2010/01/11 10:12:30 UTC
crawl result is empty
i have run nutch 1.0 in eclipse in linux every thing worked fine but in the
output folder the index file is showing as below � x��
the link database also shows empty
i have changed crawl urlfilter.txt to site which i want to crawl
but the results is empty i cant see my site is crawled can any one tell
what is the problem
--
View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27107582.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: crawl result is empty
Posted by zud <pr...@gmail.com>.
thanks for the reply it is working fine but in the crawl Database dump the
modified time field is showing incorrect
for the diffrent urls i tried but the modified time is same as shown below
http://andhravilas.com/ Version: 7
Status: 4 (db_redir_temp)
Fetch time: Thu Feb 11 13:55:51 IST 2010
Modified time: Thu Jan 01 05:30:00 IST 1970
Retries since fetch: 0
Retry interval: 2592000 seconds (30 days)
Score: 1.0
Signature: null
Metadata: _pst_: temp_moved(13), lastModified=0:
http://andhravilas.com/dynamicindex.asp
please help me out
--
View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27124281.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: crawl result is empty
Posted by Mischa Tuffield <mi...@garlik.com>.
You should be able to dump your crawldb, using a command along the lines of :
nutch readdb yourcrawldir/crawldb -dump someDumpDir -format csv
But you could quite well be having different problems...
Mischa
On 11 Jan 2010, at 12:27, zud wrote:
>
>
> thanks for reply
>
> the content folder contains some data in un readable format but the files in
> link database and index folders are empty
>
> how can i view the fetched results ........
>
>
> --
> View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27109669.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
___________________________________
Mischa Tuffield
Email: mischa.tuffield@garlik.com
Homepage - http://mmt.me.uk/
Garlik Limited, 2 Sheen Road, Richmond, TW9 1AE, UK
+44(0)20 8973 2465 http://www.garlik.com/
Registered in England and Wales 535 7233 VAT # 849 0517 11
Registered office: Thames House, Portsmouth Road, Esher, Surrey, KT10 9AD
Re: crawl result is empty
Posted by zud <pr...@gmail.com>.
thanks for reply
the content folder contains some data in un readable format but the files in
link database and index folders are empty
how can i view the fetched results ........
--
View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27109669.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: crawl result is empty
Posted by Mischa Tuffield <mi...@garlik.com>.
Hi,
Perhaps you are crawling and writing to the hdfs? Have you checked the
directory structure of the nutch user in your hadoop dfs? I was caught
out by that early on.
Mischa
Sent on the move
On 11 Jan 2010, at 09:12, zud <pr...@gmail.com> wrote:
>
> i have run nutch 1.0 in eclipse in linux every thing worked fine but
> in the
> output folder the index file is showing as below � x��
> the link database also shows empty
>
> i have changed crawl urlfilter.txt to site which i want to crawl
> but the results is empty i cant see my site is crawled can any one
> tell
> what is the problem
>
> --
> View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27107582.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>