You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by zud <pr...@gmail.com> on 2010/01/11 10:12:30 UTC

crawl result is empty

i have run nutch 1.0 in eclipse in linux every thing worked fine but in the
output folder the index file is showing as below �	x��
the link database also shows empty 

i have changed crawl urlfilter.txt to site which i want to crawl 
but the results is empty  i cant see my site is crawled can any one tell
what is the problem  

-- 
View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27107582.html
Sent from the Nutch - User mailing list archive at Nabble.com.


Re: crawl result is empty

Posted by zud <pr...@gmail.com>.
thanks for the reply it is working fine but in the crawl Database dump the
modified time field is showing  incorrect 

for the diffrent urls i tried but the modified time is same as shown below

http://andhravilas.com/	Version: 7
Status: 4 (db_redir_temp)
Fetch time: Thu Feb 11 13:55:51 IST 2010
Modified time: Thu Jan 01 05:30:00 IST 1970
Retries since fetch: 0
Retry interval: 2592000 seconds (30 days)
Score: 1.0
Signature: null
Metadata: _pst_: temp_moved(13), lastModified=0:
http://andhravilas.com/dynamicindex.asp


please help me out
 
-- 
View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27124281.html
Sent from the Nutch - User mailing list archive at Nabble.com.


Re: crawl result is empty

Posted by Mischa Tuffield <mi...@garlik.com>.
You should be able to dump your crawldb, using a command along the lines of  :

nutch readdb yourcrawldir/crawldb -dump someDumpDir -format csv 

But you could quite well be having different problems...

Mischa
On 11 Jan 2010, at 12:27, zud wrote:

> 
> 
> thanks for reply
> 
> the content folder contains some data in un readable format but the files in
> link database and index folders are empty
> 
> how can i view the fetched results ........
> 
> 
> -- 
> View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27109669.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
> 

___________________________________
Mischa Tuffield
Email: mischa.tuffield@garlik.com
Homepage - http://mmt.me.uk/
Garlik Limited, 2 Sheen Road, Richmond, TW9 1AE, UK
+44(0)20 8973 2465  http://www.garlik.com/
Registered in England and Wales 535 7233 VAT # 849 0517 11
Registered office: Thames House, Portsmouth Road, Esher, Surrey, KT10 9AD


Re: crawl result is empty

Posted by zud <pr...@gmail.com>.

thanks for reply

the content folder contains some data in un readable format but the files in
link database and index folders are empty

how can i view the fetched results ........


-- 
View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27109669.html
Sent from the Nutch - User mailing list archive at Nabble.com.


Re: crawl result is empty

Posted by Mischa Tuffield <mi...@garlik.com>.
Hi,

Perhaps you are crawling and writing to the hdfs? Have you checked the  
directory structure of the nutch user in your hadoop dfs? I was caught  
out by that early on.

Mischa

Sent on the move

On 11 Jan 2010, at 09:12, zud <pr...@gmail.com> wrote:

>
> i have run nutch 1.0 in eclipse in linux every thing worked fine but  
> in the
> output folder the index file is showing as below �    x��
> the link database also shows empty
>
> i have changed crawl urlfilter.txt to site which i want to crawl
> but the results is empty  i cant see my site is crawled can any one  
> tell
> what is the problem
>
> -- 
> View this message in context: http://old.nabble.com/crawl-result-is-empty-tp27107582p27107582.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>