You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "Feng (Michael) Ji" <fj...@yahoo.com> on 2005/07/23 19:57:17 UTC

search depth

hi there,

I found some weid behavior of Nutch when I use depth =
10.

For example,

"
bin/nutch crawl url1 -dir crawl2 -depth 10 >&
crawl2.log
"

I did a crawling based on one single web site, with
depth 10. Seems it can't finish.

The subdir with segments/ is 7, if it finished
successfully, should be 10.

And I checked the log, it ends in fetching a website
and didn't go on.

Is there a depth limitation for Nutch?

thanks,

Michael,



		
__________________________________ 
Do you Yahoo!? 
Read only the mail you want - Yahoo! Mail SpamGuard. 
http://promotions.yahoo.com/new_mail

search depth

Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.

hi there,
 
I found some weid behavior of Nutch when I use depth
=
10.
 
 For example,
 
 "
 bin/nutch crawl url1 -dir crawl2 -depth 10 >&
 crawl2.log
 "
 
 I did a crawling based on one single web site, with
 depth 10. Seems it can't finish.
 
 The subdir with segments/ is 7, if it finished
 successfully, should be 10.
 
 And I checked the log, it ends in fetching a website
 and didn't go on.
 
 Is there a depth limitation for Nutch?
 
 thanks,
 
 Michael,
 
 
 
 		



		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Find what you need with new enhanced search. 
http://info.mail.yahoo.com/mail_250