You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by Sherjeel Niazi <sh...@softmatics.com> on 2009/04/17 09:42:49 UTC

How to crawl every URL of website

Hello there,

I would like to know that if I have to crawl every URL of a web site (I
myself do not know how many URL's are there).
Website structure is like:
   www.domain.com/record/1
   www.domain.com/record/2
   www.domain.com/record/3
   ......................
   so on



Thanks,
Sherjeel