You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by "Chaushu, Shani" <sh...@intel.com> on 2015/04/29 14:50:47 UTC

crawl into the same folder twtice

Hi,
I wanted to ask what happen if I crawl into the same folder twice different seed.
I saw that in the second time even if I give different seed it crawls the previous seed as well.
If I want to crawl the same pages few times, should I do it into the same folder for merging duplications? Or it will continue to crawl from the last time segments?

Thanks,
Shani

---------------------------------------------------------------------
Intel Electronics Ltd.

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.