You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by John Thompson <jo...@gmail.com> on 2008/07/04 15:18:20 UTC

Only crawling out from pages that meet a certain criteria

Is there a way for me to prevent nutch from fetching outlinks from pages
that I decide to be irrelevant (where I make the decision that a page is
irrelevant during the parsing of that page with my parse filter)?  I realize
that I can stop nutch from indexing such pages, but I believe the index is
separate from the structure that determines what new pages should be
fetched.

Best,
John