Custom Generator or ScoringFilter (or Fetch)

[Alexis Hope <ba...@gmail.com>]

Hi All,
(happy new year!)

I've been curious this year to delve further into Nutch. I have been using
generate/fetch/parse/update but noticed some pages get re-crawled before
fetching new segments. From what I understand this is because of the
generators internal ScoringFilter?

My question is how would I prioritise certain content? For example either a
domain or content type, or just unfetched segments.

Looking at the docs for fetching I see the segment parameter to point to
the segments dir. I'm unsure how to user this with Mongo as I dont have a
segments dir (I think).
In the docs for ScoreFilter I see its used with generate, "ScoringFilter is
used within ... which selects ..... a subset of URLs due for fetching".

Should I be looking to solve this with Fetching Segments Directory or a
custom Score Filter?

Advise on either or reference material is welcomed.

Cheers,
Lex