You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Emilijan Mirceski <em...@cpuedge.com> on 2005/07/01 07:22:16 UTC

regex url filter

If in my regex-urlfilter:

>> # skip URLs containing certain characters as probable queries, etc.
>> -[?*!@=]

i skip '?' and '=', I will have more pages in my database. 

Is there any strong reason why this was disabled in the release version? 
(My segments have about ~100 thousand pages total, which is barely 1.2 GB)

Regards,
Emilijan