You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nutch.apache.org by Apache Wiki <wi...@apache.org> on 2006/10/31 12:26:54 UTC
[Nutch Wiki] Update of "Nutch - The Java Search Engine" by ThorstenScherler
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Nutch Wiki" for change notification.
The following page has been changed by ThorstenScherler:
http://wiki.apache.org/nutch/Nutch_-_The_Java_Search_Engine
The comment on the change is:
Adding more information about working with trunk
------------------------------------------------------------------------------
=== 3.2.2 Edit the file conf/crawl-urlfilter.txt ===
+ If you are using TRUNK then there is no file called conf/crawl-urlfilter.txt but conf/crawl-urlfilter.txt. Just do
+ {{{
+ cat conf/crawl-urlfilter.txt.template|sed 's/MY.DOMAIN.NAME/criaturitas.org/'g> conf/crawl-urlfilter.txt
+ }}}
- and replace the existing domain name with the name of the domain you wish to crawl. For example, if you wished to limit the crawl to the virtusa.com domain, the line should read:
+ If you already have this file then replace the existing domain name with the name of the domain you wish to crawl. For example, if you wished to limit the crawl to the virtusa.com domain, the line should read:
{{{ +^http://([a-z0-9]*\.)*virtusa.com/ }}}