You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Markus Jelsma <ma...@openindex.io> on 2011/08/01 13:12:38 UTC

Re: Change user-agent in runtime

There are no such settings in Nutch. You can adapt the fetcher to do this.

On Sunday 31 July 2011 17:50:03 kushti wrote:
> I need to change user-agent string for each crawled domain. I use standard
> Nutch crawl utility code, it crawls one domain per time. It's been started
> in multithreading mode to crawl many domains. I need to pass to domain
> string [botname]+domainID. How to implement it?
> 
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Change-user-agent-in-runtime-tp3213713p
> 3213713.html Sent from the Nutch - User mailing list archive at Nabble.com.

-- 
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536620 / 06-50258350