You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Scott Simpson <ss...@InterchangeUSA.com> on 2006/04/07 03:17:19 UTC

Confusion about the Hadoop conf/slaves file

It seems that the Hadoop conf/slaves file designates 2 things:
 
1. Where Hadoop should be running (which must be on the search nodes and
the crawl nodes at least with Nutch).
2. Which machines are used for a MapReduce operation.
 
Suppose I want to run Nutch 0.8 searches on separate machines than I
crawl on. Is there a way to separate this so my crawling operation
(MapReduce) doesn't happen on my search machines?
 
Also, is there any to specify the placement of segments on machines?
That is, what if I want all my segments on different nodes than those I
run searches on?

Re: Confusion about the Hadoop conf/slaves file

Posted by Doug Cutting <cu...@apache.org>.

Scott Simpson wrote:
> Suppose I want to run Nutch 0.8 searches on separate machines than I
> crawl on. Is there a way to separate this so my crawling operation
> (MapReduce) doesn't happen on my search machines?

You could have two different configuration directories and set 
HADOOP_CONF_DIR (or use cd).

Doug