You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nutch.apache.org by Apache Wiki <wi...@apache.org> on 2005/10/31 23:32:08 UTC

[Nutch Wiki] Update of "NutchDistributedFileSystem" by PaulBaclace

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Nutch Wiki" for change notification.

The following page has been changed by PaulBaclace:
http://wiki.apache.org/nutch/NutchDistributedFileSystem

The comment on the change is:
minor corrections

------------------------------------------------------------------------------
  
  More details on NDFS operation are coming soon. For now, take a look at the following files, all in src/org/apache/nutch/ndfs/*.java:
  
- NDFS.java has two inner classes, each with a main(). One is for the NameNode?, and one is for the DataNode?. This file has all the network-handling code. Much of the work is handed to other classes.
+ The NameNode daemon cass is NameNode.
+ 
+ A DataNode daemon cass is DataNode. 
  
  FSNamesystem.java handles all the bookkeeping for the NameNode?. It keeps track of where all the blocks are, which DataNodes? are available, etc.
  
@@ -110, +112 @@

  
  "ndfs.name.dir" - Determines where on the local filesystem the NDFS name node should store the name table.
  
- "ndfs.data.dir" - Determines where on the local filesystem the NDFS name node should store the name table.
+ "ndfs.data.dir" - Determines where on the local filesystem the NDFS data node should store the data table. 
  
  "ndfs.replication" - how many copies we try to have at all times (not present in config file)
  
@@ -134, +136 @@

  
  You now have an NDFS installation with one NameNode? and two DataNodes?. (Note, of course, you don't have to run these on different machines. It's enough to use different directories and avoid port conflicts.) DataNodes use port 7000 or greater (they probe to find free port to listen on starting from 7000).
  
- Anywhere, run the client (having fs.default.name = A:9000 in nutch config file): 
+ Anywhere, run the client (having fs.default.name = A:9000 in nutch config file):
+ 
+ (class Test``Client is under src/java not src/test; this is the class that
+ runs for bin/nutch ndfs ...)
  
  $ nutch org.apache.nutch.fs.Test``Client