[Lucene-hadoop Wiki] Update of "DistributedLucene" by MarkButler

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by MarkButler:
http://wiki.apache.org/lucene-hadoop/DistributedLucene

New page:
= Distributed Lucene =

Doug Cutting's original proposal: http://www.mail-archive.com/general@lucene.apache.org/msg00338.html

== The key classes for the API ==

{{{
public class IndexVersion implements Comparable<IndexVersion>, Writable, Constants {

  private String id;
  private int version;
}
}}}

{{{
public class IndexLocation implements Comparable<IndexLocation>, Writable {

  private IndexVersion indexVersion;
  private InetSocketAddress location;
  private IndexState state;
}
}}}

{{{
  public enum IndexState {
    UNCOMMITTED, REPLICATING, LIVE, UNKNOWN
  };
}}}


== The revised interfaces ==

=== Client to Data node ===

{{{
public interface ClientToDataNodeProtocol extends VersionedProtocol {

  void addDocument(String index, Document doc) throws IOException;

  // Change here, Doug suggested int[] but that is different
  // to current Lucene API

  int removeDocuments(String index, Term term) throws IOException;
  IndexVersion commitVersion(String index) throws IOException;

  // batch update

  void addIndex(String index) throws IOException;
  void addIndex(String index, IndexLocation indexToAdd) throws IOException;

  // search

  SearchResults search(IndexVersion i, Query query, Sort sort, int n) throws IOException;
}
}}}

=== Client to Name node ===

{{{
public interface ClientToNameNodeProtocol extends VersionedProtocol {

  IndexLocation[] getSearchableIndexes();
  IndexLocation getUpdateableIndex(String id);
}
}}}

=== Data node to Data node ===

{{{
public interface DataNodeToDataNodeProtocol extends VersionedProtocol {

  String[] getFileSet(IndexVersion indexVersion) throws IOException;
  byte[] getFileContent(IndexVersion indexVersion, String file)
      throws IOException;
  // based on experience in Hadoop we probably wouldn't really use
  // RPC to find file content, instead HTTP
}
}}}

=== Data node to Name node ===

{{{
public interface DataNodeToNameNodeProtocol extends VersionedProtocol {

  public IndexLocation[] heartbeat(DataNodeStatusInformation datanode,
      IndexLocation[] searchableIndexes) throws RemoteException;
}
}}}