You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by "Jarvis.Guo" <nt...@gmail.com> on 2008/08/22 05:04:50 UTC

Questions about lucene index on HDFS

Hi all,
    Firstly I have known that there is a FsDirectory class in Nutch-0.9 so
we can access the index on HDFS. But after I tested it, i found that we can
only read the index but can not to append or modify, I think the reason is
the one mentioned in the HDFS-file append issues, am I right?
    Secondly, now I want to merge indexes on HDFS , when there's only one
computer I must create a temp Index which as the merge result on local file
system using the merge function in Lucene package, then upload it to hdfs,
right? So, when I have a lot of pc, how can I do the index merging more
effeciently? Whatever to use, like hadoop-mapreduce framework or other
distributed platform.

Thanks
Jarvis