Converting Lucene Demo to search index in DFS

[howard chen <ho...@gmail.com>]

Hi,

I have been playing with Lucene & Hadoop.

The Lucene demo (Web) is a great tutorial for people to understand Lucene,

e.g. http://lucene.apache.org/java/docs/demo3.html

I want to ask

1. If I put the index in the DFS of Hadoop, is it easy to modify the
codes to search in the DFS, rather than local FS? (ignore abt
mapreduce first, I mean just search index in the DFS from web server)

2. More than (1), now if I want to search the index from serveral
running nodes using mapreduce, is the wordcount example a good
starting point?

Thanks for any comments and suggestions.

howa.

Re: Converting Lucene Demo to search index in DFS

[howard chen <ho...@gmail.com>]

On 12/4/06, howard chen <ho...@gmail.com> wrote:
> Hi,
>
> I have been playing with Lucene & Hadoop.
>
> The Lucene demo (Web) is a great tutorial for people to understand Lucene,
>
> e.g. http://lucene.apache.org/java/docs/demo3.html
>
> I want to ask
>
> 1. If I put the index in the DFS of Hadoop, is it easy to modify the
> codes to search in the DFS, rather than local FS? (ignore abt
> mapreduce first, I mean just search index in the DFS from web server)
>
> 2. More than (1), now if I want to search the index from serveral
> running nodes using mapreduce, is the wordcount example a good
> starting point?
>
> Thanks for any comments and suggestions.
>
> howa.
>

in fact, the problem maybe just i don't know how to split the lucene
index, if i can split the index, the flow i suppose is similar to the
word count example.

can anyone experience in nutch can tell me how to deal with index splitting?

thanks.