You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by "George A. Papayiannis" <pa...@gmail.com> on 2005/08/22 01:46:25 UTC

Combining multiple index to single index

Hi,

I am in a scenario where I have many small indexes and would like to merge 
them into a single index. I know I could do searches from multiple indexes, 
but I would rather combine them. Does anyone know of a way to do this, other 
than using the Lucene API?

Thanks,
George

Re: Combining multiple index to single index

Posted by "George A. Papayiannis" <pa...@gmail.com>.
Hi,

Thanks for the reply -- I using that script -- I see now what I was doing 
wrong -- but still I'm not completly satisfied

I have my directories of:

./db
./segments (has many segments inside each indexed in its segment)
./index (main index)

I can I would run: nutch merge indexTemp segments/* ./
This would merge all the segment index's and the main index into directory 
indexTemp

Then I would overwrite indexTemp with index

in any case, playing with it more, i'm sure i'll figure something out

George

On 8/21/05, Michael Ji <fj...@yahoo.com> wrote:
> 
> I mean using "bin/nutch merge"
> 
> --- "George A. Papayiannis" <pa...@gmail.com>
> wrote:
> 
> > Hi,
> >
> > I am in a scenario where I have many small indexes
> > and would like to merge
> > them into a single index. I know I could do searches
> > from multiple indexes,
> > but I would rather combine them. Does anyone know of
> > a way to do this, other
> > than using the Lucene API?
> >
> > Thanks,
> > George
> >
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>

Re: Combining multiple index to single index

Posted by Michael Ji <fj...@yahoo.com>.
I mean using "bin/nutch merge"

--- "George A. Papayiannis" <pa...@gmail.com>
wrote:

> Hi,
> 
> I am in a scenario where I have many small indexes
> and would like to merge 
> them into a single index. I know I could do searches
> from multiple indexes, 
> but I would rather combine them. Does anyone know of
> a way to do this, other 
> than using the Lucene API?
> 
> Thanks,
> George
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

Re: Combining multiple index to single index

Posted by Michael Ji <fj...@yahoo.com>.
You can use nutch built-in script to do index merging;

run "bin/nutch mergesegs" to see the required
parameters;

Michael Ji


--- "George A. Papayiannis" <pa...@gmail.com>
wrote:

> Hi,
> 
> I am in a scenario where I have many small indexes
> and would like to merge 
> them into a single index. I know I could do searches
> from multiple indexes, 
> but I would rather combine them. Does anyone know of
> a way to do this, other 
> than using the Lucene API?
> 
> Thanks,
> George
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com