Re: Nutch & Cluster

[Francesc Bruguera <fr...@yahoo.es>]

----- Mensaje reenviado ----

De: Francesc Bruguera <fr...@yahoo.es>
Para: nutch-user@lucene.apache.org
Enviado: lunes, 27 de octubre, 2008 18:38:06
Asunto: Re: Nutch & Cluster


My problem is that, I have various servers.
One in Spain, another in Japan, and 2 in the USA (diferent DC)

Can I use the clusering function?





________________________________
De: brainstorm <br...@gmail.com>
Para: nutch-user@lucene.apache.org
CC: francesc_bm@yahoo.es
Enviado: lunes, 27 de octubre, 2008 11:25:44
Asunto: Re: Nutch & Cluster

Hi Francesc,

There are many tutorials out there, this is the one that was most useful to me:

http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Single-Node_Cluster%29

I use rocksclusters.org as the infraestructure.

If you happen to run Solaris, you can try out this one:

http://blogs.sun.com/george/entry/creating_a_virtual_hadoop_cluster

I si vols suport en català, et puc donar un cop de mà ;)

Saluts,
Roman


On Sun, Oct 26, 2008 at 6:39 PM, Francesc Bruguera <fr...@yahoo.es> wrote:
> Hi,
> I want to run Nutch in a cluster of machines. What software do I have to use?
>
> How I can do that?
>
>
>
>


      

Re: Nutch merging options?

[Jianheng Qiu <ki...@gmail.com>]

Hi Alex:

As far as I know, if you delete the segment sub folders every time after
index, the index should be incremental.




On Mon, Nov 3, 2008 at 11:58 PM, Alex Basa <al...@yahoo.com> wrote:

> Hi,
>
> Does anyone have any tips on how to speed up merges?  Also, is there a way
> to do incremental merges?  If I have 10 indexes, is the fastest way to merge
> them into one big index just using mergecrawls.sh and merge one at a time?
>
> Thanks in advance,
>
> Alex
>
>
>
>


-- 
Best Regards,
Jianheng Qiu
kinhunt@gmail.com

Nutch merging options?

[Alex Basa <al...@yahoo.com>]

Hi,

Does anyone have any tips on how to speed up merges?  Also, is there a way to do incremental merges?  If I have 10 indexes, is the fastest way to merge them into one big index just using mergecrawls.sh and merge one at a time?

Thanks in advance,

Alex