You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pravin Karne <pr...@persistent.co.in> on 2009/10/14 14:45:08 UTC
hadoop configuarions for SOLR-1301 patch
Hi,
I am using SOLR-1301 path. I have build the solr with given patch.
But I am not able to configure Hadoop for above war.
I want to run solr(create index) with 3 nodes (1+2) cluster.
How to do the Hadoop configurations for above patch?
How to set master and slave?
Thanks
-Pravin
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: hadoop configuarions for SOLR-1301 patch
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Wed, Oct 14, 2009 at 6:15 PM, Pravin Karne <pravin_karne@persistent.co.in
> wrote:
> Hi,
> I am using SOLR-1301 path. I have build the solr with given patch.
> But I am not able to configure Hadoop for above war.
>
> I want to run solr(create index) with 3 nodes (1+2) cluster.
>
> How to do the Hadoop configurations for above patch?
> How to set master and slave?
>
>
Pravin, questions on specific patches are best asked on the Jira issue.
--
Regards,
Shalin Shekhar Mangar.
Re: hadoop configuarions for SOLR-1301 patch
Posted by Jason Rutherglen <ja...@gmail.com>.
Pravin,
Hadoop allows jaring libraries into a jar under /lib. So I have
my code, then put apache-solr-core-1.4-dev.jar, and all the
other Solr related jars into it using "ant clean jar && jar uvf
target/myhadoopsolr.jar lib/*". Then on your Hadoop cluster run
something like: "/app/hadoop/bin/hadoop jar myhadoopsolr.jar
com.company.HadoopSolrIndexer"
As far as a more comprehensive installation instructions, like say creating a
Hadoop cluster from scratch and running a this with, it
will be best to start a wiki page, which I'll do shortly.
-J
On Thu, Oct 15, 2009 at 11:01 PM, Pravin Karne
<pr...@persistent.co.in> wrote:
> Hi,
> Patch(SOLR-1301) provides distributed indexing (using Hadoop).
>
> Now I have Hadoop cluster with 1 master and 2 slaves.
>
> Also I have applied above path to solr and build solr.
>
> So how I integrate above solr executables with Hadoop cluster?
>
> Can u please tell what are the steps for this.
>
> Shall I just have copy solr war to Hadoop cluster or what else ?
>
> (Note: I have two setup :
> 1. Hadoop setup
> 2. Solr setup)
>
> So to run distributed indexing how to bridge these two setup?
>
> Thanks
> -Pravin
> -----Original Message-----
> From: Jason Rutherglen [mailto:jason.rutherglen@gmail.com]
> Sent: Friday, October 16, 2009 7:45 AM
> To: solr-user@lucene.apache.org
> Subject: Re: hadoop configuarions for SOLR-1301 patch
>
> Hi Pravin,
>
> You'll need to setup a Hadoop cluster which is independent of
> SOLR-1301. 1301 is for building Solr indexes only, so there
> isn't a master and slave. After building the indexes one needs
> to provision the indexes to Solr servers. In my case I only have
> slaves because I'm not incrementally indexing on the Hadoop
> generated shards.
>
> 1301 does need a Hadoop specific unit test, which I got started
> and need to complete, that could help a little in understanding.
>
> -J
>
> On Wed, Oct 14, 2009 at 5:45 AM, Pravin Karne
> <pr...@persistent.co.in> wrote:
>> Hi,
>> I am using SOLR-1301 path. I have build the solr with given patch.
>> But I am not able to configure Hadoop for above war.
>>
>> I want to run solr(create index) with 3 nodes (1+2) cluster.
>>
>> How to do the Hadoop configurations for above patch?
>> How to set master and slave?
>>
>>
>> Thanks
>> -Pravin
>>
>>
>>
>>
>> DISCLAIMER
>> ==========
>> This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
>>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
>
RE: hadoop configuarions for SOLR-1301 patch
Posted by Pravin Karne <pr...@persistent.co.in>.
Hi,
Patch(SOLR-1301) provides distributed indexing (using Hadoop).
Now I have Hadoop cluster with 1 master and 2 slaves.
Also I have applied above path to solr and build solr.
So how I integrate above solr executables with Hadoop cluster?
Can u please tell what are the steps for this.
Shall I just have copy solr war to Hadoop cluster or what else ?
(Note: I have two setup :
1. Hadoop setup
2. Solr setup)
So to run distributed indexing how to bridge these two setup?
Thanks
-Pravin
-----Original Message-----
From: Jason Rutherglen [mailto:jason.rutherglen@gmail.com]
Sent: Friday, October 16, 2009 7:45 AM
To: solr-user@lucene.apache.org
Subject: Re: hadoop configuarions for SOLR-1301 patch
Hi Pravin,
You'll need to setup a Hadoop cluster which is independent of
SOLR-1301. 1301 is for building Solr indexes only, so there
isn't a master and slave. After building the indexes one needs
to provision the indexes to Solr servers. In my case I only have
slaves because I'm not incrementally indexing on the Hadoop
generated shards.
1301 does need a Hadoop specific unit test, which I got started
and need to complete, that could help a little in understanding.
-J
On Wed, Oct 14, 2009 at 5:45 AM, Pravin Karne
<pr...@persistent.co.in> wrote:
> Hi,
> I am using SOLR-1301 path. I have build the solr with given patch.
> But I am not able to configure Hadoop for above war.
>
> I want to run solr(create index) with 3 nodes (1+2) cluster.
>
> How to do the Hadoop configurations for above patch?
> How to set master and slave?
>
>
> Thanks
> -Pravin
>
>
>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: hadoop configuarions for SOLR-1301 patch
Posted by Jason Rutherglen <ja...@gmail.com>.
Hi Pravin,
You'll need to setup a Hadoop cluster which is independent of
SOLR-1301. 1301 is for building Solr indexes only, so there
isn't a master and slave. After building the indexes one needs
to provision the indexes to Solr servers. In my case I only have
slaves because I'm not incrementally indexing on the Hadoop
generated shards.
1301 does need a Hadoop specific unit test, which I got started
and need to complete, that could help a little in understanding.
-J
On Wed, Oct 14, 2009 at 5:45 AM, Pravin Karne
<pr...@persistent.co.in> wrote:
> Hi,
> I am using SOLR-1301 path. I have build the solr with given patch.
> But I am not able to configure Hadoop for above war.
>
> I want to run solr(create index) with 3 nodes (1+2) cluster.
>
> How to do the Hadoop configurations for above patch?
> How to set master and slave?
>
>
> Thanks
> -Pravin
>
>
>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
>