You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Manuel Gonzalo <go...@strands.com> on 2010/11/26 18:56:57 UTC
Newbie question on distributed writes onto SolrCloud
Hi all,
while investigating SolrCloud for our distributed search needs I can't seem to find the means to perform distributed writes/updates ... Distributed access to data is available but from what I understood each node has to be accessed independently in order to write/update a value.
Is this so or am I missing something?
Thanks in advance,
Manuel Gonzalo
Software Engineer
http://recommender.strands.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Newbie question on distributed writes onto SolrCloud
Posted by Upayavira <uv...@odoko.co.uk>.
I'm not a solr developer, merely someone who has thought about
implementing some of this stuff one day.
Were I to attempt it, I'd dive in to the existing code that is used to
implement the distributed search. Look at the interfaces that are used,
work out how it manages http requests, etc. Then come up with a
ShardingStrategy interface, and build an initial implementation of that
that follows the patterns I saw when looking at the distributed search
code.
But as I say, I'm not a solr developer at this point, and am happy for
others to chime in with better ideas.
Upayavira
On Mon, 29 Nov 2010 10:57 +0100, "Manuel Gonzalo" <go...@strands.com>
wrote:
> Thanks for your answer.
>
> I would be really interested in developing this system for distributed
> writing. Could you please give me a pair of hints about where I should
> start looking into the code in order to make these modifications? Or is
> it something that should be developed completely outside the current
> code?
>
> Thanks!
>
> On 26/11/2010, at 21:16, Upayavira wrote:
>
> >
> >
> > On Fri, 26 Nov 2010 18:56 +0100, "Manuel Gonzalo" <go...@strands.com>
> > wrote:
> >> Hi all,
> >>
> >> while investigating SolrCloud for our distributed search needs I can't
> >> seem to find the means to perform distributed writes/updates ...
> >> Distributed access to data is available but from what I understood each
> >> node has to be accessed independently in order to write/update a value.
> >>
> >> Is this so or am I missing something?
> >>
> >> Thanks in advance,
> >
> > As I understand it, distributed write is a TODO as a part of SolrCloud.
> >
> > It would require a ShardStrategy interface, and probably a default
> > interface that simply does a MOD shard_count on the document ID.
> >
> > Also, it would have to split incoming posts and distrubute amongst
> > shards (an incoming block of 500 documents would maybe end up as 5 posts
> > of 100 if we had five shards).
> >
> > Upayavira
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> Manuel Gonzalo
> Software Engineer
>
> http://recommender.strands.com
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Newbie question on distributed writes onto SolrCloud
Posted by Manuel Gonzalo <go...@strands.com>.
Thanks for your answer.
I would be really interested in developing this system for distributed writing. Could you please give me a pair of hints about where I should start looking into the code in order to make these modifications? Or is it something that should be developed completely outside the current code?
Thanks!
On 26/11/2010, at 21:16, Upayavira wrote:
>
>
> On Fri, 26 Nov 2010 18:56 +0100, "Manuel Gonzalo" <go...@strands.com>
> wrote:
>> Hi all,
>>
>> while investigating SolrCloud for our distributed search needs I can't
>> seem to find the means to perform distributed writes/updates ...
>> Distributed access to data is available but from what I understood each
>> node has to be accessed independently in order to write/update a value.
>>
>> Is this so or am I missing something?
>>
>> Thanks in advance,
>
> As I understand it, distributed write is a TODO as a part of SolrCloud.
>
> It would require a ShardStrategy interface, and probably a default
> interface that simply does a MOD shard_count on the document ID.
>
> Also, it would have to split incoming posts and distrubute amongst
> shards (an incoming block of 500 documents would maybe end up as 5 posts
> of 100 if we had five shards).
>
> Upayavira
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
Manuel Gonzalo
Software Engineer
http://recommender.strands.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Newbie question on distributed writes onto SolrCloud
Posted by Upayavira <uv...@odoko.co.uk>.
On Fri, 26 Nov 2010 18:56 +0100, "Manuel Gonzalo" <go...@strands.com>
wrote:
> Hi all,
>
> while investigating SolrCloud for our distributed search needs I can't
> seem to find the means to perform distributed writes/updates ...
> Distributed access to data is available but from what I understood each
> node has to be accessed independently in order to write/update a value.
>
> Is this so or am I missing something?
>
> Thanks in advance,
As I understand it, distributed write is a TODO as a part of SolrCloud.
It would require a ShardStrategy interface, and probably a default
interface that simply does a MOD shard_count on the document ID.
Also, it would have to split incoming posts and distrubute amongst
shards (an incoming block of 500 documents would maybe end up as 5 posts
of 100 if we had five shards).
Upayavira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org