You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rafał Kuć <r....@solr.pl> on 2011/11/09 18:52:07 UTC
Distributed indexing
Hello!
I was looking for a way to implement distributed indexing in Solr.
From looking at the https://issues.apache.org/jira/browse/SOLR-2358
there was some work done to enable Solr to distribute the documents to
shards without the need of 3rd party software before Solr. What I
would like to know if this is the road Solr will take to make things
work. Maybe there was some additional work done with distributed
indexing ?
--
Regards,
Rafał Kuć
Re: Distributed indexing
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
Thanks for the answer :)
--
Regards,
Rafał Kuć
> On Nov 9, 2011, at 12:52 PM, Rafał Kuć wrote:
>> Hello!
>>
>> I was looking for a way to implement distributed indexing in Solr.
>> From looking at the https://issues.apache.org/jira/browse/SOLR-2358
>> there was some work done to enable Solr to distribute the documents to
>> shards without the need of 3rd party software before Solr. What I
>> would like to know if this is the road Solr will take to make things
>> work.
> Yup - we are slowly marching down it on the solrcloud branch in
> svn. We have some work to do to finish the basic scaffolding, and
> clean some things up - we should be able to start working on recovery soon.
>> Maybe there was some additional work done with distributed
>> indexing ?
> Yeah, we have made a bit of progress on the solrcloud branch - but there is still much to do!
> At a very simple level, there is an impl of a distributed update
> handler that works in the simple case attached to that issue, but
> it's got lots of limitations that we will hopefully address with the
> in-progress Distributed Indexing work.
>>
>> --
>> Regards,
>> Rafał Kuć
>>
> - Mark Miller
> lucidimagination.com
Re: Distributed indexing
Posted by Mark Miller <ma...@gmail.com>.
On Nov 9, 2011, at 12:52 PM, Rafał Kuć wrote:
> Hello!
>
> I was looking for a way to implement distributed indexing in Solr.
> From looking at the https://issues.apache.org/jira/browse/SOLR-2358
> there was some work done to enable Solr to distribute the documents to
> shards without the need of 3rd party software before Solr. What I
> would like to know if this is the road Solr will take to make things
> work.
Yup - we are slowly marching down it on the solrcloud branch in svn. We have some work to do to finish the basic scaffolding, and clean some things up - we should be able to start working on recovery soon.
> Maybe there was some additional work done with distributed
> indexing ?
Yeah, we have made a bit of progress on the solrcloud branch - but there is still much to do!
At a very simple level, there is an impl of a distributed update handler that works in the simple case attached to that issue, but it's got lots of limitations that we will hopefully address with the in-progress Distributed Indexing work.
>
> --
> Regards,
> Rafał Kuć
>
- Mark Miller
lucidimagination.com
Re: Distributed indexing
Posted by Mark Miller <ma...@gmail.com>.
There is a distrib indexing branch called solrcloud. Anyone that wants to contribute should check that branch out.
I have various code I have not put up yet for review because I'm still working on it - but the main action happens on that branch.
I'm not working on every issue at the moment though - just a couple. Been playing around with tests locally most recently. Also working on merging some of the Version work Yonik recently committed with the Distributed update processor. I've committed a WIP to the branch of that, but I've been cleaning it up while working on other other things.
- Mark Miller
lucidimagination.com
On Nov 11, 2011, at 2:38 PM, Otis Gospodnetic wrote:
> Hi,
>
> I think at least a part of the question here is whether what's in JIRA is the latest stuff or if Mark or Yonik or ... have some patches that they could put in JIRA, so that others can work off of them. For example, I just looked at these 3 issues:
>
> https://issues.apache.org/jira/browse/SOLR-2358
>
> https://issues.apache.org/jira/browse/SOLR-2341
>
> https://issues.apache.org/jira/browse/SOLR-2355
>
>
> All 3 have patches, but they are all from February, and presumably people have newer stuff in their local IDEs or repos?
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
>> ________________________________
>> From: Erick Erickson <er...@gmail.com>
>> To: solr-user@lucene.apache.org
>> Sent: Friday, November 11, 2011 8:47 AM
>> Subject: Re: Distributed indexing
>>
>> As you can see, there are a lot of related issues. Solr Cloud
>> is where this action is happening, and there's a lot of
>> active development work going on in this area. But the short
>> form is you'll have to wait for that release (you can pitch
>> in and help if you'd like)....
>>
>> The release date for all this is uncertain....
>>
>> Best
>> Erick
>>
>> 2011/11/9 Rafał Kuć <r....@solr.pl>:
>>> Hello!
>>>
>>> I was looking for a way to implement distributed indexing in Solr.
>>> From looking at the https://issues.apache.org/jira/browse/SOLR-2358
>>> there was some work done to enable Solr to distribute the documents to
>>> shards without the need of 3rd party software before Solr. What I
>>> would like to know if this is the road Solr will take to make things
>>> work. Maybe there was some additional work done with distributed
>>> indexing ?
>>>
>>> --
>>> Regards,
>>> Rafał Kuć
>>>
>>>
>>
>>
Re: Distributed indexing
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi,
I think at least a part of the question here is whether what's in JIRA is the latest stuff or if Mark or Yonik or ... have some patches that they could put in JIRA, so that others can work off of them. For example, I just looked at these 3 issues:
https://issues.apache.org/jira/browse/SOLR-2358
https://issues.apache.org/jira/browse/SOLR-2341
https://issues.apache.org/jira/browse/SOLR-2355
All 3 have patches, but they are all from February, and presumably people have newer stuff in their local IDEs or repos?
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
>________________________________
>From: Erick Erickson <er...@gmail.com>
>To: solr-user@lucene.apache.org
>Sent: Friday, November 11, 2011 8:47 AM
>Subject: Re: Distributed indexing
>
>As you can see, there are a lot of related issues. Solr Cloud
>is where this action is happening, and there's a lot of
>active development work going on in this area. But the short
>form is you'll have to wait for that release (you can pitch
>in and help if you'd like)....
>
>The release date for all this is uncertain....
>
>Best
>Erick
>
>2011/11/9 Rafał Kuć <r....@solr.pl>:
>> Hello!
>>
>> I was looking for a way to implement distributed indexing in Solr.
>> From looking at the https://issues.apache.org/jira/browse/SOLR-2358
>> there was some work done to enable Solr to distribute the documents to
>> shards without the need of 3rd party software before Solr. What I
>> would like to know if this is the road Solr will take to make things
>> work. Maybe there was some additional work done with distributed
>> indexing ?
>>
>> --
>> Regards,
>> Rafał Kuć
>>
>>
>
>
>
Re: Distributed indexing
Posted by Erick Erickson <er...@gmail.com>.
As you can see, there are a lot of related issues. Solr Cloud
is where this action is happening, and there's a lot of
active development work going on in this area. But the short
form is you'll have to wait for that release (you can pitch
in and help if you'd like)....
The release date for all this is uncertain....
Best
Erick
2011/11/9 Rafał Kuć <r....@solr.pl>:
> Hello!
>
> I was looking for a way to implement distributed indexing in Solr.
> From looking at the https://issues.apache.org/jira/browse/SOLR-2358
> there was some work done to enable Solr to distribute the documents to
> shards without the need of 3rd party software before Solr. What I
> would like to know if this is the road Solr will take to make things
> work. Maybe there was some additional work done with distributed
> indexing ?
>
> --
> Regards,
> Rafał Kuć
>
>