You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Don Werve <do...@madwombat.com> on 2010/03/09 02:06:19 UTC
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Too bad it requires integer (long) primary keys... :/
2010/3/8 Ian Holsman <li...@holsman.net>
>
> I just saw this on twitter, and thought you guys would be interested.. I
> haven't tried it, but it looks interesting.
>
> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin
>
> Thanks for the RT Shalin!
>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by John Wang <jo...@gmail.com>.
Sorry, have completely missed this thread until someone pointed out to me
just now.
Some clarifications:
commit=true is for the plugin only. I added it to make it consistent with
Solr's behavior.
Stand-alone Zoie does not require this, it is in more of an autocommit mode
since it assumes a constant and busy stream of indexing events.
So I think what we will do is add an autocommit parameter to the
ZoieUpdateHandler and default it to true. With this, you don't need to have
commit=true to see the new docs.
-John
On Wed, Apr 14, 2010 at 5:25 PM, qaz <df...@hotmail.com> wrote:
>
> do you mean that the plugin for solr doesn't have all the functionalities
> of
> the standalone zoie or do you mean zoie just simply cannot handle large
> indexes? by really really small, what exactly are we talking about here?
> are
> there any better ways for NRT? maybe in solr 1.5?
>
> thanks
> --
> View this message in context:
> http://n3.nabble.com/ANN-Zoie-Solr-Plugin-Zoie-Solr-Plugin-enables-real-time-update-functionality-for-Apache-Solr-1-4-tp506099p719893.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by qaz <df...@hotmail.com>.
do you mean that the plugin for solr doesn't have all the functionalities of
the standalone zoie or do you mean zoie just simply cannot handle large
indexes? by really really small, what exactly are we talking about here? are
there any better ways for NRT? maybe in solr 1.5?
thanks
--
View this message in context: http://n3.nabble.com/ANN-Zoie-Solr-Plugin-Zoie-Solr-Plugin-enables-real-time-update-functionality-for-Apache-Solr-1-4-tp506099p719893.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Posted by Grant Ingersoll <gs...@apache.org>.
On Mar 23, 2010, at 7:29 PM, brad anderson wrote:
> I see, so when you do a commit it adds it to Zoie's ramdirectory. So, could
> you just commit after every document without having a performance impact and
> have real time search?
>
Not likely, maybe on really, really small indexes. Zoie also does a writethrough, AIUI, to a file based index.
> Thanks,
> Brad
>
> On 20 March 2010 00:34, Janne Majaranta <ja...@gmail.com> wrote:
>
>> To my understanding it adds a in-memory index which holds the recent
>> commits and which is flushed to the main index based on the config options.
>> Not sure if it helps to get solr near real time. I am evaluating it
>> currently, and I am really not sure if it adds anything because of the cache
>> regeneration of solr on every commit ??
>>
>> -Janne
>>
>> Lähetetty iPodista
>>
>> brad anderson <so...@gmail.com> kirjoitti 19.3.2010 kello 20.53:
>>
>>
>> Indeed, which is why I'm wondering what is Zoie adding if you still need
>>> to
>>> commit to search recent documents. Does anyone know?
>>>
>>> Thanks,
>>> Brad
>>>
>>> On 18 March 2010 19:41, Erik Hatcher <er...@gmail.com> wrote:
>>>
>>> "When I don't do the commit, I cannot search the documents I've indexed."
>>>> -
>>>> that's exactly how Solr without Zoie works, and it's how Lucene itself
>>>> works. Gotta commit to see the documents indexed.
>>>>
>>>> Erik
>>>>
>>>>
>>>>
>>>> On Mar 18, 2010, at 5:41 PM, brad anderson wrote:
>>>>
>>>> Tried following their tutorial for plugging zoie into solr:
>>>>
>>>>> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server
>>>>>
>>>>> It appears it only allows you to search on documents after you do a
>>>>> commit?
>>>>> Am I missing something here, or does plugin not doing anything.
>>>>>
>>>>> Their tutorial tells you to do a commit when you index the docs:
>>>>>
>>>>> curl http://localhost:8983/solr/update/csv?commit=true --data-binary
>>>>> @books.csv -H 'Content-type:text/plain; charset=utf-8'
>>>>>
>>>>>
>>>>> When I don't do the commit, I cannot search the documents I've indexed.
>>>>>
>>>>> Thanks,
>>>>> Brad
>>>>>
>>>>> On 9 March 2010 23:34, Don Werve <do...@madwombat.com> wrote:
>>>>>
>>>>> 2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
>>>>>
>>>>>>
>>>>>> I think Don is talking about Zoie - it requires a long uniqueKey.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Yep; we're using UUIDs.
>>>>>>
>>>>>>
>>>>>>
>>>>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem using Solr/Lucene: http://www.lucidimagination.com/search
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by brad anderson <so...@gmail.com>.
I see, so when you do a commit it adds it to Zoie's ramdirectory. So, could
you just commit after every document without having a performance impact and
have real time search?
Thanks,
Brad
On 20 March 2010 00:34, Janne Majaranta <ja...@gmail.com> wrote:
> To my understanding it adds a in-memory index which holds the recent
> commits and which is flushed to the main index based on the config options.
> Not sure if it helps to get solr near real time. I am evaluating it
> currently, and I am really not sure if it adds anything because of the cache
> regeneration of solr on every commit ??
>
> -Janne
>
> Lähetetty iPodista
>
> brad anderson <so...@gmail.com> kirjoitti 19.3.2010 kello 20.53:
>
>
> Indeed, which is why I'm wondering what is Zoie adding if you still need
>> to
>> commit to search recent documents. Does anyone know?
>>
>> Thanks,
>> Brad
>>
>> On 18 March 2010 19:41, Erik Hatcher <er...@gmail.com> wrote:
>>
>> "When I don't do the commit, I cannot search the documents I've indexed."
>>> -
>>> that's exactly how Solr without Zoie works, and it's how Lucene itself
>>> works. Gotta commit to see the documents indexed.
>>>
>>> Erik
>>>
>>>
>>>
>>> On Mar 18, 2010, at 5:41 PM, brad anderson wrote:
>>>
>>> Tried following their tutorial for plugging zoie into solr:
>>>
>>>> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server
>>>>
>>>> It appears it only allows you to search on documents after you do a
>>>> commit?
>>>> Am I missing something here, or does plugin not doing anything.
>>>>
>>>> Their tutorial tells you to do a commit when you index the docs:
>>>>
>>>> curl http://localhost:8983/solr/update/csv?commit=true --data-binary
>>>> @books.csv -H 'Content-type:text/plain; charset=utf-8'
>>>>
>>>>
>>>> When I don't do the commit, I cannot search the documents I've indexed.
>>>>
>>>> Thanks,
>>>> Brad
>>>>
>>>> On 9 March 2010 23:34, Don Werve <do...@madwombat.com> wrote:
>>>>
>>>> 2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
>>>>
>>>>>
>>>>> I think Don is talking about Zoie - it requires a long uniqueKey.
>>>>>
>>>>>>
>>>>>>
>>>>>> Yep; we're using UUIDs.
>>>>>
>>>>>
>>>>>
>>>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Posted by Janne Majaranta <ja...@gmail.com>.
To my understanding it adds a in-memory index which holds the recent
commits and which is flushed to the main index based on the config
options. Not sure if it helps to get solr near real time. I am
evaluating it currently, and I am really not sure if it adds anything
because of the cache regeneration of solr on every commit ??
-Janne
Lähetetty iPodista
brad anderson <so...@gmail.com> kirjoitti 19.3.2010 kello 20.53:
> Indeed, which is why I'm wondering what is Zoie adding if you still
> need to
> commit to search recent documents. Does anyone know?
>
> Thanks,
> Brad
>
> On 18 March 2010 19:41, Erik Hatcher <er...@gmail.com> wrote:
>
>> "When I don't do the commit, I cannot search the documents I've
>> indexed." -
>> that's exactly how Solr without Zoie works, and it's how Lucene
>> itself
>> works. Gotta commit to see the documents indexed.
>>
>> Erik
>>
>>
>>
>> On Mar 18, 2010, at 5:41 PM, brad anderson wrote:
>>
>> Tried following their tutorial for plugging zoie into solr:
>>> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server
>>>
>>> It appears it only allows you to search on documents after you do a
>>> commit?
>>> Am I missing something here, or does plugin not doing anything.
>>>
>>> Their tutorial tells you to do a commit when you index the docs:
>>>
>>> curl http://localhost:8983/solr/update/csv?commit=true --data-binary
>>> @books.csv -H 'Content-type:text/plain; charset=utf-8'
>>>
>>>
>>> When I don't do the commit, I cannot search the documents I've
>>> indexed.
>>>
>>> Thanks,
>>> Brad
>>>
>>> On 9 March 2010 23:34, Don Werve <do...@madwombat.com> wrote:
>>>
>>> 2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
>>>>
>>>> I think Don is talking about Zoie - it requires a long uniqueKey.
>>>>>
>>>>>
>>>> Yep; we're using UUIDs.
>>>>
>>>>
>>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by brad anderson <so...@gmail.com>.
Indeed, which is why I'm wondering what is Zoie adding if you still need to
commit to search recent documents. Does anyone know?
Thanks,
Brad
On 18 March 2010 19:41, Erik Hatcher <er...@gmail.com> wrote:
> "When I don't do the commit, I cannot search the documents I've indexed." -
> that's exactly how Solr without Zoie works, and it's how Lucene itself
> works. Gotta commit to see the documents indexed.
>
> Erik
>
>
>
> On Mar 18, 2010, at 5:41 PM, brad anderson wrote:
>
> Tried following their tutorial for plugging zoie into solr:
>> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server
>>
>> It appears it only allows you to search on documents after you do a
>> commit?
>> Am I missing something here, or does plugin not doing anything.
>>
>> Their tutorial tells you to do a commit when you index the docs:
>>
>> curl http://localhost:8983/solr/update/csv?commit=true --data-binary
>> @books.csv -H 'Content-type:text/plain; charset=utf-8'
>>
>>
>> When I don't do the commit, I cannot search the documents I've indexed.
>>
>> Thanks,
>> Brad
>>
>> On 9 March 2010 23:34, Don Werve <do...@madwombat.com> wrote:
>>
>> 2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
>>>
>>> I think Don is talking about Zoie - it requires a long uniqueKey.
>>>>
>>>>
>>> Yep; we're using UUIDs.
>>>
>>>
>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Posted by Erik Hatcher <er...@gmail.com>.
"When I don't do the commit, I cannot search the documents I've
indexed." - that's exactly how Solr without Zoie works, and it's how
Lucene itself works. Gotta commit to see the documents indexed.
Erik
On Mar 18, 2010, at 5:41 PM, brad anderson wrote:
> Tried following their tutorial for plugging zoie into solr:
> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server
>
> It appears it only allows you to search on documents after you do a
> commit?
> Am I missing something here, or does plugin not doing anything.
>
> Their tutorial tells you to do a commit when you index the docs:
>
> curl http://localhost:8983/solr/update/csv?commit=true --data-binary
> @books.csv -H 'Content-type:text/plain; charset=utf-8'
>
>
> When I don't do the commit, I cannot search the documents I've
> indexed.
>
> Thanks,
> Brad
>
> On 9 March 2010 23:34, Don Werve <do...@madwombat.com> wrote:
>
>> 2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
>>
>>> I think Don is talking about Zoie - it requires a long uniqueKey.
>>>
>>
>> Yep; we're using UUIDs.
>>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by brad anderson <so...@gmail.com>.
Tried following their tutorial for plugging zoie into solr:
http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server
It appears it only allows you to search on documents after you do a commit?
Am I missing something here, or does plugin not doing anything.
Their tutorial tells you to do a commit when you index the docs:
curl http://localhost:8983/solr/update/csv?commit=true --data-binary
@books.csv -H 'Content-type:text/plain; charset=utf-8'
When I don't do the commit, I cannot search the documents I've indexed.
Thanks,
Brad
On 9 March 2010 23:34, Don Werve <do...@madwombat.com> wrote:
> 2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
>
> > I think Don is talking about Zoie - it requires a long uniqueKey.
> >
>
> Yep; we're using UUIDs.
>
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by Don Werve <do...@madwombat.com>.
2010/3/9 Shalin Shekhar Mangar <sh...@gmail.com>
> I think Don is talking about Zoie - it requires a long uniqueKey.
>
Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
I think Don is talking about Zoie - it requires a long uniqueKey.
On Tue, Mar 9, 2010 at 10:18 AM, Lance Norskog <go...@gmail.com> wrote:
> Solr unique ids can be any type. The QueryElevateComponent complains
> if the unique id is not a string, but you can comment out the QEC. I
> have one benchmark test with 2 billion documents with an integer id.
> Works great.
>
> On Mon, Mar 8, 2010 at 5:06 PM, Don Werve <do...@madwombat.com> wrote:
> > Too bad it requires integer (long) primary keys... :/
> >
> > 2010/3/8 Ian Holsman <li...@holsman.net>
> >
> >>
> >> I just saw this on twitter, and thought you guys would be interested.. I
> >> haven't tried it, but it looks interesting.
> >>
> >> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin
> >>
> >> Thanks for the RT Shalin!
> >>
> >
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
--
Regards,
Shalin Shekhar Mangar.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time
update functionality for Apache Solr 1.4+
Posted by Lance Norskog <go...@gmail.com>.
Solr unique ids can be any type. The QueryElevateComponent complains
if the unique id is not a string, but you can comment out the QEC. I
have one benchmark test with 2 billion documents with an integer id.
Works great.
On Mon, Mar 8, 2010 at 5:06 PM, Don Werve <do...@madwombat.com> wrote:
> Too bad it requires integer (long) primary keys... :/
>
> 2010/3/8 Ian Holsman <li...@holsman.net>
>
>>
>> I just saw this on twitter, and thought you guys would be interested.. I
>> haven't tried it, but it looks interesting.
>>
>> http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin
>>
>> Thanks for the RT Shalin!
>>
>
--
Lance Norskog
goksron@gmail.com