Re: Realtime Searching..

[John Wang <jo...@gmail.com>]

Hi Jon:
    We are running various LinkedIn search systems on Zoie in production.

-John

On Thu, Feb 19, 2009 at 9:11 AM, Jon Baer <jo...@gmail.com> wrote:

> This part:
>
> The part of Zoie that enables real-time searchability is the fact that
> ZoieSystem contains three IndexDataLoader objects:
>
>    * a RAMLuceneIndexDataLoader, which is a simple wrapper around a
> RAMDirectory,
>    * a DiskLuceneIndexDataLoader, which can index directly to the
> FSDirectory (followed by an optimize() call if a specified optimizeDuration
> has been exceeded) in batches via an intermediary
>    * BatchedIndexDataLoader, whose primary job is to queue up and batch
> DataEvents that need to be flushed to disk
>
> Sounds like it (might) be / (can) be layered into Solr somehow, has anyone
> been using this project or testing it?
>
> - Jon
>
>
> On Feb 19, 2009, at 9:44 AM, Genta Kaneyama wrote:
>
>  Michael,
>>
>> I think you might be get interested in "zoie".
>>
>> zoie: real-time search and indexing system built on Apache Lucene
>> http://code.google.com/p/zoie/
>>
>> Zoie is realtime search project for lucene by Linkedin.
>> Basically, I think it is similar technique to a Otis's trick.
>>
>>  In the mean time you can use the trick of one large and less frequently
>>>> updated core and one small and more frequently >>updated core + distributed
>>>> search across them.
>>>>
>>>> Otis
>>>>
>>>
>> Genta
>>
>>
>> On Sat, Feb 7, 2009 at 3:02 AM, Michael Austin <ma...@gmail.com>
>> wrote:
>>
>>> I need to find a solution for our current social application. It's low
>>> traffic now because we are early on.. However I'm expecting and want to
>>> be
>>> prepaired to grow.  We have messages of different "types" that are
>>> aggregated into one stream. Each of these message types have much
>>> different
>>> data so that our main queries have a few unions and many joins.  I know
>>> that
>>> Solr would work great for searching but we need a realtime system
>>> (twitter-like) to view user updates.  I'm not interested in a few minutes
>>> delay; I need something that will be fast updating and searchable and
>>> have n
>>> columns per record/document. Can solor do this? what is Ocean?
>>>
>>> Thanks
>>>
>>>
>

Re: Realtime Searching..

[Otis Gospodnetic <ot...@yahoo.com>]

Would it not make more sense to wait for the Lucene's IW+IR marriage and other things happening in core Lucene that will make near-real-time search possible?

 Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: John Wang <jo...@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Wednesday, March 25, 2009 2:34:04 PM
> Subject: Re: Realtime Searching..
> 
> Hi Jon:
>     We are running various LinkedIn search systems on Zoie in production.
> 
> -John
> 
> On Thu, Feb 19, 2009 at 9:11 AM, Jon Baer wrote:
> 
> > This part:
> >
> > The part of Zoie that enables real-time searchability is the fact that
> > ZoieSystem contains three IndexDataLoader objects:
> >
> >    * a RAMLuceneIndexDataLoader, which is a simple wrapper around a
> > RAMDirectory,
> >    * a DiskLuceneIndexDataLoader, which can index directly to the
> > FSDirectory (followed by an optimize() call if a specified optimizeDuration
> > has been exceeded) in batches via an intermediary
> >    * BatchedIndexDataLoader, whose primary job is to queue up and batch
> > DataEvents that need to be flushed to disk
> >
> > Sounds like it (might) be / (can) be layered into Solr somehow, has anyone
> > been using this project or testing it?
> >
> > - Jon
> >
> >
> > On Feb 19, 2009, at 9:44 AM, Genta Kaneyama wrote:
> >
> >  Michael,
> >>
> >> I think you might be get interested in "zoie".
> >>
> >> zoie: real-time search and indexing system built on Apache Lucene
> >> http://code.google.com/p/zoie/
> >>
> >> Zoie is realtime search project for lucene by Linkedin.
> >> Basically, I think it is similar technique to a Otis's trick.
> >>
> >>  In the mean time you can use the trick of one large and less frequently
> >>>> updated core and one small and more frequently >>updated core + distributed
> >>>> search across them.
> >>>>
> >>>> Otis
> >>>>
> >>>
> >> Genta
> >>
> >>
> >> On Sat, Feb 7, 2009 at 3:02 AM, Michael Austin 
> >> wrote:
> >>
> >>> I need to find a solution for our current social application. It's low
> >>> traffic now because we are early on.. However I'm expecting and want to
> >>> be
> >>> prepaired to grow.  We have messages of different "types" that are
> >>> aggregated into one stream. Each of these message types have much
> >>> different
> >>> data so that our main queries have a few unions and many joins.  I know
> >>> that
> >>> Solr would work great for searching but we need a realtime system
> >>> (twitter-like) to view user updates.  I'm not interested in a few minutes
> >>> delay; I need something that will be fast updating and searchable and
> >>> have n
> >>> columns per record/document. Can solor do this? what is Ocean?
> >>>
> >>> Thanks
> >>>
> >>>
> >