On-Demanp update if using DHI

[Ramo Karahasan <ra...@googlemail.com>]

Hi,

 

i'm using DHI for indexing my data. Currently I always to a delta import
manually as described here:
http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport

 

The data is fetched from a database. How is it possible to update the index
if new data is inserted into the database? 

We have an PHP Applicatoin where we import some new data into the database,
I would like to index these data 1.) on-demand if we don't have much updates
2.) as a batch for example 50.000 documents  

Automatically

 

Would be great get some hints

 

Best regards,

Ramo

Re: On-Demanp update if using DHI

[Erick Erickson <er...@gmail.com>]

There's nothing built into Solr or DIH that automatically looks
to see if your DB has changed. People sometimes use
cron jobs or similar to fire off the delta-import query
on a regular schedule.

Best
Erick

On Tue, Jan 31, 2012 at 3:06 PM, Ramo Karahasan
<ra...@googlemail.com> wrote:
> Hi,
>
>
>
> i'm using DHI for indexing my data. Currently I always to a delta import
> manually as described here:
> http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
>
>
>
> The data is fetched from a database. How is it possible to update the index
> if new data is inserted into the database?
>
> We have an PHP Applicatoin where we import some new data into the database,
> I would like to index these data 1.) on-demand if we don't have much updates
> 2.) as a batch for example 50.000 documents
>
> Automatically
>
>
>
> Would be great get some hints
>
>
>
> Best regards,
>
> Ramo
>

Re: On-Demanp update if using DHI

[Gora Mohanty <go...@mimirtech.com>]

On Wed, Feb 1, 2012 at 4:13 AM, Ramo Karahasan
<ra...@googlemail.com> wrote:
> This would mean, that i call somewhere in my application the url that is described in: http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport from my application?
[...]

Yes, if by application you mean the script used by a task scheduler
like cron.

Also, if you are doing frequent delta imports and/or your index is large,
you should consider how often to optimize when doing the import.
Optimization is on by default, so in order not to optimize, you need to
add &optimize=false to the URL. It is advisable to optimize at least
once in a while, say at off-peak times.

Regards,
Gora

AW: On-Demanp update if using DHI

[Ramo Karahasan <ra...@googlemail.com>]

This would mean, that i call somewhere in my application the url that is described in: http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport from my application?

Ramo

-----Ursprüngliche Nachricht-----
Von: Igor MILOVANOVIC [mailto:pletisan@gmail.com] 
Gesendet: Dienstag, 31. Januar 2012 23:15
An: solr-user@lucene.apache.org
Betreff: Re: On-Demanp update if using DHI

To have it updated on-demand you could just implement it inside your application, in form of event trigger or hook (depending how is your application's architecture).

For batch udpates it is just as simple as cron job script running as fast as every minute. Limits (50k documents) are imposed on your side, not on side of Solr...

On Tue, Jan 31, 2012 at 9:06 PM, Ramo Karahasan <ra...@googlemail.com> wrote:
>
> We have an PHP Applicatoin where we import some new data into the 
> database, I would like to index these data 1.) on-demand if we don't 
> have much updates
> 2.) as a batch for example 50.000 documents



--
Igor Milovanović
http://about.me/igor.milovanovic
http://umotvorine.com/

Re: On-Demanp update if using DHI

[Igor MILOVANOVIC <pl...@gmail.com>]

To have it updated on-demand you could just implement it inside your
application, in form of event trigger or hook (depending how is your
application's architecture).

For batch udpates it is just as simple as cron job script running as
fast as every minute. Limits (50k documents) are imposed on your side,
not on side of Solr...

On Tue, Jan 31, 2012 at 9:06 PM, Ramo Karahasan
<ra...@googlemail.com> wrote:
>
> We have an PHP Applicatoin where we import some new data into the database,
> I would like to index these data 1.) on-demand if we don't have much updates
> 2.) as a batch for example 50.000 documents



--
Igor Milovanović
http://about.me/igor.milovanovic
http://umotvorine.com/