Queuing for Solr Updates?

[William Bell <bi...@gmail.com>]

Is there a simpler way to kick off a DIH handler update when it is running?

Scenario:

1. Doing an update using DIH
2. We need to kick off another update. Cannot since DIH is already running.
So the program inserts into a table (ID=55)
3. Since the DIH is still running old update, we cannot fire an update to
DIH.

We want it to run right away. If there a simple way to queue up an update
if it is still running? We wrote the query to catch all pending, so we only
need to run it again if there is 1 or more pending updates.

Thoughts?

-- 
Bill Bell
billnbell@gmail.com
cell 720-256-8076

Re: Queuing for Solr Updates?

[Shawn Heisey <so...@elyograg.org>]

On 6/20/2013 11:42 PM, William Bell wrote:
> Is there a simpler way to kick off a DIH handler update when it is running?
> 
> Scenario:
> 
> 1. Doing an update using DIH
> 2. We need to kick off another update. Cannot since DIH is already running.
> So the program inserts into a table (ID=55)
> 3. Since the DIH is still running old update, we cannot fire an update to
> DIH.
> 
> We want it to run right away. If there a simple way to queue up an update
> if it is still running? We wrote the query to catch all pending, so we only
> need to run it again if there is 1 or more pending updates.

The reply you already got from Gora is one way.  Another is to define
additional dataimport handlers on their own URL paths (simple ideas are
/dataimport2 and /dataimport3).  If the first one is busy, use one of
the others.  One problem with this approach is that each handler won't
have access to the last index time saved by the others.

Thanks,
Shawn

Re: Queuing for Solr Updates?

[Gora Mohanty <go...@mimirtech.com>]

On 21 June 2013 11:12, William Bell <bi...@gmail.com> wrote:
> Is there a simpler way to kick off a DIH handler update when it is running?
>
> Scenario:
>
> 1. Doing an update using DIH
> 2. We need to kick off another update. Cannot since DIH is already running.
> So the program inserts into a table (ID=55)
> 3. Since the DIH is still running old update, we cannot fire an update to
> DIH.
>
> We want it to run right away. If there a simple way to queue up an update
> if it is still running? We wrote the query to catch all pending, so we only
> need to run it again if there is 1 or more pending updates.
[...]

Your best bet might be to write an external daemon that
monitors the DIH import status as well as the the update
queue, and launches a new DIH import as required.

Regards,
Gora