Caching requests to Solr

[Tommaso Teofili <to...@gmail.com>]

Hi all,

I have the following requirement where I have an application talking to
Solr via SolrJ where I don't know upfront which type of Solr instance that
will be communicating with, while this is easily solvable by using
different SolrServer implementations I also need a way to ensure that all
the indexing requests will go through in the correct order even if the Solr
instance(s) will be down for a while. This means that if the Solr instance
/ cluster is down I need to cache the requests e.g. in an ordered queue and
let them be processed out of the queue as soon as the instance / cluster
comes up again.
For this I was thinking to implementing a wrapping SolrServer which takes
the "root" SolrServer as a parameter and delegates all the requests to it
while it keeps a queue where all the (indexing) requests start going as
soon as one is failing due to a IO / Connection issue and that gets
continuously processed in order to pull requests out as soon as it's
possible to communicate again with the Solr instance / cluster.
I wonder then if there's any other approach you can think of to handle this
maybe leveraging existing stuff.

Regards,
Tommaso

Re: Caching requests to Solr

[Tommaso Teofili <to...@gmail.com>]

following up on this, I've created
https://issues.apache.org/jira/browse/SOLR-5826 , with a draft patch.
Regards,
Tommaso


2014-03-05 8:50 GMT+01:00 Tommaso Teofili <to...@gmail.com>:

> Hi all,
>
> I have the following requirement where I have an application talking to
> Solr via SolrJ where I don't know upfront which type of Solr instance that
> will be communicating with, while this is easily solvable by using
> different SolrServer implementations I also need a way to ensure that all
> the indexing requests will go through in the correct order even if the Solr
> instance(s) will be down for a while. This means that if the Solr instance
> / cluster is down I need to cache the requests e.g. in an ordered queue and
> let them be processed out of the queue as soon as the instance / cluster
> comes up again.
> For this I was thinking to implementing a wrapping SolrServer which takes
> the "root" SolrServer as a parameter and delegates all the requests to it
> while it keeps a queue where all the (indexing) requests start going as
> soon as one is failing due to a IO / Connection issue and that gets
> continuously processed in order to pull requests out as soon as it's
> possible to communicate again with the Solr instance / cluster.
> I wonder then if there's any other approach you can think of to handle
> this maybe leveraging existing stuff.
>
> Regards,
> Tommaso
>