You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Tushar Arora <ta...@gmail.com> on 2020/08/26 09:29:30 UTC
Real time index data
Hi,
One of our use cases requires real time indexing of data in solr from DB.
Approximately, 30 rows are updated in a second in DB. And I also want these
to be updated in the index simultaneously.
Is the Queuing mechanism like Rabbitmq helpful in my case?
Please suggest the ways to achieve it.
Regards,
Tushar Arora
Re: Real time index data
Posted by Jörn Franke <jo...@gmail.com>.
Maybe to add to this . Additionally try to batch the requests from the queue - don’t do it one by one , but take n items at the same time.
Look on the Solr side also on the configuration of soft commits vs hard commits . Soft commits are relevant for definition how real time this is and can be.
> Am 26.08.2020 um 11:36 schrieb Jörn Franke <jo...@gmail.com>:
>
> You do not provide many details, but a queuing mechanism seems to be appropriate for this use case.
>
>> Am 26.08.2020 um 11:30 schrieb Tushar Arora <ta...@gmail.com>:
>>
>> Hi,
>>
>> One of our use cases requires real time indexing of data in solr from DB.
>> Approximately, 30 rows are updated in a second in DB. And I also want these
>> to be updated in the index simultaneously.
>> Is the Queuing mechanism like Rabbitmq helpful in my case?
>> Please suggest the ways to achieve it.
>>
>> Regards,
>> Tushar Arora
Re: Real time index data
Posted by Jörn Franke <jo...@gmail.com>.
You do not provide many details, but a queuing mechanism seems to be appropriate for this use case.
> Am 26.08.2020 um 11:30 schrieb Tushar Arora <ta...@gmail.com>:
>
> Hi,
>
> One of our use cases requires real time indexing of data in solr from DB.
> Approximately, 30 rows are updated in a second in DB. And I also want these
> to be updated in the index simultaneously.
> Is the Queuing mechanism like Rabbitmq helpful in my case?
> Please suggest the ways to achieve it.
>
> Regards,
> Tushar Arora