Apache solr can be made near-real-Time???

[Samina <sa...@gmail.com>]

I want to use solr for enterprise level search on a large scale of data in
TB, where in  Lakh's of data will be update in an hour and approx 3 Lakh's
of data of would be seached in one hour.This is just the rough value though
nearby,so how can we achieve near -real-Time search in solr ? and how much
percent of real time search would be possible on this large data? 
Can we even achieve this doing indexing at certain
interval(automatic/Manual)?
Please help and suggest



--
View this message in context: http://lucene.472066.n3.nabble.com/Apache-solr-can-be-made-near-real-Time-tp4253808.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Apache solr can be made near-real-Time???

[Emir Arnautovic <em...@sematext.com>]

Hi Samina,
First to thank you for teaching me what "lakh" is :)

Solr is capable of handling large amount of data, but that requires 
large Solr cluster. What you need to determine is what is your real time 
- what is max time you can tolerate update to be visible; and determine 
acceptable query latency. After that you need to test with different 
shard size to achieve target latency. After that you can extrapolate it 
to your full data set and see how many shards you need.
What you can do with your data to reduce hw requirements:
* remove from index anything that is not needed
* in case you have time related data you can use time slicing
* in case of multi tenant index you can use routing

Regards,
Emir

On 28.01.2016 12:20, Samina wrote:
> I want to use solr for enterprise level search on a large scale of data in
> TB, where in  Lakh's of data will be update in an hour and approx 3 Lakh's
> of data of would be seached in one hour.This is just the rough value though
> nearby,so how can we achieve near -real-Time search in solr ? and how much
> percent of real time search would be possible on this large data?
> Can we even achieve this doing indexing at certain
> interval(automatic/Manual)?
> Please help and suggest
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Apache-solr-can-be-made-near-real-Time-tp4253808.html
> Sent from the Solr - User mailing list archive at Nabble.com.

-- 
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/