You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by rehman kahloon <mr...@yahoo.com.INVALID> on 2017/07/18 07:08:08 UTC
Need guidance for distributing data base on date interval in a
collection
Hello Sir/Madam I am new to SolrCloud, Having ORACLE technologies experience.
Now a days , i am comparing oracle and solrcloud using bigdata.
So i want to know how can i create time interval sharding.
e.g i have 10 machines, each machine for one shard and one date data, So how can i fix next day data go to next shard and so on?
search too much but not found any command/way, that handle it from some core/shard file.
So i request you please guide me.
thanks in advanced.
Kind Regards,Muhammad Rehman Kahloonmrehman_kahloon@yahoo.com
Re: Need guidance for distributing data base on date interval in a collection
Posted by Charlie Hull <ch...@flax.co.uk>.
Hi,
You should also consider how you should shard for best performance: for
example, if most of your queries are for recent documents, you could end up
with them all hitting only one shard. Here's an old blog we wrote on this
subject (it mentions another open source engine, Xapian, but ignore that as
the same principles apply to Solr).
HTH
Charlie
On 18 July 2017 at 09:16, Modassar Ather <mo...@gmail.com> wrote:
> Hi Rehman,
>
> You may want to look into how the documents are routed on different shards.
> For that you can look into following documentation.
> https://cwiki.apache.org/confluence/display/solr/
> Shards+and+Indexing+Data+in+SolrCloud
>
> Basically it is the id of the document which when prefixed with certain
> attribute helps decide which shard the document actually goes.
> So document id and date id prefix may be helpful.
>
> Best,
> Modassar
>
>
>
>
>
> On Tue, Jul 18, 2017 at 1:08 PM, Atita Arora <at...@gmail.com> wrote:
>
> > Hi Rehman,
> > I am not sure about your use case, but why wouldn't you consider
> creating
> > shard for a particular date range like within a week from current date,
> 15
> > days, a month and so on and so forth.
> >
> > I have done a similar implementation elsewhere.
> > Can you tell more about your use case?
> >
> > Atita
> >
> > On Jul 18, 2017 1:04 PM, "rehman kahloon" <mrehman_kahloon@yahoo.com.
> > invalid>
> > wrote:
> >
> > Hello Sir/Madam I am new to SolrCloud, Having ORACLE
> > technologies experience.
> > Now a days , i am comparing oracle and solrcloud using bigdata.
> > So i want to know how can i create time interval sharding.
> > e.g i have 10 machines, each machine for one shard and one date data, So
> > how can i fix next day data go to next shard and so on?
> >
> > search too much but not found any command/way, that handle it from some
> > core/shard file.
> > So i request you please guide me.
> > thanks in advanced.
> > Kind Regards,Muhammad Rehman Kahloonmrehman_kahloon@yahoo.com
> >
>
Re: Need guidance for distributing data base on date interval in a collection
Posted by Modassar Ather <mo...@gmail.com>.
Hi Rehman,
You may want to look into how the documents are routed on different shards.
For that you can look into following documentation.
https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud
Basically it is the id of the document which when prefixed with certain
attribute helps decide which shard the document actually goes.
So document id and date id prefix may be helpful.
Best,
Modassar
On Tue, Jul 18, 2017 at 1:08 PM, Atita Arora <at...@gmail.com> wrote:
> Hi Rehman,
> I am not sure about your use case, but why wouldn't you consider creating
> shard for a particular date range like within a week from current date, 15
> days, a month and so on and so forth.
>
> I have done a similar implementation elsewhere.
> Can you tell more about your use case?
>
> Atita
>
> On Jul 18, 2017 1:04 PM, "rehman kahloon" <mrehman_kahloon@yahoo.com.
> invalid>
> wrote:
>
> Hello Sir/Madam I am new to SolrCloud, Having ORACLE
> technologies experience.
> Now a days , i am comparing oracle and solrcloud using bigdata.
> So i want to know how can i create time interval sharding.
> e.g i have 10 machines, each machine for one shard and one date data, So
> how can i fix next day data go to next shard and so on?
>
> search too much but not found any command/way, that handle it from some
> core/shard file.
> So i request you please guide me.
> thanks in advanced.
> Kind Regards,Muhammad Rehman Kahloonmrehman_kahloon@yahoo.com
>
Re: Need guidance for distributing data base on date interval in a collection
Posted by Atita Arora <at...@gmail.com>.
Hi Rehman,
I am not sure about your use case, but why wouldn't you consider creating
shard for a particular date range like within a week from current date, 15
days, a month and so on and so forth.
I have done a similar implementation elsewhere.
Can you tell more about your use case?
Atita
On Jul 18, 2017 1:04 PM, "rehman kahloon" <mr...@yahoo.com.invalid>
wrote:
Hello Sir/Madam I am new to SolrCloud, Having ORACLE
technologies experience.
Now a days , i am comparing oracle and solrcloud using bigdata.
So i want to know how can i create time interval sharding.
e.g i have 10 machines, each machine for one shard and one date data, So
how can i fix next day data go to next shard and so on?
search too much but not found any command/way, that handle it from some
core/shard file.
So i request you please guide me.
thanks in advanced.
Kind Regards,Muhammad Rehman Kahloonmrehman_kahloon@yahoo.com