You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Vadim Kisselmann <v....@googlemail.com> on 2011/11/02 10:00:01 UTC
shard indexing
Hello folks,
i have an problem with shard indexing.
with an single core i use this update command:
http://localhost:8983/solr/update .....
now i have 2 shards, we can call them core0 / core1
http://localhost:8983/solr/core0/update .....
can i adjust anything to indexing in the same way like with a single core
without core-name?
thanks and regards
vadim
Re: shard indexing
Posted by Vadim Kisselmann <v....@googlemail.com>.
Hello Yury,
thanks for your response.
This is exactly my plan. But "defaultCoreName" is buggy. When i use it
(defaultCore="core_november"), the defaultCore will be deleted.
I think this here was the issue:
https://issues.apache.org/jira/browse/SOLR-2127
Do you use this feature and did it work?
Thanks and Regards
Vadim
2011/11/2 Yury Kats <yu...@yahoo.com>
> There's a "defaultCore" parameter in solr.xml that let's you specify what
> core should be used when none is specified in the URL. You can change that
> every time you create a new core.
>
>
>
> >________________________________
> >From: Vadim Kisselmann <v....@googlemail.com>
> >To: solr-user@lucene.apache.org
> >Sent: Wednesday, November 2, 2011 6:16 AM
> >Subject: Re: shard indexing
> >
> >Hello Jan,
> >
> >thanks for your quick response.
> >
> >It's quite difficult to explain:
> >We want to create new shards on the fly every month and switch the default
> >shard to the newest one.
> >We always want to index to the newest shard with the same update query
> >like http://localhost:8983/solr/update.(content stream)
> >
> >Is our idea possible to implement?
> >
> >Thanks in advance.
> >Regards
> >
> >Vadim
> >
> >
> >
> >
> >
> >2011/11/2 Jan Høydahl <ja...@cominvent.com>
> >
> >> Hi,
> >>
> >> The only difference is the core name in the URL, which should be easy
> >> enough to handle from your indexing client code. I don't really
> understand
> >> the reason behind your request. How would you control which core to
> index
> >> your document to if you did not specify it in the URL?
> >>
> >> You could name ONE of your cores as ".", meaning it would be the
> "default"
> >> core living at /solr/update, perhaps that is what you're looking for?
> >>
> >> --
> >> Jan Høydahl, search solution architect
> >> Cominvent AS - www.cominvent.com
> >> Solr Training - www.solrtraining.com
> >>
> >> On 2. nov. 2011, at 10:00, Vadim Kisselmann wrote:
> >>
> >> > Hello folks,
> >> > i have an problem with shard indexing.
> >> >
> >> > with an single core i use this update command:
> >> > http://localhost:8983/solr/update .....
> >> >
> >> > now i have 2 shards, we can call them core0 / core1
> >> > http://localhost:8983/solr/core0/update .....
> >> >
> >> >
> >> > can i adjust anything to indexing in the same way like with a single
> core
> >> > without core-name?
> >> >
> >> > thanks and regards
> >> > vadim
> >>
> >>
> >
> >
> >
>
Re: shard indexing
Posted by Yury Kats <yu...@yahoo.com>.
There's a "defaultCore" parameter in solr.xml that let's you specify what core should be used when none is specified in the URL. You can change that every time you create a new core.
>________________________________
>From: Vadim Kisselmann <v....@googlemail.com>
>To: solr-user@lucene.apache.org
>Sent: Wednesday, November 2, 2011 6:16 AM
>Subject: Re: shard indexing
>
>Hello Jan,
>
>thanks for your quick response.
>
>It's quite difficult to explain:
>We want to create new shards on the fly every month and switch the default
>shard to the newest one.
>We always want to index to the newest shard with the same update query
>like http://localhost:8983/solr/update.(content stream)
>
>Is our idea possible to implement?
>
>Thanks in advance.
>Regards
>
>Vadim
>
>
>
>
>
>2011/11/2 Jan Høydahl <ja...@cominvent.com>
>
>> Hi,
>>
>> The only difference is the core name in the URL, which should be easy
>> enough to handle from your indexing client code. I don't really understand
>> the reason behind your request. How would you control which core to index
>> your document to if you did not specify it in the URL?
>>
>> You could name ONE of your cores as ".", meaning it would be the "default"
>> core living at /solr/update, perhaps that is what you're looking for?
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Solr Training - www.solrtraining.com
>>
>> On 2. nov. 2011, at 10:00, Vadim Kisselmann wrote:
>>
>> > Hello folks,
>> > i have an problem with shard indexing.
>> >
>> > with an single core i use this update command:
>> > http://localhost:8983/solr/update .....
>> >
>> > now i have 2 shards, we can call them core0 / core1
>> > http://localhost:8983/solr/core0/update .....
>> >
>> >
>> > can i adjust anything to indexing in the same way like with a single core
>> > without core-name?
>> >
>> > thanks and regards
>> > vadim
>>
>>
>
>
>
Re: shard indexing
Posted by Vadim Kisselmann <v....@googlemail.com>.
Hello Jan,
i think personally the same (switch URL for my indexing code), but my
requirement is to use the same query.
Thanks for your suppose with this one trick. Great idea which could work in
my case, i test it.
Regards
Vadim
2011/11/2 Jan Høydahl <ja...@cominvent.com>
> Personally I think it is better to be explicit about where you index, so
> that when you create a new shard "december", you also switch the URL for
> your indexing code.
>
> I suppose one trick you could use is to have a core called "current",
> which now would be for november, and once you get to december, you create a
> "november" core, and do a SWAP between "current"<->"november". Then your
> new core would now be "current" and you don't need to change URLs on the
> index client side.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Solr Training - www.solrtraining.com
>
> On 2. nov. 2011, at 11:16, Vadim Kisselmann wrote:
>
> > Hello Jan,
> >
> > thanks for your quick response.
> >
> > It's quite difficult to explain:
> > We want to create new shards on the fly every month and switch the
> default
> > shard to the newest one.
> > We always want to index to the newest shard with the same update query
> > like http://localhost:8983/solr/update.(content stream)
> >
> > Is our idea possible to implement?
> >
> > Thanks in advance.
> > Regards
> >
> > Vadim
> >
> >
> >
> >
> >
> > 2011/11/2 Jan Høydahl <ja...@cominvent.com>
> >
> >> Hi,
> >>
> >> The only difference is the core name in the URL, which should be easy
> >> enough to handle from your indexing client code. I don't really
> understand
> >> the reason behind your request. How would you control which core to
> index
> >> your document to if you did not specify it in the URL?
> >>
> >> You could name ONE of your cores as ".", meaning it would be the
> "default"
> >> core living at /solr/update, perhaps that is what you're looking for?
> >>
> >> --
> >> Jan Høydahl, search solution architect
> >> Cominvent AS - www.cominvent.com
> >> Solr Training - www.solrtraining.com
> >>
> >> On 2. nov. 2011, at 10:00, Vadim Kisselmann wrote:
> >>
> >>> Hello folks,
> >>> i have an problem with shard indexing.
> >>>
> >>> with an single core i use this update command:
> >>> http://localhost:8983/solr/update .....
> >>>
> >>> now i have 2 shards, we can call them core0 / core1
> >>> http://localhost:8983/solr/core0/update .....
> >>>
> >>>
> >>> can i adjust anything to indexing in the same way like with a single
> core
> >>> without core-name?
> >>>
> >>> thanks and regards
> >>> vadim
> >>
> >>
>
>
Re: shard indexing
Posted by Jan Høydahl <ja...@cominvent.com>.
Personally I think it is better to be explicit about where you index, so that when you create a new shard "december", you also switch the URL for your indexing code.
I suppose one trick you could use is to have a core called "current", which now would be for november, and once you get to december, you create a "november" core, and do a SWAP between "current"<->"november". Then your new core would now be "current" and you don't need to change URLs on the index client side.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 2. nov. 2011, at 11:16, Vadim Kisselmann wrote:
> Hello Jan,
>
> thanks for your quick response.
>
> It's quite difficult to explain:
> We want to create new shards on the fly every month and switch the default
> shard to the newest one.
> We always want to index to the newest shard with the same update query
> like http://localhost:8983/solr/update.(content stream)
>
> Is our idea possible to implement?
>
> Thanks in advance.
> Regards
>
> Vadim
>
>
>
>
>
> 2011/11/2 Jan Høydahl <ja...@cominvent.com>
>
>> Hi,
>>
>> The only difference is the core name in the URL, which should be easy
>> enough to handle from your indexing client code. I don't really understand
>> the reason behind your request. How would you control which core to index
>> your document to if you did not specify it in the URL?
>>
>> You could name ONE of your cores as ".", meaning it would be the "default"
>> core living at /solr/update, perhaps that is what you're looking for?
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Solr Training - www.solrtraining.com
>>
>> On 2. nov. 2011, at 10:00, Vadim Kisselmann wrote:
>>
>>> Hello folks,
>>> i have an problem with shard indexing.
>>>
>>> with an single core i use this update command:
>>> http://localhost:8983/solr/update .....
>>>
>>> now i have 2 shards, we can call them core0 / core1
>>> http://localhost:8983/solr/core0/update .....
>>>
>>>
>>> can i adjust anything to indexing in the same way like with a single core
>>> without core-name?
>>>
>>> thanks and regards
>>> vadim
>>
>>
Re: shard indexing
Posted by Vadim Kisselmann <v....@googlemail.com>.
Hello Jan,
thanks for your quick response.
It's quite difficult to explain:
We want to create new shards on the fly every month and switch the default
shard to the newest one.
We always want to index to the newest shard with the same update query
like http://localhost:8983/solr/update.(content stream)
Is our idea possible to implement?
Thanks in advance.
Regards
Vadim
2011/11/2 Jan Høydahl <ja...@cominvent.com>
> Hi,
>
> The only difference is the core name in the URL, which should be easy
> enough to handle from your indexing client code. I don't really understand
> the reason behind your request. How would you control which core to index
> your document to if you did not specify it in the URL?
>
> You could name ONE of your cores as ".", meaning it would be the "default"
> core living at /solr/update, perhaps that is what you're looking for?
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Solr Training - www.solrtraining.com
>
> On 2. nov. 2011, at 10:00, Vadim Kisselmann wrote:
>
> > Hello folks,
> > i have an problem with shard indexing.
> >
> > with an single core i use this update command:
> > http://localhost:8983/solr/update .....
> >
> > now i have 2 shards, we can call them core0 / core1
> > http://localhost:8983/solr/core0/update .....
> >
> >
> > can i adjust anything to indexing in the same way like with a single core
> > without core-name?
> >
> > thanks and regards
> > vadim
>
>
Re: shard indexing
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,
The only difference is the core name in the URL, which should be easy enough to handle from your indexing client code. I don't really understand the reason behind your request. How would you control which core to index your document to if you did not specify it in the URL?
You could name ONE of your cores as ".", meaning it would be the "default" core living at /solr/update, perhaps that is what you're looking for?
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 2. nov. 2011, at 10:00, Vadim Kisselmann wrote:
> Hello folks,
> i have an problem with shard indexing.
>
> with an single core i use this update command:
> http://localhost:8983/solr/update .....
>
> now i have 2 shards, we can call them core0 / core1
> http://localhost:8983/solr/core0/update .....
>
>
> can i adjust anything to indexing in the same way like with a single core
> without core-name?
>
> thanks and regards
> vadim