You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Yago Riveiro <ya...@gmail.com> on 2015/09/15 18:46:47 UTC
Problem with CoreAdmin API CREATE command
Hi,
I’m having some issues with the command CREATE of the CoreAdmin API in solr
4.10.4.
When I try to load a previous unloaded core with CREATE command, the result
of this operation is 2 replicas in down state. One with the original
coreNodeName set in clusterstate.json and other with an new one.
I’m with the new solr.xml format and the core.properties of cores looks like
(I set this configuration when upgraded from 4.6.1 in legacy solr.xml to
4.10.4:
name=bucket-15_shardX_replicaY
shard=shardX
collection=bucket-15
The API command looks like:
solr/admin/cores?action=CREATE&name=bucket-15_shardX_replicaY&collection=bucket-15&shard=shardX&wt=json
What I’m doing wrong?
-----
Best regards
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-CoreAdmin-API-CREATE-command-tp4229248.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with CoreAdmin API CREATE command
Posted by Shai Erera <se...@gmail.com>.
That's definitely strange as Solr 5.x should support all Solr 4.x indexes.
Anyway, you can somewhat force an upgrade by running a forceMerge command
after you've upgraded the libraries to 5.3.0. This will rewrite the index
into one segment whose version will be 5.3. It is usually not recommended
to run this as it's a heavy operation, but if you want all index segments
to be upgraded, this is an option.
But again, I don't see any reason to do that. If you hit an error that you
can reproduce, it might be worth it to file a bug report.
Shai
On Thu, Sep 17, 2015 at 4:01 PM, Yago Riveiro <ya...@gmail.com>
wrote:
> 90% of my data was indexed in 4.6.1 or lower.
>
>
>
>
> My goal is upgrade all data to 4.10.4 and then upgrade to 5.3
>
>
>
>
> In previous test that I did with 5.3 in dev cluster, I see some strange
> behaviour with data indexed with 4.6.1 that in 4.10.4 didn’t reproduce.
>
>
>
>
> Some queries to data indexed in 4.6.1 that returns data with source 4.6.1,
> in 5.3 some times returned zero docs (in a random fashion way, I can’t find
> the pattern that produced the failure of the query). The same data upgraded
> to 4.10.4 with source code of 5.3 worked as expected with any issue.
>
>
> —/Yago Riveiro
>
> On Thu, Sep 17, 2015 at 1:08 PM, Shai Erera <se...@gmail.com> wrote:
>
> > Solr 5.3 can read Solr 4.10.4 indexes as-is. Why are you trying to
> upgrade
> > the indexes in the first place?
> > Shai
> > On Thu, Sep 17, 2015 at 3:05 PM, Yago Riveiro <ya...@gmail.com>
> > wrote:
> >> I have a very old index with more than 12T (re-index data is not an
> option
> >> ...) that I want upgrade to 5.3, I’m using lucene-core-4.10.4.jar (I’m
> in
> >> 4.10.4 right now) to upgrade old segments of data. With solr running I
> can
> >> run the command because solr has the lock of the core.
> >>
> >>
> >>
> >>
> >>
> >> I only want to unload it to perform an index upgrade command to load
> again
> >> without stop the node with the core.
> >>
> >>
> >>
> >>
> >>
> >> The issue here is that the DELETEREPLICA deletes the data … the UNLOAD
> >> command doesn’t save the original core.properties and creates a new one
> >> core.properties.unloaded that not store the shard param and creates a
> new
> >> random coreNodeName that doesn’t correspond to the name of
> >> the clusterstate, the result … the core is loaded twice or even worse,
> in
> >> some situation is attached to a wrong shard.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> —/Yago Riveiro
> >>
> >> On Wed, Sep 16, 2015 at 4:38 PM, Erick Erickson <
> erickerickson@gmail.com>
> >> wrote:
> >>
> >> > The not-very-helpful answer is that you're using the core admin API in
> >> > a SolrCloud setup. Please do not do this as (you're well aware of this
> >> by now!)
> >> > it's far too easy to get "interesting" results.
> >> > Instead, use the Collections API, specifically the ADDREPLICA and
> >> DELETEREPLICA
> >> > commands. Under the covers, they actually create a core via the core
> >> admin API,
> >> > but they insure that all the core create parameters are correct. For
> >> ADDREPLICA,
> >> > you can also easily insure that a node lands on a particular machine.
> >> > Best,
> >> > Erick
> >> > On Tue, Sep 15, 2015 at 9:46 AM, Yago Riveiro <yago.riveiro@gmail.com
> >
> >> wrote:
> >> >> Hi,
> >> >>
> >> >> I’m having some issues with the command CREATE of the CoreAdmin API
> in
> >> solr
> >> >> 4.10.4.
> >> >>
> >> >> When I try to load a previous unloaded core with CREATE command, the
> >> result
> >> >> of this operation is 2 replicas in down state. One with the original
> >> >> coreNodeName set in clusterstate.json and other with an new one.
> >> >>
> >> >> I’m with the new solr.xml format and the core.properties of cores
> looks
> >> like
> >> >> (I set this configuration when upgraded from 4.6.1 in legacy
> solr.xml
> >> to
> >> >> 4.10.4:
> >> >>
> >> >> name=bucket-15_shardX_replicaY
> >> >> shard=shardX
> >> >> collection=bucket-15
> >> >>
> >> >> The API command looks like:
> >> >>
> >> >>
> >>
> solr/admin/cores?action=CREATE&name=bucket-15_shardX_replicaY&collection=bucket-15&shard=shardX&wt=json
> >> >>
> >> >> What I’m doing wrong?
> >> >>
> >> >>
> >> >>
> >> >> -----
> >> >> Best regards
> >> >> --
> >> >> View this message in context:
> >>
> http://lucene.472066.n3.nabble.com/Problem-with-CoreAdmin-API-CREATE-command-tp4229248.html
> >> >> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with CoreAdmin API CREATE command
Posted by Yago Riveiro <ya...@gmail.com>.
90% of my data was indexed in 4.6.1 or lower.
My goal is upgrade all data to 4.10.4 and then upgrade to 5.3
In previous test that I did with 5.3 in dev cluster, I see some strange behaviour with data indexed with 4.6.1 that in 4.10.4 didn’t reproduce.
Some queries to data indexed in 4.6.1 that returns data with source 4.6.1, in 5.3 some times returned zero docs (in a random fashion way, I can’t find the pattern that produced the failure of the query). The same data upgraded to 4.10.4 with source code of 5.3 worked as expected with any issue.
—/Yago Riveiro
On Thu, Sep 17, 2015 at 1:08 PM, Shai Erera <se...@gmail.com> wrote:
> Solr 5.3 can read Solr 4.10.4 indexes as-is. Why are you trying to upgrade
> the indexes in the first place?
> Shai
> On Thu, Sep 17, 2015 at 3:05 PM, Yago Riveiro <ya...@gmail.com>
> wrote:
>> I have a very old index with more than 12T (re-index data is not an option
>> ...) that I want upgrade to 5.3, I’m using lucene-core-4.10.4.jar (I’m in
>> 4.10.4 right now) to upgrade old segments of data. With solr running I can
>> run the command because solr has the lock of the core.
>>
>>
>>
>>
>>
>> I only want to unload it to perform an index upgrade command to load again
>> without stop the node with the core.
>>
>>
>>
>>
>>
>> The issue here is that the DELETEREPLICA deletes the data … the UNLOAD
>> command doesn’t save the original core.properties and creates a new one
>> core.properties.unloaded that not store the shard param and creates a new
>> random coreNodeName that doesn’t correspond to the name of
>> the clusterstate, the result … the core is loaded twice or even worse, in
>> some situation is attached to a wrong shard.
>>
>>
>>
>>
>>
>>
>>
>>
>> —/Yago Riveiro
>>
>> On Wed, Sep 16, 2015 at 4:38 PM, Erick Erickson <er...@gmail.com>
>> wrote:
>>
>> > The not-very-helpful answer is that you're using the core admin API in
>> > a SolrCloud setup. Please do not do this as (you're well aware of this
>> by now!)
>> > it's far too easy to get "interesting" results.
>> > Instead, use the Collections API, specifically the ADDREPLICA and
>> DELETEREPLICA
>> > commands. Under the covers, they actually create a core via the core
>> admin API,
>> > but they insure that all the core create parameters are correct. For
>> ADDREPLICA,
>> > you can also easily insure that a node lands on a particular machine.
>> > Best,
>> > Erick
>> > On Tue, Sep 15, 2015 at 9:46 AM, Yago Riveiro <ya...@gmail.com>
>> wrote:
>> >> Hi,
>> >>
>> >> I’m having some issues with the command CREATE of the CoreAdmin API in
>> solr
>> >> 4.10.4.
>> >>
>> >> When I try to load a previous unloaded core with CREATE command, the
>> result
>> >> of this operation is 2 replicas in down state. One with the original
>> >> coreNodeName set in clusterstate.json and other with an new one.
>> >>
>> >> I’m with the new solr.xml format and the core.properties of cores looks
>> like
>> >> (I set this configuration when upgraded from 4.6.1 in legacy solr.xml
>> to
>> >> 4.10.4:
>> >>
>> >> name=bucket-15_shardX_replicaY
>> >> shard=shardX
>> >> collection=bucket-15
>> >>
>> >> The API command looks like:
>> >>
>> >>
>> solr/admin/cores?action=CREATE&name=bucket-15_shardX_replicaY&collection=bucket-15&shard=shardX&wt=json
>> >>
>> >> What I’m doing wrong?
>> >>
>> >>
>> >>
>> >> -----
>> >> Best regards
>> >> --
>> >> View this message in context:
>> http://lucene.472066.n3.nabble.com/Problem-with-CoreAdmin-API-CREATE-command-tp4229248.html
>> >> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with CoreAdmin API CREATE command
Posted by Shai Erera <se...@gmail.com>.
Solr 5.3 can read Solr 4.10.4 indexes as-is. Why are you trying to upgrade
the indexes in the first place?
Shai
On Thu, Sep 17, 2015 at 3:05 PM, Yago Riveiro <ya...@gmail.com>
wrote:
> I have a very old index with more than 12T (re-index data is not an option
> ...) that I want upgrade to 5.3, I’m using lucene-core-4.10.4.jar (I’m in
> 4.10.4 right now) to upgrade old segments of data. With solr running I can
> run the command because solr has the lock of the core.
>
>
>
>
>
> I only want to unload it to perform an index upgrade command to load again
> without stop the node with the core.
>
>
>
>
>
> The issue here is that the DELETEREPLICA deletes the data … the UNLOAD
> command doesn’t save the original core.properties and creates a new one
> core.properties.unloaded that not store the shard param and creates a new
> random coreNodeName that doesn’t correspond to the name of
> the clusterstate, the result … the core is loaded twice or even worse, in
> some situation is attached to a wrong shard.
>
>
>
>
>
>
>
>
> —/Yago Riveiro
>
> On Wed, Sep 16, 2015 at 4:38 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
> > The not-very-helpful answer is that you're using the core admin API in
> > a SolrCloud setup. Please do not do this as (you're well aware of this
> by now!)
> > it's far too easy to get "interesting" results.
> > Instead, use the Collections API, specifically the ADDREPLICA and
> DELETEREPLICA
> > commands. Under the covers, they actually create a core via the core
> admin API,
> > but they insure that all the core create parameters are correct. For
> ADDREPLICA,
> > you can also easily insure that a node lands on a particular machine.
> > Best,
> > Erick
> > On Tue, Sep 15, 2015 at 9:46 AM, Yago Riveiro <ya...@gmail.com>
> wrote:
> >> Hi,
> >>
> >> I’m having some issues with the command CREATE of the CoreAdmin API in
> solr
> >> 4.10.4.
> >>
> >> When I try to load a previous unloaded core with CREATE command, the
> result
> >> of this operation is 2 replicas in down state. One with the original
> >> coreNodeName set in clusterstate.json and other with an new one.
> >>
> >> I’m with the new solr.xml format and the core.properties of cores looks
> like
> >> (I set this configuration when upgraded from 4.6.1 in legacy solr.xml
> to
> >> 4.10.4:
> >>
> >> name=bucket-15_shardX_replicaY
> >> shard=shardX
> >> collection=bucket-15
> >>
> >> The API command looks like:
> >>
> >>
> solr/admin/cores?action=CREATE&name=bucket-15_shardX_replicaY&collection=bucket-15&shard=shardX&wt=json
> >>
> >> What I’m doing wrong?
> >>
> >>
> >>
> >> -----
> >> Best regards
> >> --
> >> View this message in context:
> http://lucene.472066.n3.nabble.com/Problem-with-CoreAdmin-API-CREATE-command-tp4229248.html
> >> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with CoreAdmin API CREATE command
Posted by Yago Riveiro <ya...@gmail.com>.
I have a very old index with more than 12T (re-index data is not an option ...) that I want upgrade to 5.3, I’m using lucene-core-4.10.4.jar (I’m in 4.10.4 right now) to upgrade old segments of data. With solr running I can run the command because solr has the lock of the core.
I only want to unload it to perform an index upgrade command to load again without stop the node with the core.
The issue here is that the DELETEREPLICA deletes the data … the UNLOAD command doesn’t save the original core.properties and creates a new one core.properties.unloaded that not store the shard param and creates a new random coreNodeName that doesn’t correspond to the name of the clusterstate, the result … the core is loaded twice or even worse, in some situation is attached to a wrong shard.
—/Yago Riveiro
On Wed, Sep 16, 2015 at 4:38 PM, Erick Erickson <er...@gmail.com>
wrote:
> The not-very-helpful answer is that you're using the core admin API in
> a SolrCloud setup. Please do not do this as (you're well aware of this by now!)
> it's far too easy to get "interesting" results.
> Instead, use the Collections API, specifically the ADDREPLICA and DELETEREPLICA
> commands. Under the covers, they actually create a core via the core admin API,
> but they insure that all the core create parameters are correct. For ADDREPLICA,
> you can also easily insure that a node lands on a particular machine.
> Best,
> Erick
> On Tue, Sep 15, 2015 at 9:46 AM, Yago Riveiro <ya...@gmail.com> wrote:
>> Hi,
>>
>> I’m having some issues with the command CREATE of the CoreAdmin API in solr
>> 4.10.4.
>>
>> When I try to load a previous unloaded core with CREATE command, the result
>> of this operation is 2 replicas in down state. One with the original
>> coreNodeName set in clusterstate.json and other with an new one.
>>
>> I’m with the new solr.xml format and the core.properties of cores looks like
>> (I set this configuration when upgraded from 4.6.1 in legacy solr.xml to
>> 4.10.4:
>>
>> name=bucket-15_shardX_replicaY
>> shard=shardX
>> collection=bucket-15
>>
>> The API command looks like:
>>
>> solr/admin/cores?action=CREATE&name=bucket-15_shardX_replicaY&collection=bucket-15&shard=shardX&wt=json
>>
>> What I’m doing wrong?
>>
>>
>>
>> -----
>> Best regards
>> --
>> View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-CoreAdmin-API-CREATE-command-tp4229248.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with CoreAdmin API CREATE command
Posted by Erick Erickson <er...@gmail.com>.
The not-very-helpful answer is that you're using the core admin API in
a SolrCloud setup. Please do not do this as (you're well aware of this by now!)
it's far too easy to get "interesting" results.
Instead, use the Collections API, specifically the ADDREPLICA and DELETEREPLICA
commands. Under the covers, they actually create a core via the core admin API,
but they insure that all the core create parameters are correct. For ADDREPLICA,
you can also easily insure that a node lands on a particular machine.
Best,
Erick
On Tue, Sep 15, 2015 at 9:46 AM, Yago Riveiro <ya...@gmail.com> wrote:
> Hi,
>
> I’m having some issues with the command CREATE of the CoreAdmin API in solr
> 4.10.4.
>
> When I try to load a previous unloaded core with CREATE command, the result
> of this operation is 2 replicas in down state. One with the original
> coreNodeName set in clusterstate.json and other with an new one.
>
> I’m with the new solr.xml format and the core.properties of cores looks like
> (I set this configuration when upgraded from 4.6.1 in legacy solr.xml to
> 4.10.4:
>
> name=bucket-15_shardX_replicaY
> shard=shardX
> collection=bucket-15
>
> The API command looks like:
>
> solr/admin/cores?action=CREATE&name=bucket-15_shardX_replicaY&collection=bucket-15&shard=shardX&wt=json
>
> What I’m doing wrong?
>
>
>
> -----
> Best regards
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-CoreAdmin-API-CREATE-command-tp4229248.html
> Sent from the Solr - User mailing list archive at Nabble.com.