Re: Creating shard with core.properties

[Bharath Kumar <bh...@gmail.com>]

Thanks Shawn for your inputs and the pointer to the documentation. Our
setup currently has 1 shard and 2 replicas for that shard and we do not
want a manual step which involves creating a collection since for SOLR
Cloud at least more than 50% of the shard nodes should be up and running.
Also if the zookeeper states go bad for some reason, we will need to
re-create the collection, whereas in the legacy cloud mode with manual
core.properties creation it has helped us bring up the solr cloud even
without any known zookeeper states after an upgrade and not do any
additional step.

On Wed, Jan 30, 2019 at 3:49 PM Shawn Heisey <ap...@elyograg.org> wrote:

> On 1/30/2019 3:36 PM, Bharath Kumar wrote:
> > Thanks Erick. We cleanup the zookeeper state on every installation, so
> the
> > zookeeper states are gone. So what should we do in case of a new 7.6
> > installation where we want to manually create core.properties and use the
> > non-legacy cloud option? Is it in order to use non-legacy cloud, we
> should
> > use the collections api to create a collection first and then use the
> > manual core.properties for auto-discovery?
>
> *ALL* creations and modifications to SolrCloud collections should be
> done using the Collections API.  Creating cores directly (either with
> core.properties or the CoreAdmin API) is something that will almost
> certainly bite you hard.  Based on what Erick has said, I don't think
> you can even do it at all when legacy mode is disabled.  Even when you
> can ... don't.
>
> > Because in the legacy cloud mode we were just creating the
> core.properties
> > manually and that would update the zookeeper state when the solr boots
> up.
> > Can you please help me with this?
>
> Use the Collections API.  This is the recommendation even for experts
> who really know the code.  Creating cores manually in ANY SolrCloud
> install is a recipe for problems, even in legacy mode.
>
> There is a very large warning box (red triangle with an exclamation
> point) in this section of the documentation:
>
>
> https://lucene.apache.org/solr/guide/7_6/coreadmin-api.html#coreadmin-create
>
> One of the first things it says there in that warning box is that the
> CoreAdmin API should not be used in SolrCloud.  Manually creating
> core.properties files and restarting Solr is effectively the same thing
> as using the CoreAdmin API.
>
> Thanks,
> Shawn
>


-- 
Thanks & Regards,
Bharath MV Kumar

"Life is short, enjoy every moment of it"

Re: Creating shard with core.properties

[Erick Erickson <er...@gmail.com>]

I think you're making this much more difficult for yourself than necessary.

I'd _strongly_ recommend you abandon this approach and use the
collections AP. Perhaps you'd need to create some kind of script
that handles core creation and the like. If you know where the
core.properties file should be for instance (which you must if
you're trying to create it manually) you can specify instanceDir
and/or dataDir in the ADDREPLICA command to point it to an
index for example.

You can use the EMPTY flag on the collection create command
to create the skeleton in ZK with _no_ replicas defined and use
ADDREPLICA.

Frankly, it sounds like you got started down this road with
legacyCloud quite some time ago and are unwilling to change
even though Solr has changed dramatically. It's always
hard to throw away what's worked in the past, but it's also
sometimes necessary.

Best,
Erick

On Fri, Feb 1, 2019 at 2:41 AM Bharath Kumar <bh...@gmail.com> wrote:
>
> Thanks Shawn for your inputs and the pointer to the documentation. Our
> setup currently has 1 shard and 2 replicas for that shard and we do not
> want a manual step which involves creating a collection since for SOLR
> Cloud at least more than 50% of the shard nodes should be up and running.
> Also if the zookeeper states go bad for some reason, we will need to
> re-create the collection, whereas in the legacy cloud mode with manual
> core.properties creation it has helped us bring up the solr cloud even
> without any known zookeeper states after an upgrade and not do any
> additional step.
>
> On Wed, Jan 30, 2019 at 3:49 PM Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 1/30/2019 3:36 PM, Bharath Kumar wrote:
> > > Thanks Erick. We cleanup the zookeeper state on every installation, so
> > the
> > > zookeeper states are gone. So what should we do in case of a new 7.6
> > > installation where we want to manually create core.properties and use the
> > > non-legacy cloud option? Is it in order to use non-legacy cloud, we
> > should
> > > use the collections api to create a collection first and then use the
> > > manual core.properties for auto-discovery?
> >
> > *ALL* creations and modifications to SolrCloud collections should be
> > done using the Collections API.  Creating cores directly (either with
> > core.properties or the CoreAdmin API) is something that will almost
> > certainly bite you hard.  Based on what Erick has said, I don't think
> > you can even do it at all when legacy mode is disabled.  Even when you
> > can ... don't.
> >
> > > Because in the legacy cloud mode we were just creating the
> > core.properties
> > > manually and that would update the zookeeper state when the solr boots
> > up.
> > > Can you please help me with this?
> >
> > Use the Collections API.  This is the recommendation even for experts
> > who really know the code.  Creating cores manually in ANY SolrCloud
> > install is a recipe for problems, even in legacy mode.
> >
> > There is a very large warning box (red triangle with an exclamation
> > point) in this section of the documentation:
> >
> >
> > https://lucene.apache.org/solr/guide/7_6/coreadmin-api.html#coreadmin-create
> >
> > One of the first things it says there in that warning box is that the
> > CoreAdmin API should not be used in SolrCloud.  Manually creating
> > core.properties files and restarting Solr is effectively the same thing
> > as using the CoreAdmin API.
> >
> > Thanks,
> > Shawn
> >
>
>
> --
> Thanks & Regards,
> Bharath MV Kumar
>
> "Life is short, enjoy every moment of it"