You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rachid Bouacheria <wi...@gmail.com> on 2016/07/27 18:37:44 UTC
Solr 6 managed-schema & version control
Hi All,
I am upgrading from solr 4 to 6.
In solr 4 I have a schema.xml that is under version control.
But solr 6 has the notion of a managed schema that could be modified via a
solr api call.
This seems great and flexible, but my assumption is that in this case
zookeeper becomes the authoritative copy and not SVN or Git.
And this is where things become unclear to me.
Is the expectation to download the configuration from zk the same way we do
an svn checkout to have the configuration and run locally?
How do we know who changed what and when?
I know that there still is the option to use schema.xml by using
the ClassicIndexSchemaFactory but I am curious to hear from y'all that use
managed schema how you are doing it and if there are any downside, gotchas,
or if all is just much better :-)
Seems to me that running locally is harder as you cannot just checkout a
project that contains the up to date schema.
Thank you,
Rachid.
Re: Solr 6 managed-schema & version control
Posted by Erick Erickson <er...@gmail.com>.
Rachid is exactly right. There's a long story here, I had
the bright idea of editing the schema directly from the admin UI
and one of the folks who, you know, understands browsers
did the heavy lifting.... and then we were alerted to the fact that
this is a huge security issue (i.e. uploading arbitrary XML to
a website) so had to pull it all out.
The managed schema API is a was to have front-ends that
change the schema and config.
Best,
Erick
On Fri, Jul 29, 2016 at 7:19 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 7/27/2016 12:37 PM, Rachid Bouacheria wrote:
>> I know that there still is the option to use schema.xml by using the
>> ClassicIndexSchemaFactory but I am curious to hear from y'all that use
>> managed schema how you are doing it and if there are any downside,
>> gotchas, or if all is just much better :-)
>
> You have two choices.
>
> * Switch to the classic schema and use a file named schema.xml.
>
> * Don't worry about it.
> ** Keep the config as-is.
> ** Use existing methods with a new filename.
> ** Ignore the Schema API functionality entirely.
>
> There's nothing preventing you from just editing managed-schema. Yes, I
> am aware that the first thing the example file says is "don't edit this
> file" ... but if you are not using the Schema API, then you don't have
> to worry about that.
>
> If you're extremely careful, you can combine Schema API usage and hand
> edits ... but I wouldn't recommend doing that.
>
> Thanks,
> Shawn
>
Re: Solr 6 managed-schema & version control
Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/27/2016 12:37 PM, Rachid Bouacheria wrote:
> I know that there still is the option to use schema.xml by using the
> ClassicIndexSchemaFactory but I am curious to hear from y'all that use
> managed schema how you are doing it and if there are any downside,
> gotchas, or if all is just much better :-)
You have two choices.
* Switch to the classic schema and use a file named schema.xml.
* Don't worry about it.
** Keep the config as-is.
** Use existing methods with a new filename.
** Ignore the Schema API functionality entirely.
There's nothing preventing you from just editing managed-schema. Yes, I
am aware that the first thing the example file says is "don't edit this
file" ... but if you are not using the Schema API, then you don't have
to worry about that.
If you're extremely careful, you can combine Schema API usage and hand
edits ... but I wouldn't recommend doing that.
Thanks,
Shawn
Re: Solr 6 managed-schema & version control
Posted by John Bickerstaff <jo...@johnbickerstaff.com>.
Gratzi!
On Jul 28, 2016 9:14 PM, "Rachid Bouacheria" <wi...@gmail.com> wrote:
> Jonn,
> I don't want to answer on Erick's behalf, but I am pretty sure there is no
> UI built in solr 6 that allows you to update the schema and somehow check
> it in VCS. I would guess that you could do this by exposing an MBean.
> Anyway that's how I interpreted Erick's reply.
>
>
> On Wed, Jul 27, 2016 at 6:01 PM, John Bickerstaff <
> john@johnbickerstaff.com>
> wrote:
>
> > Erick - the UI you mention -- something that exists or something that has
> > to be built? (I'm upgrading to version 6 as well and this question is
> one
> > I'll have to deal with...)
> >
> > On Wed, Jul 27, 2016 at 5:31 PM, Rachid Bouacheria <wi...@gmail.com>
> > wrote:
> >
> > > Thank you very much Erick, I appreciate your feed back.
> > >
> > > On Wed, Jul 27, 2016 at 2:24 PM, Erick Erickson <
> erickerickson@gmail.com
> > >
> > > wrote:
> > >
> > > > Using classic schema is perfectly acceptable/reasonable, you can
> > > > continue to do so freely (you'll have to change to
> > > > ClassicSchemaFactory though).
> > > >
> > > > Also, you can freely edit managed-schema just as you did schema.xml.
> > > > The "trick" here is that you have to take some care _not_ to issue
> > > > commands that modify the schema or the in-memory version will
> > > > overwrite the one in ZK. Otherwise, though, you can freely use
> > > > managed-schema just as you do classic schema.
> > > >
> > > > So you can do just what you do now, keep managed-schema in VCS and
> > > > upconfig it. Also note that Solr 6.2 has "bin/solr zk
> > > > upconfig/downconfig/cp/mv/ls" functionality.
> > > >
> > > > Managed lends itself to some kind of UI that maintains it. The
> process
> > > > (IMO) for using that in prod would be something like:
> > > > > Use the UI to build your schema
> > > > > copy from ZK to your local machine
> > > > > put the configs in VCS
> > > > > Deploy using the VCS as your system-of-record.
> > > >
> > > > But that's just my approach. If you don't want to use the
> > > > managed-schema features, switch back to classic IMO.
> > > >
> > > > Best,
> > > > Erick
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Jul 27, 2016 at 11:37 AM, Rachid Bouacheria <
> > willix44@gmail.com>
> > > > wrote:
> > > > > Hi All,
> > > > >
> > > > > I am upgrading from solr 4 to 6.
> > > > > In solr 4 I have a schema.xml that is under version control.
> > > > > But solr 6 has the notion of a managed schema that could be
> modified
> > > via
> > > > a
> > > > > solr api call.
> > > > > This seems great and flexible, but my assumption is that in this
> case
> > > > > zookeeper becomes the authoritative copy and not SVN or Git.
> > > > >
> > > > > And this is where things become unclear to me.
> > > > > Is the expectation to download the configuration from zk the same
> way
> > > we
> > > > do
> > > > > an svn checkout to have the configuration and run locally?
> > > > > How do we know who changed what and when?
> > > > >
> > > > > I know that there still is the option to use schema.xml by using
> > > > > the ClassicIndexSchemaFactory but I am curious to hear from y'all
> > that
> > > > use
> > > > > managed schema how you are doing it and if there are any downside,
> > > > gotchas,
> > > > > or if all is just much better :-)
> > > > >
> > > > > Seems to me that running locally is harder as you cannot just
> > checkout
> > > a
> > > > > project that contains the up to date schema.
> > > > >
> > > > > Thank you,
> > > > > Rachid.
> > > >
> > >
> >
>
Re: Solr 6 managed-schema & version control
Posted by Rachid Bouacheria <wi...@gmail.com>.
Jonn,
I don't want to answer on Erick's behalf, but I am pretty sure there is no
UI built in solr 6 that allows you to update the schema and somehow check
it in VCS. I would guess that you could do this by exposing an MBean.
Anyway that's how I interpreted Erick's reply.
On Wed, Jul 27, 2016 at 6:01 PM, John Bickerstaff <jo...@johnbickerstaff.com>
wrote:
> Erick - the UI you mention -- something that exists or something that has
> to be built? (I'm upgrading to version 6 as well and this question is one
> I'll have to deal with...)
>
> On Wed, Jul 27, 2016 at 5:31 PM, Rachid Bouacheria <wi...@gmail.com>
> wrote:
>
> > Thank you very much Erick, I appreciate your feed back.
> >
> > On Wed, Jul 27, 2016 at 2:24 PM, Erick Erickson <erickerickson@gmail.com
> >
> > wrote:
> >
> > > Using classic schema is perfectly acceptable/reasonable, you can
> > > continue to do so freely (you'll have to change to
> > > ClassicSchemaFactory though).
> > >
> > > Also, you can freely edit managed-schema just as you did schema.xml.
> > > The "trick" here is that you have to take some care _not_ to issue
> > > commands that modify the schema or the in-memory version will
> > > overwrite the one in ZK. Otherwise, though, you can freely use
> > > managed-schema just as you do classic schema.
> > >
> > > So you can do just what you do now, keep managed-schema in VCS and
> > > upconfig it. Also note that Solr 6.2 has "bin/solr zk
> > > upconfig/downconfig/cp/mv/ls" functionality.
> > >
> > > Managed lends itself to some kind of UI that maintains it. The process
> > > (IMO) for using that in prod would be something like:
> > > > Use the UI to build your schema
> > > > copy from ZK to your local machine
> > > > put the configs in VCS
> > > > Deploy using the VCS as your system-of-record.
> > >
> > > But that's just my approach. If you don't want to use the
> > > managed-schema features, switch back to classic IMO.
> > >
> > > Best,
> > > Erick
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Jul 27, 2016 at 11:37 AM, Rachid Bouacheria <
> willix44@gmail.com>
> > > wrote:
> > > > Hi All,
> > > >
> > > > I am upgrading from solr 4 to 6.
> > > > In solr 4 I have a schema.xml that is under version control.
> > > > But solr 6 has the notion of a managed schema that could be modified
> > via
> > > a
> > > > solr api call.
> > > > This seems great and flexible, but my assumption is that in this case
> > > > zookeeper becomes the authoritative copy and not SVN or Git.
> > > >
> > > > And this is where things become unclear to me.
> > > > Is the expectation to download the configuration from zk the same way
> > we
> > > do
> > > > an svn checkout to have the configuration and run locally?
> > > > How do we know who changed what and when?
> > > >
> > > > I know that there still is the option to use schema.xml by using
> > > > the ClassicIndexSchemaFactory but I am curious to hear from y'all
> that
> > > use
> > > > managed schema how you are doing it and if there are any downside,
> > > gotchas,
> > > > or if all is just much better :-)
> > > >
> > > > Seems to me that running locally is harder as you cannot just
> checkout
> > a
> > > > project that contains the up to date schema.
> > > >
> > > > Thank you,
> > > > Rachid.
> > >
> >
>
Re: Solr 6 managed-schema & version control
Posted by John Bickerstaff <jo...@johnbickerstaff.com>.
Erick - the UI you mention -- something that exists or something that has
to be built? (I'm upgrading to version 6 as well and this question is one
I'll have to deal with...)
On Wed, Jul 27, 2016 at 5:31 PM, Rachid Bouacheria <wi...@gmail.com>
wrote:
> Thank you very much Erick, I appreciate your feed back.
>
> On Wed, Jul 27, 2016 at 2:24 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
> > Using classic schema is perfectly acceptable/reasonable, you can
> > continue to do so freely (you'll have to change to
> > ClassicSchemaFactory though).
> >
> > Also, you can freely edit managed-schema just as you did schema.xml.
> > The "trick" here is that you have to take some care _not_ to issue
> > commands that modify the schema or the in-memory version will
> > overwrite the one in ZK. Otherwise, though, you can freely use
> > managed-schema just as you do classic schema.
> >
> > So you can do just what you do now, keep managed-schema in VCS and
> > upconfig it. Also note that Solr 6.2 has "bin/solr zk
> > upconfig/downconfig/cp/mv/ls" functionality.
> >
> > Managed lends itself to some kind of UI that maintains it. The process
> > (IMO) for using that in prod would be something like:
> > > Use the UI to build your schema
> > > copy from ZK to your local machine
> > > put the configs in VCS
> > > Deploy using the VCS as your system-of-record.
> >
> > But that's just my approach. If you don't want to use the
> > managed-schema features, switch back to classic IMO.
> >
> > Best,
> > Erick
> >
> >
> >
> >
> >
> > On Wed, Jul 27, 2016 at 11:37 AM, Rachid Bouacheria <wi...@gmail.com>
> > wrote:
> > > Hi All,
> > >
> > > I am upgrading from solr 4 to 6.
> > > In solr 4 I have a schema.xml that is under version control.
> > > But solr 6 has the notion of a managed schema that could be modified
> via
> > a
> > > solr api call.
> > > This seems great and flexible, but my assumption is that in this case
> > > zookeeper becomes the authoritative copy and not SVN or Git.
> > >
> > > And this is where things become unclear to me.
> > > Is the expectation to download the configuration from zk the same way
> we
> > do
> > > an svn checkout to have the configuration and run locally?
> > > How do we know who changed what and when?
> > >
> > > I know that there still is the option to use schema.xml by using
> > > the ClassicIndexSchemaFactory but I am curious to hear from y'all that
> > use
> > > managed schema how you are doing it and if there are any downside,
> > gotchas,
> > > or if all is just much better :-)
> > >
> > > Seems to me that running locally is harder as you cannot just checkout
> a
> > > project that contains the up to date schema.
> > >
> > > Thank you,
> > > Rachid.
> >
>
Re: Solr 6 managed-schema & version control
Posted by Rachid Bouacheria <wi...@gmail.com>.
Thank you very much Erick, I appreciate your feed back.
On Wed, Jul 27, 2016 at 2:24 PM, Erick Erickson <er...@gmail.com>
wrote:
> Using classic schema is perfectly acceptable/reasonable, you can
> continue to do so freely (you'll have to change to
> ClassicSchemaFactory though).
>
> Also, you can freely edit managed-schema just as you did schema.xml.
> The "trick" here is that you have to take some care _not_ to issue
> commands that modify the schema or the in-memory version will
> overwrite the one in ZK. Otherwise, though, you can freely use
> managed-schema just as you do classic schema.
>
> So you can do just what you do now, keep managed-schema in VCS and
> upconfig it. Also note that Solr 6.2 has "bin/solr zk
> upconfig/downconfig/cp/mv/ls" functionality.
>
> Managed lends itself to some kind of UI that maintains it. The process
> (IMO) for using that in prod would be something like:
> > Use the UI to build your schema
> > copy from ZK to your local machine
> > put the configs in VCS
> > Deploy using the VCS as your system-of-record.
>
> But that's just my approach. If you don't want to use the
> managed-schema features, switch back to classic IMO.
>
> Best,
> Erick
>
>
>
>
>
> On Wed, Jul 27, 2016 at 11:37 AM, Rachid Bouacheria <wi...@gmail.com>
> wrote:
> > Hi All,
> >
> > I am upgrading from solr 4 to 6.
> > In solr 4 I have a schema.xml that is under version control.
> > But solr 6 has the notion of a managed schema that could be modified via
> a
> > solr api call.
> > This seems great and flexible, but my assumption is that in this case
> > zookeeper becomes the authoritative copy and not SVN or Git.
> >
> > And this is where things become unclear to me.
> > Is the expectation to download the configuration from zk the same way we
> do
> > an svn checkout to have the configuration and run locally?
> > How do we know who changed what and when?
> >
> > I know that there still is the option to use schema.xml by using
> > the ClassicIndexSchemaFactory but I am curious to hear from y'all that
> use
> > managed schema how you are doing it and if there are any downside,
> gotchas,
> > or if all is just much better :-)
> >
> > Seems to me that running locally is harder as you cannot just checkout a
> > project that contains the up to date schema.
> >
> > Thank you,
> > Rachid.
>
Re: Solr 6 managed-schema & version control
Posted by Erick Erickson <er...@gmail.com>.
Using classic schema is perfectly acceptable/reasonable, you can
continue to do so freely (you'll have to change to
ClassicSchemaFactory though).
Also, you can freely edit managed-schema just as you did schema.xml.
The "trick" here is that you have to take some care _not_ to issue
commands that modify the schema or the in-memory version will
overwrite the one in ZK. Otherwise, though, you can freely use
managed-schema just as you do classic schema.
So you can do just what you do now, keep managed-schema in VCS and
upconfig it. Also note that Solr 6.2 has "bin/solr zk
upconfig/downconfig/cp/mv/ls" functionality.
Managed lends itself to some kind of UI that maintains it. The process
(IMO) for using that in prod would be something like:
> Use the UI to build your schema
> copy from ZK to your local machine
> put the configs in VCS
> Deploy using the VCS as your system-of-record.
But that's just my approach. If you don't want to use the
managed-schema features, switch back to classic IMO.
Best,
Erick
On Wed, Jul 27, 2016 at 11:37 AM, Rachid Bouacheria <wi...@gmail.com> wrote:
> Hi All,
>
> I am upgrading from solr 4 to 6.
> In solr 4 I have a schema.xml that is under version control.
> But solr 6 has the notion of a managed schema that could be modified via a
> solr api call.
> This seems great and flexible, but my assumption is that in this case
> zookeeper becomes the authoritative copy and not SVN or Git.
>
> And this is where things become unclear to me.
> Is the expectation to download the configuration from zk the same way we do
> an svn checkout to have the configuration and run locally?
> How do we know who changed what and when?
>
> I know that there still is the option to use schema.xml by using
> the ClassicIndexSchemaFactory but I am curious to hear from y'all that use
> managed schema how you are doing it and if there are any downside, gotchas,
> or if all is just much better :-)
>
> Seems to me that running locally is harder as you cannot just checkout a
> project that contains the up to date schema.
>
> Thank you,
> Rachid.