You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "mattia.martinello@gmail.com" <ma...@gmail.com> on 2015/06/08 19:56:12 UTC
SolrCloud and configuration files on ZooKeeper
Hi.
I setted up a SolrCloud cluster with 3 nodes and configuration files on
Zookeeper.
I read the document at
https://cwiki.apache.org/confluence/display/solr/Using+ZooKeeper+to+Manage+Configuration+Files
and saw that configuration files are stored on ZooKeeper of course.
But now I have to update my custom schema.xml and I cannot realize how I
can download the current config set to update it.
Could you help me please?
Thank you very much!
Bye
Re: SolrCloud and configuration files on ZooKeeper
Posted by Erick Erickson <er...@gmail.com>.
And when you're _not_ using schemaless (thanks, Shalin, didn't even notice that)
you need to specify the -n option.
But I second Shalin's comment, you shouldn't be manually editing something
a managed schema...
Best,
Erick
On Mon, Jun 8, 2015 at 1:09 PM, Shalin Shekhar Mangar
<sh...@gmail.com> wrote:
> Looks like you are using managed-schema (better known as the schemaless
> mode). In this case, the schema is inferred/managed by Solr by looking at
> the documents that you add. You can also use the Schema API to change this
> schema. Editing this schema manually is not recommended. See
> https://cwiki.apache.org/confluence/display/solr/Schema+API
>
> The schemaless mode is most helpful when you want to prototype stuff but I
> wouldn't recommend it in production. If you want full control of the schema
> and you want to manage it yourself then it's best not to choose schemaless
> configuration (data_driven_schema_confgs) while setting up solr.
>
> On Tue, Jun 9, 2015 at 1:19 AM, mattia.martinello@gmail.com <
> mattia.martinello@gmail.com> wrote:
>
>> 2015-06-08 21:42 GMT+02:00 Erick Erickson <er...@gmail.com>:
>>
>> > See "downconfig" here:
>> > https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities
>>
>>
>> Thank you Erick for your answer!
>> I've already seen that page, and I was trying with the following command:
>>
>> root@index1:/opt/zookeeper/bin# ./zkCli.sh -z zk1 -cmd downconfig -confdir
>> /tmp/zkconfig/
>> (zk1 is the first node of the Zookeeper's cluster).
>>
>> But no files are written in /tmp/zkconfig and I see this output:
>>
>> root@index1:/opt/zookeeper/bin# ./zkCli.sh -z zk1 -cmd downconfig -confdir
>> /tmp/zkconfig/
>> Connecting to localhost:2181
>> 2015-06-08 15:44:55,952 [myid:] - INFO [main:Environment@100] - Client
>> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
>> 2015-06-08 15:44:55,955 [myid:] - INFO [main:Environment@100] - Client
>> environment:host.name=index1.dc.ems.network
>> 2015-06-08 15:44:55,955 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.version=1.8.0_45
>> 2015-06-08 15:44:55,957 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.vendor=Oracle Corporation
>> 2015-06-08 15:44:55,957 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.home=/usr/lib/jvm/java-8-oracle/jre
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>>
>> environment:java.class.path=/opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.6.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf:
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>>
>> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.io.tmpdir=/tmp
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.compiler=<NA>
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>> environment:os.name=Linux
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>> environment:os.arch=amd64
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>> environment:os.version=2.6.32-39-pve
>> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>> environment:user.name=root
>> 2015-06-08 15:44:55,959 [myid:] - INFO [main:Environment@100] - Client
>> environment:user.home=/root
>> 2015-06-08 15:44:55,959 [myid:] - INFO [main:Environment@100] - Client
>> environment:user.dir=/opt/zookeeper-3.4.6/bin
>> 2015-06-08 15:44:55,960 [myid:] - INFO [main:ZooKeeper@438] - Initiating
>> client connection, connectString=localhost:2181 sessionTimeout=30000
>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@3eb07fd3
>>
>> I'm guessing you started your cloud with one of the example (i.e. -e
>> > cloud). Under the covers,
>> > it did an "upconfig" to put the configs up in ZK in the first place,
>> > this is the inverse.
>> >
>>
>> I created a new collection with the following SOLR command:
>>
>> ./solr create_collection -c test -replicationFactor 3
>>
>> I also tried to list the configuration in the ZooKeeper's CLI:
>>
>> [zk: 127.0.0.1:2181(CONNECTED) 4] ls /configs/test
>> [currency.xml, protwords.txt, managed-schema, solrconfig.xml, synonyms.txt,
>> elevate.xml, stopwords.txt, params.json, lang]
>>
>> But... Where is the schema.xml file?
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
Re: SolrCloud and configuration files on ZooKeeper
Posted by Erick Erickson <er...@gmail.com>.
The thing that's confusing is this line:
[zk: 127.0.0.1:2181(CONNECTED) 4] ls /configs/test
[currency.xml, protwords.txt, managed-schema, solrconfig.xml, synonyms.txt,
elevate.xml, stopwords.txt, params.json, lang]
it says right there "managed-schema". So my suspicion is that you uploaded this
to ZK as a config set without really intending to.
You might want to consider just creating a new configset with upconfig that
you _know_ is the one you want....
Best,
Erick
On Mon, Jun 8, 2015 at 10:55 PM, mattia.martinello@gmail.com
<ma...@gmail.com> wrote:
> 2015-06-08 22:09 GMT+02:00 Shalin Shekhar Mangar <sh...@gmail.com>:
>
>> Looks like you are using managed-schema (better known as the schemaless
>> mode).
>
>
> Thank you very much Shalin and Erick!
> I've never used schemaless before.
> I am not editing the managed schema, and I uploaded my own custom schema.
> Thank you very much again!
> Bye
Re: SolrCloud and configuration files on ZooKeeper
Posted by "mattia.martinello@gmail.com" <ma...@gmail.com>.
2015-06-08 22:09 GMT+02:00 Shalin Shekhar Mangar <sh...@gmail.com>:
> Looks like you are using managed-schema (better known as the schemaless
> mode).
Thank you very much Shalin and Erick!
I've never used schemaless before.
I am not editing the managed schema, and I uploaded my own custom schema.
Thank you very much again!
Bye
Re: SolrCloud and configuration files on ZooKeeper
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Looks like you are using managed-schema (better known as the schemaless
mode). In this case, the schema is inferred/managed by Solr by looking at
the documents that you add. You can also use the Schema API to change this
schema. Editing this schema manually is not recommended. See
https://cwiki.apache.org/confluence/display/solr/Schema+API
The schemaless mode is most helpful when you want to prototype stuff but I
wouldn't recommend it in production. If you want full control of the schema
and you want to manage it yourself then it's best not to choose schemaless
configuration (data_driven_schema_confgs) while setting up solr.
On Tue, Jun 9, 2015 at 1:19 AM, mattia.martinello@gmail.com <
mattia.martinello@gmail.com> wrote:
> 2015-06-08 21:42 GMT+02:00 Erick Erickson <er...@gmail.com>:
>
> > See "downconfig" here:
> > https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities
>
>
> Thank you Erick for your answer!
> I've already seen that page, and I was trying with the following command:
>
> root@index1:/opt/zookeeper/bin# ./zkCli.sh -z zk1 -cmd downconfig -confdir
> /tmp/zkconfig/
> (zk1 is the first node of the Zookeeper's cluster).
>
> But no files are written in /tmp/zkconfig and I see this output:
>
> root@index1:/opt/zookeeper/bin# ./zkCli.sh -z zk1 -cmd downconfig -confdir
> /tmp/zkconfig/
> Connecting to localhost:2181
> 2015-06-08 15:44:55,952 [myid:] - INFO [main:Environment@100] - Client
> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
> 2015-06-08 15:44:55,955 [myid:] - INFO [main:Environment@100] - Client
> environment:host.name=index1.dc.ems.network
> 2015-06-08 15:44:55,955 [myid:] - INFO [main:Environment@100] - Client
> environment:java.version=1.8.0_45
> 2015-06-08 15:44:55,957 [myid:] - INFO [main:Environment@100] - Client
> environment:java.vendor=Oracle Corporation
> 2015-06-08 15:44:55,957 [myid:] - INFO [main:Environment@100] - Client
> environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>
> environment:java.class.path=/opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.6.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf:
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
>
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
> environment:java.io.tmpdir=/tmp
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
> environment:java.compiler=<NA>
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
> environment:os.name=Linux
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
> environment:os.arch=amd64
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
> environment:os.version=2.6.32-39-pve
> 2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
> environment:user.name=root
> 2015-06-08 15:44:55,959 [myid:] - INFO [main:Environment@100] - Client
> environment:user.home=/root
> 2015-06-08 15:44:55,959 [myid:] - INFO [main:Environment@100] - Client
> environment:user.dir=/opt/zookeeper-3.4.6/bin
> 2015-06-08 15:44:55,960 [myid:] - INFO [main:ZooKeeper@438] - Initiating
> client connection, connectString=localhost:2181 sessionTimeout=30000
> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@3eb07fd3
>
> I'm guessing you started your cloud with one of the example (i.e. -e
> > cloud). Under the covers,
> > it did an "upconfig" to put the configs up in ZK in the first place,
> > this is the inverse.
> >
>
> I created a new collection with the following SOLR command:
>
> ./solr create_collection -c test -replicationFactor 3
>
> I also tried to list the configuration in the ZooKeeper's CLI:
>
> [zk: 127.0.0.1:2181(CONNECTED) 4] ls /configs/test
> [currency.xml, protwords.txt, managed-schema, solrconfig.xml, synonyms.txt,
> elevate.xml, stopwords.txt, params.json, lang]
>
> But... Where is the schema.xml file?
>
--
Regards,
Shalin Shekhar Mangar.
Re: SolrCloud and configuration files on ZooKeeper
Posted by "mattia.martinello@gmail.com" <ma...@gmail.com>.
2015-06-08 21:42 GMT+02:00 Erick Erickson <er...@gmail.com>:
> See "downconfig" here:
> https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities
Thank you Erick for your answer!
I've already seen that page, and I was trying with the following command:
root@index1:/opt/zookeeper/bin# ./zkCli.sh -z zk1 -cmd downconfig -confdir
/tmp/zkconfig/
(zk1 is the first node of the Zookeeper's cluster).
But no files are written in /tmp/zkconfig and I see this output:
root@index1:/opt/zookeeper/bin# ./zkCli.sh -z zk1 -cmd downconfig -confdir
/tmp/zkconfig/
Connecting to localhost:2181
2015-06-08 15:44:55,952 [myid:] - INFO [main:Environment@100] - Client
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-06-08 15:44:55,955 [myid:] - INFO [main:Environment@100] - Client
environment:host.name=index1.dc.ems.network
2015-06-08 15:44:55,955 [myid:] - INFO [main:Environment@100] - Client
environment:java.version=1.8.0_45
2015-06-08 15:44:55,957 [myid:] - INFO [main:Environment@100] - Client
environment:java.vendor=Oracle Corporation
2015-06-08 15:44:55,957 [myid:] - INFO [main:Environment@100] - Client
environment:java.home=/usr/lib/jvm/java-8-oracle/jre
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:java.class.path=/opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.6.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf:
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:java.io.tmpdir=/tmp
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:java.compiler=<NA>
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:os.name=Linux
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:os.arch=amd64
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:os.version=2.6.32-39-pve
2015-06-08 15:44:55,958 [myid:] - INFO [main:Environment@100] - Client
environment:user.name=root
2015-06-08 15:44:55,959 [myid:] - INFO [main:Environment@100] - Client
environment:user.home=/root
2015-06-08 15:44:55,959 [myid:] - INFO [main:Environment@100] - Client
environment:user.dir=/opt/zookeeper-3.4.6/bin
2015-06-08 15:44:55,960 [myid:] - INFO [main:ZooKeeper@438] - Initiating
client connection, connectString=localhost:2181 sessionTimeout=30000
watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@3eb07fd3
I'm guessing you started your cloud with one of the example (i.e. -e
> cloud). Under the covers,
> it did an "upconfig" to put the configs up in ZK in the first place,
> this is the inverse.
>
I created a new collection with the following SOLR command:
./solr create_collection -c test -replicationFactor 3
I also tried to list the configuration in the ZooKeeper's CLI:
[zk: 127.0.0.1:2181(CONNECTED) 4] ls /configs/test
[currency.xml, protwords.txt, managed-schema, solrconfig.xml, synonyms.txt,
elevate.xml, stopwords.txt, params.json, lang]
But... Where is the schema.xml file?
Re: SolrCloud and configuration files on ZooKeeper
Posted by Erick Erickson <er...@gmail.com>.
See "downconfig" here:
https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities
I'm guessing you started your cloud with one of the example (i.e. -e
cloud). Under the covers,
it did an "upconfig" to put the configs up in ZK in the first place,
this is the inverse.
Best,
Erick
On Mon, Jun 8, 2015 at 10:56 AM, mattia.martinello@gmail.com
<ma...@gmail.com> wrote:
> Hi.
> I setted up a SolrCloud cluster with 3 nodes and configuration files on
> Zookeeper.
> I read the document at
> https://cwiki.apache.org/confluence/display/solr/Using+ZooKeeper+to+Manage+Configuration+Files
> and saw that configuration files are stored on ZooKeeper of course.
> But now I have to update my custom schema.xml and I cannot realize how I
> can download the current config set to update it.
> Could you help me please?
> Thank you very much!
> Bye