You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Zheng Lin Edwin Yeo <ed...@gmail.com> on 2015/03/30 05:56:36 UTC
Same schema.xml is loaded for different cores in SolrCloud
Hi everyone,
I've created a SolrCloud with multiple core, and I have different
schema.xml for each of the core. However, when I start Solr, there's only
one version of the schema.xml that is loaded onto Solr. Regardless of which
core I go to, the schema.xml that is shown is the first one which I have
loaded.
What I did was, I have 3 cores: logmill, collection1 and collection2.
Each of the core has 2 shrads: shard1 and shard2
I first started the Solr with shard1 using the following command:
java -Dcollection.configName=logmill -DzkRun -DnumShards=2
-Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
After that I start shard2 using the following command:
java -Dcollection.configName=logmill -DzkRun -DnumShards=2
-Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
All the schema.xml loaded are from logmill core, even for the collection1
and collection2.
Even after I change the command to start shard1 with the following command,
all the schema.xml are still from logmill
java -Dcollection.configName=collection1 -DzkRun
-DnumShards=2 -Dbootstrap_confdir=./solr/collection1/conf -jar start.jar
How do I get Solr to read the different schema.xml for the different cores?
Regards,
Edwin
Re: Same schema.xml is loaded for different cores in SolrCloud
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Yes, I've delete my previous collections, and retried these using zkcli and
creating my collections using the collections API thereafter. It's working
now.
Thanks Erick.
Regards,
Edwin
On 31 March 2015 at 13:55, Erick Erickson <er...@gmail.com> wrote:
> By now, I wouldn't particularly trust my setup. I'd blow it away and start
> over.
>
> bootstrapping is _only_ required to get the configs up to Zookeeper
> the first time. In fact I suggest you don't use it at all. Just start
> SolrCloud, and use zkcli to push the configs up. Thereafter, create
> your collections using the collections API.
>
> Zookeeper is just a central repository for your configs and the
> overall state of your cluster. As far as config sets are concerned,
> think of the upconfig (or bootstrap) as copying the config files to a
> place where they can be found by any random Solr instance that starts
> up.
>
> And the same applies to parameters like numShards. It's _only_ used as
> a convenience for creating a cluster for demo purposes. Thereafter,
> any time you start up that particular cloud, it'll read the old
> cluster state and completely ignore the numShards parameter.
>
> Rather than try to untangle what you've done, I'd re-install and work
> through the tutorial step-by-step. I think you've jumped ahead and
> gotten some things mixed up as far as your cluster state is concerned.
>
> Best,
> Erick
>
> On Mon, Mar 30, 2015 at 6:52 PM, Zheng Lin Edwin Yeo
> <ed...@gmail.com> wrote:
> > Hi Erick,
> >
> > I've started shard2 with the following command instead, but it's still
> the
> > same problem.
> > java -DzkHost=localhost:9983 -Djetty.port=8984 -jar start.jar
> >
> > But you mean for shard1 we do not have to include "
> > -Dbootstrap_confdir=./solr/logmill/conf" for subsequent startup?
> >
> > Regards,
> > Edwin
> >
> >
> > On 31 March 2015 at 00:46, Erick Erickson <er...@gmail.com>
> wrote:
> >
> >> OK, this is a bit confused:
> >>
> >> 1> You're starting two embedded Zookeepers but they don't know about
> >> each other. So looking for the configsets is a bit confused.
> >> 2> There's no need to do the bootstrap thing after the first time. The
> >> _very_ first time you do this it pushes the configs up to Zookeeper,
> >> but after that you should just reference the config name.
> >> 3> you specify the config name when you _create_ the collection, not
> >> when you start it up. You may be doing this, but your startup.
> >> 4> I think you're confusing shards with collections. The equivalent of
> >> older-style cores would be just single-shard _collections_. Configs
> >> are associated on the collection level, not the shard level as all
> >> shards in a collection are presumed (indeed, _must_) use the same
> >> configuration.
> >>
> >> HTH,
> >> Erick
> >>
> >> On Mon, Mar 30, 2015 at 2:20 AM, Zheng Lin Edwin Yeo
> >> <ed...@gmail.com> wrote:
> >> > I've roughly know what is the problem from here.
> >> >
> >>
> http://stackoverflow.com/questions/23338324/zookeeper-multiple-collection-different-schema
> >> >
> >> > However, I couldn't find the zoo_data directory in all of my solr
> folder.
> >> > What could be the problem or where is the directory supposed to be
> >> located?
> >> >
> >> > Regards,
> >> > Edwin
> >> >
> >> >
> >> > On 30 March 2015 at 11:56, Zheng Lin Edwin Yeo <ed...@gmail.com>
> >> wrote:
> >> >
> >> >> Hi everyone,
> >> >>
> >> >> I've created a SolrCloud with multiple core, and I have different
> >> >> schema.xml for each of the core. However, when I start Solr, there's
> >> only
> >> >> one version of the schema.xml that is loaded onto Solr. Regardless of
> >> which
> >> >> core I go to, the schema.xml that is shown is the first one which I
> have
> >> >> loaded.
> >> >>
> >> >> What I did was, I have 3 cores: logmill, collection1 and collection2.
> >> >> Each of the core has 2 shrads: shard1 and shard2
> >> >>
> >> >> I first started the Solr with shard1 using the following command:
> >> >> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
> >> >> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
> >> >>
> >> >> After that I start shard2 using the following command:
> >> >> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
> >> >> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
> >> >>
> >> >> All the schema.xml loaded are from logmill core, even for the
> >> collection1
> >> >> and collection2.
> >> >>
> >> >> Even after I change the command to start shard1 with the following
> >> >> command, all the schema.xml are still from logmill
> >> >> java -Dcollection.configName=collection1 -DzkRun
> >> >> -DnumShards=2 -Dbootstrap_confdir=./solr/collection1/conf -jar
> start.jar
> >> >>
> >> >>
> >> >> How do I get Solr to read the different schema.xml for the different
> >> cores?
> >> >>
> >> >> Regards,
> >> >> Edwin
> >> >>
> >>
>
Re: Same schema.xml is loaded for different cores in SolrCloud
Posted by Erick Erickson <er...@gmail.com>.
By now, I wouldn't particularly trust my setup. I'd blow it away and start over.
bootstrapping is _only_ required to get the configs up to Zookeeper
the first time. In fact I suggest you don't use it at all. Just start
SolrCloud, and use zkcli to push the configs up. Thereafter, create
your collections using the collections API.
Zookeeper is just a central repository for your configs and the
overall state of your cluster. As far as config sets are concerned,
think of the upconfig (or bootstrap) as copying the config files to a
place where they can be found by any random Solr instance that starts
up.
And the same applies to parameters like numShards. It's _only_ used as
a convenience for creating a cluster for demo purposes. Thereafter,
any time you start up that particular cloud, it'll read the old
cluster state and completely ignore the numShards parameter.
Rather than try to untangle what you've done, I'd re-install and work
through the tutorial step-by-step. I think you've jumped ahead and
gotten some things mixed up as far as your cluster state is concerned.
Best,
Erick
On Mon, Mar 30, 2015 at 6:52 PM, Zheng Lin Edwin Yeo
<ed...@gmail.com> wrote:
> Hi Erick,
>
> I've started shard2 with the following command instead, but it's still the
> same problem.
> java -DzkHost=localhost:9983 -Djetty.port=8984 -jar start.jar
>
> But you mean for shard1 we do not have to include "
> -Dbootstrap_confdir=./solr/logmill/conf" for subsequent startup?
>
> Regards,
> Edwin
>
>
> On 31 March 2015 at 00:46, Erick Erickson <er...@gmail.com> wrote:
>
>> OK, this is a bit confused:
>>
>> 1> You're starting two embedded Zookeepers but they don't know about
>> each other. So looking for the configsets is a bit confused.
>> 2> There's no need to do the bootstrap thing after the first time. The
>> _very_ first time you do this it pushes the configs up to Zookeeper,
>> but after that you should just reference the config name.
>> 3> you specify the config name when you _create_ the collection, not
>> when you start it up. You may be doing this, but your startup.
>> 4> I think you're confusing shards with collections. The equivalent of
>> older-style cores would be just single-shard _collections_. Configs
>> are associated on the collection level, not the shard level as all
>> shards in a collection are presumed (indeed, _must_) use the same
>> configuration.
>>
>> HTH,
>> Erick
>>
>> On Mon, Mar 30, 2015 at 2:20 AM, Zheng Lin Edwin Yeo
>> <ed...@gmail.com> wrote:
>> > I've roughly know what is the problem from here.
>> >
>> http://stackoverflow.com/questions/23338324/zookeeper-multiple-collection-different-schema
>> >
>> > However, I couldn't find the zoo_data directory in all of my solr folder.
>> > What could be the problem or where is the directory supposed to be
>> located?
>> >
>> > Regards,
>> > Edwin
>> >
>> >
>> > On 30 March 2015 at 11:56, Zheng Lin Edwin Yeo <ed...@gmail.com>
>> wrote:
>> >
>> >> Hi everyone,
>> >>
>> >> I've created a SolrCloud with multiple core, and I have different
>> >> schema.xml for each of the core. However, when I start Solr, there's
>> only
>> >> one version of the schema.xml that is loaded onto Solr. Regardless of
>> which
>> >> core I go to, the schema.xml that is shown is the first one which I have
>> >> loaded.
>> >>
>> >> What I did was, I have 3 cores: logmill, collection1 and collection2.
>> >> Each of the core has 2 shrads: shard1 and shard2
>> >>
>> >> I first started the Solr with shard1 using the following command:
>> >> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
>> >> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
>> >>
>> >> After that I start shard2 using the following command:
>> >> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
>> >> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
>> >>
>> >> All the schema.xml loaded are from logmill core, even for the
>> collection1
>> >> and collection2.
>> >>
>> >> Even after I change the command to start shard1 with the following
>> >> command, all the schema.xml are still from logmill
>> >> java -Dcollection.configName=collection1 -DzkRun
>> >> -DnumShards=2 -Dbootstrap_confdir=./solr/collection1/conf -jar start.jar
>> >>
>> >>
>> >> How do I get Solr to read the different schema.xml for the different
>> cores?
>> >>
>> >> Regards,
>> >> Edwin
>> >>
>>
Re: Same schema.xml is loaded for different cores in SolrCloud
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Hi Erick,
I've started shard2 with the following command instead, but it's still the
same problem.
java -DzkHost=localhost:9983 -Djetty.port=8984 -jar start.jar
But you mean for shard1 we do not have to include "
-Dbootstrap_confdir=./solr/logmill/conf" for subsequent startup?
Regards,
Edwin
On 31 March 2015 at 00:46, Erick Erickson <er...@gmail.com> wrote:
> OK, this is a bit confused:
>
> 1> You're starting two embedded Zookeepers but they don't know about
> each other. So looking for the configsets is a bit confused.
> 2> There's no need to do the bootstrap thing after the first time. The
> _very_ first time you do this it pushes the configs up to Zookeeper,
> but after that you should just reference the config name.
> 3> you specify the config name when you _create_ the collection, not
> when you start it up. You may be doing this, but your startup.
> 4> I think you're confusing shards with collections. The equivalent of
> older-style cores would be just single-shard _collections_. Configs
> are associated on the collection level, not the shard level as all
> shards in a collection are presumed (indeed, _must_) use the same
> configuration.
>
> HTH,
> Erick
>
> On Mon, Mar 30, 2015 at 2:20 AM, Zheng Lin Edwin Yeo
> <ed...@gmail.com> wrote:
> > I've roughly know what is the problem from here.
> >
> http://stackoverflow.com/questions/23338324/zookeeper-multiple-collection-different-schema
> >
> > However, I couldn't find the zoo_data directory in all of my solr folder.
> > What could be the problem or where is the directory supposed to be
> located?
> >
> > Regards,
> > Edwin
> >
> >
> > On 30 March 2015 at 11:56, Zheng Lin Edwin Yeo <ed...@gmail.com>
> wrote:
> >
> >> Hi everyone,
> >>
> >> I've created a SolrCloud with multiple core, and I have different
> >> schema.xml for each of the core. However, when I start Solr, there's
> only
> >> one version of the schema.xml that is loaded onto Solr. Regardless of
> which
> >> core I go to, the schema.xml that is shown is the first one which I have
> >> loaded.
> >>
> >> What I did was, I have 3 cores: logmill, collection1 and collection2.
> >> Each of the core has 2 shrads: shard1 and shard2
> >>
> >> I first started the Solr with shard1 using the following command:
> >> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
> >> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
> >>
> >> After that I start shard2 using the following command:
> >> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
> >> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
> >>
> >> All the schema.xml loaded are from logmill core, even for the
> collection1
> >> and collection2.
> >>
> >> Even after I change the command to start shard1 with the following
> >> command, all the schema.xml are still from logmill
> >> java -Dcollection.configName=collection1 -DzkRun
> >> -DnumShards=2 -Dbootstrap_confdir=./solr/collection1/conf -jar start.jar
> >>
> >>
> >> How do I get Solr to read the different schema.xml for the different
> cores?
> >>
> >> Regards,
> >> Edwin
> >>
>
Re: Same schema.xml is loaded for different cores in SolrCloud
Posted by Erick Erickson <er...@gmail.com>.
OK, this is a bit confused:
1> You're starting two embedded Zookeepers but they don't know about
each other. So looking for the configsets is a bit confused.
2> There's no need to do the bootstrap thing after the first time. The
_very_ first time you do this it pushes the configs up to Zookeeper,
but after that you should just reference the config name.
3> you specify the config name when you _create_ the collection, not
when you start it up. You may be doing this, but your startup.
4> I think you're confusing shards with collections. The equivalent of
older-style cores would be just single-shard _collections_. Configs
are associated on the collection level, not the shard level as all
shards in a collection are presumed (indeed, _must_) use the same
configuration.
HTH,
Erick
On Mon, Mar 30, 2015 at 2:20 AM, Zheng Lin Edwin Yeo
<ed...@gmail.com> wrote:
> I've roughly know what is the problem from here.
> http://stackoverflow.com/questions/23338324/zookeeper-multiple-collection-different-schema
>
> However, I couldn't find the zoo_data directory in all of my solr folder.
> What could be the problem or where is the directory supposed to be located?
>
> Regards,
> Edwin
>
>
> On 30 March 2015 at 11:56, Zheng Lin Edwin Yeo <ed...@gmail.com> wrote:
>
>> Hi everyone,
>>
>> I've created a SolrCloud with multiple core, and I have different
>> schema.xml for each of the core. However, when I start Solr, there's only
>> one version of the schema.xml that is loaded onto Solr. Regardless of which
>> core I go to, the schema.xml that is shown is the first one which I have
>> loaded.
>>
>> What I did was, I have 3 cores: logmill, collection1 and collection2.
>> Each of the core has 2 shrads: shard1 and shard2
>>
>> I first started the Solr with shard1 using the following command:
>> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
>> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
>>
>> After that I start shard2 using the following command:
>> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
>> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
>>
>> All the schema.xml loaded are from logmill core, even for the collection1
>> and collection2.
>>
>> Even after I change the command to start shard1 with the following
>> command, all the schema.xml are still from logmill
>> java -Dcollection.configName=collection1 -DzkRun
>> -DnumShards=2 -Dbootstrap_confdir=./solr/collection1/conf -jar start.jar
>>
>>
>> How do I get Solr to read the different schema.xml for the different cores?
>>
>> Regards,
>> Edwin
>>
Re: Same schema.xml is loaded for different cores in SolrCloud
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
I've roughly know what is the problem from here.
http://stackoverflow.com/questions/23338324/zookeeper-multiple-collection-different-schema
However, I couldn't find the zoo_data directory in all of my solr folder.
What could be the problem or where is the directory supposed to be located?
Regards,
Edwin
On 30 March 2015 at 11:56, Zheng Lin Edwin Yeo <ed...@gmail.com> wrote:
> Hi everyone,
>
> I've created a SolrCloud with multiple core, and I have different
> schema.xml for each of the core. However, when I start Solr, there's only
> one version of the schema.xml that is loaded onto Solr. Regardless of which
> core I go to, the schema.xml that is shown is the first one which I have
> loaded.
>
> What I did was, I have 3 cores: logmill, collection1 and collection2.
> Each of the core has 2 shrads: shard1 and shard2
>
> I first started the Solr with shard1 using the following command:
> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
>
> After that I start shard2 using the following command:
> java -Dcollection.configName=logmill -DzkRun -DnumShards=2
> -Dbootstrap_confdir=./solr/logmill/conf -jar start.jar
>
> All the schema.xml loaded are from logmill core, even for the collection1
> and collection2.
>
> Even after I change the command to start shard1 with the following
> command, all the schema.xml are still from logmill
> java -Dcollection.configName=collection1 -DzkRun
> -DnumShards=2 -Dbootstrap_confdir=./solr/collection1/conf -jar start.jar
>
>
> How do I get Solr to read the different schema.xml for the different cores?
>
> Regards,
> Edwin
>