You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by M Skazy <ms...@gmail.com> on 2016/09/09 17:04:43 UTC
Bug with bootstrap_confdir?
Hi,
I was having an issue setting up a Solr instance w/ a external Zookeeper.
My SOLR_HOME is not set to the default location. I believe the problem is
related to the following line and I wanted to confirm if this is a bug:
https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L1383
It would seem that if we're checking if a file exists at some position
relative to SOLR_HOME that the path supplied for bootstrap_confdir should
also be rooted by SOLR_HOME instead of the working directory of the solr
process.
For me this translated into errors when solr started as it was trying to
load a configuration into ZK from a directory that did not exist.
The fix is easy and I can create a patch for this if it is decided that
this is a bug.
Thanks,
Mike
Re: Bug with bootstrap_confdir?
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,
So what you are attempting is to upgrade from a single-node SolrCloud (version?)
with built-in ZooKeeper to a Solr6.1 with external ZK. And you want to carry
over the configuration and data.
First, you must realise that all your collection config is in your old built-in
ZK, not in files in SOLR_HOME, so boostrap_confidir will not work here.
Anyway, it is not the recommended way to bootstrap Solr anymore, so if you
had a SOLR_HOME with a collection1/conf/ folder the preferred way to get
it into Zookeeper would be
1. Put your collection1 folder somewhere else
2. Install Solr with external ZK and start up without collections
3. bin/solr create -c mycollection -d /path/to/old/collection1/conf
4. Then get your data in, i.e. by stopping Solr, copying the data folder and starting
If you have a copy of your config outside of ZK you can use this approach
if your config is only in ZK, you will first have to download it from the old
built-in ZK to disk (zkcli downconfig)
If you need help in debugging any other concrete error messages, please
share exact command-line or action taken along with copy/paste of relevant error
logs, so we can know exactly what you were trying to do.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 13. sep. 2016 kl. 19.02 skrev M Skazy <ms...@gmail.com>:
>
> Hi Jan,
>
> I'll try to shed some more light on the issue I ran into.
>
> ENVIRONMENT:
>
> Solr Version 6.1.0 (however, same code exists in current master)
> SOLR_HOME=/some/directory (contains solr.xml)
> SOLR_HOME/collection1 is an existing core that I was using previously w/
> Solr configured using internal ZK
>
> PROBLEM:
>
> I then setup an external ZK instance which I wanted Solr to use. Having
> solr use the external ZK puts it into solrcloud mode:
> https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L1105
>
> This error happened right when I was booting up Solr w/ the configured
> external ZK for the first time. The command was bin/solr start (run via
> the installed service so 'sudo service solr start'). The error is that the
> directory trying to be used does not exist, eg:
>
> /some/directory/collection1 DOES exist
>
> /opt/solr/server/solr/collection1 does NOT exist
>
> MY INTERPRETATION:
>
> There is a disconnect between the condition being asserted and the value
> being used to bootstrap. My main opinion is that these two things should
> be consistent. I don't see a use case as to why one would check a
> directory as existing at one location and then use a completely unrelated
> path in the following command. Where I'm unclear is whether the
> conditional should be looking specifically in the install directory, or if
> bootstrap_confdir should be using SOLR_HOME.
>
> I would argue SOLR_HOME is the correct solution since bootsrap_conf is
> intended to be "If you pass Dbootstrap_conf=true on startup, each SolrCore
> you have configured will have it's configuration files automatically
> uploaded and linked to the collection that SolrCore is part of". If the
> user changes SOLR_HOME it is usually because the core config/data is stored
> outside the installation directory.
>
> Mike
>
>
> On Sat, Sep 10, 2016 at 5:39 PM, Jan Høydahl <ja...@cominvent.com> wrote:
>
>> Hi
>>
>> It is not clear to me what exact command you were doing when you got an
>> error.
>> Please state your solr version, the exact command you were trying to
>> execute
>> and what error msg you got.
>>
>> Why do you believe that resolving relative bootstrap_confdir relative to
>> SOLR_HOME
>> is a good idea? If a config resides relative to SOLR_HOME, Solr will try to
>> bootstrap that as a core at startup, which is probably not what you would
>> like.
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>>> 9. sep. 2016 kl. 19.04 skrev M Skazy <ms...@gmail.com>:
>>>
>>> Hi,
>>>
>>> I was having an issue setting up a Solr instance w/ a external Zookeeper.
>>> My SOLR_HOME is not set to the default location. I believe the problem
>> is
>>> related to the following line and I wanted to confirm if this is a bug:
>>>
>>> https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L1383
>>>
>>> It would seem that if we're checking if a file exists at some position
>>> relative to SOLR_HOME that the path supplied for bootstrap_confdir should
>>> also be rooted by SOLR_HOME instead of the working directory of the solr
>>> process.
>>>
>>> For me this translated into errors when solr started as it was trying to
>>> load a configuration into ZK from a directory that did not exist.
>>>
>>> The fix is easy and I can create a patch for this if it is decided that
>>> this is a bug.
>>>
>>> Thanks,
>>>
>>> Mike
>>
>>
Re: Bug with bootstrap_confdir?
Posted by M Skazy <ms...@gmail.com>.
Hi Jan,
I'll try to shed some more light on the issue I ran into.
ENVIRONMENT:
Solr Version 6.1.0 (however, same code exists in current master)
SOLR_HOME=/some/directory (contains solr.xml)
SOLR_HOME/collection1 is an existing core that I was using previously w/
Solr configured using internal ZK
PROBLEM:
I then setup an external ZK instance which I wanted Solr to use. Having
solr use the external ZK puts it into solrcloud mode:
https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L1105
This error happened right when I was booting up Solr w/ the configured
external ZK for the first time. The command was bin/solr start (run via
the installed service so 'sudo service solr start'). The error is that the
directory trying to be used does not exist, eg:
/some/directory/collection1 DOES exist
/opt/solr/server/solr/collection1 does NOT exist
MY INTERPRETATION:
There is a disconnect between the condition being asserted and the value
being used to bootstrap. My main opinion is that these two things should
be consistent. I don't see a use case as to why one would check a
directory as existing at one location and then use a completely unrelated
path in the following command. Where I'm unclear is whether the
conditional should be looking specifically in the install directory, or if
bootstrap_confdir should be using SOLR_HOME.
I would argue SOLR_HOME is the correct solution since bootsrap_conf is
intended to be "If you pass Dbootstrap_conf=true on startup, each SolrCore
you have configured will have it's configuration files automatically
uploaded and linked to the collection that SolrCore is part of". If the
user changes SOLR_HOME it is usually because the core config/data is stored
outside the installation directory.
Mike
On Sat, Sep 10, 2016 at 5:39 PM, Jan Høydahl <ja...@cominvent.com> wrote:
> Hi
>
> It is not clear to me what exact command you were doing when you got an
> error.
> Please state your solr version, the exact command you were trying to
> execute
> and what error msg you got.
>
> Why do you believe that resolving relative bootstrap_confdir relative to
> SOLR_HOME
> is a good idea? If a config resides relative to SOLR_HOME, Solr will try to
> bootstrap that as a core at startup, which is probably not what you would
> like.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 9. sep. 2016 kl. 19.04 skrev M Skazy <ms...@gmail.com>:
> >
> > Hi,
> >
> > I was having an issue setting up a Solr instance w/ a external Zookeeper.
> > My SOLR_HOME is not set to the default location. I believe the problem
> is
> > related to the following line and I wanted to confirm if this is a bug:
> >
> > https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L1383
> >
> > It would seem that if we're checking if a file exists at some position
> > relative to SOLR_HOME that the path supplied for bootstrap_confdir should
> > also be rooted by SOLR_HOME instead of the working directory of the solr
> > process.
> >
> > For me this translated into errors when solr started as it was trying to
> > load a configuration into ZK from a directory that did not exist.
> >
> > The fix is easy and I can create a patch for this if it is decided that
> > this is a bug.
> >
> > Thanks,
> >
> > Mike
>
>
Re: Bug with bootstrap_confdir?
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi
It is not clear to me what exact command you were doing when you got an error.
Please state your solr version, the exact command you were trying to execute
and what error msg you got.
Why do you believe that resolving relative bootstrap_confdir relative to SOLR_HOME
is a good idea? If a config resides relative to SOLR_HOME, Solr will try to
bootstrap that as a core at startup, which is probably not what you would like.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 9. sep. 2016 kl. 19.04 skrev M Skazy <ms...@gmail.com>:
>
> Hi,
>
> I was having an issue setting up a Solr instance w/ a external Zookeeper.
> My SOLR_HOME is not set to the default location. I believe the problem is
> related to the following line and I wanted to confirm if this is a bug:
>
> https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L1383
>
> It would seem that if we're checking if a file exists at some position
> relative to SOLR_HOME that the path supplied for bootstrap_confdir should
> also be rooted by SOLR_HOME instead of the working directory of the solr
> process.
>
> For me this translated into errors when solr started as it was trying to
> load a configuration into ZK from a directory that did not exist.
>
> The fix is easy and I can create a patch for this if it is decided that
> this is a bug.
>
> Thanks,
>
> Mike