You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jay Potharaju <js...@gmail.com> on 2016/05/26 03:31:25 UTC

Re: How to save index data to other place? [scottchu]

use property.*dataDir*=*value*
https://cwiki.apache.org/confluence/display/solr/Defining+core.properties

On Wed, May 25, 2016 at 8:20 PM, scott.chu <sc...@udngroup.com> wrote:

>
> When I create a collection, say named 'cugna'. Solr create a folder with
> same name under server\slolr, e.g. /local/solr-5.4.1/server/solr/cugna.
> Index data is also saved there. But wish to save index data on other
> folder, say /var/sc_data/cugna. How can I dothis?
>
> scott.chu,scott.chu@udngroup.com
> 2016/5/26 (週四)
>



-- 
Thanks
Jay Potharaju

Re(2): How to save index data to other place? [scottchu]

Posted by "scott.chu" <sc...@udngroup.com>.
Thanks! I'll see to it.  There're a lot to learn for CentOS, Geez.

scott.chu,scott.chu@udngroup.com
2016/5/30 (週一)
----- Original Message ----- 
From: Shawn Heisey 
To: solr-user 
CC: 
Date: 2016/5/27 (週五) 15:13
Subject: Re: How to save index data to other place? [scottchu]


On 5/26/2016 8:51 PM, scott.chu wrote: 
> I want to migrate my Solrcloud from Windows to CentOS. Because I am 
> new to CentOS, not familiar with how to install Solr on it and I did a 
> lot of config in my Solrcloud on Windows, so I use ftp to upload 
> solr-5.4.1 and zookeeper-3.4.6 folders to 3 different servers running 
> CentOS. (They are all under /local). Then I tweak something in 3rd 
> machine (See my other post titled "Can "Using cp replica and modifying 
> core.properties" rather than ADDREPLICA API work?") and make my 
> Solrcloud run with 3 replicas ok. I do wish to follow the default 
> folder/file convention of solr. So can you show me (or hint me) how 
> to: * Install solr and zookeeper with install shell script under 
> CentOS 6.4? * How to auto start solr and zookeeper under CentOS 6.4? 

Installing Solr: 

https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production#TakingSolrtoProduction-RuntheSolrInstallationScript 

Zookeeper is a completely separate project from Solr. For best results, 
you should consult their documentation: 

http://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html 

I can give you some quick install steps based on what I have done in the 
past. I have no idea whether this is how the zookeeper project would 
recommend doing it, but if it's done correctly, I know that it works. 

Extract the download tarball into some location, like /opt. If you 
choose that location and extract the latest version, you will have 
/opt/zookeeper-3.4.8 to work with. The rest of this assumes that 
directory location. You may need to adjust these steps if you choose 
another location: 

Create a shell script named /usr/local/sbin/zkrun with these contents: 

-------------------------- 
#!/bin/sh 

# chkconfig: - 75 50 
# description: Starts and stops Zookeeper 

cd /opt/zookeeper 
bin/zkServer.sh $1 
-------------------------- 

Change to the /opt directory and do these commands: 

ln -s zookeeper-3.4.8 zookeeper 
chmod +x /usr/local/sbin/zkrun 
ln -s /usr/local/sbin/zkrun /etc/init.d/zookeeper 
chkconfig --add zookeeper 
chkconfig zookeeper on 

Consult zookeeper documentation on how to configure the newly installed 
server -- which consists of creating /opt/zookeeper/conf/zoo.cfg and 
${dataDir}/myid A common setting for the dataDir config parameter is 
"zoodata". This is a path that is relative to the current working 
directory. If you use my little startup script with that value, the 
full path becomes opt/zookeeper/zoodata. Once it's configured, you can 
start the zookeeper service. This is a slightly redacted copy of my 
zookeeper config file: 

------------------------------------- 
# The number of milliseconds of each tick 
tickTime=2000 
# The number of ticks that the initial 
# synchronization phase can take 
initLimit=10 
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement 
syncLimit=5 
# the directory where the snapshot is stored. 
# do not use /tmp for storage, /tmp here is just 
# example sakes. 
dataDir=zoodata 
# the port at which the clients will connect 
clientPort=2181 
# 
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge. 
# 
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance 
# 
# The number of snapshots to retain in dataDir 
#autopurge.snapRetainCount=3 
# Purge task interval in hours 
# Set to "0" to disable auto purge feature 
#autopurge.purgeInterval=1 
server.1=zoo1.REDACTED.com:2888:3888 
server.2=zoo2.REDACTED.com:2888:3888 
server.3=zoo3.REDACTED.com:2888:3888 
------------------------------------- 

Thanks, 
Shawn 



----- 
未在此訊息中找到病毒。 
已透過 AVG 檢查 - www.avg.com 
版本: 2015.0.6201 / 病毒庫: 4568/12324 - 發佈日期: 05/29/16

Re: How to save index data to other place? [scottchu]

Posted by Shawn Heisey <ap...@elyograg.org>.
On 5/26/2016 8:51 PM, scott.chu wrote:
> I want to migrate my Solrcloud from Windows to CentOS. Because I am
> new to CentOS, not familiar with how to install Solr on it and I did a
> lot of config in my Solrcloud on Windows, so I use ftp to upload
> solr-5.4.1 and zookeeper-3.4.6 folders to 3 different servers running
> CentOS. (They are all under /local). Then I tweak something in 3rd
> machine (See my other post titled "Can "Using cp replica and modifying
> core.properties" rather than ADDREPLICA API work?") and make my
> Solrcloud run with 3 replicas ok. I do wish to follow the default
> folder/file convention of solr. So can you show me (or hint me) how
> to: * Install solr and zookeeper with install shell script under
> CentOS 6.4? * How to auto start solr and zookeeper under CentOS 6.4? 

Installing Solr:

https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production#TakingSolrtoProduction-RuntheSolrInstallationScript

Zookeeper is a completely separate project from Solr.  For best results,
you should consult their documentation:

http://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html

I can give you some quick install steps based on what I have done in the
past.  I have no idea whether this is how the zookeeper project would
recommend doing it, but if it's done correctly, I know that it works.

Extract the download tarball into some location, like /opt.  If you
choose that location and extract the latest version, you will have
/opt/zookeeper-3.4.8 to work with.  The rest of this assumes that
directory location.  You may need to adjust these steps if you choose
another location:

Create a shell script named /usr/local/sbin/zkrun with these contents:

--------------------------
#!/bin/sh

# chkconfig: - 75 50
# description: Starts and stops Zookeeper

cd /opt/zookeeper
bin/zkServer.sh $1
--------------------------

Change to the /opt directory and do these commands:

ln -s zookeeper-3.4.8 zookeeper
chmod +x /usr/local/sbin/zkrun
ln -s /usr/local/sbin/zkrun /etc/init.d/zookeeper
chkconfig --add zookeeper
chkconfig zookeeper on

Consult zookeeper documentation on how to configure the newly installed
server -- which consists of creating /opt/zookeeper/conf/zoo.cfg and
${dataDir}/myid  A common setting for the dataDir config parameter is
"zoodata".  This is a path that is relative to the current working
directory.  If you use my little startup script with that value, the
full path becomes opt/zookeeper/zoodata.  Once it's configured, you can
start the zookeeper service.  This is a slightly redacted copy of my
zookeeper config file:

-------------------------------------
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=zoodata
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=zoo1.REDACTED.com:2888:3888
server.2=zoo2.REDACTED.com:2888:3888
server.3=zoo3.REDACTED.com:2888:3888
-------------------------------------

Thanks,
Shawn


Re: How to save index data to other place? [scottchu]

Posted by "scott.chu" <sc...@udngroup.com>.
I want to migrate my Solrcloud from Windows to CentOS. Because I am new to CentOS, not familiar with how to install Solr on it and I did a lot of config in my Solrcloud on Windows, so I use ftp to upload solr-5.4.1 and zookeeper-3.4.6 folders to 3 different servers running CentOS. (They are all under /local). Then I tweak something in 3rd machine (See my other post titled "Can "Using cp replica and modifying core.properties" rather than ADDREPLICA API work?") and make my Solrcloud run with 3 replicas ok.

I do wish to follow the default folder/file convention of solr. So can you show me (or hint me) how to:

* Install solr and zookeeper with install shell script under CentOS 6.4?
* How to auto start solr and zookeeper under CentOS 6.4?

Thanks in advance!

scott.chu,scott.chu@udngroup.com
2016/5/27 (週五)
----- Original Message ----- 
From: Shawn Heisey 
To: solr-user 
CC: 
Date: 2016/5/27 (週五) 02:34
Subject: Re: How to save index data to other place? [scottchu]


On 5/25/2016 10:16 PM, scott.chu wrote: 
> Thanks! I thought I have to tune solrconfig.xml. 
> 
> scott.chu,scott.chu@udngroup.com 
> 2016/5/26 (週四) 
> ----- Original Message ----- 
> From: Jay Potharaju 
> To: solr-user ; scott(自己) 
> CC: 
> Date: 2016/5/26 (週四) 11:31 
> Subject: Re: How to save index data to other place? [scottchu] 
> 
> 
> use property.*dataDir*=*value* 
> https://cwiki.apache.org/confluence/display/solr/Defining+core.properties 

In general, I only place *relative* paths in dataDir ... but for most 
people, I would actually recommend not setting dataDir at all, and 
letting it default to "data". 

Rather, what I would do is set the solr home to another location, so 
*all* cores live there by default, and let Solr create its default 
directory structure in that location. 

You haven't indicated in this thread whether you're running on a UNIX or 
UNIX-like OS such as Linux, or whether you're running on Windows. You 
used both slashes and backslashes when you described your actual and 
desired index paths. 

For most operating systems other than Windows, I strongly recommend 
using the solr installer shell script. This script has a concept of a 
"var dir", and the solr home is a directory inside that. 

If you are using the installer shell script to install Solr and do not 
provide any options, then Solr itself will install to /opt/solr-X.Y.Z, a 
symlink at /opt/solr will be created that points to the install 
directory, and Solr will set its var dir to /var/solr. A configuration 
script will be created at /etc/default/solr.in.sh. In the configuration 
script, the solr home gets set to /var/solr/data. Each core created 
would create a directory under /var/solr/data with the core's name, and 
inside that directory would be a data directory, containing the index 
directory and the tlog directory. These paths assume you've got 5.5 or 
later. 

On a dev server, I used the installer script with a service name (-s 
option) of solr5. With this option, most of the paths get changed from 
the default. The symlink is /opt/solr5 pointing to /opt/solr-5.5.1, the 
var dir is /var/solr5, the configuration script is 
/etc/default/solr5.in.sh, and the solr home is /var/solr5/data. 

Thanks, 
Shawn 



----- 
未在此訊息中找到病毒。 
已透過 AVG 檢查 - www.avg.com 
版本: 2015.0.6201 / 病毒庫: 4568/12302 - 發佈日期: 05/26/16

Re: How to save index data to other place? [scottchu]

Posted by Shawn Heisey <ap...@elyograg.org>.
On 5/25/2016 10:16 PM, scott.chu wrote:
> Thanks! I thought I have to tune solrconfig.xml.
>
> scott.chu\uff0cscott.chu@udngroup.com
> 2016/5/26 (\u9031\u56db)
> ----- Original Message ----- 
> From: Jay Potharaju 
> To: solr-user ; scott(\u81ea\u5df1) 
> CC: 
> Date: 2016/5/26 (\u9031\u56db) 11:31
> Subject: Re: How to save index data to other place? [scottchu]
>
>
> use property.*dataDir*=*value* 
> https://cwiki.apache.org/confluence/display/solr/Defining+core.properties 

In general, I only place *relative* paths in dataDir ... but for most
people, I would actually recommend not setting dataDir at all, and
letting it default to "data".

Rather, what I would do is set the solr home to another location, so
*all* cores live there by default, and let Solr create its default
directory structure in that location.

You haven't indicated in this thread whether you're running on a UNIX or
UNIX-like OS such as Linux, or whether you're running on Windows.  You
used both slashes and backslashes when you described your actual and
desired index paths.

For most operating systems other than Windows, I strongly recommend
using the solr installer shell script.  This script has a concept of a
"var dir", and the solr home is a directory inside that.

If you are using the installer shell script to install Solr and do not
provide any options, then Solr itself will install to /opt/solr-X.Y.Z, a
symlink at /opt/solr will be created that points to the install
directory, and Solr will set its var dir to /var/solr.  A configuration
script will be created at /etc/default/solr.in.sh.  In the configuration
script, the solr home gets set to /var/solr/data.  Each core created
would create a directory under /var/solr/data with the core's name, and
inside that directory would be a data directory, containing the index
directory and the tlog directory.  These paths assume you've got 5.5 or
later.

On a dev server, I used the installer script with a service name (-s
option) of solr5.  With this option, most of the paths get changed from
the default.  The symlink is /opt/solr5 pointing to /opt/solr-5.5.1, the
var dir is /var/solr5, the configuration script is
/etc/default/solr5.in.sh, and the solr home is /var/solr5/data.

Thanks,
Shawn


Re(2): How to save index data to other place? [scottchu]

Posted by "scott.chu" <sc...@udngroup.com>.
Thanks! I thought I have to tune solrconfig.xml.

scott.chu,scott.chu@udngroup.com
2016/5/26 (週四)
----- Original Message ----- 
From: Jay Potharaju 
To: solr-user ; scott(自己) 
CC: 
Date: 2016/5/26 (週四) 11:31
Subject: Re: How to save index data to other place? [scottchu]


use property.*dataDir*=*value* 
https://cwiki.apache.org/confluence/display/solr/Defining+core.properties 

On Wed, May 25, 2016 at 8:20 PM, scott.chu <sc...@udngroup.com> wrote: 


> 
> When I create a collection, say named 'cugna'. Solr create a folder with 

> same name under server\slolr, e.g. /local/solr-5.4.1/server/solr/cugna. 
> Index data is also saved there. But wish to save index data on other 
> folder, say /var/sc_data/cugna. How can I dothis? 
> 
> scott.chu,scott.chu@udngroup.com 
> 2016/5/26 (週四) 
> 



-- 
Thanks 
Jay Potharaju 



----- 
未在此訊息中找到病毒。 
已透過 AVG 檢查 - www.avg.com 
版本: 2015.0.6201 / 病毒庫: 4568/12296 - 發佈日期: 05/25/16