Copy existing index from standalone Solr to Solr cloud

[avgxm <gi...@hotmail.com>]

Is there a correct way to take an existing Solr index (core.properties,
conf/, data/ directories from a standalone Solr instance) and copy it over
to a Solr cloud, with shards, without having to use import or re-indexing? 
Does anyone know the proper steps to accomplish this type of a move?  The
target system is zookeeper 3.4.6, tomcat 7.0.54, and solr 4.8.1.  I have
been able to copy the data and load the core by executing upconfig,
linkconfig to zookeeper, and then copying over the core.properties, and
conf/ and data/ directories, bouncing tomcat.  The core comes up and is
searchable.  The cloud pic looks like corename ---- shard1 ----
ip_addr:8080.  Then, I have tried to use split core, split shard, create
core, without success to try and add shard2 and shard3, either on the same
or different hosts.  Not sure what I'm missing or if this way of reusing the
existing data is even an option.



--
View this message in context: http://lucene.472066.n3.nabble.com/Copy-existing-index-from-standalone-Solr-to-Solr-cloud-tp4149920.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Copy existing index from standalone Solr to Solr cloud

[avgxm <gi...@hotmail.com>]

Used the admin/collections?action=SPLITSHARD, to create shard1_0, shard1_1,
and then followed this thread
http://lucene.472066.n3.nabble.com/How-can-you-move-a-shard-from-one-SolrCloud-node-to-another-td4106815.html
to move the shards to the right nodes.  Problem solved.



--
View this message in context: http://lucene.472066.n3.nabble.com/Copy-existing-index-from-standalone-Solr-to-Solr-cloud-tp4149920p4150163.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Copy existing index from standalone Solr to Solr cloud

[Anshum Gupta <an...@anshumgupta.net>]

Use the Split shard API to split an existing shard. It splits an
existing shard into 2.
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api3

You can then split the sub-shards. One thing to note though is that
the Admin UI still doesn't comprehend the difference between active
and inactive shards. You can look at the cluster state to get a better
picture of what shards are active.

Another thing to note would be to use the async mode while splitting a
rather big shard. That would help you overcome the internal timeout
issues of a long running task.

On Tue, Jul 29, 2014 at 2:10 PM, avgxm <gi...@hotmail.com> wrote:
> Thanks a lot, Shawn.  I have gotten as far as having the core come up per
> your instructions.  Since numShards was set to 1, what is the next step to
> add more shards?  Is it /admin/collections?action=CREATESHARD... or
> something else?  Ultimately, I'd like to have shard1, shard2, shard3, with
> "router":{"name":"compositeId"}, where each core is "leader":"true".
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Copy-existing-index-from-standalone-Solr-to-Solr-cloud-tp4149920p4149934.html
> Sent from the Solr - User mailing list archive at Nabble.com.



-- 

Anshum Gupta
http://www.anshumgupta.net

Re: Copy existing index from standalone Solr to Solr cloud

[avgxm <gi...@hotmail.com>]

Thanks a lot, Shawn.  I have gotten as far as having the core come up per
your instructions.  Since numShards was set to 1, what is the next step to
add more shards?  Is it /admin/collections?action=CREATESHARD... or
something else?  Ultimately, I'd like to have shard1, shard2, shard3, with
"router":{"name":"compositeId"}, where each core is "leader":"true".



--
View this message in context: http://lucene.472066.n3.nabble.com/Copy-existing-index-from-standalone-Solr-to-Solr-cloud-tp4149920p4149934.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Copy existing index from standalone Solr to Solr cloud

[Shawn Heisey <so...@elyograg.org>]

On 7/29/2014 2:23 PM, avgxm wrote:
> Is there a correct way to take an existing Solr index (core.properties,
> conf/, data/ directories from a standalone Solr instance) and copy it over
> to a Solr cloud, with shards, without having to use import or re-indexing? 
> Does anyone know the proper steps to accomplish this type of a move?  The
> target system is zookeeper 3.4.6, tomcat 7.0.54, and solr 4.8.1.  I have
> been able to copy the data and load the core by executing upconfig,
> linkconfig to zookeeper, and then copying over the core.properties, and
> conf/ and data/ directories, bouncing tomcat.  The core comes up and is
> searchable.  The cloud pic looks like corename ---- shard1 ----
> ip_addr:8080.  Then, I have tried to use split core, split shard, create
> core, without success to try and add shard2 and shard3, either on the same
> or different hosts.  Not sure what I'm missing or if this way of reusing the
> existing data is even an option.

You'll need to create a collection with the Collections API (which also
creates the cores) before you try copying anything, and then you'll want
to copy *only* the "data" directory -- the config is in zookeeper and
the core.properties file should already exist.

When you create the collection, you'll likely want numShards on the
CREATE call to be 1, and replicationFactor should be whatever you want
-- if it's 2, you'll end up with two copies of your index on different
servers.  If the collection is named "test" then the core on the first
server will be named test_shard1_replica1, the core on the second server
will be named test_shard1_replica2, and so on.  Your zookeeper ensemble
should be separate from Solr, don't use the -DzkRun option.

To put the existing data into the collection:  1) Shut down all the Solr
servers. 2) Delete the data and tlog directories from
XXXX_shard1_replicaN on all the servers.  3) Copy the data directory
from the source to the first server, then start Solr on that server.  4)
Wait a few minutes for everything to stabilize.  5) Start Solr on any
other servers.

Thanks,
Shawn