You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Joan Touzet <wo...@apache.org> on 2017/04/19 22:29:02 UTC
couchup, a 1.x->2.x migration tool, needs your testing!
Hi everyone!
I'm back with a new tool that I think you'll love, couchup.
Your feedback is really needed this time- please help me test couchup
before we add it to CouchDB!
-Joan
Overview
========
couchup makes it easy to migrate your CouchDB 1.x databases to CouchDB
2.x by providing useful 4 sub-commands:
list - lists all CouchDB 1.x databases
replicate - replicates one or more 1.x databases to CouchDB 2.x
rebuild - rebuilds one or more CouchDB 2.x views
delete - deletes one or more CouchDB 1.x databases
The idea is that you'll install CouchDB 2.x, copy the .couch files from
your 1.x installation (or, if you've upgraded in-place, do nothing),
then something like the following:
$ couchup list # Shows your unmigrated 1.x databases
$ couchup replicate -a # Replicates your 1.x DBs to 2.x
$ couchup rebuild -a # Optional; starts rebuilding your views
$ couchup delete -a # Deletes your 1.x DBs (careful!)
$ couchup list # Should show no remaining databases!
The same process works for moving from a single 1.x node to a cluster of
2.x nodes; the only difference is that you must complete cluster setup
prior to running the couchup commands.
Special Features
================
* Lots of extra help is available via:
$ couchup -h
$ couchup <sub-command> -h
* Various optional arguments provide for admin login/password,
overriding ports, quiet mode and so on.
* couchup delete will NOT delete your 1.x DBs unless the contents are
identical to the replicated 2.x DBs, or you override with the
-f/--force command (be VERY careful with this!!)
* couchup replicate supports an optional flag, -f/--filter-deleted, to
filter delete documents during the replication process. This can
improve the performance and disk-size of your database if it has a lot
of deleted documents. It is IMPORTANT that no documents be deleted
from the 1.x database during this process, or those deletions may not
successfully replicate to the 2.x database. (It's recommended that
you not access or modify the 1.x database at all during the whole
couchup process.)
Getting & Installing couchup
============================
First, backup everything. :)
couchup is currently in a Pull Request to be merged with couchdb itself.
You can download a copy from the PR itself:
https://github.com/apache/couchdb/pull/483
Click on the "Files changed" tab, then the "View" button, and finally
the "Raw" button. Save to disk. (I'm not directly linking because the
link may have changed by the time you read this.)
Place couchup in the /path/to/couchdb/bin directory, or wherever you
like; it's not picky. Be sure to set the file as executable:
chmod +x couchup
couchup requires Python 2.7 or later (and has been tested on Python 3.6)
and the Python requests library. It can optionally make use of the
Python progressbar library as well. These can be installed with:
pip install requests progressbar
For Python 3.x you need the progressbar2 library instead:
pip install requests progressbar2
Or, install these via binary packages in your friendly Linux/OSX/etc
distributions, e.g.:
apt-get install python-requests python-progressbar
I have explicitly included support for positively ancient versions of
python-requests (such as those provided by Ubuntu 12.04 and CentOS 6)
by special request.
Testing couchup
===============
1. Copy a bunch of CouchDB 1.x .couch files into the CouchDB 2.x data/
directory.
2. Start CouchDB 2.x.
3. Use the 4 couchup commands to list, replicate, rebuild and delete
the databases as desired.
4. Test your application against the migrated data.
5. Report any bugs on this mailing list, or on the PR directly.
Cat
===
Thanks for reading this far! Here's an ASCII cat for you.
|\ _,,,---,,_
/,`.-'`' -. ;-;;,_
|,4- ) )-,_..;\ ( `'_'
'---''(_/--' `-'\_) Felix Lee
Fwd: couchup, a 1.x->2.x migration tool, needs your testing!
Posted by Jan Lehnardt <ja...@apache.org>.
CC dev@ & marketing@ this is really cool! :)
> Begin forwarded message:
>
> From: Joan Touzet <wo...@apache.org>
> Subject: couchup, a 1.x->2.x migration tool, needs your testing!
> Date: 20 April 2017 at 00:29:02 GMT+2
> To: user@couchdb.apache.org
> Reply-To: user@couchdb.apache.org
> Reply-To: Joan Touzet <wo...@apache.org>
>
> Hi everyone!
>
> I'm back with a new tool that I think you'll love, couchup.
>
> Your feedback is really needed this time- please help me test couchup
> before we add it to CouchDB!
>
> -Joan
>
>
> Overview
> ========
> couchup makes it easy to migrate your CouchDB 1.x databases to CouchDB
> 2.x by providing useful 4 sub-commands:
>
> list - lists all CouchDB 1.x databases
> replicate - replicates one or more 1.x databases to CouchDB 2.x
> rebuild - rebuilds one or more CouchDB 2.x views
> delete - deletes one or more CouchDB 1.x databases
>
> The idea is that you'll install CouchDB 2.x, copy the .couch files from
> your 1.x installation (or, if you've upgraded in-place, do nothing),
> then something like the following:
>
> $ couchup list # Shows your unmigrated 1.x databases
> $ couchup replicate -a # Replicates your 1.x DBs to 2.x
> $ couchup rebuild -a # Optional; starts rebuilding your views
> $ couchup delete -a # Deletes your 1.x DBs (careful!)
> $ couchup list # Should show no remaining databases!
>
> The same process works for moving from a single 1.x node to a cluster of
> 2.x nodes; the only difference is that you must complete cluster setup
> prior to running the couchup commands.
>
>
> Special Features
> ================
> * Lots of extra help is available via:
>
> $ couchup -h
> $ couchup <sub-command> -h
>
> * Various optional arguments provide for admin login/password,
> overriding ports, quiet mode and so on.
>
> * couchup delete will NOT delete your 1.x DBs unless the contents are
> identical to the replicated 2.x DBs, or you override with the
> -f/--force command (be VERY careful with this!!)
>
> * couchup replicate supports an optional flag, -f/--filter-deleted, to
> filter delete documents during the replication process. This can
> improve the performance and disk-size of your database if it has a lot
> of deleted documents. It is IMPORTANT that no documents be deleted
> from the 1.x database during this process, or those deletions may not
> successfully replicate to the 2.x database. (It's recommended that
> you not access or modify the 1.x database at all during the whole
> couchup process.)
>
>
> Getting & Installing couchup
> ============================
> First, backup everything. :)
>
> couchup is currently in a Pull Request to be merged with couchdb itself.
> You can download a copy from the PR itself:
>
> https://github.com/apache/couchdb/pull/483
>
> Click on the "Files changed" tab, then the "View" button, and finally
> the "Raw" button. Save to disk. (I'm not directly linking because the
> link may have changed by the time you read this.)
>
> Place couchup in the /path/to/couchdb/bin directory, or wherever you
> like; it's not picky. Be sure to set the file as executable:
>
> chmod +x couchup
>
> couchup requires Python 2.7 or later (and has been tested on Python 3.6)
> and the Python requests library. It can optionally make use of the
> Python progressbar library as well. These can be installed with:
>
> pip install requests progressbar
>
> For Python 3.x you need the progressbar2 library instead:
>
> pip install requests progressbar2
>
> Or, install these via binary packages in your friendly Linux/OSX/etc
> distributions, e.g.:
>
> apt-get install python-requests python-progressbar
>
> I have explicitly included support for positively ancient versions of
> python-requests (such as those provided by Ubuntu 12.04 and CentOS 6)
> by special request.
>
>
> Testing couchup
> ===============
> 1. Copy a bunch of CouchDB 1.x .couch files into the CouchDB 2.x data/
> directory.
> 2. Start CouchDB 2.x.
> 3. Use the 4 couchup commands to list, replicate, rebuild and delete
> the databases as desired.
> 4. Test your application against the migrated data.
> 5. Report any bugs on this mailing list, or on the PR directly.
>
>
> Cat
> ===
> Thanks for reading this far! Here's an ASCII cat for you.
>
> |\ _,,,---,,_
> /,`.-'`' -. ;-;;,_
> |,4- ) )-,_..;\ ( `'_'
> '---''(_/--' `-'\_) Felix Lee
--
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/
Fwd: couchup, a 1.x->2.x migration tool, needs your testing!
Posted by Jan Lehnardt <ja...@apache.org>.
CC dev@ & marketing@ this is really cool! :)
> Begin forwarded message:
>
> From: Joan Touzet <wo...@apache.org>
> Subject: couchup, a 1.x->2.x migration tool, needs your testing!
> Date: 20 April 2017 at 00:29:02 GMT+2
> To: user@couchdb.apache.org
> Reply-To: user@couchdb.apache.org
> Reply-To: Joan Touzet <wo...@apache.org>
>
> Hi everyone!
>
> I'm back with a new tool that I think you'll love, couchup.
>
> Your feedback is really needed this time- please help me test couchup
> before we add it to CouchDB!
>
> -Joan
>
>
> Overview
> ========
> couchup makes it easy to migrate your CouchDB 1.x databases to CouchDB
> 2.x by providing useful 4 sub-commands:
>
> list - lists all CouchDB 1.x databases
> replicate - replicates one or more 1.x databases to CouchDB 2.x
> rebuild - rebuilds one or more CouchDB 2.x views
> delete - deletes one or more CouchDB 1.x databases
>
> The idea is that you'll install CouchDB 2.x, copy the .couch files from
> your 1.x installation (or, if you've upgraded in-place, do nothing),
> then something like the following:
>
> $ couchup list # Shows your unmigrated 1.x databases
> $ couchup replicate -a # Replicates your 1.x DBs to 2.x
> $ couchup rebuild -a # Optional; starts rebuilding your views
> $ couchup delete -a # Deletes your 1.x DBs (careful!)
> $ couchup list # Should show no remaining databases!
>
> The same process works for moving from a single 1.x node to a cluster of
> 2.x nodes; the only difference is that you must complete cluster setup
> prior to running the couchup commands.
>
>
> Special Features
> ================
> * Lots of extra help is available via:
>
> $ couchup -h
> $ couchup <sub-command> -h
>
> * Various optional arguments provide for admin login/password,
> overriding ports, quiet mode and so on.
>
> * couchup delete will NOT delete your 1.x DBs unless the contents are
> identical to the replicated 2.x DBs, or you override with the
> -f/--force command (be VERY careful with this!!)
>
> * couchup replicate supports an optional flag, -f/--filter-deleted, to
> filter delete documents during the replication process. This can
> improve the performance and disk-size of your database if it has a lot
> of deleted documents. It is IMPORTANT that no documents be deleted
> from the 1.x database during this process, or those deletions may not
> successfully replicate to the 2.x database. (It's recommended that
> you not access or modify the 1.x database at all during the whole
> couchup process.)
>
>
> Getting & Installing couchup
> ============================
> First, backup everything. :)
>
> couchup is currently in a Pull Request to be merged with couchdb itself.
> You can download a copy from the PR itself:
>
> https://github.com/apache/couchdb/pull/483
>
> Click on the "Files changed" tab, then the "View" button, and finally
> the "Raw" button. Save to disk. (I'm not directly linking because the
> link may have changed by the time you read this.)
>
> Place couchup in the /path/to/couchdb/bin directory, or wherever you
> like; it's not picky. Be sure to set the file as executable:
>
> chmod +x couchup
>
> couchup requires Python 2.7 or later (and has been tested on Python 3.6)
> and the Python requests library. It can optionally make use of the
> Python progressbar library as well. These can be installed with:
>
> pip install requests progressbar
>
> For Python 3.x you need the progressbar2 library instead:
>
> pip install requests progressbar2
>
> Or, install these via binary packages in your friendly Linux/OSX/etc
> distributions, e.g.:
>
> apt-get install python-requests python-progressbar
>
> I have explicitly included support for positively ancient versions of
> python-requests (such as those provided by Ubuntu 12.04 and CentOS 6)
> by special request.
>
>
> Testing couchup
> ===============
> 1. Copy a bunch of CouchDB 1.x .couch files into the CouchDB 2.x data/
> directory.
> 2. Start CouchDB 2.x.
> 3. Use the 4 couchup commands to list, replicate, rebuild and delete
> the databases as desired.
> 4. Test your application against the migrated data.
> 5. Report any bugs on this mailing list, or on the PR directly.
>
>
> Cat
> ===
> Thanks for reading this far! Here's an ASCII cat for you.
>
> |\ _,,,---,,_
> /,`.-'`' -. ;-;;,_
> |,4- ) )-,_..;\ ( `'_'
> '---''(_/--' `-'\_) Felix Lee
--
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/