[GitHub] [couchdb] nkosi23 opened a new issue, #4043: Provide a way to export a database to a file and to any S3 compatible storage

[GitBox <gi...@apache.org>]

nkosi23 opened a new issue, #4043:
URL: https://github.com/apache/couchdb/issues/4043

   [NOTE]: # ( ^^ Provide a general summary of the request in the title above. ^^ )
   
   ## Summary
   
   [NOTE]: # ( Provide a brief overview of what the new feature is all about. )
   
   The recently open-sourced yandex-db database [has a feature](https://cloud.yandex.com/en-ru/docs/ydb/reference/ydb-cli/export_import/file_structure) enabling users to export a database to both the file system and any S3-compatible storage. The import story is similarly easy. 
   
   It would be great to have the same feature in couchdb, specifically to help with the replication of very large databases (hundreds of gigabytes, dozens of terabytes or even petabytes) to a different data center. Transmitting everything over the wire is impractical in these situations. In the 1.X era, one simply needed to take a snapshot of the .couch file, but with clustering there is no clear solution to this problem.
   
   ## Desired Behaviour
   
   [NOTE]: # ( Tell us how the new feature should work. Be specific. )
   [TIP]:  # ( Do NOT give us access or passwords to your actual CouchDB! )
   
   Users should be able to both export and import and a database in one click from Fauxton or using curl. The feature only needs to support the export of a whole database (no need to support partial exports), and the import of the whole database into a new database. Supporting imports to an existing database would unnecessarily introduce the problem of conflict handling. For simplicity, users can simply import to a new database and realize filtered replications as needed to a local database. 
   
   
   ## Additional context
   
   [TIP]:  # ( Why does this feature matter to you? What unique circumstances do you have? )
   Sometimes it is needed to move or replicate very large databases to a different data center not necessarily having a 10gbps internet link. This feature would help facilitate the transfer of the bulk of data, while newest data will be able to be transferred using the replication protocol.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org