You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by jiangph <ji...@hotmail.com> on 2020/03/18 12:04:57 UTC
[DISCUSS] soft-deletion
Hi Everyone,
Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
## Proposed Data model change
In CouchDB 4.0, directories and indirection access in FoundationDB are already used to better build data model. One key/value pair is used to build reference from Dbkey to DbPrefix. All other key/value pairs are based on DbPrefix instead of DbKey. This decouples the direct relationship between DBName and data in this database. The current implementation for `DBKey -> DBPrefix` is in [2]. So you can see below information in FoundationDB using fdbcli, etc.
```
{?ALL_DBS, DbName} -> {?DBS, DbName}
{?DBS, DbName, other part of key} -> <value>
```
To support soft-deletion, especially allowing one database to be deleted/re-created multiple time, we need to use different DbPrefix for the same DbKey/DBName. The proposed change is to use a unique value allocated via High Contention Allocator(HCA) algorithm in [3].
```
DbPrefixAllocator = erlfdb_hca:create(?ERLFDB_EXTEND(DbId, <<"hca">>)),,
DbPrefix = erlfdb_hca:allocate(DBPrefixAllocator, Tx),
erlfdb:set(Tx, DbKey, DbPrefix),
```
The data in FoundationDB looks like:
```
{?ALL_DBS, DbName} -> <unique key allocated by hca>
{<unique key allocated by hca>, other part of key} -> <value>
```
Using HCA algorithm, it can acquire one unique key quickly while avoiding conflicting. The more important, it is shorter enough to save space because `DBPrefix` exists in almost every key/value pair for database.
## Soft-deletion, restore and permanent-deletion
Once database is soft-deleted, the only action is to change `DBKey -> DBPrefix` pair. All other data for this data is not changed. In order to give clear namespace management, the proposal is to move DBkey from `?ALL_DBS` to `?DELETED_DBS`. The timestamp when database was deleted is added to `DBKey` so that we can know when the data in this database can be permantenly restored. The `DBKey -> DBPrefix` pair is changed to
```
{?DELETED_DBS, DbName, TimeStamp} -> <unique key allocated by hca>
```
There is a background task to clear the ranges eventually. Depending on setting on how long the soft-deleted database will be kept, such as 48 hours, the background task will check `DELETED_DBS` namespace, and find eligible key/value pairs, and delete data associated with this `DBPrefix` and then delete DbKey/DbPrefix pair finally.
Overtime, it is possible that database can be deleted several times. The `_deleted_dbs_info` endpoint is proposed to list information about all deleted instances for the specified database, including deletion timestamp, document counts and disk size, etc. This allows users to identify which one to be restored. Also it also provides information for billing. In given period, such as 48 hours, the deletion times of same database is most likely limited, the design API is to list all instances in one time using GET method with query parameter.
After deciding which instance to be restored, users can use the `_restore` endpoint with `deletedTS` to restore database. The underlying logic is to change `DBKey -> DBPrefix` back to
```
{?ALL_DBS, DbName} -> <unique key allocated by hca>
```
Considering the sensitive actions, the `_deleted_dbs_info` and `_restore` endpoints are supposed to be an admin-only endpoints only to allow granted user to restore the database.
##view index and search index
Although view index and search index is based on the `DBPrefix`, the change of value of `DBPrefix` doesn't have impact on storage and search of view index and search index because `DBPrefix` is one opaque value. If database is soft-deleted, the `DBkey -> DBPrefix` pair is changed so that any access to view index and search index will be blocked with `"Database does not exist."` error. This is expected.
The only thing we need to care is to stop all indexing or pending requests for soft-deleted database.
## API
1) `DELETE /{db}`
There is no change on this endpoint [4] to send DELETE against one database. The soft-deletion is triggered once [couchdb][enable_database_recovery] is set to true in configuration file.
2) `GET /{db}/_deleted_dbs_info`
returning basic information of all deleted instances for the specified database, including when the instance was deleted.
Parameters:
db –Database name
Request Headers:
Content-Type –application/json
Response Headers:
Content-Type –
application/json
Status Codes:
200 OK –Request completed successfully
404 Not Found –Requested database not found
Request:
GET /db/_deleted_dbs_info HTTP/1.1
Accept: application/json
Host: localhost:5984
Response:
HTTP/1.1 200 OK
Cache-Control: must-revalidate
Content-Type: application/json
{
"total_rows": 2,
"rows": [{
"deleted_when": "20200318.071532",
"info": {
"update_seq": "0000019100b5992700000000",
"doc_del_count": 0,
"doc_count": 3,
"sizes": {
"external": 287,
"views": 0
}
}
}, {
"deleted_when": "20200318.071703",
"info": {
"update_seq": "0000019105f0e29900000000",
"doc_del_count": 0,
"doc_count": 2,
"sizes": {
"external": 200,
"views": 0
}
}
}]
}
3) `PUT /{db}/_restore/{deletedTS}`
Restore a deleted database.
Parameters:
db –Database name
deletedTS - timestamp when database was deleted
Request Headers:
Accept –
application/json
text/plain
Response Headers:
Content-Type –
application/json
text/plain; charset=utf-8
Response JSON Object:
ok (boolean) –Operation status. Available in case of success
error (string) –Error type. Available if response code is 4xx
reason (string) –Error description. Available if response code is 4xx
Status Codes:
200 Restored –Database restored successfully
400 Bad Request –Invalid database name or deleted timestamp
401 Unauthorized –CouchDB Server Administrator privileges required
412 Precondition Failed –Database already exists
What do you think of that? Any questions or thoughts on this? Once again a big acknowledgment to Nick and Paul who helped with initial design and provide consultation on this.
Cheers
Peng Hui
[1] https://github.com/apache/couchdb/blob/master/src/couch/src/couch_file.erl#L251
[2] https://github.com/apache/couchdb/blob/prototype/fdb-layer/src/fabric/src/fabric2_fdb.erl#L182-L184
[3] https://activesphere.com/blog/2018/08/05/high-contention-allocator <https://activesphere.com/blog/2018/08/05/high-contention-allocator>
[4] https://docs.couchdb.org/en/stable/api/database/common.html#delete--db
Re: [DISCUSS] soft-deletion
Posted by jiangph <ji...@hotmail.com>.
Thanks so much to Rich for comments/questions related to API design. They are quite helpful. Please see my embedded response below.
> On Mar 20, 2020, at 1:23 AM, Richard Ellis <RI...@uk.ibm.com> wrote:
>
> This is a nice proposal, but I have a few comments/questions regarding the
> proposed API:
>
> ## API
>
> 1) `DELETE /{db}`
>
> I wonder if there should be some consideration of the status code here.
> Currently for a db delete a 200/202 is returned depending if quorum is
> met. Do FDB transactional semantics mean that is likely to change at all?
> I can't really decide if a 202 always makes more sense for a soft-deletion
> (in the sense that the server has accepted the request to delete the
> database, but won't really do it for some time). On the other hand as soon
> as the database is in a soft-deleted state the {db} paths will return 404
> because the database is not operable at that path, so perhaps there isn't
> really any change.
With the soft-deletion, the semantic for `DELETE /{db}` in CouchDB 4.0 context is not changed. Especially, it will not be long-run action because just a few key/values are changed.
>
> 2) `GET /{db}/_deleted_dbs_info`
>
> It isn't shown in the example, but I'm assuming as this is a view-like
> response that the database name is the `key` for the row similarly to
> _dbs_info endpoint. Is that correct?
Different from `_dbs_info`, the `GET /{db}/_deleted_dbs_info` only returns the information for specified database. So I didn’t add database name in returned response before. If it is helpful and keeps consistent, I can add “key” in every deleted instances.
>
> Would there be any advantage to using a standard timestamp format
> (specifically ISO 8601, which I think should be used for all new internet
> date/time representations). Compared to the examples shown this would
> probably be more user friendly for people to identify the actual time
> associated with instances and would have improved interoperability with
> other systems interacting with the API that might want to display the
> deleted date/time. Having said that I think the additional complexity of
> ISO8601 would be less useful as the identifier for restore operations,
> which would make me lean towards the UUID proposed by Nick for that part.
`20200318.071532` is based on original internal name for couch file. Yes, for external usage, we can change it with ISO 8601 format. I am not sure to replace timestamp with UUID because timestamp is meaningful and our operation team ever used it to identify which database to restore.
Based on existing discussion, the modified response looks like:
{
"total_rows": 2,
"rows": [{
"key": "db01",
"timestamp": "2020-03-20T12:18:07Z",
"info": {
"update_seq":
"0000019100b5992700000000",
"doc_del_count": 0,
"doc_count": 3,
"sizes": {
"external": 287,
"views":
0
}
}
}, {
"key": "db01",
"timestamp": “2020-03-20T13:15:30Z",
"info": {
"update_seq":
"0000019105f0e29900000000",
"doc_del_count": 0,
"doc_count": 2,
"sizes": {
"external": 200,
"views":
0
}
}
}]
}
>
> 3) `PUT /{db}/_restore/{deletedTS}`
>
> It seems that this is basically identical in structure to the create
> operation. Are we expecting down the line a more complex set of options,
> different response structure, or different permission to a normal db
> create that warrants having a new endpoint? Given there is precedent for
> using a `partitioned=true` parameter to create a database in a "special"
> way, is there a good reason not to follow the same pattern here?
> e.g. PUT /{db}?restore=timestamp/uuid
>
> That would maintain the symmetry of the PUT/DELETE operations on the /{db}
> path. What I mean by this is that other `PUT /{db}/foo/bar` path
> operations always create sub-resources of a database (i.e. docs,
> design_docs, attachments etc) and 404 in the cases that the database does
> not exist. It seems strange to me to use a path structure similar to those
> that has different semantics because it doesn't 404 when the {db} part
> doesn't exist. Consider an equivalent could be the expectation that the
> `PUT /{db}/{doc_id}` operation would create the database if it doesn't
> exist instead of returning 404. Additionally usually the PUT verb is used
> for creating resources at the path of the request, but that is not the
> outcome of this call.
Good call. I ever have another alternative with `PUT /{db}/_restore?deleteTS={deletedTS}`. But your suggestion with `PUT /{db}?restore=timestamp/uuid` looks to be better. I don’t see other complex set of options.
>
>
> I also wonder a little bit about the use of "restore" here. I guess there
> maybe should be a little consideration of whether there might be some
> future backup/restore functionality and whether it be able to restore via
> the same method or not. If not then calling this "restore" might cause
> confusion with any future backup/restore API, say, compared to calling
> this something like "undelete”.
Okay, to reduce confusion with backup/restore, we can select other words. How about using “recover” instead of “undelete”? If so, endpoint will change to `PUT /{db}?recover=timestamp/uuid`
>
> Thanks,
> Rich
>
>
>
> From: jiangph <ji...@hotmail.com>
> To: dev@couchdb.apache.org
> Date: 18/03/2020 12:05
> Subject: [EXTERNAL] [DISCUSS] soft-deletion
>
>
>
> Hi Everyone,
>
> Instead of automatically and immediately removing data and index in
> database after a delete operation, soft-deletion allows to restore the
> deleted data back to original state due to a “fat finger”or undesired
> delete operation, up to defined periods, such as 48 hours.
>
> In CouchDB 3.0, soft-deletion of database is implemented in [1]. The
> .couch file is renamed with the .<timestamp>.deleted.couch file after
> soft-deletion is enabled, and such file can be changed back to .couch for
> the purpose of restore. If restore is not needed and some specified period
> passed, the .<timestamp>.deleted.couch file can be deleted to achieve
> deletion of database permanently.
>
> In CouchDB 4.0, with the introduction of FoundationDB, the data model and
> storage is changed. In order to support soft-deletion, we propose below
> solution and then implement them.
>
> ## Proposed Data model change
>
> In CouchDB 4.0, directories and indirection access in FoundationDB are
> already used to better build data model. One key/value pair is used to
> build reference from Dbkey to DbPrefix. All other key/value pairs are
> based on DbPrefix instead of DbKey. This decouples the direct relationship
> between DBName and data in this database. The current implementation for
> `DBKey -> DBPrefix` is in [2]. So you can see below information in
> FoundationDB using fdbcli, etc.
>
> ```
> {?ALL_DBS, DbName} -> {?DBS, DbName}
> {?DBS, DbName, other part of key} -> <value>
> ```
>
> To support soft-deletion, especially allowing one database to be
> deleted/re-created multiple time, we need to use different DbPrefix for
> the same DbKey/DBName. The proposed change is to use a unique value
> allocated via High Contention Allocator(HCA) algorithm in [3].
>
>
> ```
> DbPrefixAllocator = erlfdb_hca:create(?ERLFDB_EXTEND(DbId,
> <<"hca">>)),,
> DbPrefix = erlfdb_hca:allocate(DBPrefixAllocator, Tx),
> erlfdb:set(Tx, DbKey, DbPrefix),
> ```
>
> The data in FoundationDB looks like:
>
> ```
> {?ALL_DBS, DbName} -> <unique key allocated by hca>
> {<unique key allocated by hca>, other part of key} -> <value>
> ```
>
> Using HCA algorithm, it can acquire one unique key quickly while avoiding
> conflicting. The more important, it is shorter enough to save space
> because `DBPrefix` exists in almost every key/value pair for database.
>
> ## Soft-deletion, restore and permanent-deletion
>
> Once database is soft-deleted, the only action is to change `DBKey ->
> DBPrefix` pair. All other data for this data is not changed. In order to
> give clear namespace management, the proposal is to move DBkey from
> `?ALL_DBS` to `?DELETED_DBS`. The timestamp when database was deleted is
> added to `DBKey` so that we can know when the data in this database can be
> permantenly restored. The `DBKey -> DBPrefix` pair is changed to
>
> ```
> {?DELETED_DBS, DbName, TimeStamp} -> <unique key allocated by hca>
> ```
>
> There is a background task to clear the ranges eventually. Depending on
> setting on how long the soft-deleted database will be kept, such as 48
> hours, the background task will check `DELETED_DBS` namespace, and find
> eligible key/value pairs, and delete data associated with this `DBPrefix`
> and then delete DbKey/DbPrefix pair finally.
>
> Overtime, it is possible that database can be deleted several times. The
> `_deleted_dbs_info` endpoint is proposed to list information about all
> deleted instances for the specified database, including deletion
> timestamp, document counts and disk size, etc. This allows users to
> identify which one to be restored. Also it also provides information for
> billing. In given period, such as 48 hours, the deletion times of same
> database is most likely limited, the design API is to list all instances
> in one time using GET method with query parameter.
>
> After deciding which instance to be restored, users can use the `_restore`
> endpoint with `deletedTS` to restore database. The underlying logic is to
> change `DBKey -> DBPrefix` back to
>
> ```
> {?ALL_DBS, DbName} -> <unique key allocated by hca>
> ```
>
> Considering the sensitive actions, the `_deleted_dbs_info` and `_restore`
> endpoints are supposed to be an admin-only endpoints only to allow granted
> user to restore the database.
>
> ##view index and search index
>
> Although view index and search index is based on the `DBPrefix`, the
> change of value of `DBPrefix` doesn't have impact on storage and search of
> view index and search index because `DBPrefix` is one opaque value. If
> database is soft-deleted, the `DBkey -> DBPrefix` pair is changed so that
> any access to view index and search index will be blocked with `"Database
> does not exist."` error. This is expected.
>
> The only thing we need to care is to stop all indexing or pending requests
> for soft-deleted database.
>
>
> ## API
>
> 1) `DELETE /{db}`
>
> There is no change on this endpoint [4] to send DELETE against one
> database. The soft-deletion is triggered once
> [couchdb][enable_database_recovery] is set to true in configuration file.
>
>
> 2) `GET /{db}/_deleted_dbs_info`
>
> returning basic information of all deleted instances for the specified
> database, including when the instance was deleted.
> Parameters:
>
> db –Database name
>
> Request Headers:
>
> Content-Type –application/json
>
> Response Headers:
>
>
> Content-Type –
> application/json
>
> Status Codes:
>
> 200 OK –Request completed successfully
> 404 Not Found –Requested database not found
>
> Request:
>
> GET /db/_deleted_dbs_info HTTP/1.1
> Accept: application/json
> Host: localhost:5984
>
> Response:
>
> HTTP/1.1 200 OK
> Cache-Control: must-revalidate
> Content-Type: application/json
> {
> "total_rows": 2,
> "rows": [{
> "deleted_when": "20200318.071532",
> "info": {
> "update_seq":
> "0000019100b5992700000000",
> "doc_del_count": 0,
> "doc_count": 3,
> "sizes": {
> "external": 287,
> "views":
> 0
> }
> }
> }, {
> "deleted_when": "20200318.071703",
> "info": {
> "update_seq":
> "0000019105f0e29900000000",
> "doc_del_count": 0,
> "doc_count": 2,
> "sizes": {
> "external": 200,
> "views":
> 0
> }
> }
> }]
> }
>
>
> 3) `PUT /{db}/_restore/{deletedTS}`
>
> Restore a deleted database.
> Parameters:
>
> db –Database name
> deletedTS - timestamp when database was deleted
>
> Request Headers:
>
>
> Accept –
> application/json
> text/plain
>
> Response Headers:
>
>
> Content-Type –
> application/json
> text/plain; charset=utf-8
>
> Response JSON Object:
>
>
> ok (boolean) –Operation status. Available in case of success
> error (string) –Error type. Available if response code is 4xx
> reason (string) –Error description. Available if response code is
> 4xx
>
> Status Codes:
>
> 200 Restored –Database restored successfully
> 400 Bad Request –Invalid database name or deleted timestamp
> 401 Unauthorized –CouchDB Server Administrator privileges required
> 412 Precondition Failed –Database already exists
>
>
> What do you think of that? Any questions or thoughts on this? Once again a
> big acknowledgment to Nick and Paul who helped with initial design and
> provide consultation on this.
>
>
> Cheers
> Peng Hui
>
>
> [1]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_couchdb_blob_master_src_couch_src_couch-5Ffile.erl-23L251&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=91sTDh27SxPlA-ejydi3-mlMFm5qA-makaWpFPhqrlM&e=
>
> [2]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_couchdb_blob_prototype_fdb-2Dlayer_src_fabric_src_fabric2-5Ffdb.erl-23L182-2DL184&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=44Jf80PJJPAsgg2LKZXYelRa8sWTxlm2CCtJN1JWOMg&e=
>
> [3]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__activesphere.com_blog_2018_08_05_high-2Dcontention-2Dallocator&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=5jLD7_hzgxzAhJP-NRqIbQuw2e8Q1vNeQQwAqXMsPBw&e=
> <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__activesphere.com_blog_2018_08_05_high-2Dcontention-2Dallocator&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=5jLD7_hzgxzAhJP-NRqIbQuw2e8Q1vNeQQwAqXMsPBw&e=
>>
> [4]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.couchdb.org_en_stable_api_database_common.html-23delete-2D-2Ddb&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=WkBi398mDF995FmYkRw0DeqwQSMtCD4xYLmPwntrRkI&e=
>
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
Re: [DISCUSS] soft-deletion
Posted by Richard Ellis <RI...@uk.ibm.com>.
This is a nice proposal, but I have a few comments/questions regarding the
proposed API:
## API
1) `DELETE /{db}`
I wonder if there should be some consideration of the status code here.
Currently for a db delete a 200/202 is returned depending if quorum is
met. Do FDB transactional semantics mean that is likely to change at all?
I can't really decide if a 202 always makes more sense for a soft-deletion
(in the sense that the server has accepted the request to delete the
database, but won't really do it for some time). On the other hand as soon
as the database is in a soft-deleted state the {db} paths will return 404
because the database is not operable at that path, so perhaps there isn't
really any change.
2) `GET /{db}/_deleted_dbs_info`
It isn't shown in the example, but I'm assuming as this is a view-like
response that the database name is the `key` for the row similarly to
_dbs_info endpoint. Is that correct?
Would there be any advantage to using a standard timestamp format
(specifically ISO 8601, which I think should be used for all new internet
date/time representations). Compared to the examples shown this would
probably be more user friendly for people to identify the actual time
associated with instances and would have improved interoperability with
other systems interacting with the API that might want to display the
deleted date/time. Having said that I think the additional complexity of
ISO8601 would be less useful as the identifier for restore operations,
which would make me lean towards the UUID proposed by Nick for that part.
3) `PUT /{db}/_restore/{deletedTS}`
It seems that this is basically identical in structure to the create
operation. Are we expecting down the line a more complex set of options,
different response structure, or different permission to a normal db
create that warrants having a new endpoint? Given there is precedent for
using a `partitioned=true` parameter to create a database in a "special"
way, is there a good reason not to follow the same pattern here?
e.g. PUT /{db}?restore=timestamp/uuid
That would maintain the symmetry of the PUT/DELETE operations on the /{db}
path. What I mean by this is that other `PUT /{db}/foo/bar` path
operations always create sub-resources of a database (i.e. docs,
design_docs, attachments etc) and 404 in the cases that the database does
not exist. It seems strange to me to use a path structure similar to those
that has different semantics because it doesn't 404 when the {db} part
doesn't exist. Consider an equivalent could be the expectation that the
`PUT /{db}/{doc_id}` operation would create the database if it doesn't
exist instead of returning 404. Additionally usually the PUT verb is used
for creating resources at the path of the request, but that is not the
outcome of this call.
I also wonder a little bit about the use of "restore" here. I guess there
maybe should be a little consideration of whether there might be some
future backup/restore functionality and whether it be able to restore via
the same method or not. If not then calling this "restore" might cause
confusion with any future backup/restore API, say, compared to calling
this something like "undelete".
Thanks,
Rich
From: jiangph <ji...@hotmail.com>
To: dev@couchdb.apache.org
Date: 18/03/2020 12:05
Subject: [EXTERNAL] [DISCUSS] soft-deletion
Hi Everyone,
Instead of automatically and immediately removing data and index in
database after a delete operation, soft-deletion allows to restore the
deleted data back to original state due to a “fat finger”or undesired
delete operation, up to defined periods, such as 48 hours.
In CouchDB 3.0, soft-deletion of database is implemented in [1]. The
.couch file is renamed with the .<timestamp>.deleted.couch file after
soft-deletion is enabled, and such file can be changed back to .couch for
the purpose of restore. If restore is not needed and some specified period
passed, the .<timestamp>.deleted.couch file can be deleted to achieve
deletion of database permanently.
In CouchDB 4.0, with the introduction of FoundationDB, the data model and
storage is changed. In order to support soft-deletion, we propose below
solution and then implement them.
## Proposed Data model change
In CouchDB 4.0, directories and indirection access in FoundationDB are
already used to better build data model. One key/value pair is used to
build reference from Dbkey to DbPrefix. All other key/value pairs are
based on DbPrefix instead of DbKey. This decouples the direct relationship
between DBName and data in this database. The current implementation for
`DBKey -> DBPrefix` is in [2]. So you can see below information in
FoundationDB using fdbcli, etc.
```
{?ALL_DBS, DbName} -> {?DBS, DbName}
{?DBS, DbName, other part of key} -> <value>
```
To support soft-deletion, especially allowing one database to be
deleted/re-created multiple time, we need to use different DbPrefix for
the same DbKey/DBName. The proposed change is to use a unique value
allocated via High Contention Allocator(HCA) algorithm in [3].
```
DbPrefixAllocator = erlfdb_hca:create(?ERLFDB_EXTEND(DbId,
<<"hca">>)),,
DbPrefix = erlfdb_hca:allocate(DBPrefixAllocator, Tx),
erlfdb:set(Tx, DbKey, DbPrefix),
```
The data in FoundationDB looks like:
```
{?ALL_DBS, DbName} -> <unique key allocated by hca>
{<unique key allocated by hca>, other part of key} -> <value>
```
Using HCA algorithm, it can acquire one unique key quickly while avoiding
conflicting. The more important, it is shorter enough to save space
because `DBPrefix` exists in almost every key/value pair for database.
## Soft-deletion, restore and permanent-deletion
Once database is soft-deleted, the only action is to change `DBKey ->
DBPrefix` pair. All other data for this data is not changed. In order to
give clear namespace management, the proposal is to move DBkey from
`?ALL_DBS` to `?DELETED_DBS`. The timestamp when database was deleted is
added to `DBKey` so that we can know when the data in this database can be
permantenly restored. The `DBKey -> DBPrefix` pair is changed to
```
{?DELETED_DBS, DbName, TimeStamp} -> <unique key allocated by hca>
```
There is a background task to clear the ranges eventually. Depending on
setting on how long the soft-deleted database will be kept, such as 48
hours, the background task will check `DELETED_DBS` namespace, and find
eligible key/value pairs, and delete data associated with this `DBPrefix`
and then delete DbKey/DbPrefix pair finally.
Overtime, it is possible that database can be deleted several times. The
`_deleted_dbs_info` endpoint is proposed to list information about all
deleted instances for the specified database, including deletion
timestamp, document counts and disk size, etc. This allows users to
identify which one to be restored. Also it also provides information for
billing. In given period, such as 48 hours, the deletion times of same
database is most likely limited, the design API is to list all instances
in one time using GET method with query parameter.
After deciding which instance to be restored, users can use the `_restore`
endpoint with `deletedTS` to restore database. The underlying logic is to
change `DBKey -> DBPrefix` back to
```
{?ALL_DBS, DbName} -> <unique key allocated by hca>
```
Considering the sensitive actions, the `_deleted_dbs_info` and `_restore`
endpoints are supposed to be an admin-only endpoints only to allow granted
user to restore the database.
##view index and search index
Although view index and search index is based on the `DBPrefix`, the
change of value of `DBPrefix` doesn't have impact on storage and search of
view index and search index because `DBPrefix` is one opaque value. If
database is soft-deleted, the `DBkey -> DBPrefix` pair is changed so that
any access to view index and search index will be blocked with `"Database
does not exist."` error. This is expected.
The only thing we need to care is to stop all indexing or pending requests
for soft-deleted database.
## API
1) `DELETE /{db}`
There is no change on this endpoint [4] to send DELETE against one
database. The soft-deletion is triggered once
[couchdb][enable_database_recovery] is set to true in configuration file.
2) `GET /{db}/_deleted_dbs_info`
returning basic information of all deleted instances for the specified
database, including when the instance was deleted.
Parameters:
db –Database name
Request Headers:
Content-Type –application/json
Response Headers:
Content-Type –
application/json
Status Codes:
200 OK –Request completed successfully
404 Not Found –Requested database not found
Request:
GET /db/_deleted_dbs_info HTTP/1.1
Accept: application/json
Host: localhost:5984
Response:
HTTP/1.1 200 OK
Cache-Control: must-revalidate
Content-Type: application/json
{
"total_rows": 2,
"rows": [{
"deleted_when": "20200318.071532",
"info": {
"update_seq":
"0000019100b5992700000000",
"doc_del_count": 0,
"doc_count": 3,
"sizes": {
"external": 287,
"views":
0
}
}
}, {
"deleted_when": "20200318.071703",
"info": {
"update_seq":
"0000019105f0e29900000000",
"doc_del_count": 0,
"doc_count": 2,
"sizes": {
"external": 200,
"views":
0
}
}
}]
}
3) `PUT /{db}/_restore/{deletedTS}`
Restore a deleted database.
Parameters:
db –Database name
deletedTS - timestamp when database was deleted
Request Headers:
Accept –
application/json
text/plain
Response Headers:
Content-Type –
application/json
text/plain; charset=utf-8
Response JSON Object:
ok (boolean) –Operation status. Available in case of success
error (string) –Error type. Available if response code is 4xx
reason (string) –Error description. Available if response code is
4xx
Status Codes:
200 Restored –Database restored successfully
400 Bad Request –Invalid database name or deleted timestamp
401 Unauthorized –CouchDB Server Administrator privileges required
412 Precondition Failed –Database already exists
What do you think of that? Any questions or thoughts on this? Once again a
big acknowledgment to Nick and Paul who helped with initial design and
provide consultation on this.
Cheers
Peng Hui
[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_couchdb_blob_master_src_couch_src_couch-5Ffile.erl-23L251&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=91sTDh27SxPlA-ejydi3-mlMFm5qA-makaWpFPhqrlM&e=
[2]
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_couchdb_blob_prototype_fdb-2Dlayer_src_fabric_src_fabric2-5Ffdb.erl-23L182-2DL184&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=44Jf80PJJPAsgg2LKZXYelRa8sWTxlm2CCtJN1JWOMg&e=
[3]
https://urldefense.proofpoint.com/v2/url?u=https-3A__activesphere.com_blog_2018_08_05_high-2Dcontention-2Dallocator&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=5jLD7_hzgxzAhJP-NRqIbQuw2e8Q1vNeQQwAqXMsPBw&e=
<
https://urldefense.proofpoint.com/v2/url?u=https-3A__activesphere.com_blog_2018_08_05_high-2Dcontention-2Dallocator&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=5jLD7_hzgxzAhJP-NRqIbQuw2e8Q1vNeQQwAqXMsPBw&e=
>
[4]
https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.couchdb.org_en_stable_api_database_common.html-23delete-2D-2Ddb&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=CDCq0vbFWjQXx1sCFm2-iYoMZQ4i0QQj2XmPZmLvZp0&m=Axkxn5LYrVZw0h2AT61RjMT83ZlljDikPgUfCzAql5E&s=WkBi398mDF995FmYkRw0DeqwQSMtCD4xYLmPwntrRkI&e=
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] soft-deletion
Posted by Alex Miller <al...@apple.com.INVALID>.
> On Mar 18, 2020, at 14:03, Joan Touzet <wo...@apache.org> wrote:
>> And this would all rely on having a continuous backup configured and running that would hold a minimum of 48 hours of changes.
>
> People aren't in the habit of doing this in the field. I don't know if we'd be able to impress upon people the proper hygiene for this, especially if their current backup solution is "I zip/tar up the /opt/couchdb/data directory daily."
>
> If CouchDB somehow shipped out of the box with the FDB continuous backup configured and running as you say, the system load for this was minimal, and the storage required not extreme...we might be able to pass it off.
I think there’s an argument that backup to (local) file could be made automated and
managed as part of the CouchDB installation process, but it definitely would introduce
much more complexity to the install. I was honestly expecting a bit more of an objection
to the idea from Joan “CouchDB should be simple and friendly for small deployments” Touzet. ;)
> On Mar 18, 2020, at 14:06, Paul Davis <pa...@gmail.com> wrote:
>
> Alex,
>
> The first con I see for that approach is that its not soft-deletion.
> Its actual deletion with an API for restoration. Which, fair enough,
> is probably a feature we should consider supporting for CouchDB
> installations that are based on FoundationDB.
>
> The second major con is that it relies on CouchDB being based on
> FoundationDB. Part of CouchDB's design philosophy is that the internet
> may or may not exist, and if it does exist that it may or may not be
> reliable. There are lots of deployments of CouchDB that are part of a
> desktop application or POS installation that may see internet only
> periodically if at all so an S3 backup solution is out.
>
> Then the last major con I see is the time-to-restore disparity. With
> soft-deletion restoration is a few milliseconds. Streaming from S3
> will obviously depend on the size of the database and obviously be
> orders of magnitude longer.
>
> On the pro side for the soft-delete on FoundationDB is that the first
> draft of the RFC is 108 lines [1]. We obviously can't say for sure how
> big or involved the fdbrestore approach would be but I think we'd all
> agree it'd be bigger.
>
> Paul
>
> [1] https://github.com/apache/couchdb/pull/2666
Thanks for the explanation. Time-to-restore disparity would be significant,
although I’m not sure that I follow that there’s really a semantic difference
to a user between soft-delete and an instantaneous restore. Backing up to
local files could also theoretically work to avoid an internet dependency, but
you’d be getting operational complexity for not a tremendous gain, and there’s
probably other complications I’m not thinking of. I appreciate the CouchDB
design philosophy education.
And oh wow, you meant the code is 108 lines and not the document.
That's even smaller than I was expecting. Y’all keep writing surprisingly
succinct erlang. :O
> There also may come a time that there's a flavor of CouchDB that uses
> LevelDB or SQLite or FDBLite (I just made that up, any idea how hard it'd be?)
> for these sorts of embedded deployments where fdbrestore/fdbbackup
> wouldn't be feasible.
Yes! I have snipped away this thought, as I’ve actually been waiting for
someone from CouchDB to come asking how hard would it be to make
libfdb_c.so-but-actually-it’s-just-rocksdb/lmdb/sqlite. This is a project that I
would love to discuss in far more depth, but probably something we should
do outside the scope of a soft deletion RFC. I do think it’d be overall feasible
and perhaps be a nice option for embedded/mobile deployments of CouchDB.
Thanks,
Alex
Re: [DISCUSS] soft-deletion
Posted by Nick Vatamaniuc <va...@gmail.com>.
I think it looks good, Peng Hui. Nice work!
I like the API shape, and the implementation looks pretty small and
easy so far. Bonus points for using the HCA to hopefully get some
performance improvement from smaller keys overall. That was Paul's
original idea all along I believe.
Was wondering about a few minor API nits:
* Maybe use `timestamp` instead of `deleted_when` since we used
`timestamp` in the rest of the API description?
* Our db instances have unique a `uuid` (instance id) attribute
internally, we just don't surface it in the API. So when we re-create
a db with the same name it gets a new `uuid`. I could see using that
to identify individual deleted db instances when we restore them, as
opposed to using timestamps: `/{db}/_restore/{DbUuid}`. However,
because we don't already surface that attribute in the API it would be
a bit more noise too... So I think that argues for keeping timestamp
as the id, but thought I'd mentioned and see if others have thoughts
on it anyway.
Concerning the backup implementation. I think that's still an option!
In other words the soft deletion API can still be the same, and
eventually, once we get backup implemented, soft-deleted instances
could immediately (or transparently in the background) become backups.
Users might just see an extra metadata rows in `_deleted_dbs_info`
something like "backed to blobstore foo as 1 day ago .... So they know
restoring it won't be a single transaction but might take a while.
FDB backup does have a local `file://</absolute/path/to/base_dir>`
option for URLs [1] so that might be useful in embedded scenarios. And
someone has probably created some sort of local filesystem S3 shim (S0
;-) ) we could adapt perhaps....
Cheers,
-Nick
[1] https://apple.github.io/foundationdb/backups.html#backup-urls
On Wed, Mar 18, 2020 at 5:06 PM Paul Davis <pa...@gmail.com> wrote:
>
> Alex,
>
> The first con I see for that approach is that its not soft-deletion.
> Its actual deletion with an API for restoration. Which, fair enough,
> is probably a feature we should consider supporting for CouchDB
> installations that are based on FoundationDB.
>
> The second major con is that it relies on CouchDB being based on
> FoundationDB. Part of CouchDB's design philosophy is that the internet
> may or may not exist, and if it does exist that it may or may not be
> reliable. There are lots of deployments of CouchDB that are part of a
> desktop application or POS installation that may see internet only
> periodically if at all so an S3 backup solution is out. There also may
> come a time that there's a flavor of CouchDB that uses LevelDB or
> SQLite or FDBLite (I just made that up, any idea how hard it'd be?)
> for these sorts of embedded deployments where fdbrestore/fdbbackup
> wouldn't be feasible.
>
> Then the last major con I see is the time-to-restore disparity. With
> soft-deletion restoration is a few milliseconds. Streaming from S3
> will obviously depend on the size of the database and obviously be
> orders of magnitude longer.
>
> On the pro side for the soft-delete on FoundationDB is that the first
> draft of the RFC is 108 lines [1]. We obviously can't say for sure how
> big or involved the fdbrestore approach would be but I think we'd all
> agree it'd be bigger.
>
> Paul
>
> [1] https://github.com/apache/couchdb/pull/2666
>
>
> On Wed, Mar 18, 2020 at 2:31 PM Alex Miller
> <al...@apple.com.invalid> wrote:
> >
> > Let me perhaps paint an alternative RFC:
> >
> > 1) `DELETE /{db}`
> >
> > If soft-deletion is enabled, delete the database subspace, and also record into ?DELETED_DBS the timestamp of the commit and the database subspace prefix
> >
> > 2) `GET /{db}/_deleted_dbs_info`
> >
> > Return the timestamp (and whatever other info one should record) of deleted databases.
> >
> > 3) `PUT /{db}/_restore/{deletedTS}`
> >
> > Invoke `fdbrestore -k` to do a key range restricted restore into the current cluster of the deleted subspace prefix at versionstamp-1. Wait for it to complete, and return 200 when completed.
> >
> > And this would all rely on having a continuous backup configured and running that would hold a minimum of 48 hours of changes.
> >
> >
> > Now, I don’t actually deal with backups often so my memory on current caveats is a bit fuzzy. I think there might be a couple complications here that I’ve missed, like…
> > * There not being key range restricted locking of the database
> > * A key range restore is currently suboptimal in that it doesn’t do obvious filtering that it could to cut down on the amount of data it reads
> >
> > But, neither of these seem heavily blocking, as they could be tackled quickly, particularly if you leverage some upstream relationships ;). Backup and restore has been the general answer to accidental data deletion (or corruption) on FDB, and I could paint some attractive looking pros of this approach: backup files are more disk space efficient, soft deleted data could be offloaded to an S3-compatible store, it would be free if FDB is already configured to take backups. I was just curious to hear a bit more detail on your/Peng’s side of the reasons for preferring to build soft deletion on top of FDB (and thus have also intentionally withheld more of the cons of this approach, or the pros of yours).
> >
> > > On Mar 18, 2020, at 11:59, Paul Davis <pa...@gmail.com> wrote:
> > >
> > > Alex,
> > >
> > > All joking aside, soft-deletion's target use case is accidental
> > > deletions. This isn't a replacement for backup/restore which will
> > > still happen for all the usual reasons.
> > >
> > > Paul
> > >
> > > On Wed, Mar 18, 2020 at 1:42 PM Paul Davis <pa...@gmail.com> wrote:
> > >>
> > >> On Wed, Mar 18, 2020 at 1:29 PM Alex Miller
> > >> <al...@apple.com.invalid> wrote:
> > >>>
> > >>>
> > >>>> On Mar 18, 2020, at 05:04, jiangph <ji...@hotmail.com> wrote:
> > >>>>
> > >>>> Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
> > >>>>
> > >>>> In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
> > >>>>
> > >>>> In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
> > >>>
> > >>>
> > >>>
> > >>> I’ve sort of hand waved some answers to this in my head, but would you mind expanding a bit on the advantages of keeping soft-deleted data in FoundationDB as opposed to actually deleting it and relying on FoundationDB’s backup and restore to recover it if needed?
> > >>
> > >> From: Panicked User
> > >> To: Customer Support
> > >> Subject: URGENT! EMERGENCY DATABASE RESTORE!
> > >>
> > >> Dear,
> > >>
> > >> I have accidentally deleted my Very Important Database and need to
> > >> have it restored ASAP! Without this mission critical database my
> > >> company is completely offline which is costing $1B an hour!!!!!
> > >>
> > >> Please respond ASAP!
> > >>
> > >> Sincerely,
> > >> Panicky McPanics
> >
Re: [DISCUSS] soft-deletion
Posted by Paul Davis <pa...@gmail.com>.
Alex,
The first con I see for that approach is that its not soft-deletion.
Its actual deletion with an API for restoration. Which, fair enough,
is probably a feature we should consider supporting for CouchDB
installations that are based on FoundationDB.
The second major con is that it relies on CouchDB being based on
FoundationDB. Part of CouchDB's design philosophy is that the internet
may or may not exist, and if it does exist that it may or may not be
reliable. There are lots of deployments of CouchDB that are part of a
desktop application or POS installation that may see internet only
periodically if at all so an S3 backup solution is out. There also may
come a time that there's a flavor of CouchDB that uses LevelDB or
SQLite or FDBLite (I just made that up, any idea how hard it'd be?)
for these sorts of embedded deployments where fdbrestore/fdbbackup
wouldn't be feasible.
Then the last major con I see is the time-to-restore disparity. With
soft-deletion restoration is a few milliseconds. Streaming from S3
will obviously depend on the size of the database and obviously be
orders of magnitude longer.
On the pro side for the soft-delete on FoundationDB is that the first
draft of the RFC is 108 lines [1]. We obviously can't say for sure how
big or involved the fdbrestore approach would be but I think we'd all
agree it'd be bigger.
Paul
[1] https://github.com/apache/couchdb/pull/2666
On Wed, Mar 18, 2020 at 2:31 PM Alex Miller
<al...@apple.com.invalid> wrote:
>
> Let me perhaps paint an alternative RFC:
>
> 1) `DELETE /{db}`
>
> If soft-deletion is enabled, delete the database subspace, and also record into ?DELETED_DBS the timestamp of the commit and the database subspace prefix
>
> 2) `GET /{db}/_deleted_dbs_info`
>
> Return the timestamp (and whatever other info one should record) of deleted databases.
>
> 3) `PUT /{db}/_restore/{deletedTS}`
>
> Invoke `fdbrestore -k` to do a key range restricted restore into the current cluster of the deleted subspace prefix at versionstamp-1. Wait for it to complete, and return 200 when completed.
>
> And this would all rely on having a continuous backup configured and running that would hold a minimum of 48 hours of changes.
>
>
> Now, I don’t actually deal with backups often so my memory on current caveats is a bit fuzzy. I think there might be a couple complications here that I’ve missed, like…
> * There not being key range restricted locking of the database
> * A key range restore is currently suboptimal in that it doesn’t do obvious filtering that it could to cut down on the amount of data it reads
>
> But, neither of these seem heavily blocking, as they could be tackled quickly, particularly if you leverage some upstream relationships ;). Backup and restore has been the general answer to accidental data deletion (or corruption) on FDB, and I could paint some attractive looking pros of this approach: backup files are more disk space efficient, soft deleted data could be offloaded to an S3-compatible store, it would be free if FDB is already configured to take backups. I was just curious to hear a bit more detail on your/Peng’s side of the reasons for preferring to build soft deletion on top of FDB (and thus have also intentionally withheld more of the cons of this approach, or the pros of yours).
>
> > On Mar 18, 2020, at 11:59, Paul Davis <pa...@gmail.com> wrote:
> >
> > Alex,
> >
> > All joking aside, soft-deletion's target use case is accidental
> > deletions. This isn't a replacement for backup/restore which will
> > still happen for all the usual reasons.
> >
> > Paul
> >
> > On Wed, Mar 18, 2020 at 1:42 PM Paul Davis <pa...@gmail.com> wrote:
> >>
> >> On Wed, Mar 18, 2020 at 1:29 PM Alex Miller
> >> <al...@apple.com.invalid> wrote:
> >>>
> >>>
> >>>> On Mar 18, 2020, at 05:04, jiangph <ji...@hotmail.com> wrote:
> >>>>
> >>>> Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
> >>>>
> >>>> In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
> >>>>
> >>>> In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
> >>>
> >>>
> >>>
> >>> I’ve sort of hand waved some answers to this in my head, but would you mind expanding a bit on the advantages of keeping soft-deleted data in FoundationDB as opposed to actually deleting it and relying on FoundationDB’s backup and restore to recover it if needed?
> >>
> >> From: Panicked User
> >> To: Customer Support
> >> Subject: URGENT! EMERGENCY DATABASE RESTORE!
> >>
> >> Dear,
> >>
> >> I have accidentally deleted my Very Important Database and need to
> >> have it restored ASAP! Without this mission critical database my
> >> company is completely offline which is costing $1B an hour!!!!!
> >>
> >> Please respond ASAP!
> >>
> >> Sincerely,
> >> Panicky McPanics
>
Re: [DISCUSS] soft-deletion
Posted by Joan Touzet <wo...@apache.org>.
Hi Alex,
On 2020-03-18 3:31 p.m., Alex Miller wrote:
> And this would all rely on having a continuous backup configured and running that would hold a minimum of 48 hours of changes.
People aren't in the habit of doing this in the field. I don't know if
we'd be able to impress upon people the proper hygiene for this,
especially if their current backup solution is "I zip/tar up the
/opt/couchdb/data directory daily."
If CouchDB somehow shipped out of the box with the FDB continuous backup
configured and running as you say, the system load for this was minimal,
and the storage required not extreme...we might be able to pass it off.
Offering a soft-delete like this would be feature-continuous with
CouchDB 2.x/3.x and continue to provide the best approach for people in
*nix platforms. (Windows file locking introduces all sorts of other
problems.)
-Joan "people are more often lazy than incompetent" Touzet
Re: [DISCUSS] soft-deletion
Posted by Alex Miller <al...@apple.com.INVALID>.
Let me perhaps paint an alternative RFC:
1) `DELETE /{db}`
If soft-deletion is enabled, delete the database subspace, and also record into ?DELETED_DBS the timestamp of the commit and the database subspace prefix
2) `GET /{db}/_deleted_dbs_info`
Return the timestamp (and whatever other info one should record) of deleted databases.
3) `PUT /{db}/_restore/{deletedTS}`
Invoke `fdbrestore -k` to do a key range restricted restore into the current cluster of the deleted subspace prefix at versionstamp-1. Wait for it to complete, and return 200 when completed.
And this would all rely on having a continuous backup configured and running that would hold a minimum of 48 hours of changes.
Now, I don’t actually deal with backups often so my memory on current caveats is a bit fuzzy. I think there might be a couple complications here that I’ve missed, like…
* There not being key range restricted locking of the database
* A key range restore is currently suboptimal in that it doesn’t do obvious filtering that it could to cut down on the amount of data it reads
But, neither of these seem heavily blocking, as they could be tackled quickly, particularly if you leverage some upstream relationships ;). Backup and restore has been the general answer to accidental data deletion (or corruption) on FDB, and I could paint some attractive looking pros of this approach: backup files are more disk space efficient, soft deleted data could be offloaded to an S3-compatible store, it would be free if FDB is already configured to take backups. I was just curious to hear a bit more detail on your/Peng’s side of the reasons for preferring to build soft deletion on top of FDB (and thus have also intentionally withheld more of the cons of this approach, or the pros of yours).
> On Mar 18, 2020, at 11:59, Paul Davis <pa...@gmail.com> wrote:
>
> Alex,
>
> All joking aside, soft-deletion's target use case is accidental
> deletions. This isn't a replacement for backup/restore which will
> still happen for all the usual reasons.
>
> Paul
>
> On Wed, Mar 18, 2020 at 1:42 PM Paul Davis <pa...@gmail.com> wrote:
>>
>> On Wed, Mar 18, 2020 at 1:29 PM Alex Miller
>> <al...@apple.com.invalid> wrote:
>>>
>>>
>>>> On Mar 18, 2020, at 05:04, jiangph <ji...@hotmail.com> wrote:
>>>>
>>>> Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
>>>>
>>>> In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
>>>>
>>>> In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
>>>
>>>
>>>
>>> I’ve sort of hand waved some answers to this in my head, but would you mind expanding a bit on the advantages of keeping soft-deleted data in FoundationDB as opposed to actually deleting it and relying on FoundationDB’s backup and restore to recover it if needed?
>>
>> From: Panicked User
>> To: Customer Support
>> Subject: URGENT! EMERGENCY DATABASE RESTORE!
>>
>> Dear,
>>
>> I have accidentally deleted my Very Important Database and need to
>> have it restored ASAP! Without this mission critical database my
>> company is completely offline which is costing $1B an hour!!!!!
>>
>> Please respond ASAP!
>>
>> Sincerely,
>> Panicky McPanics
Re: [DISCUSS] soft-deletion
Posted by Paul Davis <pa...@gmail.com>.
Alex,
All joking aside, soft-deletion's target use case is accidental
deletions. This isn't a replacement for backup/restore which will
still happen for all the usual reasons.
Paul
On Wed, Mar 18, 2020 at 1:42 PM Paul Davis <pa...@gmail.com> wrote:
>
> On Wed, Mar 18, 2020 at 1:29 PM Alex Miller
> <al...@apple.com.invalid> wrote:
> >
> >
> > > On Mar 18, 2020, at 05:04, jiangph <ji...@hotmail.com> wrote:
> > >
> > > Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
> > >
> > > In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
> > >
> > > In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
> >
> >
> >
> > I’ve sort of hand waved some answers to this in my head, but would you mind expanding a bit on the advantages of keeping soft-deleted data in FoundationDB as opposed to actually deleting it and relying on FoundationDB’s backup and restore to recover it if needed?
>
> From: Panicked User
> To: Customer Support
> Subject: URGENT! EMERGENCY DATABASE RESTORE!
>
> Dear,
>
> I have accidentally deleted my Very Important Database and need to
> have it restored ASAP! Without this mission critical database my
> company is completely offline which is costing $1B an hour!!!!!
>
> Please respond ASAP!
>
> Sincerely,
> Panicky McPanics
Re: [DISCUSS] soft-deletion
Posted by Paul Davis <pa...@gmail.com>.
On Wed, Mar 18, 2020 at 1:29 PM Alex Miller
<al...@apple.com.invalid> wrote:
>
>
> > On Mar 18, 2020, at 05:04, jiangph <ji...@hotmail.com> wrote:
> >
> > Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
> >
> > In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
> >
> > In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
>
>
>
> I’ve sort of hand waved some answers to this in my head, but would you mind expanding a bit on the advantages of keeping soft-deleted data in FoundationDB as opposed to actually deleting it and relying on FoundationDB’s backup and restore to recover it if needed?
From: Panicked User
To: Customer Support
Subject: URGENT! EMERGENCY DATABASE RESTORE!
Dear,
I have accidentally deleted my Very Important Database and need to
have it restored ASAP! Without this mission critical database my
company is completely offline which is costing $1B an hour!!!!!
Please respond ASAP!
Sincerely,
Panicky McPanics
Re: [DISCUSS] soft-deletion
Posted by Alex Miller <al...@apple.com.INVALID>.
> On Mar 18, 2020, at 05:04, jiangph <ji...@hotmail.com> wrote:
>
> Instead of automatically and immediately removing data and index in database after a delete operation, soft-deletion allows to restore the deleted data back to original state due to a “fat finger”or undesired delete operation, up to defined periods, such as 48 hours.
>
> In CouchDB 3.0, soft-deletion of database is implemented in [1]. The .couch file is renamed with the .<timestamp>.deleted.couch file after soft-deletion is enabled, and such file can be changed back to .couch for the purpose of restore. If restore is not needed and some specified period passed, the .<timestamp>.deleted.couch file can be deleted to achieve deletion of database permanently.
>
> In CouchDB 4.0, with the introduction of FoundationDB, the data model and storage is changed. In order to support soft-deletion, we propose below solution and then implement them.
I’ve sort of hand waved some answers to this in my head, but would you mind expanding a bit on the advantages of keeping soft-deleted data in FoundationDB as opposed to actually deleting it and relying on FoundationDB’s backup and restore to recover it if needed?
Re: [DISCUSS] soft-deletion
Posted by Jan Lehnardt <ja...@apache.org>.
> On 22. Mar 2020, at 21:18, Mike Rhodes <co...@dx13.co.uk> wrote:
>
> On Sat, 21 Mar 2020, at 13:47, jiangph wrote:
>> Thanks a lot, Mike for your comments and suggestion on APIs. They are
>> quite helpful. Please see embedded response below.
>>> POST /_deleted_dbs/_restore
>>> - Restore a database at a given timestamp
>>> - Allows supplying a destination database.
>>> - I view _restore as an RPC-like call than a REST call, so it has a body
>>> containing the arguments rather than supplying as part of the path:
>>>
>>> { "source": {"db": "animaldb", "ts": "<deletedTS>"}, "destination": "animaldb"}
>>>
>>> - Fails if "destination" exists.
>>>
>>> If we choose to go with the original suggestion, I would still suggest taking the RPC approach to restore, meaning it looks like this:
>>>
>>> POST /{db}/_restore
>>>
>>> { "sourceTS": "<deletedTS>", "destination": "animaldb"}
>>>
>>> For either approach, "destination" could be optional and default to the original database name.
>>>
>>
>> From technical point of view, it is feasible to implement to restore
>> database to different destination database. I just want to bring up my
>> concern about this. If I am correct, there is no endpoint to rename
>> database in CouchDB 3.0 or before. if we support to restore database
>> to new database here, it will provide alternative way to support
>> *rename*. I am not sure whether this is expected or not.
>
> Renaming a database is a nice feature. It's not offered in CouchDB 2.0, by my understanding, because it's not possible to do easily as the rename would essentially be a distributed file rename across the cluster. In FoundationDB, per my understanding of your original email's details on the FoundationDB side of things, what soft-delete is, is essentially a rename with a specialised use-case -- and also a cheap operation. Database rename has certainly been a feature that I've seen requested a decent number of times, and one which I think it's worth re-considering given it is practically achievable with the new architecture.
>
> I am not sure on a good API for renaming a database. As REST doesn't provide us with clear guidance there, perhaps it'd suit another RPC-like call. I think that works fine for rarely-used administrative operations, and in my view it's easy to get too wedded to REST and end up with awkward APIs.
We already took COPY from the WebDAV RFC, we might as well introduce MOVE for database renames.
HTTP/1.1
MOVE /source_db
Location: /target_db
Best
Jan
—
>
>> You mentioned that restoring database might fail if there is live
>> database with same name, and it will cause data loss. But I think that
>> users may replicate live database to another database, and then delete
>> it, and then restore previously deleted database back to live database,
>> and replicate data again. The result seems to be equivalent as we
>> provide restoring to different destination.
>
> For me it's about the usability rather than the overall functionality. The scenario I have in mind is that a user has deleted a database and set up a new production database re-using the same name. Later that day they realise they need a few documents from the old database -- being able to restore to a new database path would allow them to do this safely in a few minutes. The proposed approach involves both a sequence of CouchDB replication operations, and likely a change to production application configuration part way through to point to a new database.
>
> In terms of extra code and testing required by adding options to the restore matrix, I don't know the difference here. So after outlining the scenario that I had in mind when I made the suggestion, I'll have to leave the decision here to those who better know the effort involved to set against the possible benefits.
>
> --
> Mike.
Re: [DISCUSS] soft-deletion
Posted by jiangph <ji...@hotmail.com>.
Thanks Mike for quick response. See below
> On Mar 23, 2020, at 4:18 AM, Mike Rhodes <co...@dx13.co.uk> wrote:
>
> On Sat, 21 Mar 2020, at 13:47, jiangph wrote:
>> Thanks a lot, Mike for your comments and suggestion on APIs. They are
>> quite helpful. Please see embedded response below.
>>> POST /_deleted_dbs/_restore
>>> - Restore a database at a given timestamp
>>> - Allows supplying a destination database.
>>> - I view _restore as an RPC-like call than a REST call, so it has a body
>>> containing the arguments rather than supplying as part of the path:
>>>
>>> { "source": {"db": "animaldb", "ts": "<deletedTS>"}, "destination": "animaldb"}
>>>
>>> - Fails if "destination" exists.
>>>
>>> If we choose to go with the original suggestion, I would still suggest taking the RPC approach to restore, meaning it looks like this:
>>>
>>> POST /{db}/_restore
>>>
>>> { "sourceTS": "<deletedTS>", "destination": "animaldb"}
>>>
>>> For either approach, "destination" could be optional and default to the original database name.
>>>
>>
>> From technical point of view, it is feasible to implement to restore
>> database to different destination database. I just want to bring up my
>> concern about this. If I am correct, there is no endpoint to rename
>> database in CouchDB 3.0 or before. if we support to restore database
>> to new database here, it will provide alternative way to support
>> *rename*. I am not sure whether this is expected or not.
>
> Renaming a database is a nice feature. It's not offered in CouchDB 2.0, by my understanding, because it's not possible to do easily as the rename would essentially be a distributed file rename across the cluster. In FoundationDB, per my understanding of your original email's details on the FoundationDB side of things, what soft-delete is, is essentially a rename with a specialised use-case -- and also a cheap operation. Database rename has certainly been a feature that I've seen requested a decent number of times, and one which I think it's worth re-considering given it is practically achievable with the new architecture.
>
> I am not sure on a good API for renaming a database. As REST doesn't provide us with clear guidance there, perhaps it'd suit another RPC-like call. I think that works fine for rarely-used administrative operations, and in my view it's easy to get too wedded to REST and end up with awkward APIs.
Yes, I can imagine that it will be more complex to implement *rename* in CouchDB 3.0 or before. If this is desired feature, I would like to start implement from soft-deletion piece. For purely *rename* endpoint, this can be done in separated bits.
>
>> You mentioned that restoring database might fail if there is live
>> database with same name, and it will cause data loss. But I think that
>> users may replicate live database to another database, and then delete
>> it, and then restore previously deleted database back to live database,
>> and replicate data again. The result seems to be equivalent as we
>> provide restoring to different destination.
>
> For me it's about the usability rather than the overall functionality. The scenario I have in mind is that a user has deleted a database and set up a new production database re-using the same name. Later that day they realise they need a few documents from the old database -- being able to restore to a new database path would allow them to do this safely in a few minutes. The proposed approach involves both a sequence of CouchDB replication operations, and likely a change to production application configuration part way through to point to a new database.
>
> In terms of extra code and testing required by adding options to the restore matrix, I don't know the difference here. So after outlining the scenario that I had in mind when I made the suggestion, I'll have to leave the decision here to those who better know the effort involved to set against the possible benefits.
From the effort’s point of view for restoring to another destination, it should be manage-able in coming weeks.
>
> --
> Mike.
Re: [DISCUSS] soft-deletion
Posted by Mike Rhodes <co...@dx13.co.uk>.
On Sat, 21 Mar 2020, at 13:47, jiangph wrote:
> Thanks a lot, Mike for your comments and suggestion on APIs. They are
> quite helpful. Please see embedded response below.
> > POST /_deleted_dbs/_restore
> > - Restore a database at a given timestamp
> > - Allows supplying a destination database.
> > - I view _restore as an RPC-like call than a REST call, so it has a body
> > containing the arguments rather than supplying as part of the path:
> >
> > { "source": {"db": "animaldb", "ts": "<deletedTS>"}, "destination": "animaldb"}
> >
> > - Fails if "destination" exists.
> >
> > If we choose to go with the original suggestion, I would still suggest taking the RPC approach to restore, meaning it looks like this:
> >
> > POST /{db}/_restore
> >
> > { "sourceTS": "<deletedTS>", "destination": "animaldb"}
> >
> > For either approach, "destination" could be optional and default to the original database name.
> >
>
> From technical point of view, it is feasible to implement to restore
> database to different destination database. I just want to bring up my
> concern about this. If I am correct, there is no endpoint to rename
> database in CouchDB 3.0 or before. if we support to restore database
> to new database here, it will provide alternative way to support
> *rename*. I am not sure whether this is expected or not.
Renaming a database is a nice feature. It's not offered in CouchDB 2.0, by my understanding, because it's not possible to do easily as the rename would essentially be a distributed file rename across the cluster. In FoundationDB, per my understanding of your original email's details on the FoundationDB side of things, what soft-delete is, is essentially a rename with a specialised use-case -- and also a cheap operation. Database rename has certainly been a feature that I've seen requested a decent number of times, and one which I think it's worth re-considering given it is practically achievable with the new architecture.
I am not sure on a good API for renaming a database. As REST doesn't provide us with clear guidance there, perhaps it'd suit another RPC-like call. I think that works fine for rarely-used administrative operations, and in my view it's easy to get too wedded to REST and end up with awkward APIs.
> You mentioned that restoring database might fail if there is live
> database with same name, and it will cause data loss. But I think that
> users may replicate live database to another database, and then delete
> it, and then restore previously deleted database back to live database,
> and replicate data again. The result seems to be equivalent as we
> provide restoring to different destination.
For me it's about the usability rather than the overall functionality. The scenario I have in mind is that a user has deleted a database and set up a new production database re-using the same name. Later that day they realise they need a few documents from the old database -- being able to restore to a new database path would allow them to do this safely in a few minutes. The proposed approach involves both a sequence of CouchDB replication operations, and likely a change to production application configuration part way through to point to a new database.
In terms of extra code and testing required by adding options to the restore matrix, I don't know the difference here. So after outlining the scenario that I had in mind when I made the suggestion, I'll have to leave the decision here to those who better know the effort involved to set against the possible benefits.
--
Mike.
Re: [DISCUSS] soft-deletion
Posted by jiangph <ji...@hotmail.com>.
Thanks a lot, Mike for your comments and suggestion on APIs. They are quite helpful. Please see embedded response below.
> On Mar 20, 2020, at 10:07 PM, Mike Rhodes <co...@dx13.co.uk> wrote:
>
> Looking good. I'm going to skip the database internals as I'm not at all qualified to comment. I do have some comments on the HTTP API however.
>
> First, I think this API spec is missing how you find databases which have soft-deleted versions. For example, this could be:
>
> /_all_dbs
> - lists only live databases
>
> /_deleted_dbs
> - lists only dbs with at least one soft-deleted version
> - API identical otherwise to _all_dbs
Good call for `_deleted_dbs`. I can implement this endpoint soon.
>
> Secondly, this API lacks the ability to restore a soft-deleted database to a new database, you can only restore a soft-deleted database to the original name, which could be over the top of a new database. I suspect this could be problematic in day-to-day use. I'd propose:
>
> - You can't restore a soft-deleted database on top of a "live" database. This prevents data loss from a restore.
> - Support for the ability to restore a database to a new database path.
>
> Could you also clarify whether the restore operation "removes" the soft-deleted version. My assumption is yes, that we're just switching back the database into the "live" keyspace.
Yes, it will remove the soft-deleted version and make this version back to a “live” one.
>
> An alternative API takes the conceptual model of deleted databases moving to a "recycle bin" within the API's namespace. I quite like this, but it's just how it works for me; others may hate it. The original proposal, to me, takes the approach that every database has it's own "recycle bin", so this is merely altering the conceptual containers here.
>
> Mostly, I found that the following way feels a bit more RESTful in that there is a specific set of deleted database resources, and you avoid the cognitive dissonance of being able to carry out operations on non-existent resources.
>
> In brief it roots operations on soft-deleted databases in a /_deleted_dbs namespace rather than augmenting existing endpoints:
>
> GET /_all_dbs
> - lists only live databases (noted just to clarify this)
>
> GET /_deleted_dbs
> - lists only dbs with at least one soft-deleted version
> - API identical otherwise to _all_dbs
> - It's a bit confusing to me that there can be overlap
> between _deleted_dbs and _all_dbs as a given database may
> have live and soft-deleted versions, but perhaps this is okay.
>
> GET /_deleted_dbs/{db}
> - Equivalent to `GET /{db}/_deleted_dbs_info` in original.
> - To me, this makes it very clear that a deleted db has only one
> operation -- get the metadata -- rather than hanging the _deleted_dbs_info
> on a path that could be a live database or a non-existent database.
>
> POST /_deleted_dbs/_restore
> - Restore a database at a given timestamp
> - Allows supplying a destination database.
> - I view _restore as an RPC-like call than a REST call, so it has a body
> containing the arguments rather than supplying as part of the path:
>
> { "source": {"db": "animaldb", "ts": "<deletedTS>"}, "destination": "animaldb"}
>
> - Fails if "destination" exists.
>
> If we choose to go with the original suggestion, I would still suggest taking the RPC approach to restore, meaning it looks like this:
>
> POST /{db}/_restore
>
> { "sourceTS": "<deletedTS>", "destination": "animaldb"}
>
> For either approach, "destination" could be optional and default to the original database name.
>
From technical point of view, it is feasible to implement to restore database to different destination database. I just want to bring up my concern about this. If I am correct, there is no endpoint to rename database in CouchDB 3.0 or before. if we support to restore database to new database here, it will provide alternative way to support *rename*. I am not sure whether this is expected or not.
You mentioned that restoring database might fail if there is live database with same name, and it will cause data loss. But I think that users may replicate live database to another database, and then delete it, and then restore previously deleted database back to live database, and replicate data again. The result seems to be equivalent as we provide restoring to different destination.
> --
> Mike.
>
> On Wed, 18 Mar 2020, at 12:04, jiangph wrote:
>> ## API
>>
>> 1) `DELETE /{db}`
>>
>> There is no change on this endpoint [4] to send DELETE against one
>> database. The soft-deletion is triggered once
>> [couchdb][enable_database_recovery] is set to true in configuration
>> file.
>>
>>
>> 2) `GET /{db}/_deleted_dbs_info`
>>
>> returning basic information of all deleted instances for the
>> specified database, including when the instance was deleted.
>> Parameters:
>>
>> db –Database name
>>
>> Request Headers:
>>
>> Content-Type –application/json
>>
>> Response Headers:
>>
>>
>> Content-Type –
>> application/json
>>
>> Status Codes:
>>
>> 200 OK –Request completed successfully
>> 404 Not Found –Requested database not found
>>
>> Request:
>>
>> GET /db/_deleted_dbs_info HTTP/1.1
>> Accept: application/json
>> Host: localhost:5984
>>
>> Response:
>>
>> HTTP/1.1 200 OK
>> Cache-Control: must-revalidate
>> Content-Type: application/json
>> {
>> "total_rows": 2,
>> "rows": [{
>> "deleted_when": "20200318.071532",
>> "info": {
>> "update_seq": "0000019100b5992700000000",
>> "doc_del_count": 0,
>> "doc_count": 3,
>> "sizes": {
>> "external": 287,
>> "views": 0
>> }
>> }
>> }, {
>> "deleted_when": "20200318.071703",
>> "info": {
>> "update_seq": "0000019105f0e29900000000",
>> "doc_del_count": 0,
>> "doc_count": 2,
>> "sizes": {
>> "external": 200,
>> "views": 0
>> }
>> }
>> }]
>> }
>>
>>
>> 3) `PUT /{db}/_restore/{deletedTS}`
>>
>> Restore a deleted database.
>> Parameters:
>>
>> db –Database name
>> deletedTS - timestamp when database was deleted
>>
>> Request Headers:
>>
>>
>> Accept –
>> application/json
>> text/plain
>>
>> Response Headers:
>>
>>
>> Content-Type –
>> application/json
>> text/plain; charset=utf-8
>>
>> Response JSON Object:
>>
>>
>> ok (boolean) –Operation status. Available in case of success
>> error (string) –Error type. Available if response code is 4xx
>> reason (string) –Error description. Available if response code is 4xx
>>
>> Status Codes:
>>
>> 200 Restored –Database restored successfully
>> 400 Bad Request –Invalid database name or deleted timestamp
>> 401 Unauthorized –CouchDB Server Administrator privileges required
>> 412 Precondition Failed –Database already exists
>>
>>
>> What do you think of that? Any questions or thoughts on this? Once
>> again a big acknowledgment to Nick and Paul who helped with initial
>> design and provide consultation on this.
>>
>>
>> Cheers
>> Peng Hui
>>
>>
>> [1]
>> https://github.com/apache/couchdb/blob/master/src/couch/src/couch_file.erl#L251
>> [2]
>> https://github.com/apache/couchdb/blob/prototype/fdb-layer/src/fabric/src/fabric2_fdb.erl#L182-L184
>> [3] https://activesphere.com/blog/2018/08/05/high-contention-allocator
>> <https://activesphere.com/blog/2018/08/05/high-contention-allocator>
>> [4]
>> https://docs.couchdb.org/en/stable/api/database/common.html#delete--db
>>
>>
Re: [DISCUSS] soft-deletion
Posted by Mike Rhodes <co...@dx13.co.uk>.
Looking good. I'm going to skip the database internals as I'm not at all qualified to comment. I do have some comments on the HTTP API however.
First, I think this API spec is missing how you find databases which have soft-deleted versions. For example, this could be:
/_all_dbs
- lists only live databases
/_deleted_dbs
- lists only dbs with at least one soft-deleted version
- API identical otherwise to _all_dbs
Secondly, this API lacks the ability to restore a soft-deleted database to a new database, you can only restore a soft-deleted database to the original name, which could be over the top of a new database. I suspect this could be problematic in day-to-day use. I'd propose:
- You can't restore a soft-deleted database on top of a "live" database. This prevents data loss from a restore.
- Support for the ability to restore a database to a new database path.
Could you also clarify whether the restore operation "removes" the soft-deleted version. My assumption is yes, that we're just switching back the database into the "live" keyspace.
An alternative API takes the conceptual model of deleted databases moving to a "recycle bin" within the API's namespace. I quite like this, but it's just how it works for me; others may hate it. The original proposal, to me, takes the approach that every database has it's own "recycle bin", so this is merely altering the conceptual containers here.
Mostly, I found that the following way feels a bit more RESTful in that there is a specific set of deleted database resources, and you avoid the cognitive dissonance of being able to carry out operations on non-existent resources.
In brief it roots operations on soft-deleted databases in a /_deleted_dbs namespace rather than augmenting existing endpoints:
GET /_all_dbs
- lists only live databases (noted just to clarify this)
GET /_deleted_dbs
- lists only dbs with at least one soft-deleted version
- API identical otherwise to _all_dbs
- It's a bit confusing to me that there can be overlap
between _deleted_dbs and _all_dbs as a given database may
have live and soft-deleted versions, but perhaps this is okay.
GET /_deleted_dbs/{db}
- Equivalent to `GET /{db}/_deleted_dbs_info` in original.
- To me, this makes it very clear that a deleted db has only one
operation -- get the metadata -- rather than hanging the _deleted_dbs_info
on a path that could be a live database or a non-existent database.
POST /_deleted_dbs/_restore
- Restore a database at a given timestamp
- Allows supplying a destination database.
- I view _restore as an RPC-like call than a REST call, so it has a body
containing the arguments rather than supplying as part of the path:
{ "source": {"db": "animaldb", "ts": "<deletedTS>"}, "destination": "animaldb"}
- Fails if "destination" exists.
If we choose to go with the original suggestion, I would still suggest taking the RPC approach to restore, meaning it looks like this:
POST /{db}/_restore
{ "sourceTS": "<deletedTS>", "destination": "animaldb"}
For either approach, "destination" could be optional and default to the original database name.
--
Mike.
On Wed, 18 Mar 2020, at 12:04, jiangph wrote:
> ## API
>
> 1) `DELETE /{db}`
>
> There is no change on this endpoint [4] to send DELETE against one
> database. The soft-deletion is triggered once
> [couchdb][enable_database_recovery] is set to true in configuration
> file.
>
>
> 2) `GET /{db}/_deleted_dbs_info`
>
> returning basic information of all deleted instances for the
> specified database, including when the instance was deleted.
> Parameters:
>
> db –Database name
>
> Request Headers:
>
> Content-Type –application/json
>
> Response Headers:
>
>
> Content-Type –
> application/json
>
> Status Codes:
>
> 200 OK –Request completed successfully
> 404 Not Found –Requested database not found
>
> Request:
>
> GET /db/_deleted_dbs_info HTTP/1.1
> Accept: application/json
> Host: localhost:5984
>
> Response:
>
> HTTP/1.1 200 OK
> Cache-Control: must-revalidate
> Content-Type: application/json
> {
> "total_rows": 2,
> "rows": [{
> "deleted_when": "20200318.071532",
> "info": {
> "update_seq": "0000019100b5992700000000",
> "doc_del_count": 0,
> "doc_count": 3,
> "sizes": {
> "external": 287,
> "views": 0
> }
> }
> }, {
> "deleted_when": "20200318.071703",
> "info": {
> "update_seq": "0000019105f0e29900000000",
> "doc_del_count": 0,
> "doc_count": 2,
> "sizes": {
> "external": 200,
> "views": 0
> }
> }
> }]
> }
>
>
> 3) `PUT /{db}/_restore/{deletedTS}`
>
> Restore a deleted database.
> Parameters:
>
> db –Database name
> deletedTS - timestamp when database was deleted
>
> Request Headers:
>
>
> Accept –
> application/json
> text/plain
>
> Response Headers:
>
>
> Content-Type –
> application/json
> text/plain; charset=utf-8
>
> Response JSON Object:
>
>
> ok (boolean) –Operation status. Available in case of success
> error (string) –Error type. Available if response code is 4xx
> reason (string) –Error description. Available if response code is 4xx
>
> Status Codes:
>
> 200 Restored –Database restored successfully
> 400 Bad Request –Invalid database name or deleted timestamp
> 401 Unauthorized –CouchDB Server Administrator privileges required
> 412 Precondition Failed –Database already exists
>
>
> What do you think of that? Any questions or thoughts on this? Once
> again a big acknowledgment to Nick and Paul who helped with initial
> design and provide consultation on this.
>
>
> Cheers
> Peng Hui
>
>
> [1]
> https://github.com/apache/couchdb/blob/master/src/couch/src/couch_file.erl#L251
> [2]
> https://github.com/apache/couchdb/blob/prototype/fdb-layer/src/fabric/src/fabric2_fdb.erl#L182-L184
> [3] https://activesphere.com/blog/2018/08/05/high-contention-allocator
> <https://activesphere.com/blog/2018/08/05/high-contention-allocator>
> [4]
> https://docs.couchdb.org/en/stable/api/database/common.html#delete--db
>
>