[GitHub] [couchdb-documentation] rnewson commented on issue #406: Draft of all_docs and dbinfo RFC, still WIP

[GitBox <gi...@apache.org>]

rnewson commented on issue #406: Draft of all_docs and dbinfo RFC, still WIP
URL: https://github.com/apache/couchdb-documentation/pull/406#issuecomment-484468594
 
 
   Firstly, I feel we can deprecate or change anything in the db info blob as we see fit. We are planning a major version bump, 3.0, to signal the path forward, and a further major version bump, 4.0, which is when the foundationdb backplane appears.
   
   The paragraphs on the `_all_docs` subspace are clear to me as are the rules on incrementing and decrementing `doc_count` and `doc_del_count`. Both of those values retain meaning to the user, given that tombstones will still exist.
   
   It would be good if purge_seq were to vanish, replaced by atomic removal of all trace of a document including secondary indexes, but that does seem unlikely (or, at least, it makes third party indexing difficult to impossible). Purge is a large topic of its own and so I won't say more here.
   
   On sizes, the new `couch_ejson_size:encoded_size(Body)` calculation should be the same value as today and worth preserving. Beyond that, given the absence of compaction, is there any need for other sizes? The only other obvious one that we can calculate is the sum of the lengths of all the keys and values used to store the database (that is, including all the overhead of the exploded form and the metadata). I think that's doable with atomic operations but, I'm imagining, involves housekeeping in many places. We need to decide what these values are for first, what decisions do they help users make? `external` exists, largely, for Cloudant, as it represents the data as the user understands it, and Cloudant charges by the GB for that. It is obviously also useful to other users but I suspect most users would only care for the `file` value, in order to direct compaction efforts or other cleanup (including deleting databases entirely). The distinction between `active` data and trash is eliminated by FoundationDB and our choice of update semantic (the old revision is atomically removed when the new revision supercedes it), so we should drop that item.
   
   For r/w/n/q I feel strongly we should drop them. They are intimately coupled to the current architecture and I think it would be misleading to carry these values across. The configuration of the FoundationDB cluster can take many forms and can change at runtime; we should cleanly decouple ourselves from it. I don't think we should pass-through any foundationdb status or config either, though our documentation / release notes for 4.0 should include a user guide for the basics. Others might disagree on how much handholding we should be doing. Are we, in effect, embedding FoundationDB or are we a layer above it? As code, it's emphatically the latter, but a user could legitimately hold either perspective. This matters for this RFC as its the deciding factor on the scope of the "info" in /dbname and elsewhere.
   
   +1 on removing `total_rows`, `offset`, `compact_running`, `other`, `data_size` and `disk_size` but I note that the replicator looks for `instance_start_time`. We can enhance the replicator in 3.0 to no longer do so and then state that 4.0 cannot replicate to < 3.0 systems, but perhaps it wouldn't hurt to include the field either hardcoded to 0 (as we've done from 2.0 onward for compatibility reasons) or set it to the start time of the fdb cluster if that's cheaply available. 
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services