You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "nickva (via GitHub)" <gi...@apache.org> on 2025/10/13 20:49:30 UTC
[PR] Optimize UUID generation and use UUID v7 by default [couchdb]
nickva opened a new pull request, #5699:
URL: https://github.com/apache/couchdb/pull/5699
The UUID v7 along with the `random` UUID types do not require going through the gen_server as they do not need to keep any state. This should make the system more robust and prevent clients from overwhelming the node with concurrent requests to the `/_uuid` endpoint.
Add a plain hex-encoded version of UUID v7 as `uuid_v7_hex` and use that as the new default. The `hex` version is used according to the guidance in:
https://datatracker.ietf.org/doc/rfc9562/
```
For many applications, such as databases, storing UUIDs as text is
unnecessarily verbose, requiring 288 bits to represent 128-bit UUID
values. Thus, where feasible, UUIDs SHOULD be stored within database
applications as the underlying 128-bit binary value.
```
Ideally we'd like to use just the 128 bit binary, however since we're transmitting json we'll encode it as base 16. This way its shape is also exactly compatible (same length and characters) with the previous `sequential` default.
In the future more compact encodings may be added: base32, base58 etc.
Users can still generate properly formatted UUID v7 strings (with dashes) by using the `uuid_v7` algorithm. That will produce 36 byte character strings which may look like:
```
[
"0199df37-5929-7032-b402-c3e61fbbf88f",
"0199df37-5929-7851-9492-e566433ced7f",
...
]
```
While at it, expand the test suite and add more coverage and also ensure to test the sizes of the returned values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Optimize UUID generation and use UUID v7 by default [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#discussion_r2427537145
##########
src/couch/src/couch_uuids.erl:
##########
@@ -122,37 +126,35 @@ inc() ->
rand:uniform(16#ffd).
state() ->
- AlgoStr = config:get("uuids", "algorithm", "sequential"),
+ AlgoStr = config_algorithm(),
case couch_util:to_existing_atom(AlgoStr) of
random ->
random;
utc_random ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
{utc_random, ClockSeq};
utc_id ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
UtcIdSuffix = config:get("uuids", "utc_id_suffix", ""),
{utc_id, UtcIdSuffix, ClockSeq};
sequential ->
{sequential, new_prefix(), inc()};
uuid_v7 ->
uuid_v7;
+ uuid_v7_hex ->
Review Comment:
> I saw that section of the RFC but these UUID's will not be stored as 128-bit binary
We're storing uuids as binaries from this module internally in the purge id and sequence trees. If we could store document IDs in a more compact way we would. In fact, I created a base 36 encoding version to show how it can be done. The same UUID v7 information can be encoded in only 24 bytes instead of 36. Having smaller, more compact IDs, which still have the same information content in a database is worth-while I think, especially with how we handle append-only B-trees, and keep re-writing the whole paths to the root on every commit.
> that section does not apply to document id strings.
The intent behind the section is to indicate to database they they can use a more compact representation instead of the nicely formatted strings. The dashboards and user interfaces can use the nicely formatted strings but we don't have to waste space in doc IDs repeating the same 4 extra dashes every time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Optimize UUID generation and use UUID v7 by default [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#discussion_r2427347719
##########
src/couch/src/couch_uuids.erl:
##########
@@ -122,37 +126,35 @@ inc() ->
rand:uniform(16#ffd).
state() ->
- AlgoStr = config:get("uuids", "algorithm", "sequential"),
+ AlgoStr = config_algorithm(),
case couch_util:to_existing_atom(AlgoStr) of
random ->
random;
utc_random ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
{utc_random, ClockSeq};
utc_id ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
UtcIdSuffix = config:get("uuids", "utc_id_suffix", ""),
{utc_id, UtcIdSuffix, ClockSeq};
sequential ->
{sequential, new_prefix(), inc()};
uuid_v7 ->
uuid_v7;
+ uuid_v7_hex ->
Review Comment:
If we're going with changing the default I think its shape should be consistent with previous algorithm to be least disruptive.
Also the RFC specifically mentions databases https://datatracker.ietf.org/doc/rfc9562/ :
```
6.13. DBMS and Database Considerations
For many applications, such as databases, storing UUIDs as text is
unnecessarily verbose, requiring 288 bits to represent 128-bit UUID
values. Thus, where feasible, UUIDs SHOULD be stored within database
applications as the underlying 128-bit binary value.
```
We can't transmit binaries in JSON so we use what we already do for these IDs -- we hex-encode them. I can see having `uuid_v7_base32` or something in the future and that will certainly not have any dashes in it.
> I don't see a reason to provide two formats for essentially the same thing, so I think it has to come to a dev@ vote to get a decision.
Well the fact that we can't agree exactly on it, probably indicate it's not bad idea to have both?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Optimize UUID generation and use UUID v7 by default [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on code in PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#discussion_r2427325053
##########
src/couch/src/couch_uuids.erl:
##########
@@ -122,37 +126,35 @@ inc() ->
rand:uniform(16#ffd).
state() ->
- AlgoStr = config:get("uuids", "algorithm", "sequential"),
+ AlgoStr = config_algorithm(),
case couch_util:to_existing_atom(AlgoStr) of
random ->
random;
utc_random ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
{utc_random, ClockSeq};
utc_id ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
UtcIdSuffix = config:get("uuids", "utc_id_suffix", ""),
{utc_id, UtcIdSuffix, ClockSeq};
sequential ->
{sequential, new_prefix(), inc()};
uuid_v7 ->
uuid_v7;
+ uuid_v7_hex ->
Review Comment:
I'm still not comfortable calling something an UUID v7 that doesn't conform to the RFC 9562 formatting (when strings). I appreciate couch_uuids has always returned malformed uuids but this is the first time we're claiming a particular 'version', and that only makes sense to me in RFC 9562 terms.
I don't see a reason to provide two formats for essentially the same thing, so I think it has to come to a dev@ vote to get a decision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add UUID v4 and v7. Make v7 the default. Optimize UUID generation [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva merged PR #5699:
URL: https://github.com/apache/couchdb/pull/5699
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add UUID v4 and v7. Make v7 the default. Optimize UUID generation [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on code in PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#discussion_r2435146926
##########
rel/overlay/etc/default.ini:
##########
@@ -615,12 +615,23 @@ partitioned||* = true
; First 14 characters are the time in hex. Last 18 are random.
; utc_id - Time since Jan 1, 1970 UTC with microseconds, plus utc_id_suffix string
; First 14 characters are the time in hex. uuids/utc_id_suffix string value is appended to these.
-;algorithm = sequential
+; uuid_v7 - UUID v7. The returned format depends on the [uuids] format setting
+; uuid_v4 - UUID v4. The returned format depends on the [uuids] format setting
+;algorithm = uuid_v7
; The utc_id_suffix value will be appended to uuids generated by the utc_id algorithm.
; Replicating instances should have unique utc_id_suffix values to ensure uniqueness of utc_id ids.
;utc_id_suffix =
+; Encoding format for UUID v4 and v7. The default is base_16 (aka hex). Other
+; accepted formats are "base_36" and "string". Only v4 and v7 algorithm have
+; configurable formats, other algorithms only return base_16 encoded values.
+; Examples of the same UUID value encoded in all the supported formats:
+; "base_16" : "0199df3759297032b402c3e61fbbf88f"
+; "base_36" : "03eudcyamunnfraqdgmopx09b"
+; "string" : "0199df37-5929-7032-b402-c3e61fbbf88f"
Review Comment:
let's call this option `rfc9562` for clarity?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add UUID v4 and v7. Make v7 the default. Optimize UUID generation [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#discussion_r2435727739
##########
rel/overlay/etc/default.ini:
##########
@@ -615,12 +615,23 @@ partitioned||* = true
; First 14 characters are the time in hex. Last 18 are random.
; utc_id - Time since Jan 1, 1970 UTC with microseconds, plus utc_id_suffix string
; First 14 characters are the time in hex. uuids/utc_id_suffix string value is appended to these.
-;algorithm = sequential
+; uuid_v7 - UUID v7. The returned format depends on the [uuids] format setting
+; uuid_v4 - UUID v4. The returned format depends on the [uuids] format setting
+;algorithm = uuid_v7
; The utc_id_suffix value will be appended to uuids generated by the utc_id algorithm.
; Replicating instances should have unique utc_id_suffix values to ensure uniqueness of utc_id ids.
;utc_id_suffix =
+; Encoding format for UUID v4 and v7. The default is base_16 (aka hex). Other
+; accepted formats are "base_36" and "string". Only v4 and v7 algorithm have
+; configurable formats, other algorithms only return base_16 encoded values.
+; Examples of the same UUID value encoded in all the supported formats:
+; "base_16" : "0199df3759297032b402c3e61fbbf88f"
+; "base_36" : "03eudcyamunnfraqdgmopx09b"
+; "string" : "0199df37-5929-7032-b402-c3e61fbbf88f"
Review Comment:
Good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add UUID v4 and v7. Make v7 the default. Optimize UUID generation [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#issuecomment-3411387127
Updated. Thanks for the suggestion to use a separate format section, it's looks much neater now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Optimize UUID generation and use UUID v7 by default [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on code in PR #5699:
URL: https://github.com/apache/couchdb/pull/5699#discussion_r2427438871
##########
src/couch/src/couch_uuids.erl:
##########
@@ -122,37 +126,35 @@ inc() ->
rand:uniform(16#ffd).
state() ->
- AlgoStr = config:get("uuids", "algorithm", "sequential"),
+ AlgoStr = config_algorithm(),
case couch_util:to_existing_atom(AlgoStr) of
random ->
random;
utc_random ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
{utc_random, ClockSeq};
utc_id ->
- ClockSeq = micros_since_epoch(os:timestamp()),
+ ClockSeq = micros_since_epoch(),
UtcIdSuffix = config:get("uuids", "utc_id_suffix", ""),
{utc_id, UtcIdSuffix, ClockSeq};
sequential ->
{sequential, new_prefix(), inc()};
uuid_v7 ->
uuid_v7;
+ uuid_v7_hex ->
Review Comment:
I saw that section of the RFC but these UUID's will _not_ be stored as 128-bit binary values anywhere in couchdb; that section does not apply to document id strings. It _would_ make sense if we were populating a database column with such values, which is the clear intention of ยง6.13.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org