You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "nastra (via GitHub)" <gi...@apache.org> on 2025/10/30 09:26:54 UTC
[PR] OpenAPI: Make namespace separator configurable by server [iceberg]
nastra opened a new pull request, #14448:
URL: https://github.com/apache/iceberg/pull/14448
The [REST spec](https://github.com/apache/iceberg/blob/6319712b612b724fedbc5bed41942ac3426ffe48/open-api/rest-catalog-open-api.yaml#L225) currently uses `%1F` as the UTF-8 encoded namespace separator for multi-part namespaces.
This causes [issues](https://github.com/apache/iceberg/issues/10338), since it's a [control character](https://www.compart.com/en/unicode/category/Cc) and the [Servlet spec](https://jakarta.ee/specifications/servlet/6.0/jakarta-servlet-spec-6.0.html#uri-path-canonicalization) can reject such characters.
This PR makes the hard-coded namespace separator configurable by giving servers an option to send an **optional** namespace separator instead of `%1F`. The configuration part is entirely optional for REST server implementers and there's no behavioral change for existing installations.
The actual implementation for this can be seen at https://github.com/apache/iceberg/pull/10877
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2477285718
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
Review Comment:
how about we name this conf `nested-namespace-separator` ?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
+ To be compatible with older clients, servers must use both the advertised separator and `0x1F` as valid separators when decoding namespaces.
Review Comment:
[doubt] I see is then, there no way then `0x1F` can be surely considered as a non-seperator ? even when the client supports adhering to advertised seperator and is doing that ?
Or may be if servers wanna support that they can capture `X-Iceberg-Version` header and see if the client supports this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#issuecomment-3609371324
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2481690390
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
+ To be compatible with older clients, servers must use both the advertised separator and `0x1F` as valid separators when decoding namespaces.
Review Comment:
Agree, my question was if the server knows the client is new (for example client send the iceberg sdk version to server as part of header) can they choose not to treat `0x1F` as seperator ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2477502903
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
+ To be compatible with older clients, servers must use both the advertised separator and `0x1F` as valid separators when decoding namespaces.
Review Comment:
sorry I don't think I'm following. Could you rephrase your question please?
The reason why the server must use both separators is because an older client will always use `0x1F` while a newer client will use the advertised separator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2477508498
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
Review Comment:
I'm not too strong on the naming and either is fine, so let's see what others think this should be named
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra merged PR #14448:
URL: https://github.com/apache/iceberg/pull/14448
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#issuecomment-3627556975
thanks @kamcheungting-db, @amogh-jahagirdar, @singhpk234 for reviewing!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2477508498
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
Review Comment:
I'm not too strong on the naming, so let's see what others think this should be named
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2481690390
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
+ To be compatible with older clients, servers must use both the advertised separator and `0x1F` as valid separators when decoding namespaces.
Review Comment:
Agree, my question was if the server knows the client is new i.e respects the configurable seperator (for example client send the iceberg sdk version to server as part of header) can they choose not to treat `0x1F` as seperator ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2481928848
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
+ To be compatible with older clients, servers must use both the advertised separator and `0x1F` as valid separators when decoding namespaces.
Review Comment:
while this would technically be possible with the java client, it would be more challenging with other client implementations as you'd need to keep track of each client version where you know that this is a newer client. Hence I think we should always treat `0x1F` as the legacy separator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] OpenAPI: Make namespace separator configurable by server [iceberg]
Posted by "kamcheungting-db (via GitHub)" <gi...@apache.org>.
kamcheungting-db commented on code in PR #14448:
URL: https://github.com/apache/iceberg/pull/14448#discussion_r2485418324
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -262,7 +262,9 @@ paths:
An optional namespace, underneath which to list namespaces.
If not provided, all top-level namespaces should be listed.
For backward compatibility, empty string is treated as absent for now.
- If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte.
+ If parent is a multipart namespace, the parts must be separated by the namespace separator as
+ indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte.
Review Comment:
could you provide an example inside the comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org