You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "sundapeng (via GitHub)" <gi...@apache.org> on 2026/01/22 03:38:52 UTC
[PR] add new signer for dlf [paimon]
sundapeng opened a new pull request, #7100:
URL: https://github.com/apache/paimon/pull/7100
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #xxx
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #7100:
URL: https://github.com/apache/paimon/pull/7100#discussion_r2739971197
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
Review Comment:
If automatic derivation works well, is it necessary for us to expose this configuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #7100:
URL: https://github.com/apache/paimon/pull/7100#issuecomment-3816157976
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #7100:
URL: https://github.com/apache/paimon/pull/7100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "sundapeng (via GitHub)" <gi...@apache.org>.
sundapeng commented on code in PR #7100:
URL: https://github.com/apache/paimon/pull/7100#discussion_r2740188501
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
Review Comment:
Document updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "sundapeng (via GitHub)" <gi...@apache.org>.
sundapeng commented on code in PR #7100:
URL: https://github.com/apache/paimon/pull/7100#discussion_r2739999299
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
Review Comment:
All known scenarios don't require configuring it, but it's hard to guarantee the OpenAPI infrastructure won't change in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "sundapeng (via GitHub)" <gi...@apache.org>.
sundapeng commented on code in PR #7100:
URL: https://github.com/apache/paimon/pull/7100#discussion_r2739893098
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
Review Comment:
> DLF Default Signing (Recommended)
Hi @JingsongLi , Here's a background: Each endpoint URI can only use the specific signer, so introduce the auto-selection:
- For domain names containing "dlfnext", the OpenAPI signing is selected by default.
- For other domain names, the default signing is selected by default.
For dlfnext endpoint, default signer is not work. "DLF Default Signing is Recommended" may be misleading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #7100:
URL: https://github.com/apache/paimon/pull/7100#discussion_r2740004461
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
Review Comment:
It is OK to introduce it, it is not necessary to document it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [rest] Add pluggable signer architecture for REST API authentication [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #7100:
URL: https://github.com/apache/paimon/pull/7100#discussion_r2739646095
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
+
+By default, Paimon automatically selects the appropriate signing algorithm based on the endpoint URI:
+
+- **DLF endpoints** (e.g., `cn-hangzhou-vpc.dlf.aliyuncs.com`): Automatically uses `dlf-default`
+ (backward compatible). Recommended for VPC environments with better performance.
+- **OpenAPI endpoints** (e.g., `dlfnext.cn-hangzhou.aliyuncs.com`): Automatically uses
+ `dlf-openapi` for DlfNext/2026-01-18 OpenAPI. Supports public network access through Alibaba Cloud API infrastructure
+ for special scenarios.
+
+```sql
+CREATE CATALOG `paimon-rest-catalog`
+WITH (
+ 'type' = 'paimon',
+ 'uri' = 'https://dlfnext.cn-hangzhou.aliyuncs.com', -- Auto-detected as dlf-openapi
+ 'metastore' = 'rest',
+ 'warehouse' = 'my_instance_name',
+ 'token.provider' = 'dlf',
+ 'dlf.access-key-id'='<access-key-id>',
+ 'dlf.access-key-secret'='<access-key-secret>'
+ -- 'dlf.signing-algorithm' is not set, will be auto-detected
+);
+```
+
+### Explicit Configuration
+
Review Comment:
Explain why this is needed? Public network support? Cross-VPC support?
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
+
+By default, Paimon automatically selects the appropriate signing algorithm based on the endpoint URI:
+
+- **DLF endpoints** (e.g., `cn-hangzhou-vpc.dlf.aliyuncs.com`): Automatically uses `dlf-default`
+ (backward compatible). Recommended for VPC environments with better performance.
+- **OpenAPI endpoints** (e.g., `dlfnext.cn-hangzhou.aliyuncs.com`): Automatically uses
+ `dlf-openapi` for DlfNext/2026-01-18 OpenAPI. Supports public network access through Alibaba Cloud API infrastructure
+ for special scenarios.
+
+```sql
+CREATE CATALOG `paimon-rest-catalog`
+WITH (
+ 'type' = 'paimon',
+ 'uri' = 'https://dlfnext.cn-hangzhou.aliyuncs.com', -- Auto-detected as dlf-openapi
+ 'metastore' = 'rest',
+ 'warehouse' = 'my_instance_name',
+ 'token.provider' = 'dlf',
+ 'dlf.access-key-id'='<access-key-id>',
+ 'dlf.access-key-secret'='<access-key-secret>'
+ -- 'dlf.signing-algorithm' is not set, will be auto-detected
+);
+```
+
+### Explicit Configuration
Review Comment:
DLF OpenAPI Signing
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
+
Review Comment:
Can you elaborate on the benefits of DLF-Default, such as high performance and low latency?
Also the downside, only work for VPC?
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
Review Comment:
DLF Default Signing (Recommended)
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
+
+By default, Paimon automatically selects the appropriate signing algorithm based on the endpoint URI:
+
+- **DLF endpoints** (e.g., `cn-hangzhou-vpc.dlf.aliyuncs.com`): Automatically uses `dlf-default`
+ (backward compatible). Recommended for VPC environments with better performance.
+- **OpenAPI endpoints** (e.g., `dlfnext.cn-hangzhou.aliyuncs.com`): Automatically uses
+ `dlf-openapi` for DlfNext/2026-01-18 OpenAPI. Supports public network access through Alibaba Cloud API infrastructure
+ for special scenarios.
+
+```sql
+CREATE CATALOG `paimon-rest-catalog`
+WITH (
+ 'type' = 'paimon',
+ 'uri' = 'https://dlfnext.cn-hangzhou.aliyuncs.com', -- Auto-detected as dlf-openapi
+ 'metastore' = 'rest',
+ 'warehouse' = 'my_instance_name',
+ 'token.provider' = 'dlf',
+ 'dlf.access-key-id'='<access-key-id>',
+ 'dlf.access-key-secret'='<access-key-secret>'
+ -- 'dlf.signing-algorithm' is not set, will be auto-detected
Review Comment:
remove this, by default is `'dlf-default'`.
##########
paimon-api/src/main/java/org/apache/paimon/rest/RESTCatalogOptions.java:
##########
@@ -104,6 +104,15 @@ public class RESTCatalogOptions {
.noDefaultValue()
.withDescription("REST Catalog DLF OSS endpoint.");
+ public static final ConfigOption<String> DLF_SIGNING_ALGORITHM =
+ ConfigOptions.key("dlf.signing-algorithm")
+ .stringType()
+ .defaultValue("dlf-default")
Review Comment:
Just `default` and `openapi`?
##########
docs/content/concepts/rest/dlf.md:
##########
@@ -115,3 +115,59 @@ WITH (
-- 'dlf.token-ecs-role-name' = 'my_ecs_role_name'
);
```
+
+## Signing Algorithm Configuration
+
+Paimon supports multiple signing algorithms for DLF authentication. You can configure the signing algorithm explicitly,
+or let Paimon automatically select it based on the endpoint host.
+
+### Automatic Selection (Recommended)
+
+By default, Paimon automatically selects the appropriate signing algorithm based on the endpoint URI:
+
+- **DLF endpoints** (e.g., `cn-hangzhou-vpc.dlf.aliyuncs.com`): Automatically uses `dlf-default`
+ (backward compatible). Recommended for VPC environments with better performance.
+- **OpenAPI endpoints** (e.g., `dlfnext.cn-hangzhou.aliyuncs.com`): Automatically uses
+ `dlf-openapi` for DlfNext/2026-01-18 OpenAPI. Supports public network access through Alibaba Cloud API infrastructure
+ for special scenarios.
+
+```sql
+CREATE CATALOG `paimon-rest-catalog`
+WITH (
+ 'type' = 'paimon',
+ 'uri' = 'https://dlfnext.cn-hangzhou.aliyuncs.com', -- Auto-detected as dlf-openapi
+ 'metastore' = 'rest',
+ 'warehouse' = 'my_instance_name',
+ 'token.provider' = 'dlf',
+ 'dlf.access-key-id'='<access-key-id>',
+ 'dlf.access-key-secret'='<access-key-secret>'
+ -- 'dlf.signing-algorithm' is not set, will be auto-detected
+);
+```
+
+### Explicit Configuration
+
+You can explicitly specify the signing algorithm:
+
+```sql
+CREATE CATALOG `paimon-rest-catalog`
+WITH (
+ 'type' = 'paimon',
+ 'uri' = '<catalog server url>',
+ 'metastore' = 'rest',
+ 'warehouse' = 'my_instance_name',
+ 'token.provider' = 'dlf',
+ 'dlf.access-key-id'='<access-key-id>',
+ 'dlf.access-key-secret'='<access-key-secret>',
+ 'dlf.signing-algorithm' = 'dlf-default' -- or 'dlf-openapi'
Review Comment:
'dlf.signing-algorithm' = 'openapi'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org