You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "pvary (via GitHub)" <gi...@apache.org> on 2026/05/13 09:25:20 UTC
[PR] Flink: Filter CatalogTable properties [iceberg]
pvary opened a new pull request, #16313:
URL: https://github.com/apache/iceberg/pull/16313
Filter out `CatalogTable` properties to contain only relevant info
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "talatuyarer (via GitHub)" <gi...@apache.org>.
talatuyarer commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3240031718
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
+ catalogProps.entrySet().stream()
+ .filter(e -> !GlobalConfiguration.isSensitive(e.getKey()))
+ .collect(ImmutableMap.toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
Review Comment:
@pvary The `isSensitive` list doesn't cover all Iceberg configurations. For example, the `credential` keyword used for REST catalog OAuth2 client credentials or S3 access key was only added to Flink's list in a later release than the one we're using. See [https://github.com/apache/flink/blob/release-2.1.1/flink-core/src/main/java/org/apache/flink/configuration/GlobalConfiguration.java#L49](https://github.com/apache/flink/blob/release-2.1.1/flink-core/src/main/java/org/apache/flink/configuration/GlobalConfiguration.java#L49)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "talatuyarer (via GitHub)" <gi...@apache.org>.
talatuyarer commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3240031718
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
+ catalogProps.entrySet().stream()
+ .filter(e -> !GlobalConfiguration.isSensitive(e.getKey()))
+ .collect(ImmutableMap.toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
Review Comment:
@pvary The `isSensitive` list doesn't cover all Iceberg configurations. For example, the `credential` keyword used for REST catalog OAuth2 client credentials or S3 access key was only added to Flink's list in a later release than the one we're using. See [https://github.com/apache/flink/blob/release-2.2.1/flink-core/src/main/java/org/apache/flink/configuration/GlobalConfiguration.java#L49](https://github.com/apache/flink/blob/release-2.2.1/flink-core/src/main/java/org/apache/flink/configuration/GlobalConfiguration.java#L49)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3239495854
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
+ catalogProps.entrySet().stream()
+ .filter(e -> !GlobalConfiguration.isSensitive(e.getKey()))
+ .collect(ImmutableMap.toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
Review Comment:
`ConfigurationUtils.hideSensitiveValues` is even more simple.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#issuecomment-4760474887
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#issuecomment-4700267063
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3240178457
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
+ catalogProps.entrySet().stream()
+ .filter(e -> !GlobalConfiguration.isSensitive(e.getKey()))
+ .collect(ImmutableMap.toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
Review Comment:
This is true for `ConfigurationUtils.hideSensitiveValues` also...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "swapna267 (via GitHub)" <gi...@apache.org>.
swapna267 commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3237110267
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
Review Comment:
If this property is needed in order to create Iceberg catalog, then filtering it here breaks CREATE TABLE LIKE functionality.
https://github.com/apache/iceberg/blob/main/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkDynamicTableFactory.java#L190-L204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3240175787
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
+ catalogProps.entrySet().stream()
+ .filter(e -> !GlobalConfiguration.isSensitive(e.getKey()))
+ .collect(ImmutableMap.toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
Review Comment:
On master which will be 2.4 later I've added. https://github.com/apache/flink/blob/6c60f4fee2a8da6990738dc0c5b479f7f2accba8/flink-core/src/main/java/org/apache/flink/configuration/GlobalConfiguration.java#L276
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #16313:
URL: https://github.com/apache/iceberg/pull/16313#discussion_r3235779719
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:
##########
@@ -111,7 +112,10 @@ public FlinkCatalog(
long cacheExpirationIntervalMs) {
super(catalogName, defaultDatabase);
this.catalogLoader = catalogLoader;
- this.catalogProps = catalogProps;
+ this.catalogProps =
+ catalogProps.entrySet().stream()
+ .filter(e -> !GlobalConfiguration.isSensitive(e.getKey()))
+ .collect(ImmutableMap.toImmutableMap(Map.Entry::getKey, Map.Entry::getValue));
Review Comment:
This seems like a reasonable short-term fix to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Filter CatalogTable properties [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #16313: Flink: Filter CatalogTable properties
URL: https://github.com/apache/iceberg/pull/16313
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org