You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Shekharrajak (via GitHub)" <gi...@apache.org> on 2026/07/21 17:07:20 UTC
[PR] [FLINK-38262][table] show create connection operation [flink]
Shekharrajak opened a new pull request, #28791:
URL: https://github.com/apache/flink/pull/28791
## What is the purpose of the change
Adds resolved connection lookup and SHOW CREATE CONNECTION support, formatting connection DDL while filtering internal/sensitive options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r4006793004
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/catalog/CatalogManager.java:
##########
@@ -1924,6 +1924,30 @@ public Optional<CatalogConnection> getConnection(ObjectIdentifier objectIdentifi
}
}
+ /** Get a connection from the catalog with contextual metadata. */
+ public Optional<ContextResolvedConnection> getResolvedConnection(
Review Comment:
Updated https://github.com/apache/flink/pull/28791/commits/b415a7e644dccc9c5f834e4a39c7b48798c9ef16
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on PR #28791:
URL: https://github.com/apache/flink/pull/28791#issuecomment-5667480946
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r4007441476
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/operations/SqlConnectionOperationConverterTest.java:
##########
@@ -121,4 +123,50 @@ void testCreateConnectionWithEmptyOptionsRejected() {
.isInstanceOf(SqlValidateException.class)
.hasMessageContaining("Connection property list can not be empty.");
}
+
+ @Test
+ void testShowCreateConnection() {
Review Comment:
Test added https://github.com/apache/flink/pull/28791/changes/798b2c9e807c91fc53c3411e8a72d1bef9e74d4d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r3655006663
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -64,6 +65,7 @@ public class ShowCreateUtil {
private static final DateTimeFormatter TIMESTAMP_FORMATTER =
DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss");
private static final String PRINT_INDENT = " ";
+ private static final String CONNECTION_SECRET_REFERENCE_KEY = "__flink.encrypted-secret-key__";
Review Comment:
Can you elaborate what this const is? Maybe add a one line comment why we filtering it out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on PR #28791:
URL: https://github.com/apache/flink/pull/28791#issuecomment-5340723568
> if you change ShowCreateUtil then there should be tests in ShowCreateUtilTests with pure sql
Added tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r3655009991
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/catalog/CatalogManager.java:
##########
@@ -1910,6 +1910,36 @@ public Optional<CatalogConnection> getConnection(ObjectIdentifier objectIdentifi
}
}
+ /**
+ * Get a connection from the catalog with contextual metadata.
+ *
+ * @param objectIdentifier The fully qualified path of the connection.
+ * @return The requested connection wrapped in Optional.
+ */
Review Comment:
The param and return value is clear enough
```suggestion
/**
* Get a connection from the catalog with contextual metadata.
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r3812088744
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -64,6 +65,7 @@ public class ShowCreateUtil {
private static final DateTimeFormatter TIMESTAMP_FORMATTER =
DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss");
private static final String PRINT_INDENT = " ";
+ private static final String CONNECTION_SECRET_REFERENCE_KEY = "__flink.encrypted-secret-key__";
Review Comment:
Added comment https://github.com/apache/flink/pull/28791/changes/d498c26a0fcd45c12053997d31a0c22ca32fa7fb
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -349,6 +374,18 @@ static Optional<String> extractComment(ResolvedCatalogModel model) {
: Optional.of(model.getComment());
}
+ static Optional<String> extractComment(CatalogConnection connection) {
+ return StringUtils.isEmpty(connection.getComment())
+ ? Optional.empty()
+ : Optional.of(connection.getComment());
+ }
+
+ static Map<String, String> withoutConnectionInternalOptions(Map<String, String> options) {
+ return options.entrySet().stream()
+ .filter(entry -> !CONNECTION_SECRET_REFERENCE_KEY.equals(entry.getKey()))
+ .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
+ }
Review Comment:
updated https://github.com/apache/flink/pull/28791/changes/d498c26a0fcd45c12053997d31a0c22ca32fa7fb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "lihaosky (via GitHub)" <gi...@apache.org>.
lihaosky merged PR #28791:
URL: https://github.com/apache/flink/pull/28791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r3655015328
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/catalog/CatalogManager.java:
##########
@@ -1910,6 +1910,36 @@ public Optional<CatalogConnection> getConnection(ObjectIdentifier objectIdentifi
}
}
+ /**
+ * Get a connection from the catalog with contextual metadata.
+ *
+ * @param objectIdentifier The fully qualified path of the connection.
+ * @return The requested connection wrapped in Optional.
+ */
+ public Optional<ContextResolvedConnection> getResolvedConnection(
+ ObjectIdentifier objectIdentifier) {
+ CatalogConnection temporaryConnection = temporaryConnections.get(objectIdentifier);
+ if (temporaryConnection != null) {
+ return Optional.of(
+ ContextResolvedConnection.of(objectIdentifier, temporaryConnection, true));
+ }
+
+ Optional<Catalog> catalog = getCatalog(objectIdentifier.getCatalogName());
+ if (catalog.isPresent()) {
+ try {
+ return Optional.of(
+ ContextResolvedConnection.of(
+ objectIdentifier,
+ catalog.get().getConnection(objectIdentifier.toObjectPath()),
+ false));
+ } catch (ConnectionNotExistException | UnsupportedOperationException e) {
+ return Optional.empty();
+ }
+ } else {
+ return Optional.empty();
+ }
Review Comment:
`else` not needed
```suggestion
}
return Optional.empty();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on PR #28791:
URL: https://github.com/apache/flink/pull/28791#issuecomment-5340929496
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "lihaosky (via GitHub)" <gi...@apache.org>.
lihaosky commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r3993932385
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -64,6 +65,8 @@ public class ShowCreateUtil {
private static final DateTimeFormatter TIMESTAMP_FORMATTER =
DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss");
private static final String PRINT_INDENT = " ";
+ // Internal secret-store metadata must not be emitted in recreated connection SQL.
+ private static final String CONNECTION_SECRET_REFERENCE_KEY = "__flink.encrypted-secret-key__";
Review Comment:
**Hardcoded secret-reference key.**
This literal (also duplicated in `ShowCreateUtilTest` and `CreateConnectionITCase`) is the same value as `DefaultConnectionFactory.SECRET_REFERENCE_KEY`. If the factory's key ever changes, this filter silently stops matching and the reserved key leaks into output. Consider making the factory constant `public` and referencing it here and in the tests.
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/catalog/CatalogManager.java:
##########
@@ -1924,6 +1924,30 @@ public Optional<CatalogConnection> getConnection(ObjectIdentifier objectIdentifi
}
}
+ /** Get a connection from the catalog with contextual metadata. */
+ public Optional<ContextResolvedConnection> getResolvedConnection(
Review Comment:
**`getResolvedConnection` duplicates `getConnection`.**
This re-implements `getConnection`'s temporary-then-catalog lookup (including the `ConnectionNotExistException` / `UnsupportedOperationException` handling) verbatim, only adding the `ContextResolvedConnection` wrapper and temporary flag. It could delegate to avoid keeping two copies in sync:
```java
return getConnection(objectIdentifier)
.map(c -> ContextResolvedConnection.of(
objectIdentifier, c, temporaryConnections.containsKey(objectIdentifier)));
```
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -98,6 +101,29 @@ public static String buildShowCreateModelRow(
return sb.toString();
}
+ public static String buildShowCreateConnectionRow(
+ CatalogConnection connection,
+ ObjectIdentifier connectionIdentifier,
+ boolean isTemporary,
+ List<String> additionalSensitiveKeys) {
+ StringBuilder sb =
+ new StringBuilder()
+ .append(
+ buildCreateFormattedPrefix(
+ "CONNECTION",
+ isTemporary,
+ connectionIdentifier,
+ false,
+ false));
+ extractComment(connection).ifPresent(c -> sb.append(formatComment(c)).append("\n"));
+ extractFormattedOptions(
+ withoutConnectionInternalOptions(connection.getOptions()),
+ PRINT_INDENT,
+ additionalSensitiveKeys)
+ .ifPresent(v -> sb.append("WITH (\n").append(v).append("\n)\n"));
Review Comment:
**Non-re-runnable DDL when the only option is the filtered secret reference.**
When a connection's stored options consist solely of the internal secret reference (e.g. `CREATE CONNECTION c WITH ('password'='x')` — `DefaultConnectionFactory` extracts the password to the secret store, leaving `{__flink.encrypted-secret-key__: id}`), `withoutConnectionInternalOptions` strips that key and `extractFormattedOptions` returns empty, so no `WITH` clause is appended. The output is then ``CREATE CONNECTION `cat`.`db`.`c` `` — but the `SqlCreateConnection` grammar makes `WITH` mandatory and rejects an empty property list ("Connection property list can not be empty."), so the `SHOW CREATE` output can't be re-executed. Worth handling this edge case (e.g. retain the credential keys masked, or emit a placeholder).
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/operations/SqlConnectionOperationConverterTest.java:
##########
@@ -121,4 +123,50 @@ void testCreateConnectionWithEmptyOptionsRejected() {
.isInstanceOf(SqlValidateException.class)
.hasMessageContaining("Connection property list can not be empty.");
}
+
+ @Test
+ void testShowCreateConnection() {
Review Comment:
**Permanent-connection path untested.**
All `SHOW CREATE CONNECTION` happy-path tests (here and in `CreateConnectionITCase`) use temporary connections, so the permanent branch of `getResolvedConnection` (`catalog.getConnection(...)`, `isTemporary() == false`) and its `CREATE CONNECTION` (no `TEMPORARY`) output are never exercised. This is harder to set up in the current harness (no `WritableSecretStore`, and `GenericInMemoryCatalog` doesn't implement `getConnection`), so a fake connection-supporting catalog fixture would be one way to cover it. Non-blocking, but worth a follow-up.
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -349,6 +375,19 @@ static Optional<String> extractComment(ResolvedCatalogModel model) {
: Optional.of(model.getComment());
}
+ static Optional<String> extractComment(CatalogConnection connection) {
Review Comment:
**Third near-identical `extractComment` overload.**
This has the same body as the existing `extractComment(ResolvedCatalogBaseTable)` / `extractComment(ResolvedCatalogModel)` overloads. Consider folding them into one helper taking the comment string/supplier to avoid a growing set of copy-paste overloads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r4007447012
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -98,6 +101,29 @@ public static String buildShowCreateModelRow(
return sb.toString();
}
+ public static String buildShowCreateConnectionRow(
+ CatalogConnection connection,
+ ObjectIdentifier connectionIdentifier,
+ boolean isTemporary,
+ List<String> additionalSensitiveKeys) {
+ StringBuilder sb =
+ new StringBuilder()
+ .append(
+ buildCreateFormattedPrefix(
+ "CONNECTION",
+ isTemporary,
+ connectionIdentifier,
+ false,
+ false));
+ extractComment(connection).ifPresent(c -> sb.append(formatComment(c)).append("\n"));
+ extractFormattedOptions(
+ withoutConnectionInternalOptions(connection.getOptions()),
+ PRINT_INDENT,
+ additionalSensitiveKeys)
+ .ifPresent(v -> sb.append("WITH (\n").append(v).append("\n)\n"));
Review Comment:
https://github.com/apache/flink/pull/28791/commits/12b6e681dd6c4bd8a6364ddd6f625e19f61df971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "Shekharrajak (via GitHub)" <gi...@apache.org>.
Shekharrajak commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r4007444105
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -64,6 +65,8 @@ public class ShowCreateUtil {
private static final DateTimeFormatter TIMESTAMP_FORMATTER =
DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss");
private static final String PRINT_INDENT = " ";
+ // Internal secret-store metadata must not be emitted in recreated connection SQL.
+ private static final String CONNECTION_SECRET_REFERENCE_KEY = "__flink.encrypted-secret-key__";
Review Comment:
https://github.com/apache/flink/pull/28791/commits/e85171be248cb383702a7f790a337513785c598c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #28791:
URL: https://github.com/apache/flink/pull/28791#issuecomment-5036903800
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1683711a4c095d5d963d3800ed4fdef6429f7780",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1683711a4c095d5d963d3800ed4fdef6429f7780",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1683711a4c095d5d963d3800ed4fdef6429f7780 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38262][table] show create connection operation [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #28791:
URL: https://github.com/apache/flink/pull/28791#discussion_r3655003915
##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/ShowCreateUtil.java:
##########
@@ -349,6 +374,18 @@ static Optional<String> extractComment(ResolvedCatalogModel model) {
: Optional.of(model.getComment());
}
+ static Optional<String> extractComment(CatalogConnection connection) {
+ return StringUtils.isEmpty(connection.getComment())
+ ? Optional.empty()
+ : Optional.of(connection.getComment());
+ }
+
+ static Map<String, String> withoutConnectionInternalOptions(Map<String, String> options) {
+ return options.entrySet().stream()
+ .filter(entry -> !CONNECTION_SECRET_REFERENCE_KEY.equals(entry.getKey()))
+ .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
+ }
Review Comment:
These can be private
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org