You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "dataengineervishal (via GitHub)" <gi...@apache.org> on 2026/03/18 22:19:44 UTC
[PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
dataengineervishal opened a new pull request, #27784:
URL: https://github.com/apache/flink/pull/27784
This PR depends on https://issues.apache.org/jira/browse/FLINK-38815
## What is the purpose of the change
This pull request fixes a security issue where sensitive values in the Pekko RPC configuration were logged in plain text.
The Pekko configuration is based on Typesafe Config and was directly logged, bypassing Flink’s existing masking logic. This change ensures that sensitive values (such as passwords and secrets) are masked before being logged.
## Brief change log
- Converted Pekko Config into a flat Map<String, String> using entrySet()
- Reused ConfigurationUtils.hideSensitiveValues() to mask sensitive values
- Updated debug logging to print masked configuration instead of raw config
## Verifying this change
This change is already covered by existing tests.
Additionally, the fix was manually verified by enabling debug logging and confirming that sensitive fields such as key-password and trust-store-password are masked (replaced with ******) in the logs.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4164537557
This seems to be a CI-related issue. The failure is due to a dependency download error during spotless-check:
`Premature end of Content-Length delimited message body while fetching scala-library-2.13.8`
I’ve already run `mvn spotless:apply`, and the code formatting is aligned.
I also noticed a similar failure in another pipeline that ran around the same time, which suggests a transient CI/network issue.
Pushing an empty commit to retrigger the CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "ferenc-csaky (via GitHub)" <gi...@apache.org>.
ferenc-csaky commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4154648737
@dataengineervishal can you also open backport PRs against at least `release-1.20`, `release-2.1`, and `release-2.2` branches? For those it's enough if you reference this PR in the description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "ferenc-csaky (via GitHub)" <gi...@apache.org>.
ferenc-csaky merged PR #27784:
URL: https://github.com/apache/flink/pull/27784
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4154555159
Hi @ferenc-csaky,
Would you mind helping take a look?
Thank you very much in advance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "ferenc-csaky (via GitHub)" <gi...@apache.org>.
ferenc-csaky commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4162311483
@dataengineervishal pls. rerun `mvn spotless:apply` to fix CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4167945770
Thanks a lot for the quick review and merge, @ferenc-csaky !
I see the backport PRs have been created as well—happy to take a look and address anything needed there.
Really appreciate your support!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4143362020
@davidradl Could you please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4134110112
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r2958318712
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -251,7 +268,8 @@ public static ActorSystem startLocalActorSystem(
*/
private static ActorSystem startActorSystem(
Config config, String actorSystemName, Logger logger) {
- logger.debug("Using pekko configuration\n {}", config);
+ Map<String, String> masked = ConfigurationUtils.hideSensitiveValues(toMap(config));
Review Comment:
Good catch, that makes sense. I’ve wrapped it with isDebugEnabled() so the masking and map conversion only happen when debug logging is enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4085958675
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74ca658bf962954da5983b58c1ab1565d8bfb0c0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "74ca658bf962954da5983b58c1ab1565d8bfb0c0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74ca658bf962954da5983b58c1ab1565d8bfb0c0 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "mukul-8 (via GitHub)" <gi...@apache.org>.
mukul-8 commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r2957796898
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -251,7 +268,8 @@ public static ActorSystem startLocalActorSystem(
*/
private static ActorSystem startActorSystem(
Config config, String actorSystemName, Logger logger) {
- logger.debug("Using pekko configuration\n {}", config);
+ Map<String, String> masked = ConfigurationUtils.hideSensitiveValues(toMap(config));
Review Comment:
Converting the entire config to a map and masking values is an expensive operation that runs on every actor system startup, even when debug logging is disabled. Consider wrapping this in an isDebugEnabled() check:
if (logger.isDebugEnabled()) {
Map<String, String> masked = ConfigurationUtils.hideSensitiveValues(toMap(config));
logger.debug("Using pekko configuration\n {}", masked);
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "mukul-8 (via GitHub)" <gi...@apache.org>.
mukul-8 commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r2957796898
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -251,7 +268,8 @@ public static ActorSystem startLocalActorSystem(
*/
private static ActorSystem startActorSystem(
Config config, String actorSystemName, Logger logger) {
- logger.debug("Using pekko configuration\n {}", config);
+ Map<String, String> masked = ConfigurationUtils.hideSensitiveValues(toMap(config));
Review Comment:
Converting the entire config to a map and masking values is an expensive operation that runs on every actor
system startup, even when debug logging is disabled. Consider wrapping this in an isDebugEnabled() check:
if (logger.isDebugEnabled()) {
Map<String, String> masked = ConfigurationUtils.hideSensitiveValues(toMap(config));
logger.debug("Using pekko configuration\n {}", masked);
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r2983775638
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -241,6 +244,20 @@ public static ActorSystem startLocalActorSystem(
}
}
+ /**
+ * Converts the given Pekko {@link Config} into a flattened {@link Map}.
+ *
+ * @param config The Pekko configuration
+ * @return A map of configuration keys to string values
+ */
+ private static Map<String, String> toMap(Config config) {
Review Comment:
Good suggestion, thanks! I've moved the masking logic into the method to encapsulate it and added a unit test to verify that sensitive values are properly masked.
Updated the PR, could you please have a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "Samrat002 (via GitHub)" <gi...@apache.org>.
Samrat002 commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r2964180102
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -251,7 +268,10 @@ public static ActorSystem startLocalActorSystem(
*/
private static ActorSystem startActorSystem(
Config config, String actorSystemName, Logger logger) {
- logger.debug("Using pekko configuration\n {}", config);
+ if (logger.isDebugEnabled()) {
+ Map<String, String> masked = ConfigurationUtils.hideSensitiveValues(toMap(config));
+ logger.debug("Using pekko configuration\n {}", masked);
Review Comment:
NIT:
```suggestion
logger.debug("Using pekko configuration\n {}", ConfigurationUtils.hideSensitiveValues(toMap(config)));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r3015367739
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -241,6 +244,21 @@ public static ActorSystem startLocalActorSystem(
}
}
+ /**
+ * Converts the given Pekko {@link Config} into a flattened {@link Map}.
+ *
+ * @param config The Pekko configuration
+ * @return A map of configuration keys to string values
+ */
+ static Map<String, String> toMaskedMap(Config config) {
Review Comment:
Sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "davidradl (via GitHub)" <gi...@apache.org>.
davidradl commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r2982204869
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -241,6 +244,20 @@ public static ActorSystem startLocalActorSystem(
}
}
+ /**
+ * Converts the given Pekko {@link Config} into a flattened {@link Map}.
+ *
+ * @param config The Pekko configuration
+ * @return A map of configuration keys to string values
+ */
+ private static Map<String, String> toMap(Config config) {
Review Comment:
I wonder if we could have the ConfigurationUtils.hideSensitiveValues in this method , then you could add a unit test to ensure the config comes out as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4090051503
@XComp Can you please take a look and approve this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "dataengineervishal (via GitHub)" <gi...@apache.org>.
dataengineervishal commented on PR #27784:
URL: https://github.com/apache/flink/pull/27784#issuecomment-4164393978
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38815] Mask sensitive values in Pekko configuration logs [flink]
Posted by "ferenc-csaky (via GitHub)" <gi...@apache.org>.
ferenc-csaky commented on code in PR #27784:
URL: https://github.com/apache/flink/pull/27784#discussion_r3009439097
##########
flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/ActorSystemBootstrapTools.java:
##########
@@ -241,6 +244,21 @@ public static ActorSystem startLocalActorSystem(
}
}
+ /**
+ * Converts the given Pekko {@link Config} into a flattened {@link Map}.
+ *
+ * @param config The Pekko configuration
+ * @return A map of configuration keys to string values
+ */
+ static Map<String, String> toMaskedMap(Config config) {
Review Comment:
Let's annotate this with `@VisibleForTesting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org