You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "liyang1206 (via GitHub)" <gi...@apache.org> on 2025/08/17 12:56:54 UTC
[PR] [FLINK-38254][table] Add ITCase to verify sensitive table option reda… [flink]
liyang1206 opened a new pull request, #26916:
URL: https://github.com/apache/flink/pull/26916
## What is the purpose of the change
Add ITCase to for testing the sensitive information has been hidden when creating source and sink table
## Verifying this change
This change added tests and can be verified as follows:
- Added testSensitiveInfoMasking() to CatalogTableITCase
## Does this pull request potentially affect one of the following parts:
Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? (no)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38254][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #26916:
URL: https://github.com/apache/flink/pull/26916#issuecomment-3194373696
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7f1ff469f5bb34eed114113d1a879e621f6106fc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7f1ff469f5bb34eed114113d1a879e621f6106fc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7f1ff469f5bb34eed114113d1a879e621f6106fc UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281512773
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,101 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'secret1',
+ | 'secret' = 'secret2',
+ | 'fs.azure.account.key' = 'secret3',
+ | 'apikey' = 'secret4',
+ | 'api-key' = 'secret5',
+ | 'auth-params' = 'secret6',
+ | 'service-key' = 'secret7',
+ | 'token' = 'secret8',
+ | 'basic-auth' = 'secret9',
+ | 'jaas.config' = 'secret10',
+ | 'http-headers' = '{"Authorization": "secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
+ tableEnv.executeSql("select * from t1")
+ fail("Expected ValidationException for source table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify source table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
+ assert(!errorMsg.contains("secret2"))
+ assert(!errorMsg.contains("secret3"))
+ assert(!errorMsg.contains("secret4"))
+ assert(!errorMsg.contains("secret5"))
+ assert(!errorMsg.contains("secret6"))
+ assert(!errorMsg.contains("secret7"))
+ assert(!errorMsg.contains("secret8"))
+ assert(!errorMsg.contains("secret9"))
+ assert(!errorMsg.contains("secret10"))
+ assert(!errorMsg.contains("{\"Authorization\": \"secret11\"}"))
Review Comment:
I get it, now I have simplified this part. Thanks for your comment!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281855466
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,81 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'Secret1',
+ | 'secret' = 'Secret2',
+ | 'fs.azure.account.key' = 'Secret3',
+ | 'apikey' = 'Secret4',
+ | 'api-key' = 'Secret5',
+ | 'auth-params' = 'Secret6',
+ | 'service-key' = 'Secret7',
+ | 'token' = 'Secret8',
+ | 'basic-auth' = 'Secret9',
+ | 'jaas.config' = 'Secret10',
+ | 'http-headers' = '{"Authorization": "Secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
Review Comment:
Seems like the diff between the 2 try blocks the difference is the SQL statement and source/sink word. If that so we can just extract it into a function with 2 params, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281482519
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,101 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'secret1',
+ | 'secret' = 'secret2',
+ | 'fs.azure.account.key' = 'secret3',
+ | 'apikey' = 'secret4',
+ | 'api-key' = 'secret5',
+ | 'auth-params' = 'secret6',
+ | 'service-key' = 'secret7',
+ | 'token' = 'secret8',
+ | 'basic-auth' = 'secret9',
+ | 'jaas.config' = 'secret10',
+ | 'http-headers' = '{"Authorization": "secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
+ tableEnv.executeSql("select * from t1")
+ fail("Expected ValidationException for source table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify source table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
+ assert(!errorMsg.contains("secret2"))
+ assert(!errorMsg.contains("secret3"))
+ assert(!errorMsg.contains("secret4"))
+ assert(!errorMsg.contains("secret5"))
+ assert(!errorMsg.contains("secret6"))
+ assert(!errorMsg.contains("secret7"))
+ assert(!errorMsg.contains("secret8"))
+ assert(!errorMsg.contains("secret9"))
+ assert(!errorMsg.contains("secret10"))
+ assert(!errorMsg.contains("{\"Authorization\": \"secret11\"}"))
Review Comment:
Then we can use anything else on value side but such lengthy asserts are not helping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281472005
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,101 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'secret1',
+ | 'secret' = 'secret2',
+ | 'fs.azure.account.key' = 'secret3',
+ | 'apikey' = 'secret4',
+ | 'api-key' = 'secret5',
+ | 'auth-params' = 'secret6',
+ | 'service-key' = 'secret7',
+ | 'token' = 'secret8',
+ | 'basic-auth' = 'secret9',
+ | 'jaas.config' = 'secret10',
+ | 'http-headers' = '{"Authorization": "secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
+ tableEnv.executeSql("select * from t1")
+ fail("Expected ValidationException for source table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify source table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
+ assert(!errorMsg.contains("secret2"))
+ assert(!errorMsg.contains("secret3"))
+ assert(!errorMsg.contains("secret4"))
+ assert(!errorMsg.contains("secret5"))
+ assert(!errorMsg.contains("secret6"))
+ assert(!errorMsg.contains("secret7"))
+ assert(!errorMsg.contains("secret8"))
+ assert(!errorMsg.contains("secret9"))
+ assert(!errorMsg.contains("secret10"))
+ assert(!errorMsg.contains("{\"Authorization\": \"secret11\"}"))
Review Comment:
‘secret’ is already used as a key
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #26916:
URL: https://github.com/apache/flink/pull/26916#issuecomment-3196650768
Please kick the tests though otherwise we can't merge it. Maybe rebase to latest master helps or just flaky and restart needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281943796
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,81 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'Secret1',
+ | 'secret' = 'Secret2',
+ | 'fs.azure.account.key' = 'Secret3',
+ | 'apikey' = 'Secret4',
+ | 'api-key' = 'Secret5',
+ | 'auth-params' = 'Secret6',
+ | 'service-key' = 'Secret7',
+ | 'token' = 'Secret8',
+ | 'basic-auth' = 'Secret9',
+ | 'jaas.config' = 'Secret10',
+ | 'http-headers' = '{"Authorization": "Secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
Review Comment:
You are right! I can reduce redundant code by this way. So I extract them to a function
validateSensitiveInfoMasking(sql: String, tableType: String) PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on PR #26916:
URL: https://github.com/apache/flink/pull/26916#issuecomment-3199482555
pipeline as passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281459072
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,101 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'secret1',
+ | 'secret' = 'secret2',
+ | 'fs.azure.account.key' = 'secret3',
+ | 'apikey' = 'secret4',
+ | 'api-key' = 'secret5',
+ | 'auth-params' = 'secret6',
+ | 'service-key' = 'secret7',
+ | 'token' = 'secret8',
+ | 'basic-auth' = 'secret9',
+ | 'jaas.config' = 'secret10',
+ | 'http-headers' = '{"Authorization": "secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
+ tableEnv.executeSql("select * from t1")
+ fail("Expected ValidationException for source table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify source table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
+ assert(!errorMsg.contains("secret2"))
+ assert(!errorMsg.contains("secret3"))
+ assert(!errorMsg.contains("secret4"))
+ assert(!errorMsg.contains("secret5"))
+ assert(!errorMsg.contains("secret6"))
+ assert(!errorMsg.contains("secret7"))
+ assert(!errorMsg.contains("secret8"))
+ assert(!errorMsg.contains("secret9"))
+ assert(!errorMsg.contains("secret10"))
+ assert(!errorMsg.contains("{\"Authorization\": \"secret11\"}"))
Review Comment:
'secret' = 'xxxxx' is as expected, this test only guarantee the value of option is hidden.
'password' = 'secret1',
'secret' = 'secret2',
In this case, 'secret1', 'secret2', is hidden. And the secret you mentioned is a key, don't need to hide it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table][tests] Add ITCase to verify sensitive table option redaction [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi merged PR #26916:
URL: https://github.com/apache/flink/pull/26916
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281943796
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,81 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'Secret1',
+ | 'secret' = 'Secret2',
+ | 'fs.azure.account.key' = 'Secret3',
+ | 'apikey' = 'Secret4',
+ | 'api-key' = 'Secret5',
+ | 'auth-params' = 'Secret6',
+ | 'service-key' = 'Secret7',
+ | 'token' = 'Secret8',
+ | 'basic-auth' = 'Secret9',
+ | 'jaas.config' = 'Secret10',
+ | 'http-headers' = '{"Authorization": "Secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
Review Comment:
You are right! I can reduce redundant code by this way. So I extract them to a function
validateSensitiveInfoMasking(sql: String, tableType: String)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on PR #26916:
URL: https://github.com/apache/flink/pull/26916#issuecomment-3196733190
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "liyang1206 (via GitHub)" <gi...@apache.org>.
liyang1206 commented on PR #26916:
URL: https://github.com/apache/flink/pull/26916#issuecomment-3196929713
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38255][table] Add ITCase to verify sensitive table option reda… [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #26916:
URL: https://github.com/apache/flink/pull/26916#discussion_r2281433108
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,101 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'secret1',
+ | 'secret' = 'secret2',
+ | 'fs.azure.account.key' = 'secret3',
+ | 'apikey' = 'secret4',
+ | 'api-key' = 'secret5',
+ | 'auth-params' = 'secret6',
+ | 'service-key' = 'secret7',
+ | 'token' = 'secret8',
+ | 'basic-auth' = 'secret9',
+ | 'jaas.config' = 'secret10',
+ | 'http-headers' = '{"Authorization": "secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
+ tableEnv.executeSql("select * from t1")
+ fail("Expected ValidationException for source table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify source table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
+ assert(!errorMsg.contains("secret2"))
+ assert(!errorMsg.contains("secret3"))
+ assert(!errorMsg.contains("secret4"))
+ assert(!errorMsg.contains("secret5"))
+ assert(!errorMsg.contains("secret6"))
+ assert(!errorMsg.contains("secret7"))
+ assert(!errorMsg.contains("secret8"))
+ assert(!errorMsg.contains("secret9"))
+ assert(!errorMsg.contains("secret10"))
+ assert(!errorMsg.contains("{\"Authorization\": \"secret11\"}"))
+
+ assert(errorMsg.contains("'password'='******'"))
+ assert(errorMsg.contains("'secret'='******'"))
+ assert(errorMsg.contains("'fs.azure.account.key'='******'"))
+ assert(errorMsg.contains("'apikey'='******'"))
+ assert(errorMsg.contains("'api-key'='******'"))
+ assert(errorMsg.contains("'auth-params'='******'"))
+ assert(errorMsg.contains("'service-key'='******'"))
+ assert(errorMsg.contains("'token'='******'"))
+ assert(errorMsg.contains("'basic-auth'='******'"))
+ assert(errorMsg.contains("'jaas.config'='******'"))
+ assert(errorMsg.contains("'http-headers'='******'"))
+ }
+
+ // Test sink table validation
+ try {
+ tableEnv.executeSql("insert into t1 select 1, 'test', 1")
+ fail("Expected ValidationException for sink table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify sink table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
Review Comment:
Same here
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/catalog/CatalogTableITCase.scala:
##########
@@ -1375,6 +1375,101 @@ class CatalogTableITCase(isStreamingMode: Boolean) extends TableITCaseBase {
"+I[bob]",
tableEnv.executeSql("select lowerUdf(a) from t1").collect().next().toString)
}
+
+ @TestTemplate
+ def testSensitiveInfoMasking(): Unit = {
+ // Create tables with sensitive info
+ val tableDDL =
+ """
+ |create table t1(
+ | a bigint,
+ | b varchar,
+ | c int
+ |) with (
+ | 'connector' = 'values',
+ | 'password' = 'secret1',
+ | 'secret' = 'secret2',
+ | 'fs.azure.account.key' = 'secret3',
+ | 'apikey' = 'secret4',
+ | 'api-key' = 'secret5',
+ | 'auth-params' = 'secret6',
+ | 'service-key' = 'secret7',
+ | 'token' = 'secret8',
+ | 'basic-auth' = 'secret9',
+ | 'jaas.config' = 'secret10',
+ | 'http-headers' = '{"Authorization": "secret11"}'
+ |)
+ """.stripMargin
+
+ tableEnv.executeSql(tableDDL)
+
+ // Test source table validation
+ try {
+ tableEnv.executeSql("select * from t1")
+ fail("Expected ValidationException for source table")
+ } catch {
+ case e: ValidationException =>
+ val errorMsg = e.getMessage
+ // Verify source table sensitive info is masked
+ assert(!errorMsg.contains("secret1"))
+ assert(!errorMsg.contains("secret2"))
+ assert(!errorMsg.contains("secret3"))
+ assert(!errorMsg.contains("secret4"))
+ assert(!errorMsg.contains("secret5"))
+ assert(!errorMsg.contains("secret6"))
+ assert(!errorMsg.contains("secret7"))
+ assert(!errorMsg.contains("secret8"))
+ assert(!errorMsg.contains("secret9"))
+ assert(!errorMsg.contains("secret10"))
+ assert(!errorMsg.contains("{\"Authorization\": \"secret11\"}"))
Review Comment:
Maybe `assert(!errorMsg.contains("secret"))`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org