You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/12/28 12:06:55 UTC

[GitHub] [incubator-kyuubi] minyk opened a new pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

minyk opened a new pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640


   <!--
   Thanks for sending a pull request!
   
   Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
     2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
   -->
   
   ### _Why are the changes needed?_
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you add a feature, you can talk about the use case of it.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   SparkSQLEngine's Kyuubi UI tab shows SQL statements. In some cases, these informations are secret information like S3 access/secret key. Spark SQL UI tab already redact with `spark.redaction.string.regex` configuration. Kyuubi also should redact secret informations with this property.
   
   ### _How was this patch tested?_
   - [X] Add some test cases that check the changes thoroughly including negative and positive cases if possible
   
   - [X] Add screenshots for manual tests if appropriate
   with below property and query:
   spark-defaults.conf:
   ```
   spark.redaction.string.regex (?i)url|access.*|secret.*|password
   ```
   query:
   ```
   SET fs.s3a.access.key=testkey
   ```
   
   ![Screen Shot 2021-12-28 at 9 00 07 PM](https://user-images.githubusercontent.com/1802676/147564262-bf06869f-2c2f-4e76-8380-34a92b2a390c.png)
   
   
   
   - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776145035



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
##########
@@ -98,6 +101,8 @@ abstract class SparkOperation(opType: OperationType, session: Session)
   protected def withLocalProperties[T](f: => T): T = {
     try {
       spark.sparkContext.setJobGroup(statementId, statement, forceCancel)
+      spark.sparkContext.setJobDescription(

Review comment:
       yes we can. apply and action confirmed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r775905399



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
##########
@@ -69,13 +70,18 @@ class ExecuteStatement(
     OperationLog.removeCurrentOperationLog()
   }
 
+  override val redactedStatement: String =
+    redact(spark.sessionState.conf.stringRedactionPattern, statement)
+
   private def executeStatement(): Unit = withLocalProperties {
     try {
       setState(OperationState.RUNNING)
       info(diagnostics)
       Thread.currentThread().setContextClassLoader(spark.sharedState.jarClassLoader)
       // TODO: Make it configurable
       spark.sparkContext.addSparkListener(operationListener)
+      spark.sparkContext.setJobDescription(
+        redact(spark.sessionState.conf.stringRedactionPattern, statement))

Review comment:
       +1 to set there




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776145836



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
##########
@@ -98,6 +101,7 @@ abstract class SparkOperation(opType: OperationType, session: Session)
   protected def withLocalProperties[T](f: => T): T = {
     try {
       spark.sparkContext.setJobGroup(statementId, statement, forceCancel)
+      spark.sparkContext.setJobDescription(redactedStatement)

Review comment:
       we can combine this two line by:
   
   `spark.sparkContext.setJobGroup(statementId, redactedStatement, forceCancel)`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r775905689



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -59,6 +59,7 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
   def getBackgroundHandle: Future[_] = _backgroundHandle
 
   def statement: String = opType.toString
+  def redactedStatement: String = opType.toString

Review comment:
       statement？




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002383150


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1640](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cb8721f) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/2af105a417342c4e51ec550df7259d90e1fe341e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2af105a) will **increase** coverage by `1.27%`.
   > The diff coverage is `58.13%`.
   
   > :exclamation: Current head cb8721f differs from pull request most recent head c84778e. Consider uploading reports for the commit c84778e to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1640      +/-   ##
   ============================================
   + Coverage     58.32%   59.60%   +1.27%     
   - Complexity      164      193      +29     
   ============================================
     Files           260      264       +4     
     Lines         12930    13081     +151     
     Branches       1633     1650      +17     
   ============================================
   + Hits           7542     7797     +255     
   + Misses         4756     4625     -131     
   - Partials        632      659      +27     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...g/apache/kyuubi/engine/flink/result/Constants.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/engine/flink/schema/RowSet.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9zY2hlbWEvUm93U2V0LnNjYWxh) | `50.00% <24.24%> (+36.13%)` | :arrow_up: |
   | [.../apache/kyuubi/engine/flink/FlinkEngineUtils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua0VuZ2luZVV0aWxzLnNjYWxh) | `29.41% <29.41%> (ø)` | |
   | [...rg/apache/kyuubi/engine/flink/FlinkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEVuZ2luZS5zY2FsYQ==) | `42.10% <42.10%> (ø)` | |
   | [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `77.17% <66.66%> (+0.50%)` | :arrow_up: |
   | [...yuubi/engine/flink/FlinkEngineProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvZmxpbmsvRmxpbmtFbmdpbmVQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `75.90% <75.90%> (ø)` | |
   | [...ain/scala/org/apache/kyuubi/engine/EngineRef.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvRW5naW5lUmVmLnNjYWxh) | `86.00% <87.50%> (+1.05%)` | :arrow_up: |
   | [.../apache/kyuubi/engine/flink/result/ResultKind.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9SZXN1bHRLaW5kLmphdmE=) | `100.00% <100.00%> (ø)` | |
   | [...e/kyuubi/engine/flink/FlinkSQLBackendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEJhY2tlbmRTZXJ2aWNlLnNjYWxh) | `100.00% <100.00%> (ø)` | |
   | [...uubi/engine/flink/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `80.64% <100.00%> (+12.90%)` | :arrow_up: |
   | ... and [28 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2af105a...c84778e](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776116348



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
##########
@@ -69,13 +70,18 @@ class ExecuteStatement(
     OperationLog.removeCurrentOperationLog()
   }
 
+  override val redactedStatement: String =
+    redact(spark.sessionState.conf.stringRedactionPattern, statement)
+
   private def executeStatement(): Unit = withLocalProperties {
     try {
       setState(OperationState.RUNNING)
       info(diagnostics)
       Thread.currentThread().setContextClassLoader(spark.sharedState.jarClassLoader)
       // TODO: Make it configurable
       spark.sparkContext.addSparkListener(operationListener)
+      spark.sparkContext.setJobDescription(
+        redact(spark.sessionState.conf.stringRedactionPattern, statement))

Review comment:
       let me try




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002383150


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1640](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3a8e0e9) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/2af105a417342c4e51ec550df7259d90e1fe341e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2af105a) will **increase** coverage by `1.27%`.
   > The diff coverage is `100.00%`.
   
   > :exclamation: Current head 3a8e0e9 differs from pull request most recent head 4051522. Consider uploading reports for the commit 4051522 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1640      +/-   ##
   ============================================
   + Coverage     58.32%   59.60%   +1.27%     
   - Complexity      164      193      +29     
   ============================================
     Files           260      265       +5     
     Lines         12930    13085     +155     
     Branches       1633     1650      +17     
   ============================================
   + Hits           7542     7799     +257     
   + Misses         4756     4625     -131     
   - Partials        632      661      +29     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...uubi/engine/spark/events/SparkOperationEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtPcGVyYXRpb25FdmVudC5zY2FsYQ==) | `88.88% <100.00%> (ø)` | |
   | [...kyuubi/engine/spark/operation/SparkOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtPcGVyYXRpb24uc2NhbGE=) | `83.33% <100.00%> (+0.40%)` | :arrow_up: |
   | [...ala/org/apache/spark/kyuubi/SparkUtilsHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL1NwYXJrVXRpbHNIZWxwZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
   | [.../org/apache/kyuubi/operation/KyuubiOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uLnNjYWxh) | `64.78% <0.00%> (-9.86%)` | :arrow_down: |
   | [...ala/org/apache/kyuubi/operation/LaunchEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vTGF1bmNoRW5naW5lLnNjYWxh) | `86.36% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.35% <0.00%> (ø)` | |
   | [...g/apache/kyuubi/engine/flink/result/Constants.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../apache/kyuubi/engine/flink/result/ResultKind.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9SZXN1bHRLaW5kLmphdmE=) | `100.00% <0.00%> (ø)` | |
   | [...e/kyuubi/engine/flink/FlinkSQLBackendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEJhY2tlbmRTZXJ2aWNlLnNjYWxh) | `100.00% <0.00%> (ø)` | |
   | [.../apache/kyuubi/engine/flink/FlinkEngineUtils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua0VuZ2luZVV0aWxzLnNjYWxh) | `29.41% <0.00%> (ø)` | |
   | ... and [25 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2af105a...4051522](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002383150


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1640](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c84778e) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/2af105a417342c4e51ec550df7259d90e1fe341e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2af105a) will **increase** coverage by `1.27%`.
   > The diff coverage is `100.00%`.
   
   > :exclamation: Current head c84778e differs from pull request most recent head 3a8e0e9. Consider uploading reports for the commit 3a8e0e9 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1640      +/-   ##
   ============================================
   + Coverage     58.32%   59.60%   +1.27%     
   - Complexity      164      193      +29     
   ============================================
     Files           260      265       +5     
     Lines         12930    13086     +156     
     Branches       1633     1650      +17     
   ============================================
   + Hits           7542     7800     +258     
   + Misses         4756     4625     -131     
   - Partials        632      661      +29     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...uubi/engine/spark/events/SparkOperationEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtPcGVyYXRpb25FdmVudC5zY2FsYQ==) | `88.88% <100.00%> (ø)` | |
   | [...kyuubi/engine/spark/operation/SparkOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtPcGVyYXRpb24uc2NhbGE=) | `83.52% <100.00%> (+0.60%)` | :arrow_up: |
   | [...ala/org/apache/spark/kyuubi/SparkUtilsHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL1NwYXJrVXRpbHNIZWxwZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
   | [.../org/apache/kyuubi/operation/KyuubiOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uLnNjYWxh) | `64.78% <0.00%> (-9.86%)` | :arrow_down: |
   | [...ala/org/apache/kyuubi/operation/LaunchEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vTGF1bmNoRW5naW5lLnNjYWxh) | `86.36% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.35% <0.00%> (ø)` | |
   | [...g/apache/kyuubi/engine/flink/result/Constants.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../apache/kyuubi/engine/flink/result/ResultKind.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9SZXN1bHRLaW5kLmphdmE=) | `100.00% <0.00%> (ø)` | |
   | [.../apache/kyuubi/engine/flink/FlinkEngineUtils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua0VuZ2luZVV0aWxzLnNjYWxh) | `29.41% <0.00%> (ø)` | |
   | [...e/kyuubi/engine/flink/FlinkSQLBackendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEJhY2tlbmRTZXJ2aWNlLnNjYWxh) | `100.00% <0.00%> (ø)` | |
   | ... and [25 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2af105a...3a8e0e9](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776123563



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -59,6 +59,7 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
   def getBackgroundHandle: Future[_] = _backgroundHandle
 
   def statement: String = opType.toString
+  def redactedStatement: String = opType.toString

Review comment:
       Thanks. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r775880218



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
##########
@@ -69,13 +70,18 @@ class ExecuteStatement(
     OperationLog.removeCurrentOperationLog()
   }
 
+  override val redactedStatement: String =
+    redact(spark.sessionState.conf.stringRedactionPattern, statement)
+
   private def executeStatement(): Unit = withLocalProperties {
     try {
       setState(OperationState.RUNNING)
       info(diagnostics)
       Thread.currentThread().setContextClassLoader(spark.sharedState.jarClassLoader)
       // TODO: Make it configurable
       spark.sparkContext.addSparkListener(operationListener)
+      spark.sparkContext.setJobDescription(
+        redact(spark.sessionState.conf.stringRedactionPattern, statement))

Review comment:
       Can we make redacted statment at:
   
   https://github.com/apache/incubator-kyuubi/blob/cb8721fef130771db330c0dd51844ebb13bca7b3/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala#L98-L100
   
   And we can also define the `redactedStatement` at `SparkOperation`.
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002383150


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1640](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c84778e) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/2af105a417342c4e51ec550df7259d90e1fe341e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2af105a) will **increase** coverage by `1.27%`.
   > The diff coverage is `100.00%`.
   
   > :exclamation: Current head c84778e differs from pull request most recent head 4051522. Consider uploading reports for the commit 4051522 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1640      +/-   ##
   ============================================
   + Coverage     58.32%   59.60%   +1.27%     
   - Complexity      164      193      +29     
   ============================================
     Files           260      265       +5     
     Lines         12930    13086     +156     
     Branches       1633     1650      +17     
   ============================================
   + Hits           7542     7800     +258     
   + Misses         4756     4625     -131     
   - Partials        632      661      +29     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...uubi/engine/spark/events/SparkOperationEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtPcGVyYXRpb25FdmVudC5zY2FsYQ==) | `88.88% <100.00%> (ø)` | |
   | [...kyuubi/engine/spark/operation/SparkOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtPcGVyYXRpb24uc2NhbGE=) | `83.52% <100.00%> (+0.60%)` | :arrow_up: |
   | [...ala/org/apache/spark/kyuubi/SparkUtilsHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL1NwYXJrVXRpbHNIZWxwZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
   | [.../org/apache/kyuubi/operation/KyuubiOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uLnNjYWxh) | `64.78% <0.00%> (-9.86%)` | :arrow_down: |
   | [...ala/org/apache/kyuubi/operation/LaunchEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vTGF1bmNoRW5naW5lLnNjYWxh) | `86.36% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.35% <0.00%> (ø)` | |
   | [...g/apache/kyuubi/engine/flink/result/Constants.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../apache/kyuubi/engine/flink/result/ResultKind.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9SZXN1bHRLaW5kLmphdmE=) | `100.00% <0.00%> (ø)` | |
   | [...yuubi/engine/flink/FlinkEngineProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvZmxpbmsvRmxpbmtFbmdpbmVQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `75.90% <0.00%> (ø)` | |
   | [...rg/apache/kyuubi/engine/flink/FlinkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEVuZ2luZS5zY2FsYQ==) | `42.10% <0.00%> (ø)` | |
   | ... and [25 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2af105a...4051522](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776142299



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
##########
@@ -98,6 +101,8 @@ abstract class SparkOperation(opType: OperationType, session: Session)
   protected def withLocalProperties[T](f: => T): T = {
     try {
       spark.sparkContext.setJobGroup(statementId, statement, forceCancel)
+      spark.sparkContext.setJobDescription(

Review comment:
       we can use redactedStatement here?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776141267



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -59,6 +59,7 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
   def getBackgroundHandle: Future[_] = _backgroundHandle
 
   def statement: String = opType.toString
+  def redactedStatement: String = opType.toString

Review comment:
       `redactedStatement` moves to `SparkOperation`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776141210



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
##########
@@ -69,13 +70,18 @@ class ExecuteStatement(
     OperationLog.removeCurrentOperationLog()
   }
 
+  override val redactedStatement: String =
+    redact(spark.sessionState.conf.stringRedactionPattern, statement)
+
   private def executeStatement(): Unit = withLocalProperties {
     try {
       setState(OperationState.RUNNING)
       info(diagnostics)
       Thread.currentThread().setContextClassLoader(spark.sharedState.jarClassLoader)
       // TODO: Make it configurable
       spark.sparkContext.addSparkListener(operationListener)
+      spark.sparkContext.setJobDescription(
+        redact(spark.sessionState.conf.stringRedactionPattern, statement))

Review comment:
       move codes to `SparkOperation`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002383150


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1640](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cb8721f) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/2af105a417342c4e51ec550df7259d90e1fe341e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2af105a) will **increase** coverage by `1.27%`.
   > The diff coverage is `58.13%`.
   
   > :exclamation: Current head cb8721f differs from pull request most recent head 3a8e0e9. Consider uploading reports for the commit 3a8e0e9 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1640      +/-   ##
   ============================================
   + Coverage     58.32%   59.60%   +1.27%     
   - Complexity      164      193      +29     
   ============================================
     Files           260      264       +4     
     Lines         12930    13081     +151     
     Branches       1633     1650      +17     
   ============================================
   + Hits           7542     7797     +255     
   + Misses         4756     4625     -131     
   - Partials        632      659      +27     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...g/apache/kyuubi/engine/flink/result/Constants.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/engine/flink/schema/RowSet.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9zY2hlbWEvUm93U2V0LnNjYWxh) | `50.00% <24.24%> (+36.13%)` | :arrow_up: |
   | [.../apache/kyuubi/engine/flink/FlinkEngineUtils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua0VuZ2luZVV0aWxzLnNjYWxh) | `29.41% <29.41%> (ø)` | |
   | [...rg/apache/kyuubi/engine/flink/FlinkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEVuZ2luZS5zY2FsYQ==) | `42.10% <42.10%> (ø)` | |
   | [...apache/kyuubi/server/api/v1/SessionsResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL1Nlc3Npb25zUmVzb3VyY2Uuc2NhbGE=) | `77.17% <66.66%> (+0.50%)` | :arrow_up: |
   | [...yuubi/engine/flink/FlinkEngineProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvZmxpbmsvRmxpbmtFbmdpbmVQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `75.90% <75.90%> (ø)` | |
   | [...ain/scala/org/apache/kyuubi/engine/EngineRef.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvRW5naW5lUmVmLnNjYWxh) | `86.00% <87.50%> (+1.05%)` | :arrow_up: |
   | [.../apache/kyuubi/engine/flink/result/ResultKind.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9SZXN1bHRLaW5kLmphdmE=) | `100.00% <100.00%> (ø)` | |
   | [...e/kyuubi/engine/flink/FlinkSQLBackendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9GbGlua1NRTEJhY2tlbmRTZXJ2aWNlLnNjYWxh) | `100.00% <100.00%> (ø)` | |
   | [...uubi/engine/flink/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9mbGluay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `80.64% <100.00%> (+12.90%)` | :arrow_up: |
   | ... and [28 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2af105a...3a8e0e9](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776147434



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
##########
@@ -98,6 +101,7 @@ abstract class SparkOperation(opType: OperationType, session: Session)
   protected def withLocalProperties[T](f: => T): T = {
     try {
       spark.sparkContext.setJobGroup(statementId, statement, forceCancel)
+      spark.sparkContext.setJobDescription(redactedStatement)

Review comment:
       Thanks. it will be combined. ;-)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] pan3793 commented on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
pan3793 commented on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002405631


   Thanks, merging to master


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] pan3793 closed pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
pan3793 closed pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776120894



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -59,6 +59,7 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
   def getBackgroundHandle: Future[_] = _backgroundHandle
 
   def statement: String = opType.toString
+  def redactedStatement: String = opType.toString

Review comment:
       def redactedStatement: String = statement




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] minyk commented on a change in pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
minyk commented on a change in pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#discussion_r776116329



##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -59,6 +59,7 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
   def getBackgroundHandle: Future[_] = _backgroundHandle
 
   def statement: String = opType.toString
+  def redactedStatement: String = opType.toString

Review comment:
       do you mean `redactedstatment`? all lowercase?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #1640: [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1640:
URL: https://github.com/apache/incubator-kyuubi/pull/1640#issuecomment-1002383150


   # [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1640](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4051522) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/2af105a417342c4e51ec550df7259d90e1fe341e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2af105a) will **increase** coverage by `1.25%`.
   > The diff coverage is `100.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/graphs/tree.svg?width=650&height=150&src=pr&token=925D4tb9AH&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #1640      +/-   ##
   ============================================
   + Coverage     58.32%   59.58%   +1.25%     
   - Complexity      164      193      +29     
   ============================================
     Files           260      265       +5     
     Lines         12930    13084     +154     
     Branches       1633     1650      +17     
   ============================================
   + Hits           7542     7796     +254     
   + Misses         4756     4626     -130     
   - Partials        632      662      +30     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...uubi/engine/spark/events/SparkOperationEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtPcGVyYXRpb25FdmVudC5zY2FsYQ==) | `88.88% <100.00%> (ø)` | |
   | [...kyuubi/engine/spark/operation/SparkOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtPcGVyYXRpb24uc2NhbGE=) | `83.13% <100.00%> (+0.20%)` | :arrow_up: |
   | [...ala/org/apache/spark/kyuubi/SparkUtilsHelper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL1NwYXJrVXRpbHNIZWxwZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
   | [.../org/apache/kyuubi/operation/KyuubiOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uLnNjYWxh) | `64.78% <0.00%> (-9.86%)` | :arrow_down: |
   | [...he/kyuubi/engine/spark/repl/KyuubiSparkILoop.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9yZXBsL0t5dXViaVNwYXJrSUxvb3Auc2NhbGE=) | `90.16% <0.00%> (-3.28%)` | :arrow_down: |
   | [...ala/org/apache/kyuubi/operation/LaunchEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vTGF1bmNoRW5naW5lLnNjYWxh) | `86.36% <0.00%> (ø)` | |
   | [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `87.35% <0.00%> (ø)` | |
   | [...g/apache/kyuubi/engine/flink/result/Constants.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../apache/kyuubi/engine/flink/result/ResultKind.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1mbGluay1zcWwtZW5naW5lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2ZsaW5rL3Jlc3VsdC9SZXN1bHRLaW5kLmphdmE=) | `100.00% <0.00%> (ø)` | |
   | [...yuubi/engine/flink/FlinkEngineProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvZmxpbmsvRmxpbmtFbmdpbmVQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `75.90% <0.00%> (ø)` | |
   | ... and [26 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2af105a...4051522](https://codecov.io/gh/apache/incubator-kyuubi/pull/1640?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org