You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/04/26 11:07:48 UTC
[GitHub] [incubator-kyuubi] wForget opened a new pull request, #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
wForget opened a new pull request, #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
close #2484
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [X] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] turboFei closed pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
turboFei closed pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
URL: https://github.com/apache/incubator-kyuubi/pull/2485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#issuecomment-1109714300
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2485](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5c3bb30) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/618732148f546e3489a876d17e708e06a9f48c27?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6187321) will **decrease** coverage by `0.07%`.
> The diff coverage is `7.69%`.
```diff
@@ Coverage Diff @@
## master #2485 +/- ##
============================================
- Coverage 63.45% 63.37% -0.08%
Complexity 69 69
============================================
Files 371 371
Lines 17585 17608 +23
Branches 2348 2348
============================================
+ Hits 11158 11159 +1
- Misses 5392 5416 +24
+ Partials 1035 1033 -2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../scala/org/apache/spark/ui/EngineSessionPage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3BhcmsvdWkvRW5naW5lU2Vzc2lvblBhZ2Uuc2NhbGE=) | `2.15% <0.00%> (-0.75%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/events/SessionEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU2Vzc2lvbkV2ZW50LnNjYWxh) | `84.00% <100.00%> (+1.39%)` | :arrow_up: |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `77.38% <0.00%> (+0.77%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [6187321...5c3bb30](https://codecov.io/gh/apache/incubator-kyuubi/pull/2485?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
wForget commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r859314351
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/ui/EngineSessionPage.scala:
##########
@@ -40,6 +45,34 @@ case class EngineSessionPage(parent: EngineTab)
val sessionStat = store.getSession(parameterId).getOrElse(null)
require(sessionStat != null, "Invalid sessionID[" + parameterId + "]")
+ val redactionPattern = parent.sparkUI match {
+ case Some(ui) => Some(ui.conf.get(SECRET_REDACTION_PATTERN))
+ case None => SECRET_REDACTION_PATTERN.defaultValue
+ }
+
+ val sessionPropertiesTable =
+ if (sessionStat.conf != null && !sessionStat.conf.isEmpty) {
+ val table = UIUtils.listingTable(
+ propertyHeader,
+ propertyRow,
+ Utils.redact(redactionPattern, sessionStat.conf.toArray),
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] turboFei commented on pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#issuecomment-1109995379
nice feature, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r859303603
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/ui/EngineSessionPage.scala:
##########
@@ -40,6 +45,34 @@ case class EngineSessionPage(parent: EngineTab)
val sessionStat = store.getSession(parameterId).getOrElse(null)
require(sessionStat != null, "Invalid sessionID[" + parameterId + "]")
+ val redactionPattern = parent.sparkUI match {
+ case Some(ui) => Some(ui.conf.get(SECRET_REDACTION_PATTERN))
+ case None => SECRET_REDACTION_PATTERN.defaultValue
+ }
+
+ val sessionPropertiesTable =
+ if (sessionStat.conf != null && !sessionStat.conf.isEmpty) {
+ val table = UIUtils.listingTable(
+ propertyHeader,
+ propertyRow,
+ Utils.redact(redactionPattern, sessionStat.conf.toArray),
Review Comment:
nit: sort the conf to make the ordering more readable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
wForget commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r858669008
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/events/SessionEvent.scala:
##########
@@ -43,6 +43,7 @@ case class SessionEvent(
username: String,
ip: String,
serverIp: String,
+ conf: Map[String, String],
Review Comment:
> I think this is connection initializing config map, how about the configs from `set` command
The set statement will be displayed in the statement list, we don't need to override it here, it might be better to show the initial configurations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] gabrywu commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
gabrywu commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r858614150
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/events/SessionEvent.scala:
##########
@@ -43,6 +43,7 @@ case class SessionEvent(
username: String,
ip: String,
serverIp: String,
+ conf: Map[String, String],
Review Comment:
I think this is connection initializing config map, how about the configs from `set` command
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
wForget commented on PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#issuecomment-1109755054
> need more unit tests
Thanks, I'll add it later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] gabrywu commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
gabrywu commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r858682856
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/events/SessionEvent.scala:
##########
@@ -43,6 +43,7 @@ case class SessionEvent(
username: String,
ip: String,
serverIp: String,
+ conf: Map[String, String],
Review Comment:
Yes, can't agree more. Comment on purpose 😁
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] wForget commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
wForget commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r858671224
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/ui/EngineSessionPage.scala:
##########
@@ -49,6 +82,7 @@ case class EngineSessionPage(parent: EngineTab)
Session created at {formatDate(sessionStat.startTime)},
Total run {sessionStat.totalOperations} SQL
</h4> ++
+ sessionPropertiesTable ++
Review Comment:
> Is it better to be shown after SQL stats?
I think it would be better to put this in front of `SQL stats` as part of the details of the session.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] gabrywu commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
gabrywu commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r858614835
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/ui/EngineSessionPage.scala:
##########
@@ -49,6 +82,7 @@ case class EngineSessionPage(parent: EngineTab)
Session created at {formatDate(sessionStat.startTime)},
Total run {sessionStat.totalOperations} SQL
</h4> ++
+ sessionPropertiesTable ++
Review Comment:
Is it better to be shown after SQL stats?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] gabrywu commented on a diff in pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
gabrywu commented on code in PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#discussion_r858680081
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/ui/EngineSessionPage.scala:
##########
@@ -49,6 +82,7 @@ case class EngineSessionPage(parent: EngineTab)
Session created at {formatDate(sessionStat.startTime)},
Total run {sessionStat.totalOperations} SQL
</h4> ++
+ sessionPropertiesTable ++
Review Comment:
OK, let keep it before `SQL stats`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] turboFei commented on pull request #2485: [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
Posted by GitBox <gi...@apache.org>.
turboFei commented on PR #2485:
URL: https://github.com/apache/incubator-kyuubi/pull/2485#issuecomment-1110507068
thanks, merged to master and branch-1.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org