You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/08/09 09:54:08 UTC
[GitHub] [incubator-kyuubi] hddong opened a new pull request #908: [KYUUBI#891]Add audit log for sql statement
hddong opened a new pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Add a additional log file to audit sql statement.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [X] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (595784f) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.15%`.
> The diff coverage is `85.93%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 79.28% -0.16%
- Complexity 87 101 +14
============================================
Files 176 184 +8
Lines 6552 6709 +157
Branches 775 789 +14
============================================
+ Hits 5205 5319 +114
- Misses 897 931 +34
- Partials 450 459 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/events/Log4jEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvTG9nNGpFdmVudExvZ2dlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `80.00% <75.00%> (-10.00%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.46% <100.00%> (+0.15%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `96.29% <100.00%> (+0.64%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.15% <100.00%> (+0.36%)` | :arrow_up: |
| [.../apache/kyuubi/server/api/v1/ApiRootResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0FwaVJvb3RSZXNvdXJjZS5zY2FsYQ==) | `40.00% <0.00%> (-26.67%)` | :arrow_down: |
| [...pache/kyuubi/service/AbstractFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2aWNlL0Fic3RyYWN0RnJvbnRlbmRTZXJ2aWNlLnNjYWxh) | `57.14% <0.00%> (-9.53%)` | :arrow_down: |
| ... and [40 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...595784f](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r732547757
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -862,6 +876,22 @@ object KyuubiConf {
"Unsupported event loggers")
.createWithDefault(Nil)
+
+ val AUDIT_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
Review comment:
Shall we not support this config for now？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-927264101
> we can do filtering based on the contents of the events, e.g. state?
@yaooqinn ： Sorry for feedback, been busy with other things lately.
`onEvent` log all events, but not all `KyuubiServerEvent` have `state`. We can reuse loggers, and add `auditLoggers` and `onAuditEvent` in `EventLoggingService` to make events and audit relatively independent. Then, we can choose to use one or all of them as needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r725759905
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -272,6 +272,36 @@ object KyuubiConf {
.booleanConf
.createWithDefault(true)
+ val AUDIT_LOG_ENABLE: ConfigEntry[Boolean] =
+ buildConf("audit.log.enable")
+ .doc("When true, store sql statement for audit.")
+ .version("1.4.0")
+ .booleanConf
+ .createWithDefault(true)
+
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
Review comment:
Does it means that this is just a variant of JSON logger
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #908: [KYUUBI#891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (44c192d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/c86767f33679674de11b4a3ec23ced478ee766d8?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c86767f) will **decrease** coverage by `0.02%`.
> The diff coverage is `78.43%`.
> :exclamation: Current head 44c192d differs from pull request most recent head 8156554. Consider uploading reports for the commit 8156554 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.19% 79.16% -0.03%
Complexity 11 11
============================================
Files 143 144 +1
Lines 5384 5410 +26
Branches 651 654 +3
============================================
+ Hits 4264 4283 +19
- Misses 756 758 +2
- Partials 364 369 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../scala/org/apache/kyuubi/operation/Operation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uLnNjYWxh) | `100.00% <ø> (ø)` | |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `64.06% <61.53%> (-0.65%)` | :arrow_down: |
| [...e/kyuubi/engine/spark/events/JsonEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvSnNvbkV2ZW50TG9nZ2VyLnNjYWxh) | `83.78% <79.31%> (-6.22%)` | :arrow_down: |
| [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `82.60% <100.00%> (ø)` | |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.60% <100.00%> (+0.14%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `94.54% <100.00%> (+0.05%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/operation/log/AuditLog.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vbG9nL0F1ZGl0TG9nLnNjYWxh) | `100.00% <100.00%> (ø)` | |
| [...pache/kyuubi/sql/KyuubiQueryStagePreparation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGV2L2t5dXViaS1leHRlbnNpb24tc3BhcmstMy0xL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL3NxbC9LeXV1YmlRdWVyeVN0YWdlUHJlcGFyYXRpb24uc2NhbGE=) | `80.39% <0.00%> (+0.98%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [c86767f...8156554](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-939637964
I am really confused now about this feature.
Why do we need to bind it to Log4j?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-926311965
> maybe we can reuse the current event logger system and only create a sink type for `log4j`?
Yeh, I am thinking about this too. But, if only as a sink, the event from initize to end will be logged, it's redundance for audit.
How about add a aditional func`auditEvent` in `EventLoggingService` to logger audit event.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6a06892) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/6dd8dfb70c2cbe7f92cf42ca68444c404767f699?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6dd8dfb) will **decrease** coverage by `0.13%`.
> The diff coverage is `79.16%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.23% 79.10% -0.14%
Complexity 11 11
============================================
Files 145 146 +1
Lines 5423 5427 +4
Branches 651 658 +7
============================================
- Hits 4297 4293 -4
- Misses 759 762 +3
- Partials 367 372 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../scala/org/apache/kyuubi/operation/Operation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uLnNjYWxh) | `100.00% <ø> (ø)` | |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `65.67% <70.58%> (+0.96%)` | :arrow_up: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.60% <100.00%> (+0.14%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `94.71% <100.00%> (+0.05%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/operation/log/AuditLog.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vbG9nL0F1ZGl0TG9nLnNjYWxh) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXpvb2tlZXBlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS96b29rZWVwZXIvRW1iZWRkZWRab29rZWVwZXIuc2NhbGE=) | `86.66% <0.00%> (-3.58%)` | :arrow_down: |
| [...ommon/src/main/scala/org/apache/kyuubi/Utils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9VdGlscy5zY2FsYQ==) | `70.23% <0.00%> (-3.52%)` | :arrow_down: |
| [...la/org/apache/kyuubi/zookeeper/ZookeeperConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXpvb2tlZXBlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS96b29rZWVwZXIvWm9va2VlcGVyQ29uZi5zY2FsYQ==) | `100.00% <0.00%> (ø)` | |
| [...pache/kyuubi/sql/KyuubiQueryStagePreparation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGV2L2t5dXViaS1leHRlbnNpb24tc3BhcmstMy0xL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL3NxbC9LeXV1YmlRdWVyeVN0YWdlUHJlcGFyYXRpb24uc2NhbGE=) | `80.39% <0.00%> (+0.98%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [6dd8dfb...6a06892](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-943201710
@yaooqinn @timothy65535 : What makes you confused? Audit log or Log4j logger? if just log4j logger, we can remove and just audit event use json file first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731658776
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
it will only audit statements not other behaviors like opening or closing a session
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6a06892) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.33%`.
> The diff coverage is `79.16%`.
> :exclamation: Current head 6a06892 differs from pull request most recent head 6f52d1a. Consider uploading reports for the commit 6f52d1a to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 79.10% -0.34%
+ Complexity 87 11 -76
============================================
Files 176 146 -30
Lines 6552 5427 -1125
Branches 775 658 -117
============================================
- Hits 5205 4293 -912
+ Misses 897 762 -135
+ Partials 450 372 -78
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../scala/org/apache/kyuubi/operation/Operation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uLnNjYWxh) | `100.00% <ø> (ø)` | |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `65.67% <70.58%> (+0.96%)` | :arrow_up: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.60% <100.00%> (-0.42%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `94.71% <100.00%> (-0.60%)` | :arrow_down: |
| [...ala/org/apache/kyuubi/operation/log/AuditLog.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vbG9nL0F1ZGl0TG9nLnNjYWxh) | `100.00% <100.00%> (ø)` | |
| [...scala/org/apache/kyuubi/operation/GetColumns.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vR2V0Q29sdW1ucy5zY2FsYQ==) | `0.00% <0.00%> (-88.89%)` | :arrow_down: |
| [...a/org/apache/kyuubi/operation/OperationState.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uU3RhdGUuc2NhbGE=) | `27.27% <0.00%> (-27.02%)` | :arrow_down: |
| [...he/kyuubi/engine/spark/shim/CatalogShim\_v3\_0.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zaGltL0NhdGFsb2dTaGltX3YzXzAuc2NhbGE=) | `61.84% <0.00%> (-26.32%)` | :arrow_down: |
| [.../apache/kyuubi/client/KyuubiSyncThriftClient.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jbGllbnQvS3l1dWJpU3luY1RocmlmdENsaWVudC5zY2FsYQ==) | `72.72% <0.00%> (-10.26%)` | :arrow_down: |
| [.../scala/org/apache/kyuubi/server/KyuubiServer.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpU2VydmVyLnNjYWxh) | `50.00% <0.00%> (-6.53%)` | :arrow_down: |
| ... and [82 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...6f52d1a](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r686450641
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -69,14 +77,18 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
this.operationException = opEx
}
- protected def setState(newState: OperationState): Unit = {
+ protected def setState(newState: OperationState, audit: Boolean = true): Unit = {
Review comment:
> why do we need `audit: Boolean` ?
`SparkOperation` also execute this code. We just need audit for `KyuubiOperation`, so it is used to ignore `SparkOperation`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r732595791
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -862,6 +876,22 @@ object KyuubiConf {
"Unsupported event loggers")
.createWithDefault(Nil)
+
+ val AUDIT_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
Review comment:
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r732642000
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
> should we write all audit log in a file
Maybe into a single file is a good practice. Do we need a common schema for auditing, like who did this? when did he/she do this? what is the operation？ which type is this op？How long was this op last? For both SQL statements/sessions, and so on
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r736172802
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -252,6 +252,13 @@ object KyuubiConf {
.booleanConf
.createWithDefault(true)
+ val AUDIT_LOG_ENABLE: ConfigEntry[Boolean] =
+ buildConf("audit.log.enable")
+ .doc("When true, store sql statement for audit.")
+ .version("1.4.0")
+ .booleanConf
+ .createWithDefault(false)
Review comment:
hi, `true` by default is better if introduce this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-929101464
> > we can do filtering based on the contents of the events, e.g. state?
>
> @yaooqinn ： Sorry for feedback, been busy with other things lately. `onEvent` log all events, but not all `KyuubiServerEvent` have `state`. We can reuse loggers, and add `auditLoggers` and `onAuditEvent` in `EventLoggingService` to make events and audit relatively independent. Then, we can choose to use one or all of them as needed.
SGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731630256
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
You mean elapsedTime？ why it necessary when there are createTime and completedTime already？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-940612419
> I am really confused now about this feature.
+1, just a little bit confused too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-931059212
> SGTM.
Has done. The mainly changes:
1. add a new logger: `Log4jEventLogger`
2. add `auditLoggers` and `onAuditEvent ` in `EventLoggingService`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (deff62e) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.58%`.
> The diff coverage is `82.60%`.
> :exclamation: Current head deff62e differs from pull request most recent head fcdbf9b. Consider uploading reports for the commit fcdbf9b to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 78.85% -0.59%
- Complexity 87 93 +6
============================================
Files 176 184 +8
Lines 6552 6736 +184
Branches 775 789 +14
============================================
+ Hits 5205 5312 +107
- Misses 897 972 +75
- Partials 450 452 +2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/events/Log4jEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvTG9nNGpFdmVudExvZ2dlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `75.00% <68.00%> (-15.00%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.54% <100.00%> (+0.24%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `96.29% <100.00%> (+0.64%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.15% <100.00%> (+0.36%)` | :arrow_up: |
| [...main/java/org/apache/kyuubi/jdbc/KyuubiDriver.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhpdmUtamRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3l1dWJpL2pkYmMvS3l1dWJpRHJpdmVyLmphdmE=) | `0.00% <0.00%> (-57.15%)` | :arrow_down: |
| [.../apache/kyuubi/server/api/v1/ApiRootResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0FwaVJvb3RSZXNvdXJjZS5zY2FsYQ==) | `40.00% <0.00%> (-26.67%)` | :arrow_down: |
| ... and [49 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...fcdbf9b](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r726691345
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -272,6 +272,36 @@ object KyuubiConf {
.booleanConf
.createWithDefault(true)
+ val AUDIT_LOG_ENABLE: ConfigEntry[Boolean] =
+ buildConf("audit.log.enable")
+ .doc("When true, store sql statement for audit.")
+ .version("1.4.0")
+ .booleanConf
+ .createWithDefault(true)
+
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
Review comment:
Yep, they are the same, but the recorded event is different.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6f52d1a) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.14%`.
> The diff coverage is `83.87%`.
> :exclamation: Current head 6f52d1a differs from pull request most recent head 595784f. Consider uploading reports for the commit 595784f to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 79.29% -0.15%
- Complexity 87 90 +3
============================================
Files 176 181 +5
Lines 6552 6681 +129
Branches 775 786 +11
============================================
+ Hits 5205 5298 +93
- Misses 897 926 +29
- Partials 450 457 +7
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `81.00% <57.14%> (-1.80%)` | :arrow_down: |
| [...org/apache/kyuubi/server/AuditLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvQXVkaXRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `77.27% <77.27%> (ø)` | |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/audit/ServerAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9TZXJ2ZXJBdWRpdEV2ZW50LnNjYWxh) | `90.00% <90.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.47% <100.00%> (+0.16%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...la/org/apache/kyuubi/audit/ServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9TZXJ2ZXJBdWRpdExvZ2dlci5zY2FsYQ==) | `100.00% <100.00%> (ø)` | |
| [.../scala/org/apache/kyuubi/server/KyuubiServer.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpU2VydmVyLnNjYWxh) | `58.33% <100.00%> (+1.81%)` | :arrow_up: |
| [.../apache/kyuubi/server/api/v1/ApiRootResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0FwaVJvb3RSZXNvdXJjZS5zY2FsYQ==) | `40.00% <0.00%> (-26.67%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `85.36% <0.00%> (-6.53%)` | :arrow_down: |
| ... and [12 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...595784f](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8afe861) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/4f8f4d9c6b0b6da805d767dfb07534a906b0b2e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4f8f4d9) will **decrease** coverage by `78.76%`.
> The diff coverage is `0.00%`.
> :exclamation: Current head 8afe861 differs from pull request most recent head 3517ddb. Consider uploading reports for the commit 3517ddb to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 78.76% 0.00% -78.77%
============================================
Files 182 157 -25
Lines 6754 6053 -701
Branches 805 639 -166
============================================
- Hits 5320 0 -5320
- Misses 979 6053 +5074
+ Partials 455 0 -455
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (-95.34%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `0.00% <0.00%> (-64.71%)` | :arrow_down: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `0.00% <0.00%> (-82.80%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `0.00% <0.00%> (-90.48%)` | :arrow_down: |
| [...main/scala/org/apache/kyuubi/KyuubiException.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9LeXV1YmlFeGNlcHRpb24uc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...main/scala/org/apache/kyuubi/session/package.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL3BhY2thZ2Uuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ain/scala/org/apache/kyuubi/util/ThreadUtils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS91dGlsL1RocmVhZFV0aWxzLnNjYWxh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/engine/ShareLevel.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvU2hhcmVMZXZlbC5zY2FsYQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...n/scala/org/apache/kyuubi/config/ConfigEntry.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvQ29uZmlnRW50cnkuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [162 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4f8f4d9...3517ddb](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731830607
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
it's called `KyuubiAuditEvent`, how to follow this naming for session
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r737951480
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -846,6 +853,13 @@ object KyuubiConf {
.stringConf
.createWithDefault("/tmp/kyuubi/events")
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
+ .version("1.4.0")
+ .stringConf
+ .createWithDefault("/tmp/kyuubi/events")
Review comment:
`/tmp/kyuubi/events` to `/tmp/kyuubi/events/audit` will be more clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3517ddb) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d0d5fb6d4fd177c5e4325bbb3cf5a602c10a9be3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0d5fb6) will **increase** coverage by `0.17%`.
> The diff coverage is `90.00%`.
> :exclamation: Current head 3517ddb differs from pull request most recent head 743b82b. Consider uploading reports for the commit 743b82b to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 73.10% 73.28% +0.17%
Complexity 41 41
============================================
Files 175 176 +1
Lines 6846 6917 +71
Branches 835 838 +3
============================================
+ Hits 5005 5069 +64
- Misses 1441 1443 +2
- Partials 400 405 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `88.23% <88.88%> (-2.25%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.49% <94.73%> (+0.25%)` | :arrow_up: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `95.65% <95.65%> (ø)` | |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `89.89% <0.00%> (-1.02%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `89.15% <0.00%> (-0.62%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d0d5fb6...743b82b](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r739090541
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/server/EventLoggingService.scala
##########
@@ -48,9 +62,23 @@ class EventLoggingService extends AbstractEventLoggingService[KyuubiServerEvent]
// TODO: Add more implementations
throw new IllegalArgumentException(s"Unrecognized event logger: $logger")
}
+
+ // load audit logger
+ if (conf.get(AUDIT_LOG_ENABLE)) {
+ val jsonAuditLogger = new JsonEventLogger[KyuubiServerEvent](s"server-$hostName",
+ SERVER_AUDIT_EVENT_JSON_LOG_PATH, new Configuration())
+ jsonAuditLogger.createEventLogRootDir(conf, KyuubiHadoopUtils.newHadoopConf(conf))
+ addService(jsonAuditLogger)
Review comment:
> hi, what different between `addService(jsonAuditLogger)` and `addAuditEventLogger(jsonAuditLogger)`
`addService` used to init logger(create FileSystem), `addAuditEventLogger` used to add this logger to logger list which decide decide which loggers to use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fd6a755) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/c78fe427edf3ea2e4d94e9cc1801058818438db3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c78fe42) will **increase** coverage by `0.07%`.
> The diff coverage is `88.31%`.
> :exclamation: Current head fd6a755 differs from pull request most recent head 8e6481b. Consider uploading reports for the commit 8e6481b to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 78.51% 78.59% +0.07%
Complexity 110 110
============================================
Files 178 179 +1
Lines 7241 7291 +50
Branches 887 884 -3
============================================
+ Hits 5685 5730 +45
+ Misses 1071 1069 -2
- Partials 485 492 +7
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `86.66% <75.00%> (-3.13%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `87.09% <81.81%> (-3.38%)` | :arrow_down: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `97.14% <97.14%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.20% <100.00%> (+0.04%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (-4.43%)` | :arrow_down: |
| [...ubi/server/KyuubiThriftBinaryFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpVGhyaWZ0QmluYXJ5RnJvbnRlbmRTZXJ2aWNlLnNjYWxh) | `88.88% <0.00%> (-1.12%)` | :arrow_down: |
| [...ngine/spark/SparkThriftBinaryFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1RocmlmdEJpbmFyeUZyb250ZW5kU2VydmljZS5zY2FsYQ==) | `79.03% <0.00%> (-0.34%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [c78fe42...8e6481b](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r741574285
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/server/EventLoggingService.scala
##########
@@ -48,9 +62,23 @@ class EventLoggingService extends AbstractEventLoggingService[KyuubiServerEvent]
// TODO: Add more implementations
throw new IllegalArgumentException(s"Unrecognized event logger: $logger")
}
+
+ // load audit logger
+ if (conf.get(AUDIT_LOG_ENABLE)) {
+ val jsonAuditLogger = new JsonEventLogger[KyuubiServerEvent](s"server-$hostName",
+ SERVER_AUDIT_EVENT_JSON_LOG_PATH, new Configuration())
+ jsonAuditLogger.createEventLogRootDir(conf, KyuubiHadoopUtils.newHadoopConf(conf))
+ addService(jsonAuditLogger)
Review comment:
the `jsonAuditLogger` is transfered two places seems not a good practice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-899382668
Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r732595503
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
> it's called `KyuubiAuditEvent`, how to follow this naming for session
Related naming, should we write all audit log in a file, or ,like events, write to different files? I prefer to write to the same file.
If write to th same file, how about `KyuubiAuditStatementEvent` and `KyuubiAuditSessionEvent` with the same `eventType`: `KyuubiAuditEvent`.
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -862,6 +876,22 @@ object KyuubiConf {
"Unsupported event loggers")
.createWithDefault(Nil)
+
+ val AUDIT_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
Review comment:
Yep
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r737952314
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/server/EventLoggingService.scala
##########
@@ -48,9 +62,23 @@ class EventLoggingService extends AbstractEventLoggingService[KyuubiServerEvent]
// TODO: Add more implementations
throw new IllegalArgumentException(s"Unrecognized event logger: $logger")
}
+
+ // load audit logger
+ if (conf.get(AUDIT_LOG_ENABLE)) {
+ val jsonAuditLogger = new JsonEventLogger[KyuubiServerEvent](s"server-$hostName",
+ SERVER_AUDIT_EVENT_JSON_LOG_PATH, new Configuration())
+ jsonAuditLogger.createEventLogRootDir(conf, KyuubiHadoopUtils.newHadoopConf(conf))
+ addService(jsonAuditLogger)
Review comment:
hi, what different between `addService(jsonAuditLogger)` and `addAuditEventLogger(jsonAuditLogger)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r736181716
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -252,6 +252,13 @@ object KyuubiConf {
.booleanConf
.createWithDefault(true)
+ val AUDIT_LOG_ENABLE: ConfigEntry[Boolean] =
+ buildConf("audit.log.enable")
+ .doc("When true, store sql statement for audit.")
+ .version("1.4.0")
+ .booleanConf
+ .createWithDefault(false)
Review comment:
we can turn it to true in future releases, I guess
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6f52d1a) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.14%`.
> The diff coverage is `83.87%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 79.29% -0.15%
- Complexity 87 90 +3
============================================
Files 176 181 +5
Lines 6552 6681 +129
Branches 775 786 +11
============================================
+ Hits 5205 5298 +93
- Misses 897 926 +29
- Partials 450 457 +7
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `81.00% <57.14%> (-1.80%)` | :arrow_down: |
| [...org/apache/kyuubi/server/AuditLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvQXVkaXRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `77.27% <77.27%> (ø)` | |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/audit/ServerAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9TZXJ2ZXJBdWRpdEV2ZW50LnNjYWxh) | `90.00% <90.00%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.47% <100.00%> (+0.16%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...la/org/apache/kyuubi/audit/ServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9TZXJ2ZXJBdWRpdExvZ2dlci5zY2FsYQ==) | `100.00% <100.00%> (ø)` | |
| [.../scala/org/apache/kyuubi/server/KyuubiServer.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpU2VydmVyLnNjYWxh) | `58.33% <100.00%> (+1.81%)` | :arrow_up: |
| [.../apache/kyuubi/server/api/v1/ApiRootResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0FwaVJvb3RSZXNvdXJjZS5zY2FsYQ==) | `40.00% <0.00%> (-26.67%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `85.36% <0.00%> (-6.53%)` | :arrow_down: |
| ... and [12 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...6f52d1a](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r725462906
##########
File path: bin/kyuubi
##########
@@ -96,6 +96,8 @@ if [[ -z "$KYUUBI_JAR_DIR" ]]; then
fi
KYUUBI_CLASSPATH="${KYUUBI_JAR_DIR}/*:${KYUUBI_CONF_DIR}:${HADOOP_CONF_DIR}"
+# Add kyuubi.home for log4j
+KYUUBI_JAVA_OPTS="-Dkyuubi.home=${KYUUBI_HOME} ${KYUUBI_JAVA_OPTS}"
Review comment:
Temporarily removed, found that `operation_log` and `metrics` also have the problem. When we exec `kyuubi` in any directory, the log file will be writted there. Maybe we can resolve it with other PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r724659512
##########
File path: bin/kyuubi
##########
@@ -96,6 +96,8 @@ if [[ -z "$KYUUBI_JAR_DIR" ]]; then
fi
KYUUBI_CLASSPATH="${KYUUBI_JAR_DIR}/*:${KYUUBI_CONF_DIR}:${HADOOP_CONF_DIR}"
+# Add kyuubi.home for log4j
+KYUUBI_JAVA_OPTS="-Dkyuubi.home=${KYUUBI_HOME} ${KYUUBI_JAVA_OPTS}"
Review comment:
This is hard to use, can we follow the ideas from operation log and kyuubi-metircs, the both use log4j to output messages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-924562823
Not very familiar with SQL auditing, I don't know how to review this PR. Can we elaborate more in the PR description? why we need this? what is the relationship between the current event implementation? which systems support this and what are their best practices?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r719251954
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/EventLoggerType.scala
##########
@@ -22,5 +22,5 @@ object EventLoggerType extends Enumeration {
type EventLoggerType = Value
// TODO: Only SPARK is done now
- val SPARK, JSON, DB, CUSTOM = Value
+ val SPARK, JSON, DB, LOG4J, CUSTOM = Value
Review comment:
we can name it `audit`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (595784f) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.15%`.
> The diff coverage is `85.93%`.
> :exclamation: Current head 595784f differs from pull request most recent head deff62e. Consider uploading reports for the commit deff62e to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 79.28% -0.16%
- Complexity 87 101 +14
============================================
Files 176 184 +8
Lines 6552 6709 +157
Branches 775 789 +14
============================================
+ Hits 5205 5319 +114
- Misses 897 931 +34
- Partials 450 459 +9
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/events/Log4jEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvTG9nNGpFdmVudExvZ2dlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `80.00% <75.00%> (-10.00%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.46% <100.00%> (+0.15%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `96.29% <100.00%> (+0.64%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.15% <100.00%> (+0.36%)` | :arrow_up: |
| [.../apache/kyuubi/server/api/v1/ApiRootResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0FwaVJvb3RSZXNvdXJjZS5zY2FsYQ==) | `40.00% <0.00%> (-26.67%)` | :arrow_down: |
| [...pache/kyuubi/service/AbstractFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2aWNlL0Fic3RyYWN0RnJvbnRlbmRTZXJ2aWNlLnNjYWxh) | `57.14% <0.00%> (-9.53%)` | :arrow_down: |
| ... and [38 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...deff62e](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #908: [KYUUBI#891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r685084314
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -69,14 +77,18 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
this.operationException = opEx
}
- protected def setState(newState: OperationState): Unit = {
+ protected def setState(newState: OperationState, audit: Boolean = true): Unit = {
Review comment:
why do we need `audit: Boolean` ?
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -118,6 +130,18 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
}
}
+ def logAuditEvent(sql: String, state: OperationState): Unit = {
+ if (sql != null) {
+ val user = session.user
+ val address = session.ipAddress match {
+ case s if s != null => s
+ case _ => "unknown-ip-addr"
+ }
+
+ getAuditLog.get.info(s"user=$user\tip=$address\tstate=${state.name}\tsql=$sql\t")
Review comment:
shall we log start time and elapsed time ? and also log app id I think it can help debug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r725462096
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/EventLoggerType.scala
##########
@@ -22,5 +22,5 @@ object EventLoggerType extends Enumeration {
type EventLoggerType = Value
// TODO: Only SPARK is done now
- val SPARK, JSON, DB, CUSTOM = Value
+ val SPARK, JSON, DB, LOG4J, CUSTOM = Value
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r725467876
##########
File path: bin/kyuubi
##########
@@ -96,6 +96,8 @@ if [[ -z "$KYUUBI_JAR_DIR" ]]; then
fi
KYUUBI_CLASSPATH="${KYUUBI_JAR_DIR}/*:${KYUUBI_CONF_DIR}:${HADOOP_CONF_DIR}"
+# Add kyuubi.home for log4j
+KYUUBI_JAVA_OPTS="-Dkyuubi.home=${KYUUBI_HOME} ${KYUUBI_JAVA_OPTS}"
Review comment:
yes, that's a long standing bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r726690873
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -272,6 +272,36 @@ object KyuubiConf {
.booleanConf
.createWithDefault(true)
+ val AUDIT_LOG_ENABLE: ConfigEntry[Boolean] =
+ buildConf("audit.log.enable")
+ .doc("When true, store sql statement for audit.")
+ .version("1.4.0")
+ .booleanConf
+ .createWithDefault(true)
+
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
Review comment:
> why we use both json and log4j pattern for auditting
Whether JSON, log4j or CUSTOM is just a method of auditing, we can choose one or more of them.
Actually, it is not must bind to log4j. Other components typically use log4j, so we may need support it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b1521ce) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/c78fe427edf3ea2e4d94e9cc1801058818438db3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c78fe42) will **increase** coverage by `0.10%`.
> The diff coverage is `88.46%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 78.51% 78.61% +0.10%
Complexity 110 110
============================================
Files 178 179 +1
Lines 7241 7317 +76
Branches 887 891 +4
============================================
+ Hits 5685 5752 +67
- Misses 1071 1073 +2
- Partials 485 492 +7
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `86.88% <75.00%> (-2.92%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `87.09% <81.81%> (-3.38%)` | :arrow_down: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `97.14% <97.14%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.24% <100.00%> (+0.08%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.11% <100.00%> (+0.36%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [c78fe42...b1521ce](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (743b82b) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d0d5fb6d4fd177c5e4325bbb3cf5a602c10a9be3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0d5fb6) will **increase** coverage by `4.30%`.
> The diff coverage is `88.15%`.
> :exclamation: Current head 743b82b differs from pull request most recent head d6e2415. Consider uploading reports for the commit d6e2415 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 73.10% 77.41% +4.30%
- Complexity 41 110 +69
============================================
Files 175 179 +4
Lines 6846 7291 +445
Branches 835 884 +49
============================================
+ Hits 5005 5644 +639
+ Misses 1441 1158 -283
- Partials 400 489 +89
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `85.00% <66.66%> (-4.59%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `83.87% <72.72%> (-6.61%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `97.14% <97.14%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.20% <100.00%> (-0.04%)` | :arrow_down: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [...che/kyuubi/ha/client/ZooKeeperClientProvider.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2hhL2NsaWVudC9ab29LZWVwZXJDbGllbnRQcm92aWRlci5zY2FsYQ==) | `70.17% <0.00%> (-11.88%)` | :arrow_down: |
| [...g/apache/kyuubi/engine/spark/udf/KDFRegistry.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay91ZGYvS0RGUmVnaXN0cnkuc2NhbGE=) | `90.47% <0.00%> (-9.53%)` | :arrow_down: |
| [...uubi/engine/spark/events/SparkStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtTdGF0ZW1lbnRFdmVudC5zY2FsYQ==) | `76.47% <0.00%> (-6.87%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `86.86% <0.00%> (-2.91%)` | :arrow_down: |
| ... and [31 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d0d5fb6...d6e2415](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r730641904
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
What's the difference between this and the statement event? can we reuse?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731623200
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
We need to save them to a different path. If in the same path, there will be some duplication records.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3517ddb) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/4f8f4d9c6b0b6da805d767dfb07534a906b0b2e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4f8f4d9) will **decrease** coverage by `5.48%`.
> The diff coverage is `90.00%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 78.76% 73.28% -5.49%
+ Complexity 93 41 -52
============================================
Files 182 176 -6
Lines 6754 6917 +163
Branches 805 838 +33
============================================
- Hits 5320 5069 -251
- Misses 979 1443 +464
+ Partials 455 405 -50
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `88.23% <88.88%> (-2.25%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.49% <94.73%> (+0.15%)` | :arrow_up: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `95.65% <95.65%> (ø)` | |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [.../scala/org/apache/spark/kyuubi/ui/EnginePage.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsva3l1dWJpL3VpL0VuZ2luZVBhZ2Uuc2NhbGE=) | `80.76% <0.00%> (-14.47%)` | :arrow_down: |
| [.../engine/spark/session/SparkSQLSessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU1FMU2Vzc2lvbk1hbmFnZXIuc2NhbGE=) | `86.84% <0.00%> (-5.27%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `85.88% <0.00%> (-2.87%)` | :arrow_down: |
| [...g/apache/kyuubi/engine/spark/KyuubiSparkUtil.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9LeXV1YmlTcGFya1V0aWwuc2NhbGE=) | `88.46% <0.00%> (-2.85%)` | :arrow_down: |
| [...ine/spark/operation/SparkSQLOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vU3BhcmtTUUxPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `84.44% <0.00%> (-0.93%)` | :arrow_down: |
| ... and [49 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4f8f4d9...3517ddb](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3517ddb) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d0d5fb6d4fd177c5e4325bbb3cf5a602c10a9be3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0d5fb6) will **increase** coverage by `0.17%`.
> The diff coverage is `90.00%`.
> :exclamation: Current head 3517ddb differs from pull request most recent head a39b45e. Consider uploading reports for the commit a39b45e to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 73.10% 73.28% +0.17%
Complexity 41 41
============================================
Files 175 176 +1
Lines 6846 6917 +71
Branches 835 838 +3
============================================
+ Hits 5005 5069 +64
- Misses 1441 1443 +2
- Partials 400 405 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `88.23% <88.88%> (-2.25%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.49% <94.73%> (+0.25%)` | :arrow_up: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `95.65% <95.65%> (ø)` | |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `89.89% <0.00%> (-1.02%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `89.15% <0.00%> (-0.62%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d0d5fb6...a39b45e](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r737951480
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -846,6 +853,13 @@ object KyuubiConf {
.stringConf
.createWithDefault("/tmp/kyuubi/events")
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
+ .version("1.4.0")
+ .stringConf
+ .createWithDefault("/tmp/kyuubi/events")
Review comment:
maybe `/tmp/kyuubi/events` to `/tmp/kyuubi/events/audit` will be more clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r733261246
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
sessions ans statements have many state type, we really need to consider the his impact on performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-958596444
hi, if kyuubi integrates flink, do we need add audit service again? the audit service is hign level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-958596444
hi, if kyuubi integrates flink, do we need add audit service again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-958596444
hi, if kyuubi integrates flink, do we need add audit service again? the audit service is hign level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-958596444
hi, if kyuubi integrates flink, do we need add audit service again? the audit service is hign level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-926312946
> , the event from initize to end will be logged,
we can do filtering based on the contents of the events, e.g. state?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r740084270
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -846,6 +853,13 @@ object KyuubiConf {
.stringConf
.createWithDefault("/tmp/kyuubi/events")
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
+ .version("1.4.0")
+ .stringConf
+ .createWithDefault("/tmp/kyuubi/events")
Review comment:
Had add it to doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731638524
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
If `AuditEvent` reuse `KyuubiStatementEvent`, they will all stored in `kyuubi_statement` path. When enable events and audit both, there will be some duplication event records.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (deff62e) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/aa1a7d63beef1c4d5dbe39f5baab592955686bd7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (aa1a7d6) will **decrease** coverage by `0.58%`.
> The diff coverage is `82.60%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.44% 78.85% -0.59%
- Complexity 87 93 +6
============================================
Files 176 184 +8
Lines 6552 6736 +184
Branches 775 789 +14
============================================
+ Hits 5205 5312 +107
- Misses 897 972 +75
- Partials 450 452 +2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/events/Log4jEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvTG9nNGpFdmVudExvZ2dlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `75.00% <68.00%> (-15.00%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.54% <100.00%> (+0.24%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `96.29% <100.00%> (+0.64%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.15% <100.00%> (+0.36%)` | :arrow_up: |
| [...main/java/org/apache/kyuubi/jdbc/KyuubiDriver.java](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhpdmUtamRiYy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3l1dWJpL2pkYmMvS3l1dWJpRHJpdmVyLmphdmE=) | `0.00% <0.00%> (-57.15%)` | :arrow_down: |
| [.../apache/kyuubi/server/api/v1/ApiRootResource.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvYXBpL3YxL0FwaVJvb3RSZXNvdXJjZS5zY2FsYQ==) | `40.00% <0.00%> (-26.67%)` | :arrow_down: |
| ... and [49 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [aa1a7d6...deff62e](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-943378146
> @yaooqinn @timothy65535 : What makes you confused? Audit log or Log4j logger? if just log4j logger, we can remove and just audit event use json file first.
removing the log4j part temporarily will make this PR more clear to understand
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r719251954
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/events/EventLoggerType.scala
##########
@@ -22,5 +22,5 @@ object EventLoggerType extends Enumeration {
type EventLoggerType = Value
// TODO: Only SPARK is done now
- val SPARK, JSON, DB, CUSTOM = Value
+ val SPARK, JSON, DB, LOG4J, CUSTOM = Value
Review comment:
we can name it `AUDITLOG `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-899329773
For auditing, can we have a full design? This PR seems to be inflexible, which only focus on statements and only dump the log to log4j
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r725759708
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -272,6 +272,36 @@ object KyuubiConf {
.booleanConf
.createWithDefault(true)
+ val AUDIT_LOG_ENABLE: ConfigEntry[Boolean] =
+ buildConf("audit.log.enable")
+ .doc("When true, store sql statement for audit.")
+ .version("1.4.0")
+ .booleanConf
+ .createWithDefault(true)
+
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
Review comment:
why we use both json and log4j pattern for auditting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (743b82b) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d0d5fb6d4fd177c5e4325bbb3cf5a602c10a9be3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0d5fb6) will **increase** coverage by `4.30%`.
> The diff coverage is `88.15%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 73.10% 77.41% +4.30%
- Complexity 41 110 +69
============================================
Files 175 179 +4
Lines 6846 7291 +445
Branches 835 884 +49
============================================
+ Hits 5005 5644 +639
+ Misses 1441 1158 -283
- Partials 400 489 +89
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `85.00% <66.66%> (-4.59%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `83.87% <72.72%> (-6.61%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `97.14% <97.14%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.20% <100.00%> (-0.04%)` | :arrow_down: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [...che/kyuubi/ha/client/ZooKeeperClientProvider.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2hhL2NsaWVudC9ab29LZWVwZXJDbGllbnRQcm92aWRlci5zY2FsYQ==) | `70.17% <0.00%> (-11.88%)` | :arrow_down: |
| [...g/apache/kyuubi/engine/spark/udf/KDFRegistry.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay91ZGYvS0RGUmVnaXN0cnkuc2NhbGE=) | `90.47% <0.00%> (-9.53%)` | :arrow_down: |
| [...uubi/engine/spark/events/SparkStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtTdGF0ZW1lbnRFdmVudC5zY2FsYQ==) | `76.47% <0.00%> (-6.87%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `86.86% <0.00%> (-2.91%)` | :arrow_down: |
| ... and [31 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d0d5fb6...743b82b](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-953336298
cc @yaooqinn @timothy65535
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fd6a755) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d0d5fb6d4fd177c5e4325bbb3cf5a602c10a9be3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0d5fb6) will **increase** coverage by `5.48%`.
> The diff coverage is `88.31%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 73.10% 78.59% +5.48%
- Complexity 41 110 +69
============================================
Files 175 179 +4
Lines 6846 7291 +445
Branches 835 884 +49
============================================
+ Hits 5005 5730 +725
+ Misses 1441 1069 -372
- Partials 400 492 +92
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `86.66% <75.00%> (-2.92%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `87.09% <81.81%> (-3.38%)` | :arrow_down: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `97.14% <97.14%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.20% <100.00%> (-0.04%)` | :arrow_down: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [...che/kyuubi/ha/client/ZooKeeperClientProvider.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWhhL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2hhL2NsaWVudC9ab29LZWVwZXJDbGllbnRQcm92aWRlci5zY2FsYQ==) | `70.17% <0.00%> (-11.88%)` | :arrow_down: |
| [...g/apache/kyuubi/engine/spark/udf/KDFRegistry.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay91ZGYvS0RGUmVnaXN0cnkuc2NhbGE=) | `90.47% <0.00%> (-9.53%)` | :arrow_down: |
| [...uubi/engine/spark/events/SparkStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvU3BhcmtTdGF0ZW1lbnRFdmVudC5zY2FsYQ==) | `76.47% <0.00%> (-6.87%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `86.86% <0.00%> (-2.91%)` | :arrow_down: |
| ... and [37 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d0d5fb6...fd6a755](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-958596444
hi, if kyuubi integrates flink, do we need add audit service again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r730639062
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/operation/AbstractOperation.scala
##########
@@ -84,6 +94,10 @@ abstract class AbstractOperation(opType: OperationType, session: Session)
lastAccessTime = System.currentTimeMillis()
}
+ def logAuditEvent(state: OperationState, elapsed: Double): Unit = {
Review comment:
nit: protected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731623200
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
> What's the difference between this and the statement event? can we reuse?
We need to save them to a different path. If in the same path, there will be some duplication records.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-925530265
maybe we can reuse the current event logger system and only create a sink type for `log4j`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (44c192d) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/6dd8dfb70c2cbe7f92cf42ca68444c404767f699?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6dd8dfb) will **decrease** coverage by `0.06%`.
> The diff coverage is `75.00%`.
> :exclamation: Current head 44c192d differs from pull request most recent head 6a06892. Consider uploading reports for the commit 6a06892 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
- Coverage 79.23% 79.16% -0.07%
Complexity 11 11
============================================
Files 145 144 -1
Lines 5423 5410 -13
Branches 651 654 +3
============================================
- Hits 4297 4283 -14
+ Misses 759 758 -1
- Partials 367 369 +2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../scala/org/apache/kyuubi/operation/Operation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vT3BlcmF0aW9uLnNjYWxh) | `100.00% <ø> (ø)` | |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `64.06% <61.53%> (-0.65%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.60% <100.00%> (+0.14%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `94.54% <100.00%> (-0.12%)` | :arrow_down: |
| [...ala/org/apache/kyuubi/operation/log/AuditLog.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vbG9nL0F1ZGl0TG9nLnNjYWxh) | `100.00% <100.00%> (ø)` | |
| [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `82.60% <0.00%> (-4.90%)` | :arrow_down: |
| [...g/apache/kyuubi/engine/spark/KyuubiSparkUtil.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9LeXV1YmlTcGFya1V0aWwuc2NhbGE=) | `90.90% <0.00%> (-0.40%)` | :arrow_down: |
| [...ommon/src/main/scala/org/apache/kyuubi/Utils.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9VdGlscy5zY2FsYQ==) | `73.41% <0.00%> (-0.34%)` | :arrow_down: |
| [.../scala/org/apache/kyuubi/metrics/MetricsConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLW1ldHJpY3Mvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvbWV0cmljcy9NZXRyaWNzQ29uZi5zY2FsYQ==) | `97.43% <0.00%> (-0.07%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [6dd8dfb...6a06892](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-899268587
hi @hddong , Can you explain the difference between the audit log and statement event (see https://github.com/apache/incubator-kyuubi/pull/903) ? As we are going to add metrics in Kyuubi (see umbrella https://github.com/apache/incubator-kyuubi/issues/852). We can improve the eixsted metrics if there are anything missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (deff62e) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/4f8f4d9c6b0b6da805d767dfb07534a906b0b2e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4f8f4d9) will **increase** coverage by `0.09%`.
> The diff coverage is `82.60%`.
> :exclamation: Current head deff62e differs from pull request most recent head d9d40fe. Consider uploading reports for the commit d9d40fe to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 78.76% 78.85% +0.09%
Complexity 93 93
============================================
Files 182 184 +2
Lines 6754 6736 -18
Branches 805 789 -16
============================================
- Hits 5320 5312 -8
+ Misses 979 972 -7
+ Partials 455 452 -3
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/events/Log4jEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvTG9nNGpFdmVudExvZ2dlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `75.00% <68.00%> (-15.48%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.54% <100.00%> (+0.20%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `96.29% <100.00%> (+0.64%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.15% <100.00%> (+0.36%)` | :arrow_up: |
| [...main/scala/org/apache/kyuubi/session/package.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL3BhY2thZ2Uuc2NhbGE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ala/org/apache/kyuubi/session/SessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25NYW5hZ2VyLnNjYWxh) | `56.36% <0.00%> (-6.57%)` | :arrow_down: |
| ... and [12 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4f8f4d9...d9d40fe](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r724659512
##########
File path: bin/kyuubi
##########
@@ -96,6 +96,8 @@ if [[ -z "$KYUUBI_JAR_DIR" ]]; then
fi
KYUUBI_CLASSPATH="${KYUUBI_JAR_DIR}/*:${KYUUBI_CONF_DIR}:${HADOOP_CONF_DIR}"
+# Add kyuubi.home for log4j
+KYUUBI_JAVA_OPTS="-Dkyuubi.home=${KYUUBI_HOME} ${KYUUBI_JAVA_OPTS}"
Review comment:
This is hard to use, can we follow the ideas from operation log and kyuubi-metircs, they both use log4j to output messages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-899295510
@ulysses-you : Seems that statement events contians audit information, but audit log focus on who did what, used to audit users's actions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-899258771
@ulysses-you: please review this again when free.
cc @yaooqinn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-929101464
> > we can do filtering based on the contents of the events, e.g. state?
>
> @yaooqinn ： Sorry for feedback, been busy with other things lately. `onEvent` log all events, but not all `KyuubiServerEvent` have `state`. We can reuse loggers, and add `auditLoggers` and `onAuditEvent` in `EventLoggingService` to make events and audit relatively independent. Then, we can choose to use one or all of them as needed.
SGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d9d40fe) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/4f8f4d9c6b0b6da805d767dfb07534a906b0b2e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4f8f4d9) will **increase** coverage by `0.03%`.
> The diff coverage is `85.48%`.
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 78.76% 78.80% +0.03%
Complexity 93 93
============================================
Files 182 184 +2
Lines 6754 6809 +55
Branches 805 809 +4
============================================
+ Hits 5320 5366 +46
- Misses 979 984 +5
- Partials 455 459 +4
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ala/org/apache/kyuubi/events/EventLoggerType.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvRXZlbnRMb2dnZXJUeXBlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/events/Log4jEventLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvTG9nNGpFdmVudExvZ2dlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `80.55% <75.00%> (-9.93%)` | :arrow_down: |
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `66.66% <85.71%> (+1.96%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.53% <100.00%> (+0.19%)` | :arrow_up: |
| [...apache/kyuubi/audit/DefaultServerAuditLogger.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9hdWRpdC9EZWZhdWx0U2VydmVyQXVkaXRMb2dnZXIuc2NhbGE=) | `100.00% <100.00%> (ø)` | |
| [...rg/apache/kyuubi/events/KyuubiStatementEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpU3RhdGVtZW50RXZlbnQuc2NhbGE=) | `96.29% <100.00%> (+0.64%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.15% <100.00%> (+0.36%)` | :arrow_up: |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `77.31% <0.00%> (-1.04%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4f8f4d9...d9d40fe](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-925470623
@yaooqinn : sorry for feedback late.
Actually, current events contain audit events. But, the current events are too complicated and huge, it record the beginning to the end of all the event. Audit log focus on who did what(one action one record). Others, such as hive and MYSQL, also have audit logs, It is mainly used for information security review.
For this PR, `AuditLoggingService` is similar to `EventLoggingService`, used to load audit loggers (we can consider merging the two later). `DefaultServerAuditLogger` use log4j to logging audit events to file, it extends of `ServerAuditLogger`, and users can define self's logger by extends ServerAuditLogger.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on pull request #908: [KYUUBI #891]Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-924519110
cc @yaooqinn @ulysses-you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r732148819
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
An idea: we should have a clear motivation to introduce the audit log feature, don't like #1223
These loggers will write events synchronously, they will affect the profermance.
**Events**
- SessionEvent at both engine side and server side.
- SparkStatementEvent at engine side and server side.
- Kyuubi Server Event
- Ongoing kyuubi audit log
**Logger**
- SparkLogger
- JsonLogger
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r731682279
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/events/KyuubiAuditEvent.scala
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.events
+
+import org.apache.kyuubi.Utils
+
+/**
+ *
+ * @param createTime: the create time of this statement
+ * @param user: who connect to kyuubi server
+ * @param remoteIp: the ip of user
+ * @param statementId: the identifier of operationHandler
+ * @param sessionId: the identifier of a session
+ * @param state: store each state that the sql has
+ * @param completedTime: the time that the sql's state change
+ * @param statement: the sql that you execute
+ * @param elapsedTime: the time that the sql elapsed
+ */
+case class KyuubiAuditEvent(
Review comment:
Yes, now is only for statements. I will add for session later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r739095870
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -846,6 +853,13 @@ object KyuubiConf {
.stringConf
.createWithDefault("/tmp/kyuubi/events")
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
+ .version("1.4.0")
+ .stringConf
+ .createWithDefault("/tmp/kyuubi/events")
Review comment:
Got, seems it will be more clear if we use `/tmp/kyuubi/events/audit/kyuubi_audit`, or we can add the default subdir `kyuubi_audit` to the doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fd6a755) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/c78fe427edf3ea2e4d94e9cc1801058818438db3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c78fe42) will **increase** coverage by `0.07%`.
> The diff coverage is `88.31%`.
> :exclamation: Current head fd6a755 differs from pull request most recent head b1521ce. Consider uploading reports for the commit b1521ce to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 78.51% 78.59% +0.07%
Complexity 110 110
============================================
Files 178 179 +1
Lines 7241 7291 +50
Branches 887 884 -3
============================================
+ Hits 5685 5730 +45
+ Misses 1071 1069 -2
- Partials 485 492 +7
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [.../org/apache/kyuubi/session/KyuubiSessionImpl.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25JbXBsLnNjYWxh) | `86.66% <75.00%> (-3.13%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `87.09% <81.81%> (-3.38%)` | :arrow_down: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `97.14% <97.14%> (ø)` | |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.20% <100.00%> (+0.04%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (-4.43%)` | :arrow_down: |
| [...ubi/server/KyuubiThriftBinaryFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpVGhyaWZ0QmluYXJ5RnJvbnRlbmRTZXJ2aWNlLnNjYWxh) | `88.88% <0.00%> (-1.12%)` | :arrow_down: |
| [...ngine/spark/SparkThriftBinaryFrontendService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1RocmlmdEJpbmFyeUZyb250ZW5kU2VydmljZS5zY2FsYQ==) | `79.03% <0.00%> (-0.34%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [c78fe42...b1521ce](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r739086900
##########
File path: kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
##########
@@ -846,6 +853,13 @@ object KyuubiConf {
.stringConf
.createWithDefault("/tmp/kyuubi/events")
+ val SERVER_AUDIT_EVENT_JSON_LOG_PATH: ConfigEntry[String] =
+ buildConf("audit.event.json.log.path")
+ .doc("The location of server events go for the builtin JSON logger")
+ .version("1.4.0")
+ .stringConf
+ .createWithDefault("/tmp/kyuubi/events")
Review comment:
> maybe `/tmp/kyuubi/events` to `/tmp/kyuubi/events/audit` will be more clear.
`KyuubiAuditEvent` will generate sub dir `kyuubi_audit` under location.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
hddong commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r739090541
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/server/EventLoggingService.scala
##########
@@ -48,9 +62,23 @@ class EventLoggingService extends AbstractEventLoggingService[KyuubiServerEvent]
// TODO: Add more implementations
throw new IllegalArgumentException(s"Unrecognized event logger: $logger")
}
+
+ // load audit logger
+ if (conf.get(AUDIT_LOG_ENABLE)) {
+ val jsonAuditLogger = new JsonEventLogger[KyuubiServerEvent](s"server-$hostName",
+ SERVER_AUDIT_EVENT_JSON_LOG_PATH, new Configuration())
+ jsonAuditLogger.createEventLogRootDir(conf, KyuubiHadoopUtils.newHadoopConf(conf))
+ addService(jsonAuditLogger)
Review comment:
> hi, what different between `addService(jsonAuditLogger)` and `addAuditEventLogger(jsonAuditLogger)`
`addService` used to init logger(create FileSystem), `addAuditEventLogger` used to add this logger to logger list which decide which loggers to use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 commented on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 commented on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-958596444
hi, if kyuubi integrates flink, do we need add audit service again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] leo65535 commented on a change in pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
leo65535 commented on a change in pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#discussion_r741574285
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/server/EventLoggingService.scala
##########
@@ -48,9 +62,23 @@ class EventLoggingService extends AbstractEventLoggingService[KyuubiServerEvent]
// TODO: Add more implementations
throw new IllegalArgumentException(s"Unrecognized event logger: $logger")
}
+
+ // load audit logger
+ if (conf.get(AUDIT_LOG_ENABLE)) {
+ val jsonAuditLogger = new JsonEventLogger[KyuubiServerEvent](s"server-$hostName",
+ SERVER_AUDIT_EVENT_JSON_LOG_PATH, new Configuration())
+ jsonAuditLogger.createEventLogRootDir(conf, KyuubiHadoopUtils.newHadoopConf(conf))
+ addService(jsonAuditLogger)
Review comment:
the `jsonAuditLogger` is transfered two places seems not a good practice.
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/server/EventLoggingService.scala
##########
@@ -48,9 +62,23 @@ class EventLoggingService extends AbstractEventLoggingService[KyuubiServerEvent]
// TODO: Add more implementations
throw new IllegalArgumentException(s"Unrecognized event logger: $logger")
}
+
+ // load audit logger
+ if (conf.get(AUDIT_LOG_ENABLE)) {
+ val jsonAuditLogger = new JsonEventLogger[KyuubiServerEvent](s"server-$hostName",
+ SERVER_AUDIT_EVENT_JSON_LOG_PATH, new Configuration())
+ jsonAuditLogger.createEventLogRootDir(conf, KyuubiHadoopUtils.newHadoopConf(conf))
+ addService(jsonAuditLogger)
Review comment:
the `jsonAuditLogger` is transfered two places seems not a good practice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter edited a comment on pull request #908: [KYUUBI #891] Add audit log for sql statement
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #908:
URL: https://github.com/apache/incubator-kyuubi/pull/908#issuecomment-895116067
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#908](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3517ddb) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/d0d5fb6d4fd177c5e4325bbb3cf5a602c10a9be3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0d5fb6) will **increase** coverage by `0.17%`.
> The diff coverage is `90.00%`.
> :exclamation: Current head 3517ddb differs from pull request most recent head f7c9937. Consider uploading reports for the commit f7c9937 to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #908 +/- ##
============================================
+ Coverage 73.10% 73.28% +0.17%
Complexity 41 41
============================================
Files 175 176 +1
Lines 6846 6917 +71
Branches 835 838 +3
============================================
+ Hits 5005 5069 +64
- Misses 1441 1443 +2
- Partials 400 405 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/kyuubi/operation/AbstractOperation.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vQWJzdHJhY3RPcGVyYXRpb24uc2NhbGE=) | `63.15% <57.14%> (-1.55%)` | :arrow_down: |
| [...org/apache/kyuubi/server/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `88.23% <88.88%> (-2.25%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.49% <94.73%> (+0.25%)` | :arrow_up: |
| [...la/org/apache/kyuubi/events/KyuubiAuditEvent.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9ldmVudHMvS3l1dWJpQXVkaXRFdmVudC5zY2FsYQ==) | `95.65% <95.65%> (ø)` | |
| [...org/apache/kyuubi/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `83.33% <100.00%> (+0.53%)` | :arrow_up: |
| [...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvUHJvY0J1aWxkZXIuc2NhbGE=) | `89.89% <0.00%> (-1.02%)` | :arrow_down: |
| [...ache/kyuubi/engine/spark/SparkProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/908/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvc3BhcmsvU3BhcmtQcm9jZXNzQnVpbGRlci5zY2FsYQ==) | `89.15% <0.00%> (-0.62%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d0d5fb6...f7c9937](https://codecov.io/gh/apache/incubator-kyuubi/pull/908?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org