You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2022/10/19 09:32:10 UTC
[incubator-kyuubi] branch master updated: [KYUUBI #3640] Change the default event logger description of hive and trino to json instead of spark
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 17df2428e [KYUUBI #3640] Change the default event logger description of hive and trino to json instead of spark
17df2428e is described below
commit 17df2428e6f2e119cf9a9e0aa62c3fe8a46d2271
Author: jiaoqingbo <11...@qq.com>
AuthorDate: Wed Oct 19 17:32:00 2022 +0800
[KYUUBI #3640] Change the default event logger description of hive and trino to json instead of spark
### _Why are the changes needed?_
fix #3640
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes #3641 from jiaoqingbo/kyuubi3640.
Closes #3640
c1b2d4a6 [jiaoqingbo] code review
a65bc132 [jiaoqingbo] code review
ab04f135 [jiaoqingbo] [KYUUBI #3640] Change the default event logger description of hive and trino to json instead of spark
Authored-by: jiaoqingbo <11...@qq.com>
Signed-off-by: Cheng Pan <ch...@apache.org>
---
docs/deployment/settings.md | 8 ++++----
.../src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala | 8 ++++----
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/docs/deployment/settings.md b/docs/deployment/settings.md
index 5905f8689..8c019d2b1 100644
--- a/docs/deployment/settings.md
+++ b/docs/deployment/settings.md
@@ -222,11 +222,11 @@ kyuubi.engine.deregister.exception.messages||A comma separated list of exception
kyuubi.engine.deregister.exception.ttl|PT30M|Time to live(TTL) for exceptions pattern specified in kyuubi.engine.deregister.exception.classes and kyuubi.engine.deregister.exception.messages to deregister engines. Once the total error count hits the kyuubi.engine.deregister.job.max.failures within the TTL, an engine will deregister itself and wait for self-terminated. Otherwise, we suppose that the engine has recovered from temporary failures.|duration|1.2.0
kyuubi.engine.deregister.job.max.failures|4|Number of failures of job before deregistering the engine.|int|1.2.0
kyuubi.engine.event.json.log.path|file:///tmp/kyuubi/events|The location of all the engine events go for the builtin JSON logger.<ul><li>Local Path: start with 'file://'</li><li>HDFS Path: start with 'hdfs://'</li></ul>|string|1.3.0
-kyuubi.engine.event.loggers|SPARK|A comma separated list of engine history loggers, where engine/session/operation etc events go. We use spark logger by default.<ul> <li>SPARK: the events will be written to the spark listener bus.</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.3.0
+kyuubi.engine.event.loggers|SPARK|A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>SPARK: the events will be written to the spark listener bus.</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.3.0
kyuubi.engine.flink.extra.classpath|<undefined>|The extra classpath for the flink sql engine, for configuring location of hadoop client jars, etc|string|1.6.0
kyuubi.engine.flink.java.options|<undefined>|The extra java options for the flink sql engine|string|1.6.0
kyuubi.engine.flink.memory|1g|The heap memory for the flink sql engine|string|1.6.0
-kyuubi.engine.hive.event.loggers|JSON|A comma separated list of engine history loggers, where engine/session/operation etc events go. We use spark logger by default.<ul> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.7.0
+kyuubi.engine.hive.event.loggers|JSON|A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.7.0
kyuubi.engine.hive.extra.classpath|<undefined>|The extra classpath for the hive query engine, for configuring location of hadoop client jars, etc|string|1.6.0
kyuubi.engine.hive.java.options|<undefined>|The extra java options for the hive query engine|string|1.6.0
kyuubi.engine.hive.memory|1g|The heap memory for the hive query engine|string|1.6.0
@@ -251,8 +251,8 @@ kyuubi.engine.share.level|USER|Engines will be shared in different levels, avail
kyuubi.engine.share.level.sub.domain|<undefined>|(deprecated) - Using kyuubi.engine.share.level.subdomain instead|string|1.2.0
kyuubi.engine.share.level.subdomain|<undefined>|Allow end-users to create a subdomain for the share level of an engine. A subdomain is a case-insensitive string values that must be a valid zookeeper sub path. For example, for `USER` share level, an end-user can share a certain engine within a subdomain, not for all of its clients. End-users are free to create multiple engines in the `USER` share level. When disable engine pool, use 'default' if absent.|string|1.4.0
kyuubi.engine.single.spark.session|false|When set to true, this engine is running in a single session mode. All the JDBC/ODBC connections share the temporary views, function registries, SQL configuration and the current database.|boolean|1.3.0
-kyuubi.engine.spark.event.loggers|SPARK|A comma separated list of engine loggers, where engine/session/operation etc events go. We use spark logger by default.<ul> <li>SPARK: the events will be written to the spark listener bus.</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.7.0
-kyuubi.engine.trino.event.loggers|JSON|A comma separated list of engine history loggers, where engine/session/operation etc events go. We use spark logger by default.<ul> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.7.0
+kyuubi.engine.spark.event.loggers|SPARK|A comma separated list of engine loggers, where engine/session/operation etc events go.<ul> <li>SPARK: the events will be written to the spark listener bus.</li> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.7.0
+kyuubi.engine.trino.event.loggers|JSON|A comma separated list of engine history loggers, where engine/session/operation etc events go.<ul> <li>JSON: the events will be written to the location of kyuubi.engine.event.json.log.path</li> <li>JDBC: to be done</li> <li>CUSTOM: to be done.</li></ul>|seq|1.7.0
kyuubi.engine.trino.extra.classpath|<undefined>|The extra classpath for the trino query engine, for configuring other libs which may need by the trino engine |string|1.6.0
kyuubi.engine.trino.java.options|<undefined>|The extra java options for the trino query engine|string|1.6.0
kyuubi.engine.trino.memory|1g|The heap memory for the trino query engine|string|1.6.0
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
index f83866c4a..3779cbe84 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
@@ -1531,7 +1531,7 @@ object KyuubiConf {
val ENGINE_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
buildConf("kyuubi.engine.event.loggers")
.doc("A comma separated list of engine history loggers, where engine/session/operation etc" +
- " events go. We use spark logger by default.<ul>" +
+ " events go.<ul>" +
" <li>SPARK: the events will be written to the spark listener bus.</li>" +
" <li>JSON: the events will be written to the location of" +
s" ${ENGINE_EVENT_JSON_LOG_PATH.key}</li>" +
@@ -2080,7 +2080,7 @@ object KyuubiConf {
val ENGINE_SPARK_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
buildConf("kyuubi.engine.spark.event.loggers")
.doc("A comma separated list of engine loggers, where engine/session/operation etc" +
- " events go. We use spark logger by default.<ul>" +
+ " events go.<ul>" +
" <li>SPARK: the events will be written to the spark listener bus.</li>" +
" <li>JSON: the events will be written to the location of" +
s" ${ENGINE_EVENT_JSON_LOG_PATH.key}</li>" +
@@ -2092,7 +2092,7 @@ object KyuubiConf {
val ENGINE_HIVE_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
buildConf("kyuubi.engine.hive.event.loggers")
.doc("A comma separated list of engine history loggers, where engine/session/operation etc" +
- " events go. We use spark logger by default.<ul>" +
+ " events go.<ul>" +
" <li>JSON: the events will be written to the location of" +
s" ${ENGINE_EVENT_JSON_LOG_PATH.key}</li>" +
" <li>JDBC: to be done</li>" +
@@ -2109,7 +2109,7 @@ object KyuubiConf {
val ENGINE_TRINO_EVENT_LOGGERS: ConfigEntry[Seq[String]] =
buildConf("kyuubi.engine.trino.event.loggers")
.doc("A comma separated list of engine history loggers, where engine/session/operation etc" +
- " events go. We use spark logger by default.<ul>" +
+ " events go.<ul>" +
" <li>JSON: the events will be written to the location of" +
s" ${ENGINE_EVENT_JSON_LOG_PATH.key}</li>" +
" <li>JDBC: to be done</li>" +