You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/04/15 04:51:38 UTC
[GitHub] [incubator-kyuubi] zhaomin1423 opened a new pull request, #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
zhaomin1423 opened a new pull request, #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
Configuring Hive engine heap memory and java opts
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] deadwind4 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
deadwind4 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854883596
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
Now, HiveSQLEngine is coupled of `kyuubi.engine.hive.extra.classpath` in production environment. If we don't set `kyuubi.engine.hive.extra.classpath` the HiveSQLEngine will error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#issuecomment-1099868436
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2371](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b1556ba) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/86c6b1f872b18f7e027f20bbd052041919998704?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86c6b1f) will **increase** coverage by `0.04%`.
> The diff coverage is `76.00%`.
```diff
@@ Coverage Diff @@
## master #2371 +/- ##
============================================
+ Coverage 62.62% 62.66% +0.04%
Complexity 69 69
============================================
Files 357 358 +1
Lines 16964 16988 +24
Branches 2298 2299 +1
============================================
+ Hits 10623 10645 +22
Misses 5346 5346
- Partials 995 997 +2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...apache/kyuubi/engine/hive/HiveProcessBuilder.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvaGl2ZS9IaXZlUHJvY2Vzc0J1aWxkZXIuc2NhbGE=) | `78.26% <50.00%> (-6.36%)` | :arrow_down: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `96.40% <100.00%> (+0.05%)` | :arrow_up: |
| [...ache/kyuubi/operation/KyuubiOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vS3l1dWJpT3BlcmF0aW9uTWFuYWdlci5zY2FsYQ==) | `94.11% <0.00%> (-1.97%)` | :arrow_down: |
| [...che/kyuubi/engine/hive/operation/GetTypeInfo.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvb3BlcmF0aW9uL0dldFR5cGVJbmZvLnNjYWxh) | `100.00% <0.00%> (ø)` | |
| [...rg/apache/kyuubi/engine/trino/TrinoStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS10cmluby1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL3RyaW5vL1RyaW5vU3RhdGVtZW50LnNjYWxh) | `71.26% <0.00%> (+2.29%)` | :arrow_up: |
| [...i/engine/hive/operation/HiveOperationManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1oaXZlLXNxbC1lbmdpbmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvZW5naW5lL2hpdmUvb3BlcmF0aW9uL0hpdmVPcGVyYXRpb25NYW5hZ2VyLnNjYWxh) | `63.04% <0.00%> (+3.04%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86c6b1f...b1556ba](https://codecov.io/gh/apache/incubator-kyuubi/pull/2371?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854904854
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
it is an expectant behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
yaooqinn closed pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
URL: https://github.com/apache/incubator-kyuubi/pull/2371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] deadwind4 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
deadwind4 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854908755
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
The `kyuubi.engine.hive.extra.classpath` is required in production environment. I'm not sure it's user-friendly. We can discuss in this issue. https://github.com/apache/incubator-kyuubi/issues/2433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#issuecomment-1100866488
thanks, merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] deadwind4 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
deadwind4 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854883473
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
Now, HiveSQLEngine is coupled of `kyuubi.engine.hive.extra.classpath` in production environment. If we don't set `kyuubi.engine.hive.extra.classpath` the HiveSQLEngine will error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] deadwind4 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
deadwind4 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854908755
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
The` kyuubi.engine.hive.extra.classpath` is required in production environment. I'm not sure it's user-friendly. We can discuss in this issue. https://github.com/apache/incubator-kyuubi/issues/2433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#issuecomment-1100677306
it conflicts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#issuecomment-1104890358
hadoop classpath has some unnecessary dependicies for hive engine, it may be lead to some class conflicts in the future.
| |
***@***.***
|
|
***@***.***
|
---- 回复的原邮件 ----
| 发件人 | Ada ***@***.***> |
| 日期 | 2022年04月21日 16:13 |
| 收件人 | ***@***.***> |
| 抄送至 | Xiao ***@***.******@***.***> |
| 主题 | Re: [apache/incubator-kyuubi] [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts (PR #2371) |
@deadwind4 commented on this pull request.
In kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
> @@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
The kyuubi.engine.hive.extra.classpath is required in production environment. I'm not sure it's user-friendly. We can discuss in this issue. #2433
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were assigned.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhaomin1423 commented on pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#issuecomment-1100792894
> it conflicts
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] deadwind4 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
deadwind4 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854908755
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
The `kyuubi.engine.hive.extra.classpath` is required in production environment. I'm not sure it's user-friendly. We can discuss it in this issue. https://github.com/apache/incubator-kyuubi/issues/2433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] deadwind4 commented on a diff in pull request #2371: [KYUUBI #2360] [Subtask] Configuring Hive engine heap memory and java opts
Posted by GitBox <gi...@apache.org>.
deadwind4 commented on code in PR #2371:
URL: https://github.com/apache/incubator-kyuubi/pull/2371#discussion_r854883473
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/engine/hive/HiveProcessBuilder.scala:
##########
@@ -75,20 +81,21 @@ class HiveProcessBuilder(
env.get("YARN_CONF_DIR").foreach(classpathEntries.add)
// jars from hive distribution
classpathEntries.add(s"$hiveHome${File.separator}lib${File.separator}*")
- val hadoopCp = env.get("HIVE_HADOOP_CLASSPATH").orElse(env.get("HADOOP_CLASSPATH"))
- hadoopCp.foreach(path => classpathEntries.add(s"$path${File.separator}*"))
- if (hadoopCp.isEmpty) {
- warn(s"HIVE_HADOOP_CLASSPATH or HADOOP_CLASSPATH don't export.")
+ val extraCp = conf.get(ENGINE_HIVE_EXTRA_CLASSPATH)
Review Comment:
Now, HiveSQLEngine is coupled of `kyuubi.engine.hive.extra.classpath` in production environment. If we don't set `kyuubi.engine.hive.extra.classpath` the HiveSQLEngine will error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org