You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by "hcl3039359 (via GitHub)" <gi...@apache.org> on 2023/02/16 03:48:41 UTC
[GitHub] [linkis] hcl3039359 opened a new issue, #4226: [Bug] hive on tez 管理台队列不生效，依然为客户端默认队列
hcl3039359 opened a new issue, #4226:
URL: https://github.com/apache/linkis/issues/4226
### Search before asking
- [X] I searched the [issues](https://github.com/apache/linkis/issues) and found no similar issues.
### Linkis Component
linkis-engineconnn-plugin
### Steps to reproduce
tez-site.xml 默认队列为default、管理台自定义队列为root.default
在代码中增加以下配置，设置tez.queue.name
```
class HiveEngineConnFactory extends ComputationSingleExecutorEngineConnFactory with Logging {
private val HIVE_QUEUE_NAME: String = "mapreduce.job.queuename"
private val TEZ_QUEUE_NAME: String = "tez.queue.name"
private val BDP_QUEUE_NAME: String = "wds.linkis.rm.yarnqueue"
```
```
.foreach { case (k, v) =>
logger.info(s"key is $k, value is $v")
if (BDP_QUEUE_NAME.equals(k)) {
hiveConf.set(HIVE_QUEUE_NAME, v)
hiveConf.set(TEZ_QUEUE_NAME, v)
} else hiveConf.set(k, v)
}
```
运行hql语句，管理台队列无法生效，依然为默认配置
观察日志队列先被设置为root.default 后 又修改为default
`2023-02-16 03:26:38.916 INFO [hiveEngineEngine-Thread-2] org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager 447 canWorkWithSameSession - Current queue name is root.default incoming queue name is null
2023-02-16 03:26:38.916 INFO [hiveEngineEngine-Thread-2] org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager 351 closeIfNotDefault - Closing tez session if not default: sessionId=6fe7175a-c725-48e8-a899-d16fe45a8329, queueName=root.default, user=hive, doAs=true, isOpen=true, isDefault=false`
`2023-02-16 03:26:39.185 INFO [hiveEngineEngine-Thread-2] org.apache.hadoop.hive.ql.exec.tez.TezSessionState 562 setupTezParamsBasedOnMR - Config: mr(unset):mapreduce.job.queuename, mr initial value=root.default, tez(original):tez.queue.name=default, tez(final):tez.queue.name=default`
最后报错，因为default队列不存在
`2023-02-16 03:26:39.185 INFO [hiveEngineEngine-Thread-2] org.apache.hadoop.hive.ql.exec.tez.TezSessionState 562 setupTezParamsBasedOnMR - Config: mr(unset):mapreduce.job.queuename, mr initial value=root.default, tez(original):tez.queue.name=default, tez(final):tez.queue.name=default`
### Expected behavior
hive on tez 管理台队列生效，且其他tez-site配置也希望可以支持不同用户自定义
### Your environment
<!-- Please describe the linkis version you are using and basic environment information -->
<!-- 请描述您使用的linkis版本和基本环境信息 -->
- Linkis version used: 1.1.2
- Environment name and version:
- cdh-5.14.2
- hdp-3.1.5
- hive-2.1.1
- spark-3.2.1
- scala-2.12.2
- jdk 1.8.0_121
- ....
### Anything else
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] casionone commented on issue #4226: [Bug] hive on tez control panel queue does not take effect and remains the default queue for clients
Posted by "casionone (via GitHub)" <gi...@apache.org>.
casionone commented on issue #4226:
URL: https://github.com/apache/linkis/issues/4226#issuecomment-1445293809
Without further feedback, I closed this issue. You can reopen if necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] hcl3039359 commented on issue #4226: [Bug] hive on tez control panel queue does not take effect and remains the default queue for clients
Posted by "hcl3039359 (via GitHub)" <gi...@apache.org>.
hcl3039359 commented on issue #4226:
URL: https://github.com/apache/linkis/issues/4226#issuecomment-1621509075
the question is tez queue name is different with console setting, It is same with tez-site.xml whatever console setting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] casionone closed issue #4226: [Bug] hive on tez control panel queue does not take effect and remains the default queue for clients
Posted by "casionone (via GitHub)" <gi...@apache.org>.
casionone closed issue #4226: [Bug] hive on tez control panel queue does not take effect and remains the default queue for clients
URL: https://github.com/apache/linkis/issues/4226
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] hcl3039359 commented on issue #4226: [Bug] hive on tez control panel queue does not take effect and remains the default queue for clients
Posted by "hcl3039359 (via GitHub)" <gi...@apache.org>.
hcl3039359 commented on issue #4226:
URL: https://github.com/apache/linkis/issues/4226#issuecomment-1445537378
> Hello, this needs to confirm whether the parameter passed to Hive Ec is the correct queue?
The parameter is correct queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] github-actions[bot] commented on issue #4226: [Bug] hive on tez 管理台队列不生效，依然为客户端默认队列
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4226:
URL: https://github.com/apache/linkis/issues/4226#issuecomment-1432466399
## :blush: Welcome to the Apache Linkis community!!
We are glad that you are contributing by opening this issue.
Please make sure to include all the relevant context.
We will be here shortly.
If you are interested in contributing to our website project, please let us know!
You can check out our contributing guide on
:point_right: [How to Participate in Project Contribution](https://linkis.apache.org/community/how-to-contribute).
### Community
|WeChat Assistant|WeChat Public Account|
|-|-|
|<img src="https://linkis.apache.org/Images/wedatasphere_contact_01.png" width="128"/>|<img src="https://linkis.apache.org/Images/gzh_01.png" width="128"/>|
### Mailing Lists
|Name|Description|Subscribe|Unsubscribe|Archive|
|:-----|:--------|:------|:-------|:-----|
| [dev@linkis.apache.org](mailto:dev@linkis.apache.org) | community activity information | [subscribe](mailto:dev-subscribe@linkis.apache.org) | [unsubscribe](mailto:dev-unsubscribe@linkis.apache.org) | [archive](http://mail-archives.apache.org/mod_mbox/linkis-dev) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] peacewong commented on issue #4226: [Bug] The hive on tez management console queue does not take effect, and it is still the default queue of the client
Posted by "peacewong (via GitHub)" <gi...@apache.org>.
peacewong commented on issue #4226:
URL: https://github.com/apache/linkis/issues/4226#issuecomment-1434044199
Hello, this needs to confirm whether the parameter passed to Hive Ec is the correct queue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] hcl3039359 commented on issue #4226: [Bug] hive on tez control panel queue does not take effect and remains the default queue for clients
Posted by "hcl3039359 (via GitHub)" <gi...@apache.org>.
hcl3039359 commented on issue #4226:
URL: https://github.com/apache/linkis/issues/4226#issuecomment-1445541121
> Without further feedback, I closed this issue. You can reopen if necessary
It maybe changed during executing phase, i'm trying to resolve it . Please reopen this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org