[DISCUSS]Submitting tasks through linkis is relatively slow

[Andy <an...@163.com>]

————— 2022-5-25 —————

Xu Ling 20:36
I have a question, submitting tasks through linkis is relatively slow, what is the specific process of submitting tasks in linkis?

Amphibian 20:36
Look at the architecture diagram, yes

Xu Ling 20:37
Ok

peacewong@WDS 20:38
Does it mean which link needs to be positioned slowly? Is it slower than native spark-sql/hive?

Xu Ling 20:39
Much slower, task submission stage

Xu Ling 20:39
Yes

peacewong@WDS 20:44
Can you see if it is stuck on Yarn task execution?

peacewong@WDS 20:45
Other links do not feel that there is any possibility of slowness. It should be that the parameters are different when the underlying engine is submitted to the ec side for execution.

Xu Ling 20:52
Okay, check it out online

peacewong@WDS 20:53
Yes, you can post it at that time and follow up on this issue together


—————  2022-5-25  —————

许灵 20:36
我有一个问题，通过linkis提交任务比较慢，在link is 里提交任务具体的流程是怎么样的？

两栖狼 20:36
你看看架构图 有的

许灵 20:37
Ok

peacewong@WDS 20:38
是指需要定位慢在哪个环节吗？和原生spark-sql/hive比慢吗？

许灵 20:39
慢很多，任务提交阶段

许灵 20:39
是的

peacewong@WDS 20:44
可以看看是不是卡在Yarn任务执行上面吗？

peacewong@WDS 20:45
其他环节感觉不会有慢的可能，应该是在ec端提交底层引擎执行时参数不一样导致慢了

许灵 20:52
好的，到线上排查一下

peacewong@WDS 20:53
可以的，到时候可以发一下，一起跟进下这个问题

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For additional commands, e-mail: dev-help@linkis.apache.org