[DISCUSS]Submit Spark jar tasks in one go through restful submission

[Andy <an...@163.com>]

————— 2022-5-19 —————

peacewong@WDS 14:23
Hi @Xi  Do you want to submit Spark jar tasks in one go through restful submission?

peacewong@WDS 14:23
Don't want to go to linkis-client for submission?

Xi 14:23
yes

Xi 14:24
Currently under development, according to the logic of the flink plugin, so I want to find a sample request

peacewong@WDS 14:27
Now the shell commands of linkis-cli and linkis-client sdk support submission, which is not very compatible, right?

Xi 14:30
We all use restful now, shell commands can't calculate resources, right?

peacewong@WDS 14:33
It can be calculated. You call linkis directly restful, right? You didn't go to the SDK, right?

Xi 14:33
yes

peacewong@WDS 14:35
Ok, let's give the corresponding interface later, restful corresponds to this: EngineRestfulApi.createEngineConn, and the difference from other modes is that it mainly brings the LabelKeyUtils.ENGINE_CONN_MODE_LABEL_KEY=once label.

Xi 14:37
OK, mainly the flink submission jar package sample that requires the submit interface of entry

peacewong@WDS 14:37
did not take the entry ha

peacewong@WDS 14:37
It is submitted directly through linkis manager

peacewong@WDS 14:37
refer here

Xi 14:38
Does flink not take entry when submitting?

peacewong@WDS 14:38
Once tasks such as jar packages are submitted, they are not interactive, but run directly through the manager to submit the task.

Xi 14:40
[covers face]

peacewong@WDS 14:41
Because the task is started with the start of the engine, and the end of the engine is completed. No need to go through entry

Xi 14:46
Well, I'll take a look again, maybe there is a problem with the idea, I haven't paid attention to this.

peacewong@WDS 14:49
OK Communicate anytime if you have any questions. In fact, it's almost just that the call link is much less.

Xi 14:50
ok, 3q

—————  2022-5-19  —————

peacewong@WDS 14:23
@Xi 你好 你们是希望通过restful提交一次性提交Spark jar任务吗？

peacewong@WDS 14:23
不想走linkis-client进行提交吗？

Xi 14:23
是的

Xi 14:24
现在正在开发，按照flink插件的逻辑开发的，所以想找个请求的样例

peacewong@WDS 14:27
现在linkis-cli的shell命令和linkis-client sdk都是支持提交的，是不太好兼容进去是吧？

Xi 14:30
我们现在都用restful，shell命令没法计算资源吧

peacewong@WDS 14:33
都可以计算的，你们是直接restful调用linkis是吧，没有走sdk是吧

Xi 14:33
是的

peacewong@WDS 14:35
好的，晚点我们给下对应的接口下，restful是对应这个：EngineRestfulApi.createEngineConn,和其他模式不同的就是主要带上LabelKeyUtils.ENGINE_CONN_MODE_LABEL_KEY=once标签。

Xi 14:37
好的，主要是需要entrance的submit接口的flink 提交jar包样例

peacewong@WDS 14:37
没有走entrance哈

peacewong@WDS 14:37
是直接走linkis manager进行提交的

peacewong@WDS 14:37
参考这里

Xi 14:38
flink提交的时候不走entrance吗

peacewong@WDS 14:38
jar包等once任务提交，不走交互式的，直接走manager提交任务后进行运行。

Xi 14:40
[捂脸]

peacewong@WDS 14:41
因为任务是随着引擎的启动开始，引擎结束完成。没必要经过entrance的

Xi 14:46
好吧，我再看看，可能思路有问题了，一直没注意这个

peacewong@WDS 14:49
好的。有问题随时沟通。其实差不多只是调用环节少了很多。

Xi 14:50
好的，3q

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For additional commands, e-mail: dev-help@linkis.apache.org