[DISCUSS] Does distributed deployment need to go through ssh remote execution

[Andy <an...@163.com>]

—————  2022-5-11  —————

徐飞 16:52
请教大佬们一个问题，我启用一个AM，设置的资源大小，主任务启动后又启动的子进程任务是否受到这个resource配置的资源的限制（比如内存）

casion 22:14
https://github.com/apache/incubator-linkis/pull/2069 prometheus集成的pr 可以帮review下@Adamyuanyuan @徐杰 

casion 22:27

https://github.com/apache/incubator-linkis/pull/1570/files 

Swagger2 引入配置类Knife4jConfig.scala 感觉好像没法放到  linkis-commons/linkis-module 模块中，
因为Knife4jConfig.scala 是extends WebMvcConfigurer （WebMvcConfigurer需要依赖spring-webmvc.jar包）

gateway依赖linkis-module
gateway如果exclude掉spring-webmvc.jar 包 会导致Knife4jConfig类中 WebMvcConfigurer 的class not found 异常。
gateway如果保留spring-webmvc.jar 会导致 webmvc和webflux冲突 gateaway启动异常。 @岸林 

casion 22:34
https://github.com/apache/incubator-linkis/issues/2096 分布式一键部署安装脚本 
https://github.com/apache/incubator-linkis/issues/2098 license的git check action 增加对Binaries /Archives 类型的筛查 @iture123_陈振华  这两个任务有感兴趣的吗[呲牙]

🍀 默默向上游 22:41
好，我看下。

🍀 默默向上游 22:42
分布式部署是要走ssh远程执行吗

casion 22:44
嗯，我理解是。 可以参考下开源其他项目的做法

🍀 默默向上游 22:47
我感觉这个需求有点奇怪。如果是部署的节点很多，那这公司规模不小，公司会有专门的发布平台，支持在多台服务器上传和执行命令。如果是小公司，就几个节点，一台台操作也不麻烦。

🍀 默默向上游 22:48
而且处于安全考虑，不同机器之间禁止互相ssh才对。

casion 22:51
是的 如果公司有自己的运维发布平台的话，基本走的是公司内部的发布平台。 分布式部署的话 只是给一参考样例，对于有些试用的新手，是有这方面的诉求。 

🍀 默默向上游 22:53
明白。有知道哪些开源项目这么干的吗？

Sargent Ti 22:59
基于ssh 信任的部署吗？hadoop 就是这么干的吧

Sargent Ti 23:00
基于ssh 免密信任的start/stop all

🍀 默默向上游 23:19
是的

—————  2022-5-12  —————

岸林 05:45
收到[OK]



————— 2022-5-11 —————

Xu Fei 16:52
Ask the big guys a question, I enable an AM, set the resource size, and whether the sub-process tasks that are started after the main task is started are limited by the resources configured by this resource (such as memory)

case 22:14
https://github.com/apache/incubator-linkis/pull/2069 Prometheus integrated pr can help review @Adamyuanyuan @徐杰 

case 22:27

https://github.com/apache/incubator-linkis/pull/1570/files

Swagger2 introduces the configuration class Knife4jConfig.scala and it seems that it cannot be placed in the linkis-commons/linkis-module module.
Because Knife4jConfig.scala extends WebMvcConfigurer (WebMvcConfigurer needs to depend on the spring-webmvc.jar package)

gateway depends on linkis-module
If the gateway excludes the spring-webmvc.jar package, it will cause the class not found exception of WebMvcConfigurer in the Knife4jConfig class.
If the gateway retains spring-webmvc.jar, it will cause a conflict between webmvc and webflux. Gateaway starts abnormally. @shore forest 

case 22:34
https://github.com/apache/incubator-linkis/issues/2096 Distributed one-click deployment installation script
https://github.com/apache/incubator-linkis/issues/2098 license's git check action adds the screening of Binaries /Archives type @iture123_Chen Zhenhua  Are you interested in these two tasks?

🍀 Silently upstream 22:41
OK, I'll take a look.

🍀 Silently upstream 22:42
Does distributed deployment need to go through ssh remote execution?

case 22:44
Well, I understand yes. You can refer to the practices of other open source projects

🍀 silently upstream 22:47
I find this requirement a bit odd. If there are many nodes deployed, then the company is not small, and the company will have a dedicated publishing platform that supports uploading and executing commands on multiple servers. If it is a small company, there are only a few nodes, and it is not troublesome to operate one by one.

🍀 Silently upstream 22:48
And for security reasons, it is forbidden to ssh each other between different machines.

case 22:51
Yes, if the company has its own operation and maintenance release platform, it basically uses the company's internal release platform. In the case of distributed deployment, it is just a reference example. For some newbies who try it, there are demands in this regard.

🍀 Silently upstream 22:53
clear. Does anyone know of any open source projects that do this?

Sargent Ti 22:59
ssh trust based deployment? That's what hadoop does

Sargent Ti 23:00
Start/stop all based on ssh password-free trust

🍀 silently upstream 23:19
yes

————— 2022-5-12 —————

Shore Forest 05:45
received [OK]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@linkis.apache.org
For additional commands, e-mail: dev-help@linkis.apache.org