[DISCUSS]Spark v3.1.1 DSS one-click installation

[康悦Rita <ri...@163.com>]

 Dear:

Wechat group "Apache Linkis(incubating)  community Development group "chat records are as follows:  微信群"Apache Linkis(incubating)  社区开发群"的聊天记录如下




—————  2023-2-2  —————

Hua 15:04

华  15:04




If the spark version is 3.1.1, can I use this one-click installation package for dss?

请教下， 如果spark的版本是3.1.1， 这个dss的一键安装包，可以用吗  ？







Mr. Flash 15:09

闪电先生  15:09




Should be unable to see the official website of the adaptation instructions.应该是不能的 看下官网的适配说明。 The general practice is to download the installation package, and then replace the self-compiled package

一般的做法是下载这个安装包，然后替换里面自己编译好的包







Jeremiah 15:11

华  15:11




Feel like a new plugin, specifically adapted to the 3.* version.感觉得新增一个插件， 专门适配3.*的版本。   The current version of scala is 2.11, and the spark3.* version of scala is 2.12, which has changed the version of scala, and the impact is very large

现在scala的版本是2.11， spark3.*的版本scala用2.12， 改了scala的版本， 影响范围就很大了







Jeremiah 15:12

华  15:12




Let me look at the adaptation instructions first

我先看看适配说明







Jeremiah 15:12

华  15:12




@utopianet_Guangyin Credit card _ Zhang Huajin thank you

@utopianet_广银信用卡_张华金   感谢







Shore Forest 15:13

岸林  15:13




scala should be able to adjust to 2.12 for 1.0.3 and above, which we did on 1.0.3

1.0.3以上版本应该scala是可以调整为2.12的，我们在1.0.3版本上调整过







Mr. Lightning 15:14

闪电先生  15:14




Multi-version support is expected in version 1.3.2, passing in the version number.预计在1.3.2版本中将进行多版本支持，传入版本号。 At present, Guo Feixiong is working on it, and you can see the discussion on GitHub

目前郭飞兄在做，可以看看GitHub上面的讨论







Jeremiah 15:15

华  15:15




I'm on version 1.1.1

我是在1.1.1版本







Jeremiah 15:15

华  15:15




When you change the scala version, you have to change these dependencies

修改了scala版本， 有这些依赖也要修改







Mr. Flash 15:16

闪电先生  15:16




See the official website

看官网







Mr. Flash 15:17

闪电先生  15:17




Look at the documentation section for guidance

多看文档 这块有指引







Hua 16:41

华  16:41




Will this change support spark 3.*

这个修改后， 可以支持spark  3.*的版本吗







Little Guo Fei Flying Knife 20:13

小郭飞飞刀  20:13




Look, these two are supposed to support multiple versions of scala, spark3 may need additional support, spark2/3 some dependencies may change看了下，这两个应该是支持scala 的多版本，spark3可能还需要另外支持下, spark2/3有些依赖可能会发生变化










--

Best Regards
------
康悦 ritakang 
GitHub：Ritakang0451
E-mail：rita0451@163.com