2022-04-20 Minutes of the bi-weekly meeting of Apache Linkis(incubating)

[Zhen Wang <wa...@apache.org>]

2022-04-20 Minutes of the bi-weekly meeting of Apache Linkis(incubating)

1. Apache Linkis incubation & version progress is synchronized. —— Wang Zhen
(1) Each version needs to regularly pull and merge the master code to
resolve some conflicts between versions in time
(2) Using the joint test development environment, the home IP is not fixed
and the company VPN has multiple exit IPs. For users with multiple egress
IPs, it is recommended to consult the company's network department to
provide a complete IP list. At the same time, the community will
investigate the method of cloud server proxy, and Jie Longping is
responsible for follow-up.
(3) The git action unit test coverage tool has been used, and community
users need to pay attention to some indicators of unit test coverage when
submitting PR.

2. Apache Linkis1.1.1 progress synchronization. —— Xia Chen
(1) Progress: The self-testing test is completed. When sorting out the
documents, it is necessary to provide the upgrade guide document (due to
the change of the udf table). It is expected to start voting before 5.1.
(2) Related discussion: If each version involves major changes and changes
with tables, it is necessary to provide an upgrade guide process document

3. Apache Linkis1.1.2 progress synchronization. - Wang Heping
(1) Progress: Go to HDFS and pass the test, most of the features have been
submitted for testing, and are expected to be released in May
(2) Related discussion: After implementing HDFS, you can start to start
debugging linkis on windows

4. Apache Linkis1.2.0 progress synchronization. —— Wang Zhen
(1) Progress: Most of the code has been completed, the test encountered
problems to be solved, lack of documentation and unit tests
(2) Related discussion: The Presto engine will be compatible with Presto
and Trino by implementing different EngineConnExecutors

5. Discussion on some features of Linkis - Wang Heping
(1) The parameter supports real-time modification to take effect, and there
is no need to restart the service. Now modifying the linkis parameter
requires restarting the service. It is recommended to support refresh
Discussion: Investigating Spring Cloud config and the nacos configuration
center. (Wang Zhen is responsible for follow-up)
(2) The task supports interface re-pulling. Now, if the entry service hangs
up, the task will generally be marked as canceled. However, in many
scenarios, these tasks need to be re-pulled. It is recommended to add this
function.
Discussion: Automatically or provide an interface to restart the abnormally
failed task. (Wang Heping is responsible for follow-up)
(3) The client also supports task retry. Now Orchestrator supports
LinkisRetryException to retry 10 times, but it is best to also support
retry when calling through the client. Retry is performed by judging
whether the task supports retry. Need to modify the task record table plus
retry field.
Discussion: Since task submission may also fail, all the best clients also
add retries (Xu Jie is responsible for follow-up)
(4) Linkis supports grayscale update, based on eureka offline mechanism
Discussion: Through the eureka offline mechanism, the service is offline in
a grayscale. It should be noted that the Entrance service may not be
directly offline due to the status. (Wang Zhen is responsible for follow-up)
(5) The management console supports the update and viewing of engine
material resources, and the interfaced installation of ec
Discussion: It is mainly about the development of front-end pages. Further
design and discussion are needed to consider whether it should be
considered as a project topic in the summer of open source. (Xia Chen is
responsible for follow-up)

6. The progress of Apache Linkis community operations is synchronized.
Li Wen explained and analyzed recent indicators related to community
operations.

7. The host of the next regular meeting, welcome to claim.
Lu Xiaolong will be the host of the next regular meeting.

8. The dynamic synchronization on GitHub in the past two weeks, whether it
should be discussed as a fixed item in the future (such as those new bugs,
the newly merged pr feature repair instructions, etc.)
In the later stage of the discussion, some needs, features or problems can
be collected in the community group in advance and discussed in the
follow-up meeting. And considering the scope of the open meeting, you can
try to notify the user group as well.

---

2022-04-20 Apache Linkis(incubating) 双周例会会议纪要

1. Apache Linkis 孵化&版本 进展同步。 —— 王震
（1）各个版本需要定期拉取合并 master 代码，及时解决版本间的一些冲突
（2）使用联合测试开发环境，遇到了家庭 IP 不固定，公司 VPN 存在多个出口 IP 的情况。对于有多个出口 IP
有的用户，建议向公司网络部门进行咨询，提供完整的IP列表。同时社区将调研云服务器代理的方法，介龙平负责跟进。
（3）git action 单元测试覆盖率工具开始使用，社区用户提交 PR 时需要关注单元测试覆盖率的一些指标。

2. Apache Linkis1.1.1 进展同步。 —— 夏晨
（1）进度：自测测试完成，在梳理文档，需要提供升级指引文档（由于 udf 表变动），预计 5.1 之前开始投票
（2）相关讨论：每个版本如果涉及大的变动，以及有表的变更时，需要提供升级指引流程文档

3. Apache Linkis1.1.2 进展同步。 —— 王和平
（1）进展：去 HDFS 测试通过，大部分特性已经提交带测试，预计 5 月份发布
（2）相关讨论：实现去 HDFS 后，可以开始实现 windows 上面启动调试 linkis

4. Apache Linkis1.2.0 进展同步。 —— 王震
（1）进展：大部分代码已经完成，测试遇到问题待解决，缺少文档和单元测试
（2）相关讨论：Presto 引擎将通过实现，不同的 EngineConnExecutor 来兼容 Presto 和 Trino

5. Linkis 部分特性讨论  —— 王和平
（1）参数支持实时修改生效，不需要重启服务，现在修改linkis参数需要重启服务，建议支持刷新
讨论：调研 Spring Cloud config 和 nacos 配置中心。（王震负责跟进）
（2）任务支持接口重拉，现在如果entrance服务挂掉后，一般会将任务标记为取消，但是很多场景需要将这些任务进行重拉，建议加上这个功能
讨论：自动或者提供接口，重新拉起异常失败任务。（王和平负责跟进）
（3）客户端也支持任务重试，现在Orchestrator支持LinkisRetryException重试10次，但是很多时候通过客户端调用的时候最好也支持重试，通过判断任务是否支持重试进行重试。需要修改任务记录表加上重试字段。
讨论：由于任务提交也可能会失败，所有最好客户端也添加重试（徐杰负责跟进）
（4）Linkis支持灰度更新，基于 eureka 下线机制
讨论：通过 eureka 下线机制，对服务进行灰度下线，需要注意 Entrance 服务有状态可能不能直接下线。（王震负责跟进）
（5）管理台支持引擎物料资源更新、查看，ec的界面化安装
讨论：主要是前端页面的开发，需要进一步设计讨论，考虑是否作为开源之夏的项目议题。（夏晨负责跟进）

6. Apache Linkis 社区运营 进展同步。
李文对近期的社区运营相关指标进行说明分析。

7. 下一场例会的主持人，欢迎认领。
陆小龙将作为下一场例会的主持人。

8. 最近两周的GitHub上的动态同步，后续要不要作为一个固定事项讨论(如新增了那些bug，新合入的pr特性修复说明等)
讨论后期可以提前在社区群中收集一些需求、特性或问题，在后续的会议中进行讨论。并考虑开放会议的范围，可尝试在用户群里面也进行通知。

-- 
Kind Regards,
Zhen Wang