You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@dolphinscheduler.apache.org by CalvinKirs <ac...@163.com> on 2020/09/01 17:04:03 UTC

[VOTE]next version what to do

Hi, everybody. In the email about what we need to do in the next version, we collected the following work. We hope that everyone can discuss the priority of related issues and the specific implementation of the plan. Each job is accompanied by related proposers and advisers. If you are interested, you can openly communicate with them. (You can communicate via email for a certain sub-item, and the email is only for voting)
1:Api Server communicate for Master instand of scan command table(@qiaozhanwei)
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
4:Suport passing parameters between tasks (@qiaozhaowei)
5:Sort out and simplify the license process
6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
7:Expect dolphinScheduler to provide SSO to facilitate seamless integration with existing platforms.((@yinyong))
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)
9:Worker Poll slot(@yangyichao_mango)
10:state machine processor of worker and master optimize(@yangyichao_mango)
11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)
12:Adding a batch stop on the process instance page(@xingchun_chen)
13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@all)
You can list the digital subscripts of the functions you expect, and then reply directly to the email. We will determine the main work content for the next issue based on your opinions. We look forward to your suggestions. thanks
Best wishes！
CalvinKirs

Re: [VOTE]next version what to do

Posted by qingzhong li <qi...@gmail.com>.

3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]

+1

On 2020/09/01 17:04:03, CalvinKirs <ac...@163.com> wrote: 
> 
> 
> Hi, everybody. In the email about what we need to do in the next version, we collected the following work. We hope that everyone can discuss the priority of related issues and the specific implementation of the plan. Each job is accompanied by related proposers and advisers. If you are interested, you can openly communicate with them. (You can communicate via email for a certain sub-item, and the email is only for voting)
> 1:Api Server communicate for Master instand of scan command table(@qiaozhanwei)
> 2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
> 3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
> 4:Suport passing parameters between tasks (@qiaozhaowei)
> 5:Sort out and simplify the license process
> 6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
> 7:Expect dolphinScheduler to provide SSO to facilitate seamless integration with existing platforms.((@yinyong))
> 8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)
> 9:Worker Poll slot(@yangyichao_mango)
> 10:state machine processor of worker and master optimize(@yangyichao_mango)
> 11:How to do project migration.
> Including workflow definitions, resource files, and data sources.
> The development environment and the production environment are two sets of environments.(@BoYiZhang)
> 12:Adding a batch stop on the process instance page(@xingchun_chen)
> 13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@all)
> You can list the digital subscripts of the functions you expect, and then reply directly to the email. We will determine the main work content for the next issue based on your opinions. We look forward to your suggestions. thanks
> Best  wishes！
> CalvinKirs
> 
>

回复：[VOTE]next version what to do

Posted by "xingchun.chen" <xi...@qq.com>.

good job!
Welcome everyone to vote





------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "dev"                                                                                    <acm_master@163.com&gt;;
发送时间:&nbsp;2020年9月2日(星期三) 凌晨1:04
收件人:&nbsp;"dev"<dev@dolphinscheduler.apache.org&gt;;

主题:&nbsp;[VOTE]next version what to do





Hi, everybody. In the email about what we need to do in the next version, we collected the following work. We hope that everyone can discuss the priority of related issues and the specific implementation of the plan. Each job is accompanied by related proposers and advisers. If you are interested, you can openly communicate with them. (You can communicate via email for a certain sub-item, and the email is only for voting)
1:Api Server communicate for Master instand of scan command table(@qiaozhanwei)
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
4:Suport passing parameters between tasks (@qiaozhaowei)
5:Sort out and simplify the license process
6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
7:Expect dolphinScheduler to provide SSO to facilitate seamless integration with existing platforms.((@yinyong))
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)
9:Worker Poll slot(@yangyichao_mango)
10:state machine processor of worker and master optimize(@yangyichao_mango)
11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)
12:Adding a batch stop on the process instance page(@xingchun_chen)
13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@all)
You can list the digital subscripts of the functions you expect, and then reply directly to the email. We will determine the main work content for the next issue based on your opinions. We look forward to your suggestions. thanks
Best&nbsp; wishes！
CalvinKirs

Re: [VOTE]next version what to do

Posted by leon bao <le...@apache.org>.

Glad to see so many new features, which can make DolphinScheduler better
and better.
At the same time, i think it may be difficult to put so many features into
one version.
it might be better to release multiple versions to support these functions.

CalvinKirs <ac...@163.com> 于2020年9月8日周二 下午10:20写道：

> Hello everyone, voting has ended.  The following tasks will be the main
> content of this version:
>
> 1:Api Server communicate for Master instand of scan command table(
> @qiaozhanwei )
>
> 2:Alert module support service for other service invoke( @qiaozhanwei
> @gaojun2048 )
>
> 3:SQL task support multiple sqls and parameter passing optimization(we
> should add a new task type, such as SqlScript @gabrywu @qiaozhanwei
> @yangyichao-mango )[need discuss]
>
> 4:Support passing parameters between tasks ( @qiaozhanwei )
>
> 5:Sort out and simplify the license process
>
> 6:Focus on fixing bugs base on the branch some new features can be
> released in 1.4.0_release base on branch 'dev'.(@gabrywu )
>
> 8:SPI requirements are collected and developed. We need to have a
> discussion to find out the modules that currently need SPI most(
> @gaojun2048 @lenboo @CalvinKirs )
>
> 10:state machine processor of worker and master optimize(
> @yangyichao-mango )
>
> 11:How to do project migration.Including workflow definitions, resource
> files, and data sources.The development environment and the production
> environment are two sets of environments.(@BoYiZhang)
>
> 12:Adding a batch stop on the process instance page(@xingchun-chen )
>
> 13:Many new functions have been added in the dev branch and some issues
> have been solved. It is recommended to merge them into the next
> version(@ALL)
>
> The proponents of each task please create an issue of the task, discuss
> the specific design plan and task assignment.
>
>
>
>
>
>
> | |
> Kirs
> |
> |
> 邮箱：acm_master@163.com
> |
>
> Signature is customized by Netease Mail Master
>
> On 09/08/2020 09:43, CalvinKirs wrote:
> Hi all:
>
>
> The voting has ended, and the voting results will be summarized later.
> Thank you very much for your participation.
>
>
> Best  wishes！
> CalvinKirs
>
>
> On 09/7/2020 17:14，boyi<zh...@163.com> wrote：
> hi:
>
>
>
>
> 11:How to do project migration.
> Including workflow definitions, resource files, and data sources.
> The development environment and the production environment are two sets of
> environments.(@BoYiZhang)
>
>
>
>
> +1
> --------------------------------------
> BoYi ZhangE-mail : zhangboyi_mx@163.com
> On 09/2/2020 01:04，CalvinKirs<ac...@163.com> wrote：
>
>
> Hi, everybody. In the email about what we need to do in the next version,
> we collected the following work. We hope that everyone can discuss the
> priority of related issues and the specific implementation of the plan.
> Each job is accompanied by related proposers and advisers. If you are
> interested, you can openly communicate with them. (You can communicate via
> email for a certain sub-item, and the email is only for voting)
> 1:Api Server communicate for Master instand of scan command
> table(@qiaozhanwei)
> 2:Alert module support service for other service invoke(@qiaozhanwei
> @gaojun_2048)
> 3:SQL task support multiple sqls and parameter passing optimization(we
> should add a new task type, such as SqlScript @gabrywu @qiaozhanwei
> @yangyichao_mango)[need discuss]
> 4:Suport passing parameters between tasks (@qiaozhaowei)
> 5:Sort out and simplify the license process
> 6:Focus on fixing bugs base on the branch some new features can be
> released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
> 7:Expect dolphinScheduler to provide SSO to facilitate seamless
> integration with existing platforms.((@yinyong))
> 8:SPI requirements are collected and developed. We need to have a
> discussion to find out the modules that currently need SPI
> most(@gaojun_2048 @lenboo @CalvinKirs)
> 9:Worker Poll slot(@yangyichao_mango)
> 10:state machine processor of worker and master optimize(@yangyichao_mango)
> 11:How to do project migration.
> Including workflow definitions, resource files, and data sources.
> The development environment and the production environment are two sets of
> environments.(@BoYiZhang)
> 12:Adding a batch stop on the process instance page(@xingchun_chen)
> 13:Many new functions have been added in the dev branch and some issues
> have been solved. It is recommended to merge them into the next
> version(@all)
> You can list the digital subscripts of the functions you expect, and then
> reply directly to the email. We will determine the main work content for
> the next issue based on your opinions. We look forward to your suggestions.
> thanks
> Best  wishes！
> CalvinKirs
>
>

-- 
DolphinScheduler(Incubator)  PPMC
BaoLiang 鲍亮
leonbao@apache.org

Re: [VOTE]next version what to do

Posted by CalvinKirs <ac...@163.com>.

Hello everyone, voting has ended.  The following tasks will be the main content of this version:

1:Api Server communicate for Master instand of scan command table( @qiaozhanwei )

2:Alert module support service for other service invoke( @qiaozhanwei @gaojun2048 )

3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao-mango )[need discuss]

4:Support passing parameters between tasks ( @qiaozhanwei )

5:Sort out and simplify the license process

6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@gabrywu )

8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most( @gaojun2048 @lenboo @CalvinKirs )

10:state machine processor of worker and master optimize( @yangyichao-mango )

11:How to do project migration.Including workflow definitions, resource files, and data sources.The development environment and the production environment are two sets of environments.(@BoYiZhang)

12:Adding a batch stop on the process instance page(@xingchun-chen )

13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@ALL)

The proponents of each task please create an issue of the task, discuss the specific design plan and task assignment.






| |
Kirs
|
|
邮箱：acm_master@163.com
|

Signature is customized by Netease Mail Master

On 09/08/2020 09:43, CalvinKirs wrote:
Hi all:


The voting has ended, and the voting results will be summarized later. Thank you very much for your participation.


Best  wishes！
CalvinKirs


On 09/7/2020 17:14，boyi<zh...@163.com> wrote：
hi:




11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)




+1
--------------------------------------
BoYi ZhangE-mail : zhangboyi_mx@163.com
On 09/2/2020 01:04，CalvinKirs<ac...@163.com> wrote：


Hi, everybody. In the email about what we need to do in the next version, we collected the following work. We hope that everyone can discuss the priority of related issues and the specific implementation of the plan. Each job is accompanied by related proposers and advisers. If you are interested, you can openly communicate with them. (You can communicate via email for a certain sub-item, and the email is only for voting)
1:Api Server communicate for Master instand of scan command table(@qiaozhanwei)
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
4:Suport passing parameters between tasks (@qiaozhaowei)
5:Sort out and simplify the license process
6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
7:Expect dolphinScheduler to provide SSO to facilitate seamless integration with existing platforms.((@yinyong))
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)
9:Worker Poll slot(@yangyichao_mango)
10:state machine processor of worker and master optimize(@yangyichao_mango)
11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)
12:Adding a batch stop on the process instance page(@xingchun_chen)
13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@all)
You can list the digital subscripts of the functions you expect, and then reply directly to the email. We will determine the main work content for the next issue based on your opinions. We look forward to your suggestions. thanks
Best  wishes！
CalvinKirs

Re:[VOTE]next version what to do

Posted by CalvinKirs <ac...@163.com>.

Hi all:

The voting has ended, and the voting results will be summarized later. Thank you very much for your participation.

Best wishes！
CalvinKirs

On 09/7/2020 17:14，boyi<zh...@163.com> wrote：
hi:

11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)

+1
--------------------------------------
BoYi ZhangE-mail : zhangboyi_mx@163.com
On 09/2/2020 01:04，CalvinKirs<ac...@163.com> wrote：

Re:[VOTE]next version what to do

Posted by boyi <zh...@163.com>.

hi:

11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)

+1
--------------------------------------
BoYi ZhangE-mail : zhangboyi_mx@163.com
On 09/2/2020 01:04，CalvinKirs<ac...@163.com> wrote：

Re: [VOTE]next version what to do

Posted by CalvinKirs <ac...@163.com>.

The voting will end at 0:00 on September 8. Please hurry up, thank you very much for your participation


Best  wishes！
CalvinKirs


On 09/7/2020 10:27，leon bao<le...@apache.org> wrote：
@xingchun.chen
good job!

i will check the issues:

3.[bug][master] After subtask fault tolerance, 2 task instances are
generated，The process instance status always is executing.
https://github.com/apache/incubator-dolphinscheduler/issues/3617 [@baoliang]

5.[bug][master] After the task is executed successfully, but the next task
has not been submitted, stop the master，the workflow will fail.
https://github.com/apache/incubator-dolphinscheduler/issues/3615 [@baoliang]

6.[bug][api] potential horizontal unauthorized access.
https://github.com/apache/incubator-dolphinscheduler/issues/3573 [@baoliang]
&nbsp; solution： move out "and process_definition_id=#{processDefinitionId}"

11.[Bug][Master] there exists some problems in checking task dependency.
https://github.com/apache/incubator-dolphinscheduler/issues/3324 [@baoliang]



xingchun.chen <xi...@qq.com> 于2020年9月3日周四 下午6:03写道：

Hi, all:
I have sort out&nbsp; the bug to 1.3.3-release of the milestone,
address:&nbsp;
https://github.com/apache/incubator-dolphinscheduler/milestone/10

The bug fixers are assigned as follows , 12、14、16、17、23 are not processed,
If you have any questions, please feedback in time:


1.[bug][master] After batch deleting the executing process instances, the
master cannot get the worker feedback results.
https://github.com/apache/incubator-dolphinscheduler/issues/3621
[@yangyichao]


2.[bug][worker] Run tens of thousands of workflows, too many open files.
https://github.com/apache/incubator-dolphinscheduler/issues/3618
[@dailidong]
&nbsp; [Bug][api] delete the task instance and apI-Server handles explode.
https://github.com/apache/incubator-dolphinscheduler/issues/3363
[@dailidong]


3.[bug][master] After subtask fault tolerance, 2 task instances are
generated，The process instance status always is executing.
https://github.com/apache/incubator-dolphinscheduler/issues/3617
[@baoliang]


4.[bug][master] When the master receives the result from the worker, the
master just stops. After the master restarts, the workflow status is always
executing.
https://github.com/apache/incubator-dolphinscheduler/issues/3616
[@qiaozhanwei]
&nbsp; solution：Worker updates the master status synchronously


5.[bug][master] After the task is executed successfully, but the next task
has not been submitted, stop the master，the workflow will fail.
https://github.com/apache/incubator-dolphinscheduler/issues/3615
[@baoliang]


6.[bug][api] potential horizontal unauthorized access.
https://github.com/apache/incubator-dolphinscheduler/issues/3573
[@baoliang]
&nbsp; solution： move out "and
process_definition_id=#{processDefinitionId}"


7.[Bug][api] worker group manage ui could not display createtime and
updatetime.
https://github.com/apache/incubator-dolphinscheduler/issues/3493
[@qiaozhanwei]


8.[Bug][api] ClassCastException: AppClassLoader cannot be cast to class
java.net.URLClassLoader.
https://github.com/apache/incubator-dolphinscheduler/issues/3233
[@dailidong]


9.[Bug][server] if resultset has null value of some column,
fastxml.jackson may ocurr exception.
https://github.com/apache/incubator-dolphinscheduler/issues/3436 [@ligang]


10.[Bug][api] Execute Hive UDF to report errors with tez.
https://github.com/apache/incubator-dolphinscheduler/issues/3455 [@ligang]


11.[Bug][Master] there exists some problems in checking task dependency.
https://github.com/apache/incubator-dolphinscheduler/issues/3324
[@baoliang]


13.[Bug][Resource Center] Can not create folder in docker with standalone
mode. https://github.com/apache/incubator-dolphinscheduler/issues/3238
[@dailidong]
&nbsp; &nbsp;reason：hdfs is not started in docker


15.[BUG][worker] python task bug.
https://github.com/apache/incubator-dolphinscheduler/issues/2769 [@ligang]


18.[bug][worker] Implementation Flink problem.
https://github.com/apache/incubator-dolphinscheduler/issues/3457
[@dailidong]


19.[Bug][k8s] when recreate or upgrade helm release, the host ip changed.
https://github.com/apache/incubator-dolphinscheduler/issues/3298
[@dailidong]


20.[Bug][docker][k8s] when use externalDatabase ,helm run error.
https://github.com/apache/incubator-dolphinscheduler/issues/3239
[@dailidong]


21.[Bug][docker] the file requirements.yaml in helm mode has error flag.
https://github.com/apache/incubator-dolphinscheduler/issues/3237 [@ligang]


22.[BUG][docker] a error when build a docker image.
https://github.com/apache/incubator-dolphinscheduler/issues/3124 [@ligang]



============================================================================================
The issues that do not need to be resolved are as follows:


12.[Bug][Server] when the worker and master are configured with different
data.basedir.path, it will cause the task to fail.
https://github.com/apache/incubator-dolphinscheduler/issues/3257 [No
processing, worker and master need to be configured the same]


14.[BUG][api] The shellTask scheduler reruns the date expression in the
script to the current system time.
https://github.com/apache/incubator-dolphinscheduler/issues/3177 [No
processing]
&nbsp; &nbsp;solution： Resolve in global variables



16.[BUG][master] MasterExecThread frequently create/destroy thread pool.
https://github.com/apache/incubator-dolphinscheduler/issues/2711 [No
processing]
&nbsp; &nbsp;It is fetature



17.[Bug][API] OOM occurs when uploading large files.
https://github.com/apache/incubator-dolphinscheduler/issues/3643 [No
processing]
&nbsp; &nbsp;solution：modify JVM parameters



23.[BUG][docker] DS1.3 in docker, ApiApplicationServer can't listen to
12345 , but 9090.
https://github.com/apache/incubator-dolphinscheduler/issues/3023 [No
processing]



Best Regards!
Xingchun Chen


------------------&nbsp;原始邮件&nbsp;------------------
发件人:
"dev"
<
acm_master@163.com&gt;;
发送时间:&nbsp;2020年9月3日(星期四) 上午10:47
收件人:&nbsp;"dev@dolphinscheduler.apache.org"<
dev@dolphinscheduler.apache.org&gt;;

主题:&nbsp;Re: [VOTE]next version what to do



2:Alert module support service for other service invoke( @qiaozhanwei
@gaojun2048 )


5:Sort out and simplify the license process


6:Focus on fixing bugs base on the branch some new features can be
released in 1.4.0_release base on branch 'dev'.(@gabrywu )


8:SPI requirements are collected and developed. We need to have a
discussion to find out the modules that currently need SPI most(
@gaojun2048 @lenboo @CalvinKirs )


+1
Best&nbsp; wishes！
CalvinKirs


On 09/2/2020 11:03，JUN GAO<gaojun2048@gmail.com&gt; wrote：
2:Alert module support service for other service invoke(@qiaozhanwei
@gaojun_2048)
8:SPI requirements are collected and developed. We need to have a
discussion to find out the modules that currently need SPI
most(@gaojun_2048 @lenboo @CalvinKirs)

+1


Best Regards
---------------
DolphinScheduler(Incubator)&nbsp; PPMC
Jun Gao 高俊
gaojun2048@gmail.com
---------------

2020年9月2日 上午1:04，CalvinKirs <acm_master@163.com&gt; 写道：

2:Alert module support service for other service invoke(@qiaozhanwei
@gaojun_2048)



--
DolphinScheduler(Incubator)  PPMC
BaoLiang 鲍亮
leonbao@apache.org

Re: [VOTE]next version what to do

Posted by leon bao <le...@apache.org>.

@xingchun.chen
good job!

i will check the issues:

3.[bug][master] After subtask fault tolerance, 2 task instances are
generated，The process instance status always is executing.
https://github.com/apache/incubator-dolphinscheduler/issues/3617 [@baoliang]

5.[bug][master] After the task is executed successfully, but the next task
has not been submitted, stop the master，the workflow will fail.
https://github.com/apache/incubator-dolphinscheduler/issues/3615 [@baoliang]

6.[bug][api] potential horizontal unauthorized access.
https://github.com/apache/incubator-dolphinscheduler/issues/3573 [@baoliang]
&nbsp; solution： move out "and process_definition_id=#{processDefinitionId}"

11.[Bug][Master] there exists some problems in checking task dependency.
https://github.com/apache/incubator-dolphinscheduler/issues/3324 [@baoliang]



xingchun.chen <xi...@qq.com> 于2020年9月3日周四 下午6:03写道：

> Hi, all:
> I have sort out&nbsp; the bug to 1.3.3-release of the milestone,
> address:&nbsp;
> https://github.com/apache/incubator-dolphinscheduler/milestone/10
>
> The bug fixers are assigned as follows , 12、14、16、17、23 are not processed,
> If you have any questions, please feedback in time:
>
>
> 1.[bug][master] After batch deleting the executing process instances, the
> master cannot get the worker feedback results.
> https://github.com/apache/incubator-dolphinscheduler/issues/3621
> [@yangyichao]
>
>
> 2.[bug][worker] Run tens of thousands of workflows, too many open files.
> https://github.com/apache/incubator-dolphinscheduler/issues/3618
> [@dailidong]
> &nbsp; [Bug][api] delete the task instance and apI-Server handles explode.
> https://github.com/apache/incubator-dolphinscheduler/issues/3363
> [@dailidong]
>
>
> 3.[bug][master] After subtask fault tolerance, 2 task instances are
> generated，The process instance status always is executing.
> https://github.com/apache/incubator-dolphinscheduler/issues/3617
> [@baoliang]
>
>
> 4.[bug][master] When the master receives the result from the worker, the
> master just stops. After the master restarts, the workflow status is always
> executing.
> https://github.com/apache/incubator-dolphinscheduler/issues/3616
> [@qiaozhanwei]
> &nbsp; solution：Worker updates the master status synchronously
>
>
> 5.[bug][master] After the task is executed successfully, but the next task
> has not been submitted, stop the master，the workflow will fail.
> https://github.com/apache/incubator-dolphinscheduler/issues/3615
> [@baoliang]
>
>
> 6.[bug][api] potential horizontal unauthorized access.
> https://github.com/apache/incubator-dolphinscheduler/issues/3573
> [@baoliang]
> &nbsp; solution： move out "and
> process_definition_id=#{processDefinitionId}"
>
>
> 7.[Bug][api] worker group manage ui could not display createtime and
> updatetime.
> https://github.com/apache/incubator-dolphinscheduler/issues/3493
> [@qiaozhanwei]
>
>
> 8.[Bug][api] ClassCastException: AppClassLoader cannot be cast to class
> java.net.URLClassLoader.
> https://github.com/apache/incubator-dolphinscheduler/issues/3233
> [@dailidong]
>
>
> 9.[Bug][server] if resultset has null value of some column,
> fastxml.jackson may ocurr exception.
> https://github.com/apache/incubator-dolphinscheduler/issues/3436 [@ligang]
>
>
> 10.[Bug][api] Execute Hive UDF to report errors with tez.
> https://github.com/apache/incubator-dolphinscheduler/issues/3455 [@ligang]
>
>
> 11.[Bug][Master] there exists some problems in checking task dependency.
> https://github.com/apache/incubator-dolphinscheduler/issues/3324
> [@baoliang]
>
>
> 13.[Bug][Resource Center] Can not create folder in docker with standalone
> mode. https://github.com/apache/incubator-dolphinscheduler/issues/3238
> [@dailidong]
> &nbsp; &nbsp;reason：hdfs is not started in docker
>
>
> 15.[BUG][worker] python task bug.
> https://github.com/apache/incubator-dolphinscheduler/issues/2769 [@ligang]
>
>
> 18.[bug][worker] Implementation Flink problem.
> https://github.com/apache/incubator-dolphinscheduler/issues/3457
> [@dailidong]
>
>
> 19.[Bug][k8s] when recreate or upgrade helm release, the host ip changed.
> https://github.com/apache/incubator-dolphinscheduler/issues/3298
> [@dailidong]
>
>
> 20.[Bug][docker][k8s] when use externalDatabase ,helm run error.
> https://github.com/apache/incubator-dolphinscheduler/issues/3239
> [@dailidong]
>
>
> 21.[Bug][docker] the file requirements.yaml in helm mode has error flag.
> https://github.com/apache/incubator-dolphinscheduler/issues/3237 [@ligang]
>
>
> 22.[BUG][docker] a error when build a docker image.
> https://github.com/apache/incubator-dolphinscheduler/issues/3124 [@ligang]
>
>
>
> ============================================================================================
> The issues that do not need to be resolved are as follows:
>
>
> 12.[Bug][Server] when the worker and master are configured with different
> data.basedir.path, it will cause the task to fail.
> https://github.com/apache/incubator-dolphinscheduler/issues/3257 [No
> processing, worker and master need to be configured the same]
>
>
> 14.[BUG][api] The shellTask scheduler reruns the date expression in the
> script to the current system time.
> https://github.com/apache/incubator-dolphinscheduler/issues/3177 [No
> processing]
> &nbsp; &nbsp;solution： Resolve in global variables
>
>
>
> 16.[BUG][master] MasterExecThread frequently create/destroy thread pool.
> https://github.com/apache/incubator-dolphinscheduler/issues/2711 [No
> processing]
> &nbsp; &nbsp;It is fetature
>
>
>
> 17.[Bug][API] OOM occurs when uploading large files.
> https://github.com/apache/incubator-dolphinscheduler/issues/3643 [No
> processing]
> &nbsp; &nbsp;solution：modify JVM parameters
>
>
>
> 23.[BUG][docker] DS1.3 in docker, ApiApplicationServer can't listen to
> 12345 , but 9090.
> https://github.com/apache/incubator-dolphinscheduler/issues/3023 [No
> processing]
>
>
>
> Best Regards!
> Xingchun Chen
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
>                                                   "dev"
>                                                                 <
> acm_master@163.com&gt;;
> 发送时间:&nbsp;2020年9月3日(星期四) 上午10:47
> 收件人:&nbsp;"dev@dolphinscheduler.apache.org"<
> dev@dolphinscheduler.apache.org&gt;;
>
> 主题:&nbsp;Re: [VOTE]next version what to do
>
>
>
> 2:Alert module support service for other service invoke( @qiaozhanwei
> @gaojun2048 )
>
>
> 5:Sort out and simplify the license process
>
>
> 6:Focus on fixing bugs base on the branch some new features can be
> released in 1.4.0_release base on branch 'dev'.(@gabrywu )
>
>
> 8:SPI requirements are collected and developed. We need to have a
> discussion to find out the modules that currently need SPI most(
> @gaojun2048 @lenboo @CalvinKirs )
>
>
> +1
> Best&nbsp; wishes！
> CalvinKirs
>
>
> On 09/2/2020 11:03，JUN GAO<gaojun2048@gmail.com&gt; wrote：
> 2:Alert module support service for other service invoke(@qiaozhanwei
> @gaojun_2048)
> 8:SPI requirements are collected and developed. We need to have a
> discussion to find out the modules that currently need SPI
> most(@gaojun_2048 @lenboo @CalvinKirs)
>
> +1
>
>
> Best Regards
> ---------------
> DolphinScheduler(Incubator)&nbsp; PPMC
> Jun Gao 高俊
> gaojun2048@gmail.com
> ---------------
>
> 2020年9月2日 上午1:04，CalvinKirs <acm_master@163.com&gt; 写道：
>
> 2:Alert module support service for other service invoke(@qiaozhanwei
> @gaojun_2048)



-- 
DolphinScheduler(Incubator)  PPMC
BaoLiang 鲍亮
leonbao@apache.org

re： [VOTE]next version what to do

Posted by "xingchun.chen" <xi...@qq.com>.

Hi, all:
I have sort out&nbsp; the bug to 1.3.3-release of the milestone, address:&nbsp; https://github.com/apache/incubator-dolphinscheduler/milestone/10

The bug fixers are assigned as follows , 12、14、16、17、23 are not processed, If you have any questions, please feedback in time:


1.[bug][master] After batch deleting the executing process instances, the master cannot get the worker feedback results. https://github.com/apache/incubator-dolphinscheduler/issues/3621 [@yangyichao]


2.[bug][worker] Run tens of thousands of workflows, too many open files. https://github.com/apache/incubator-dolphinscheduler/issues/3618 [@dailidong]
&nbsp; [Bug][api] delete the task instance and apI-Server handles explode. https://github.com/apache/incubator-dolphinscheduler/issues/3363 [@dailidong]


3.[bug][master] After subtask fault tolerance, 2 task instances are generated，The process instance status always is executing. https://github.com/apache/incubator-dolphinscheduler/issues/3617 [@baoliang]


4.[bug][master] When the master receives the result from the worker, the master just stops. After the master restarts, the workflow status is always executing. https://github.com/apache/incubator-dolphinscheduler/issues/3616 [@qiaozhanwei]
&nbsp; solution：Worker updates the master status synchronously


5.[bug][master] After the task is executed successfully, but the next task has not been submitted, stop the master，the workflow will fail. https://github.com/apache/incubator-dolphinscheduler/issues/3615 [@baoliang]


6.[bug][api] potential horizontal unauthorized access. https://github.com/apache/incubator-dolphinscheduler/issues/3573 [@baoliang]
&nbsp; solution： move out "and process_definition_id=#{processDefinitionId}"


7.[Bug][api] worker group manage ui could not display createtime and updatetime. https://github.com/apache/incubator-dolphinscheduler/issues/3493 [@qiaozhanwei]


8.[Bug][api] ClassCastException: AppClassLoader cannot be cast to class java.net.URLClassLoader. https://github.com/apache/incubator-dolphinscheduler/issues/3233 [@dailidong]


9.[Bug][server] if resultset has null value of some column, fastxml.jackson may ocurr exception. https://github.com/apache/incubator-dolphinscheduler/issues/3436 [@ligang]


10.[Bug][api] Execute Hive UDF to report errors with tez. https://github.com/apache/incubator-dolphinscheduler/issues/3455 [@ligang]


11.[Bug][Master] there exists some problems in checking task dependency. https://github.com/apache/incubator-dolphinscheduler/issues/3324 [@baoliang]


13.[Bug][Resource Center] Can not create folder in docker with standalone mode. https://github.com/apache/incubator-dolphinscheduler/issues/3238 [@dailidong]
&nbsp; &nbsp;reason：hdfs is not started in docker


15.[BUG][worker] python task bug. https://github.com/apache/incubator-dolphinscheduler/issues/2769 [@ligang]


18.[bug][worker] Implementation Flink problem. https://github.com/apache/incubator-dolphinscheduler/issues/3457 [@dailidong]


19.[Bug][k8s] when recreate or upgrade helm release, the host ip changed. https://github.com/apache/incubator-dolphinscheduler/issues/3298 [@dailidong]


20.[Bug][docker][k8s] when use externalDatabase ,helm run error. https://github.com/apache/incubator-dolphinscheduler/issues/3239 [@dailidong]


21.[Bug][docker] the file requirements.yaml in helm mode has error flag. https://github.com/apache/incubator-dolphinscheduler/issues/3237 [@ligang]


22.[BUG][docker] a error when build a docker image. https://github.com/apache/incubator-dolphinscheduler/issues/3124 [@ligang]


============================================================================================
The issues that do not need to be resolved are as follows:


12.[Bug][Server] when the worker and master are configured with different data.basedir.path, it will cause the task to fail. https://github.com/apache/incubator-dolphinscheduler/issues/3257 [No processing, worker and master need to be configured the same]


14.[BUG][api] The shellTask scheduler reruns the date expression in the script to the current system time. https://github.com/apache/incubator-dolphinscheduler/issues/3177 [No processing]
&nbsp; &nbsp;solution： Resolve in global variables



16.[BUG][master] MasterExecThread frequently create/destroy thread pool. https://github.com/apache/incubator-dolphinscheduler/issues/2711 [No processing]
&nbsp; &nbsp;It is fetature



17.[Bug][API] OOM occurs when uploading large files. https://github.com/apache/incubator-dolphinscheduler/issues/3643 [No processing]
&nbsp; &nbsp;solution：modify JVM parameters



23.[BUG][docker] DS1.3 in docker, ApiApplicationServer can't listen to 12345 , but 9090.https://github.com/apache/incubator-dolphinscheduler/issues/3023 [No processing]



Best Regards!
Xingchun Chen


------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "dev"                                                                                    <acm_master@163.com&gt;;
发送时间:&nbsp;2020年9月3日(星期四) 上午10:47
收件人:&nbsp;"dev@dolphinscheduler.apache.org"<dev@dolphinscheduler.apache.org&gt;;

主题:&nbsp;Re: [VOTE]next version what to do



2:Alert module support service for other service invoke( @qiaozhanwei @gaojun2048 )


5:Sort out and simplify the license process


6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@gabrywu )


8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most( @gaojun2048 @lenboo @CalvinKirs )


+1
Best&nbsp; wishes！
CalvinKirs


On 09/2/2020 11:03，JUN GAO<gaojun2048@gmail.com&gt; wrote：
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)

+1


Best Regards
---------------
DolphinScheduler(Incubator)&nbsp; PPMC
Jun Gao 高俊
gaojun2048@gmail.com
---------------

2020年9月2日 上午1:04，CalvinKirs <acm_master@163.com&gt; 写道：

2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)

Re: [VOTE]next version what to do

Posted by CalvinKirs <ac...@163.com>.

2:Alert module support service for other service invoke( @qiaozhanwei @gaojun2048 )


5:Sort out and simplify the license process


6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@gabrywu )


8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most( @gaojun2048 @lenboo @CalvinKirs )


+1
Best  wishes！
CalvinKirs


On 09/2/2020 11:03，JUN GAO<ga...@gmail.com> wrote：
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)

+1


Best Regards
---------------
DolphinScheduler(Incubator)  PPMC
Jun Gao 高俊
gaojun2048@gmail.com
---------------

2020年9月2日 上午1:04，CalvinKirs <ac...@163.com> 写道：

2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)

Re: [VOTE]next version what to do

Posted by JUN GAO <ga...@gmail.com>.

2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)

+1


Best Regards
---------------
DolphinScheduler(Incubator)  PPMC
Jun Gao 高俊
gaojun2048@gmail.com
---------------

> 2020年9月2日 上午1:04，CalvinKirs <ac...@163.com> 写道：
> 
> 2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)

Re: [VOTE]next version what to do

Posted by leon bao <le...@apache.org>.

6:Focus on fixing bugs base on the branch some new features can be released
in 1.4.0_release base on branch 'dev'.(@Gabrywu)

+1
1.3.X focus on fixing bugs
1.4.0 release new features

xingchun.chen <xi...@qq.com> 于2020年9月2日周三 上午10:37写道：

> I think the 1, 2, 3, 4, 5, 11, 12, 13 can be done
>
>
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
>                                                   "dev"
>                                                                 <
> acm_master@163.com&gt;;
> 发送时间:&nbsp;2020年9月2日(星期三) 凌晨1:04
> 收件人:&nbsp;"dev"<dev@dolphinscheduler.apache.org&gt;;
>
> 主题:&nbsp;[VOTE]next version what to do
>
>
>
>
>
> Hi, everybody. In the email about what we need to do in the next version,
> we collected the following work. We hope that everyone can discuss the
> priority of related issues and the specific implementation of the plan.
> Each job is accompanied by related proposers and advisers. If you are
> interested, you can openly communicate with them. (You can communicate via
> email for a certain sub-item, and the email is only for voting)
> 1:Api Server communicate for Master instand of scan command
> table(@qiaozhanwei)
> 2:Alert module support service for other service invoke(@qiaozhanwei
> @gaojun_2048)
> 3:SQL task support multiple sqls and parameter passing optimization(we
> should add a new task type, such as SqlScript @gabrywu @qiaozhanwei
> @yangyichao_mango)[need discuss]
> 4:Suport passing parameters between tasks (@qiaozhaowei)
> 5:Sort out and simplify the license process
> 6:Focus on fixing bugs base on the branch some new features can be
> released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
> 7:Expect dolphinScheduler to provide SSO to facilitate seamless
> integration with existing platforms.((@yinyong))
> 8:SPI requirements are collected and developed. We need to have a
> discussion to find out the modules that currently need SPI
> most(@gaojun_2048 @lenboo @CalvinKirs)
> 9:Worker Poll slot(@yangyichao_mango)
> 10:state machine processor of worker and master optimize(@yangyichao_mango)
> 11:How to do project migration.
> Including workflow definitions, resource files, and data sources.
> The development environment and the production environment are two sets of
> environments.(@BoYiZhang)
> 12:Adding a batch stop on the process instance page(@xingchun_chen)
> 13:Many new functions have been added in the dev branch and some issues
> have been solved. It is recommended to merge them into the next
> version(@all)
> You can list the digital subscripts of the functions you expect, and then
> reply directly to the email. We will determine the main work content for
> the next issue based on your opinions. We look forward to your suggestions.
> thanks
> Best&nbsp; wishes！
> CalvinKirs



-- 
DolphinScheduler(Incubator)  PPMC
BaoLiang 鲍亮
leonbao@apache.org

回复：[VOTE]next version what to do

Posted by "xingchun.chen" <xi...@qq.com>.

I think the 1, 2, 3, 4, 5, 11, 12, 13 can be done




------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "dev"                                                                                    <acm_master@163.com&gt;;
发送时间:&nbsp;2020年9月2日(星期三) 凌晨1:04
收件人:&nbsp;"dev"<dev@dolphinscheduler.apache.org&gt;;

主题:&nbsp;[VOTE]next version what to do





Hi, everybody. In the email about what we need to do in the next version, we collected the following work. We hope that everyone can discuss the priority of related issues and the specific implementation of the plan. Each job is accompanied by related proposers and advisers. If you are interested, you can openly communicate with them. (You can communicate via email for a certain sub-item, and the email is only for voting)
1:Api Server communicate for Master instand of scan command table(@qiaozhanwei)
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
4:Suport passing parameters between tasks (@qiaozhaowei)
5:Sort out and simplify the license process
6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
7:Expect dolphinScheduler to provide SSO to facilitate seamless integration with existing platforms.((@yinyong))
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)
9:Worker Poll slot(@yangyichao_mango)
10:state machine processor of worker and master optimize(@yangyichao_mango)
11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)
12:Adding a batch stop on the process instance page(@xingchun_chen)
13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@all)
You can list the digital subscripts of the functions you expect, and then reply directly to the email. We will determine the main work content for the next issue based on your opinions. We look forward to your suggestions. thanks
Best&nbsp; wishes！
CalvinKirs

回复：[VOTE]next version what to do

Posted by zixi0825 <64...@qq.com>.

3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)


+1


------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "dev"                                                                                    <acm_master@163.com&gt;;
发送时间:&nbsp;2020年9月2日(星期三) 凌晨1:04
收件人:&nbsp;"dev"<dev@dolphinscheduler.apache.org&gt;;

主题:&nbsp;[VOTE]next version what to do





Hi, everybody. In the email about what we need to do in the next version, we collected the following work. We hope that everyone can discuss the priority of related issues and the specific implementation of the plan. Each job is accompanied by related proposers and advisers. If you are interested, you can openly communicate with them. (You can communicate via email for a certain sub-item, and the email is only for voting)
1:Api Server communicate for Master instand of scan command table(@qiaozhanwei)
2:Alert module support service for other service invoke(@qiaozhanwei @gaojun_2048)
3:SQL task support multiple sqls and parameter passing optimization(we should add a new task type, such as SqlScript @gabrywu @qiaozhanwei @yangyichao_mango)[need discuss]
4:Suport passing parameters between tasks (@qiaozhaowei)
5:Sort out and simplify the license process
6:Focus on fixing bugs base on the branch some new features can be released in 1.4.0_release base on branch 'dev'.(@Gabrywu)
7:Expect dolphinScheduler to provide SSO to facilitate seamless integration with existing platforms.((@yinyong))
8:SPI requirements are collected and developed. We need to have a discussion to find out the modules that currently need SPI most(@gaojun_2048 @lenboo @CalvinKirs)
9:Worker Poll slot(@yangyichao_mango)
10:state machine processor of worker and master optimize(@yangyichao_mango)
11:How to do project migration.
Including workflow definitions, resource files, and data sources.
The development environment and the production environment are two sets of environments.(@BoYiZhang)
12:Adding a batch stop on the process instance page(@xingchun_chen)
13:Many new functions have been added in the dev branch and some issues have been solved. It is recommended to merge them into the next version(@all)
You can list the digital subscripts of the functions you expect, and then reply directly to the email. We will determine the main work content for the next issue based on your opinions. We look forward to your suggestions. thanks
Best&nbsp; wishes！
CalvinKirs