You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dolphinscheduler.apache.org by yongqing <59...@qq.com> on 2020/09/14 05:39:00 UTC
回复： 回复：[Feature][process_definition] process definition need to support parallel or serial choose. and parallel mode support wait queue length
why I add two cloums in table t_ds_process_definition:
1、in process,need to solve serial run and serial run command can set wait queue length.
2、after later，we need to statistics serial or parallel commands for every projet or every processdef.
------------------ 原始邮件 ------------------
发件人: "dev" <gabrywu@apache.org>;
发送时间: 2020年9月14日(星期一) 中午1:20
收件人: "dev@dolphinscheduler.apache.org"<dev@dolphinscheduler.apache.org>;
主题: Re: 回复：[Feature][process_definition] process definition need to support parallel or serial choose. and parallel mode support wait queue length
I think there’s no need to add the is_parallel column, because the parallelism is the base function of the workflow or task.
Serial is just the speciall case when parallelism is 1.
From: yongqing <597365581@qq.com>
Reply-To: "dev@dolphinscheduler.apache.org" <dev@dolphinscheduler.apache.org>
Date: Monday, September 14, 2020 at 11:28
To: dev <dev@dolphinscheduler.apache.org>
Subject: 回复：[Feature][process_definition] process definition need to support parallel or serial choose. and parallel mode support wait queue length
other company need this function too？
[cid:02EA6762@C129EC69.E5E25E5F.png]
------------------ 原始邮件 ------------------
发件人: "yongqing" <597365581@qq.com>;
发送时间: 2020年9月14日(星期一) 上午10:32
收件人: "dev"<dev@dolphinscheduler.apache.org>;
主题: [Feature][process_definition] process definition need to support parallel or serial choose. and parallel mode support wait queue length
hi,all
our company need this funcation. so we add this funcation into apache. and our company ‘s all old tasks will be switched to incubator-dolphinscheduler system.
process definition need to support parallel or serial choose.
if is serial in dispatch，need to the previous process_instance execute complete，next process_instance can to run。
流程实例中支持并发并行和串行。 如果是串行处理，需要等待上一个流程实例运行完成后，下一个流程实例才可以运行。
如果是并行处理，则不需要。当然在串行的时候，也可以设置串行等待的队列的长度.
in database, table t_ds_process_definition need add cloum:is_parallel,serial_command_lengh. sql script need to exec in database.
alter table t_ds_process_definition add is_parallel tinyint default '1' comment '0:serial,1:parallel';
alter table t_ds_process_definition add serial_command_lengh bigint default '0' comment 'serial exec command lengh';
why I add two cloums in table t_ds_process_definition:
1、in process,need to solve serial run and serial run command can set wait queue length.
2、after later，we need to statistics serial commands for every projet or every processdef.
this function has uesd in our company. I hope can merger this function to apache，and in future our company can use more new apache release version。
[Image removed by sender.]
[Image removed by sender.]
[Image removed by sender.]
[Image removed by sender.]
------------------ Original ------------------
From: "dev-help" <dev-help@dolphinscheduler.apache.org>;
Date: Mon, Sep 14, 2020 10:30 AM
To: "yongqing"<597365581@qq.com>;
Subject: WELCOME to dev@dolphinscheduler.apache.org
Hi! This is the ezmlm program. I'm managing the
dev@dolphinscheduler.apache.org mailing list.
I'm working for my owner, who can be reached
at dev-owner@dolphinscheduler.apache.org.
Acknowledgment: I have added the address
597365581@qq.com
to the dev mailing list.
Welcome to dev@dolphinscheduler.apache.org!
Please save this message so that you know the address you are
subscribed under, in case you later want to unsubscribe or change your
subscription address.
--- Administrative commands for the dev list ---
I can handle administrative requests automatically. Please
do not send them to the list address! Instead, send
your message to the correct command address:
To subscribe to the list, send a message to:
<dev-subscribe@dolphinscheduler.apache.org>
To remove your address from the list, send a message to:
<dev-unsubscribe@dolphinscheduler.apache.org>
Send mail to the following for info and FAQ for this list:
<dev-info@dolphinscheduler.apache.org>
<dev-faq@dolphinscheduler.apache.org>
Similar addresses exist for the digest list:
<dev-digest-subscribe@dolphinscheduler.apache.org>
<dev-digest-unsubscribe@dolphinscheduler.apache.org>
To get messages 123 through 145 (a maximum of 100 per request), mail:
<dev-get.123_145@dolphinscheduler.apache.org>
To get an index with subject and author for messages 123-456 , mail:
<dev-index.123_456@dolphinscheduler.apache.org>
They are always returned as sets of 100, max 2000 per request,
so you'll actually get 100-499.
To receive all messages with the same subject as message 12345,
send a short message to:
<dev-thread.12345@dolphinscheduler.apache.org>
The messages should contain one line or word of text to avoid being
treated as sp@m, but I will ignore their content.
Only the ADDRESS you send to is important.
You can start a subscription for an alternate address,
for example "john@host.domain", just add a hyphen and your
address (with '=' instead of '@') after the command word:
<dev-subscribe-john=host.domain@dolphinscheduler.apache.org>
To stop subscription for this address, mail:
<dev-unsubscribe-john=host.domain@dolphinscheduler.apache.org>
In both cases, I'll send a confirmation message to that address. When
you receive it, simply reply to it to complete your subscription.
If despite following these instructions, you do not get the
desired results, please contact my owner at
dev-owner@dolphinscheduler.apache.org. Please be patient, my owner is a
lot slower than I am ;-)
--- Enclosed is a copy of the request I received.
Return-Path: <597365581@qq.com>
Received: (qmail 63012 invoked by uid 99); 14 Sep 2020 02:30:58 -0000
Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100)
by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Sep 2020 02:30:58 +0000
Received: from localhost (localhost [127.0.0.1])
by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id F158D1FF42F
for <dev-sc.1600050589.nejmeipdgampgajmnign-597365581=qq.com@dolphinscheduler.apache.org>; Mon, 14 Sep 2020 02:30:57 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org
X-Spam-Flag: NO
X-Spam-Score: 0.251
X-Spam-Level:
X-Spam-Status: No, score=0.251 tagged_above=-999 required=6.31
tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25,
FROM_EXCESS_BASE64=0.001, HTML_MESSAGE=0.2,
MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_MSPIKE_H2=-0.001,
SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled
Authentication-Results: spamproc1-he-de.apache.org (amavisd-new);
dkim=pass (1024-bit key) header.d=qq.com
Received: from mx1-he-de.apache.org ([116.203.227.195])
by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024)
with ESMTP id rY5rK6RNgRn8
for <dev-sc.1600050589.nejmeipdgampgajmnign-597365581=qq.com@dolphinscheduler.apache.org>;
Mon, 14 Sep 2020 02:30:56 +0000 (UTC)
Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=203.205.250.33; helo=smtpbg506.qq.com; envelope-from=597365581@qq.com; receiver=<UNKNOWN>
Received: from smtpbg506.qq.com (smtpbg506.qq.com [203.205.250.33])
by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 918777F981
for <dev-sc.1600050589.nejmeipdgampgajmnign-597365581=qq.com@dolphinscheduler.apache.org>; Mon, 14 Sep 2020 02:30:52 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qq.com; s=s201512;
t=1600050644; bh=D2MtfdqVTxoUoQmH8WlfF8kJ7sSgPT0WH1fQQI+tZio=;
h=From:To:Subject:Mime-Version:Date:Message-ID;
b=UEkJkZ6Y0IGCKg18m1uorLOOgtAgZUZ56Aw+Dlq+oQpfXimlaqzUXt0T1RVXAkFK6
V2IXRuKPZqcsoi89sEAs4arBR87QaXM0TzeZur90MGca6H826CsVJHgqZKVYo4jxkH
d6pzT035XOnYtROn+cmy0scJIi1xkYlWeQXZ9KaU=
X-QQ-FEAT: I1MaI21uPakL5leDgr2Av3GuL20khCzx0PEShbXmOWDj4VUpqbw5pRSBRUT9i
qfVppOMk9eD+AJVXnPoj2ZqS+ho3Y+FCFAMHmrSvYj+h7Yjbn1QMqugHs1gILDyog57vQss
Nw0k4FqVd+sw0vKgf+zHcDwJe4smtTPgVUp10djo+ktfxLfeXCeWArZ94XG/l1OPNqjzhUx
ICMYiTcPye9WruDP+SnRdL4PmXDnVg8BQ2Y4SPMY9hkgU694EClb11jGrOMOcRfW+AcRp9P
eqV9NzZAaaPE72ri+K7YdRBpz21qsPo+fn8OZVfOyWnB2YZDSekhDx6vk=
X-QQ-SSF: 00000000000000F000000000000000Z
X-QQ-XMAILINFO: OKV2ZMpe2FwgxneRb9v5pXv3/xLVYQnH3+sn/NPLv1H9tNkcDLw8f2ivo5MU8a
8pzFZV58A2XdwZlXPnTLWrM8sO915dLPfpOkYelcwrDFUNoj4B5jkUga0fVhjVczNGsdSVpvix2d6
2eEp4EtKdsEqXVQfNgZi/atwYRmY53quss8mvKa9GwF/3yHeTwUSmJeZIA8WXxRP5WdNtpmStFndM
yM0UETduUv1tfDUAlOvshnjj/QJrNSQHOXC330QgkSlDZJZupmMQ8i/ZcmBh5BP6LbD8csecXYDR6
q6sB9xNMWahzjuzKyaEdsyooTmS0zGv1U9JuZ8tSoPMx7bMg5rOh0HhVdtJOovtfqTiqdGQ+D0zNk
Ej8kAuqlSzuHV+zTe8VeYeBIoNuEvB2dTlxzalQQju2CuQ1iqViuw8wtJtJIUsA9YY7eGcdJ8Esj5
Yhyrs72Iwq4/LKGASaf8WI8Qk62H/H3EByFN+eTZxFm9fUAb/S3z0DbBKaEWIleeT5RhOfdhcbSco
765NPrtU9TQJHSAbnZeMKfsgD22RDB/K4TjI7W/qZ4I/0O+weUEosfbhlKtncOB6PiIcCeWZVyMlz
DNPQYBgTE7hUADDGu7PBgdSYxPLCZ3Qqhsg4NDZhX6KqEynMF0277HpBCYJOwTR/iCM7XhVzhp+0I
vW/xx0zoPUKTqMvQivqhc1wGJ56CNKqTWfp7CxMv9OKm9d3Nl9VvvTcdP2XPGKLXpuajq7FXRqmwv
2tsUrXagsSmRzCQrekXddfPPwozRTgDiH0gX0bFHag2JNaA==
X-HAS-ATTACH: no
X-QQ-BUSINESS-ORIGIN: 2
X-Originating-IP: 58.213.71.137
In-Reply-To: <1600050589.61547.ezmlm@dolphinscheduler.apache.org>
References: <1600050589.61547.ezmlm@dolphinscheduler.apache.org>
X-QQ-STYLE:
X-QQ-mid: webmail701t1600050643t2037358
From: "=?ISO-8859-1?B?eW9uZ3Fpbmc=?=" <597365581@qq.com>
To: "=?ISO-8859-1?B?ZGV2LXNjLjE2MDAwNTA1ODkubmVqbWVpcGRnYW1wZ2FqbW5pZ24tNTk3MzY1NTgxPXFxLmNvbQ==?=" <dev-sc.1600050589.nejmeipdgampgajmnign-597365581=qq.com@dolphinscheduler.apache.org>
Subject: Re:confirm subscribe to dev@dolphinscheduler.apache.org
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_5F5ED5D3_10876F28_16C1B710"
Content-Transfer-Encoding: 8Bit
Date: Mon, 14 Sep 2020 10:30:43 +0800
X-Priority: 3
Message-ID: <tencent_9622083678E52565EE6A8CFC4951980A580A@qq.com>
X-QQ-MIME: TCMime 1.0 by Tencent
X-Mailer: QQMail 2.x
X-QQ-Mailer: QQMail 2.x
X-QQ-ReplyHash: 291485276
X-QQ-SENDSIZE: 520
Received: from qq.com (unknown [127.0.0.1])
by smtp.qq.com (ESMTP) with SMTP
id ; Mon, 14 Sep 2020 10:30:44 +0800 (CST)
Feedback-ID: webmail:qq.com:bgforeign:bgforeign11
X-QQ-Bgrelay: 1