You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/02/04 08:42:03 UTC

[GitHub] [incubator-dolphinscheduler] wangxj3 opened a new pull request #4678: [FIX-#3805][server-master] global params of master

wangxj3 opened a new pull request #4678:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/4678

The function of global params has three parts,This part is about master.

At present, we have implemented the global params function
1. The params takes effect in the process instance, each node can accept the variable, SQL and SHELL nodes can modify the params value, and subsequent nodes use the params as the modified value. (requirement 1)
2. The subprocess node passes in variables, and the subprocess can receive params values (requirement 2)

The detail of the scheme as follows

a： requirement 1
1. When saving workflow, users will add global params definition (global definition) in the pop-up box after clicking save button, and save the defined params name and default value (if there is an initial value) to t_ds_process_definition.global_Params field (using the existing function). When saving, it will verify that the params names used by the node are all from global definition.
2. When the workflow is called up for instantiation, get the value of globalparams from definition and initialize the params value to t_ds_process_instance.global_Params field. (at this time, the initial value of the workflow instance is the default value defined globally)
3. About input parameters: when creating TaskInstance, take variables from localparam, and set the value from t_ds_process_instance.global_params to localparam. After storing the task instance, the task instance is saved in the database and distributed to the worker, and the subsequent parameter processing follows the original part.
4. About output parameters: after the task execution of SQL node and SHELL node is finished(SQL query results, only support one field one value, if multiple fields or rows of data will be formatted as JSON;The SHELL uses the output of the last line as the result), write the result of task execution into TaskExecuteResponseCommand, return to master together with the worker execution result information, and the master will update the output t_ds_process_instance.global_params and update it to the database.

b： requirement 2
1. About subprocess node: judge whether there is parameter passing when generating subprocess command. If there is parameter passing, the parameter will be written into the command table.
2. When the subprocess is instantiated, judge that the command contains the parameter passed down by the main process, and update the value to t_ds_process_instance.global_params field. The subsequent part is consistent with requirement 1
To implement requirement 2, you need to add COLUMN son_Params in t_ds_command table.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org