You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/12/16 12:31:27 UTC
[GitHub] [dolphinscheduler-sdk-python] jieguangzhou opened a new pull request, #53: [improve] More pythonic way for parameter local_params in class task
jieguangzhou opened a new pull request, #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53
<!--Thanks for you contribute to Apache DolphinScheduler Python API, You can see more detail about contributing in https://github.com/apache/dolphinscheduler-sdk-python/DEVELOP.md .-->
## Brief Summary of The Change
close: #21
```python
with Workflow(
name="tutorial",
) as workflow:
task = Shell(name="task_parent",
command="echo hello pydolphinscheduler",
input_params={
"value_VARCHAR": "abc",
"value_LONG": ParameterType.LONG("1000000"),
"value_INTEGER": 123,
"value_FLOAT": 0.1,
"value_DATE": ParameterType.DATE("2022-01-02"),
"value_TIME": ParameterType.TIME("2022-01-01"),
"value_TIMESTAMP": ParameterType.TIMESTAMP(123123124125),
"value_BOOLEAN": True,
"value_LIST": ParameterType.LIST("123123"),
},
output_params={
"output_INTEGER": ParameterType.INTEGER(100),
"output_LIST": ParameterType.LIST()
})
pprint(task.local_params)
workflow.submit()
```
<img width="454" alt="image" src="https://user-images.githubusercontent.com/31528124/208098749-ef0fde3f-c610-4b03-b36d-77c23417824c.png">
## Pull Request checklist
I confirm that the following checklist has been completed.
- [ ] Add/Change **test cases** for the changes.
- [ ] Add/Change the related **documentation**, should also change `docs/source/config.rst` when you change file `default_config.yaml`.
- [ ] (Optional) Add your change to `UPDATING.md` when it is an incompatible change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] zhongjiajie commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1053075669
##########
.gitignore:
##########
@@ -15,3 +15,6 @@ build/
.coverage
coverage.xml
htmlcov/
+
+# the pydolphinscheduler config
+config.yaml
Review Comment:
good catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] zhongjiajie commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1050748534
##########
src/pydolphinscheduler/examples/parameter_example.py:
##########
@@ -0,0 +1,79 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+r"""
+A tutorial example take you to experience pydolphinscheduler.
+
+After tutorial.py file submit to Apache DolphinScheduler server a DAG would be create,
+and workflow DAG graph as below:
+
+ --> task_child_one
+ / \
+task_parent --> --> task_union
+ \ /
+ --> task_child_two
+
+it will instantiate and run all the task it have.
+"""
+
+# [start tutorial]
+# [start package_import]
+# Import Workflow object to define your workflow attributes
+from pydolphinscheduler.core.workflow import Workflow
+from pydolphinscheduler.core.parameter import ParameterType
+
+# Import task Shell object cause we would create some shell tasks later
+from pydolphinscheduler.tasks.shell import Shell
+from pprint import pprint
+
+# [end package_import]
+
+# [start workflow_declare]
+with Workflow(
+ name="tutorial",
+) as workflow:
+ task = Shell(name="task_parent",
+ command="echo hello pydolphinscheduler",
+ input_params={
+ "value_VARCHAR": "abc",
+ "value_LONG": ParameterType.LONG("1000000"),
Review Comment:
such as long, list, date?
##########
src/pydolphinscheduler/examples/parameter_example.py:
##########
@@ -0,0 +1,79 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+r"""
+A tutorial example take you to experience pydolphinscheduler.
+
+After tutorial.py file submit to Apache DolphinScheduler server a DAG would be create,
+and workflow DAG graph as below:
+
+ --> task_child_one
+ / \
+task_parent --> --> task_union
+ \ /
+ --> task_child_two
+
+it will instantiate and run all the task it have.
+"""
+
+# [start tutorial]
+# [start package_import]
+# Import Workflow object to define your workflow attributes
+from pydolphinscheduler.core.workflow import Workflow
+from pydolphinscheduler.core.parameter import ParameterType
+
+# Import task Shell object cause we would create some shell tasks later
+from pydolphinscheduler.tasks.shell import Shell
+from pprint import pprint
+
+# [end package_import]
+
+# [start workflow_declare]
+with Workflow(
+ name="tutorial",
+) as workflow:
+ task = Shell(name="task_parent",
+ command="echo hello pydolphinscheduler",
+ input_params={
+ "value_VARCHAR": "abc",
+ "value_LONG": ParameterType.LONG("1000000"),
Review Comment:
can we also find some way to auto-convert these parameter to java map?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] jieguangzhou commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1052849183
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -144,6 +145,8 @@ def __init__(
condition_result: Optional[Dict] = None,
resource_plugin: Optional[ResourcePlugin] = None,
is_cache: Optional[bool] = False,
+ input_params: Optional[Dict] = None,
+ output_params: Optional[Dict] = None,
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] jieguangzhou merged pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
jieguangzhou merged PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] zhongjiajie commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1052887580
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -415,3 +453,31 @@ def environment_code(self) -> str:
if self._environment_name is None:
return None
return gateway.query_environment_info(self._environment_name)
+
+ @property
+ def local_params(self):
+ """Convert local params."""
+ local_params = copy.deepcopy(self._local_params)
+ local_params.extend(
+ ParameterHelper.convert_params(self._input_params, Direction.IN)
+ )
+ local_params.extend(
+ ParameterHelper.convert_params(self._output_params, Direction.OUT)
+ )
+ return local_params
+
+ def add_in(self, name, value=None):
+ """Add input parameters.
+
+ :param name: str, name of the input parameter.
+ :param value: [int | str | float | bool | ParameterType ], value of the input parameter.
Review Comment:
The type hint should be better in parameter declare, it will render the type in API document when in param typing
```diff
- def add_in(self, name, value=None):
+ def add_in(self, name: str, value: Optional[Union[int, str, float, bool, ParameterType]] = None):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] jieguangzhou commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1050860327
##########
src/pydolphinscheduler/examples/parameter_example.py:
##########
@@ -0,0 +1,79 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+r"""
+A tutorial example take you to experience pydolphinscheduler.
+
+After tutorial.py file submit to Apache DolphinScheduler server a DAG would be create,
+and workflow DAG graph as below:
+
+ --> task_child_one
+ / \
+task_parent --> --> task_union
+ \ /
+ --> task_child_two
+
+it will instantiate and run all the task it have.
+"""
+
+# [start tutorial]
+# [start package_import]
+# Import Workflow object to define your workflow attributes
+from pydolphinscheduler.core.workflow import Workflow
+from pydolphinscheduler.core.parameter import ParameterType
+
+# Import task Shell object cause we would create some shell tasks later
+from pydolphinscheduler.tasks.shell import Shell
+from pprint import pprint
+
+# [end package_import]
+
+# [start workflow_declare]
+with Workflow(
+ name="tutorial",
+) as workflow:
+ task = Shell(name="task_parent",
+ command="echo hello pydolphinscheduler",
+ input_params={
+ "value_VARCHAR": "abc",
+ "value_LONG": ParameterType.LONG("1000000"),
Review Comment:
Yes, but I have no experience in using these parameters, so I don't know how to convert them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] zhongjiajie commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1052885382
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -169,6 +195,18 @@ def __init__(
self.workflow = kwargs.pop("process_definition")
else:
self.workflow: Workflow = workflow or WorkflowContext.get()
+
+ if "local_params" in kwargs:
Review Comment:
we should directly remove the `local_params` in function `__init__` in L162, otherwise local_params will never in `kwargs`
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -82,7 +83,28 @@ def __hash__(self):
class Task(Base):
- """Task object, parent class for all exactly task type."""
+ """Task object, parent class for all exactly task type.
+
+ :param name: The name of the task. Node names within the same workflow must be unique.
+ :param task_type: task type
+ :param description: str, Describing the function of this node.
+ :param flag: str, default TaskFlag.YES,
Review Comment:
we should remove the type here, because the parameter type will auto parser from class function `__init__` type hint from code, you can see in https://dolphinscheduler.apache.org/python/main/api.html#pydolphinscheduler.tasks.DataX
```suggestion
:param flag: default TaskFlag.YES,
```
##########
src/pydolphinscheduler/core/yaml_workflow.py:
##########
@@ -75,6 +76,24 @@ def parse_string_param_if_config(string_param: str, **kwargs):
return string_param
+ @staticmethod
+ def parse_string_param_if_parameter(string_param: str, **kwargs):
+ """Use TYPE(value) to set local params."""
+ key_path = kwargs.get("key_path")
+ if key_path.split("-")[0] not in {"input_params", "output_params"}:
Review Comment:
can we add `-` to `pydolphinscheduler.constants.Symbol` ?
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -169,6 +195,18 @@ def __init__(
self.workflow = kwargs.pop("process_definition")
else:
self.workflow: Workflow = workflow or WorkflowContext.get()
+
+ if "local_params" in kwargs:
Review Comment:
also for L226, and when we remove both of them, we should add attr check in function `local_params`, because `self._local_params` only assign in L206 when `local_params` in `kwargs`, means it is optional, so we should check wether it exists or not first
```diff
- local_params = copy.deepcopy(self._local_params)
+ local_params = copy.deepcopy(self._local_params) if hasattr(self, "_local_params") or []
```
##########
docs/source/concept.rst:
##########
@@ -260,10 +260,21 @@ After that, we could see new file named ``bare-create.py`` is be created in reso
Both parameter ``resource_list`` in workflow and task is list of string which mean you could upload and reference
multiple files. For more complex usage, you could read :doc:`howto/multi-resources`.
+Define Local Parameters
+-----------------------
+
+There are two ways to define local parameters
Review Comment:
could we add more detail about what parameters is, and what it works for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler-sdk-python] zhongjiajie commented on a diff in pull request #53: [improve] More pythonic way for parameter local_params in class task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #53:
URL: https://github.com/apache/dolphinscheduler-sdk-python/pull/53#discussion_r1052004625
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -415,3 +420,23 @@ def environment_code(self) -> str:
if self._environment_name is None:
return None
return gateway.query_environment_info(self._environment_name)
+
+ @property
+ def local_params(self):
+ """Convert local params."""
+ local_params = copy.deepcopy(self._local_params)
+ local_params.extend(
+ ParameterHelper.convert_params(self._input_params, Direction.IN)
+ )
+ local_params.extend(
+ ParameterHelper.convert_params(self._output_params, Direction.OUT)
+ )
+ return local_params
+
+ def add_in(self, name, value=None):
+ """Add input parameters."""
Review Comment:
could we also add a docstring for both `name` and `value`?
##########
docs/source/tutorial.rst:
##########
@@ -317,3 +317,13 @@ Furthermore, this feature supports recursion all the way down.
.. _`DolphinScheduler project page`: https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/guide/project.html
.. _`Python context manager`: https://docs.python.org/3/library/stdtypes.html#context-manager-types
+
+Define Local Parameters
Review Comment:
tutorial should keep as simple as possible, could your please add those content to https://dolphinscheduler.apache.org/python/main/concept.html page please
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -144,6 +145,8 @@ def __init__(
condition_result: Optional[Dict] = None,
resource_plugin: Optional[ResourcePlugin] = None,
is_cache: Optional[bool] = False,
+ input_params: Optional[Dict] = None,
+ output_params: Optional[Dict] = None,
Review Comment:
BTW, please add doctoring for this two new parameter
##########
src/pydolphinscheduler/examples/local_parameter_example.py:
##########
@@ -0,0 +1,81 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# [start workflow_declare]
+r"""
+A tutorial example set local parameter in pydolphinscheduler.
+
+Method 1:
+ task = Shell(..., input_params={"input":"a"}, output_params={"output": "b"})
+
+Method 2:
+ task = Shell(...)
+ task.add_in("input", "a")
+ task.add_out("output", "b")
+"""
+
+from pydolphinscheduler.core.parameter import ParameterType
+from pydolphinscheduler.core.workflow import Workflow
+from pydolphinscheduler.tasks.shell import Shell
+
+with Workflow(name="local_parameter_example", release_state="offline") as workflow:
+
+ # Add in task __init__ func
+ task_1 = Shell(
+ name="task_1",
+ command="echo hello pydolphinscheduler",
+ input_params={
+ "value_VARCHAR": "abc",
+ "value_INTEGER": 123,
+ "value_FLOAT": 0.1,
+ "value_BOOLEAN": True,
+ },
+ output_params={
+ "value_EMPTY": None,
+ },
+ )
+
+ # Add in task instance
+ task_2 = Shell(name="task_2", command="echo hello pydolphinscheduler")
+
+ task_2.add_in("value_VARCHAR", "abc")
+ task_2.add_in("value_INTEGER", 123)
+ task_2.add_in("value_FLOAT", 0.1)
+ task_2.add_in("value_BOOLEAN", True)
+ task_2.add_out("value_EMPTY")
+
+ # Task 1 is the same as task 2
+
+ # Other parameter types corresponding to DolphinScheduler
Review Comment:
```suggestion
# Others parameter types which cannot be converted automatically, must declare type explicitly
```
##########
tests/core/test_task.py:
##########
@@ -509,3 +510,66 @@ def test_python_resource_list(
resource_list=resources,
)
assert task.resource_list == expect
+
+
+@patch(
+ "pydolphinscheduler.core.task.Task.gen_code_and_version",
+ return_value=(123, 1),
+)
+def test_local_parameter(m_code_version):
+ """Test task local_params."""
+ base_local_params = [
+ {"prop": "base", "direct": "IN", "type": "VARCHAR", "value": "2022"},
+ ]
+
+ task = Task(name="test", task_type="task_type", local_params=base_local_params)
Review Comment:
why should we still keep `local_params`?
##########
src/pydolphinscheduler/core/task.py:
##########
@@ -415,3 +420,23 @@ def environment_code(self) -> str:
if self._environment_name is None:
return None
return gateway.query_environment_info(self._environment_name)
+
+ @property
+ def local_params(self):
+ """Convert local params."""
+ local_params = copy.deepcopy(self._local_params)
+ local_params.extend(
+ ParameterHelper.convert_params(self._input_params, Direction.IN)
+ )
+ local_params.extend(
+ ParameterHelper.convert_params(self._output_params, Direction.OUT)
+ )
+ return local_params
+
+ def add_in(self, name, value=None):
+ """Add input parameters."""
Review Comment:
same as func `add_out`
##########
src/pydolphinscheduler/examples/local_parameter_example.py:
##########
@@ -0,0 +1,81 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# [start workflow_declare]
+r"""
+A tutorial example set local parameter in pydolphinscheduler.
+
+Method 1:
+ task = Shell(..., input_params={"input":"a"}, output_params={"output": "b"})
+
+Method 2:
+ task = Shell(...)
+ task.add_in("input", "a")
+ task.add_out("output", "b")
+"""
+
+from pydolphinscheduler.core.parameter import ParameterType
+from pydolphinscheduler.core.workflow import Workflow
+from pydolphinscheduler.tasks.shell import Shell
+
+with Workflow(name="local_parameter_example", release_state="offline") as workflow:
+
+ # Add in task __init__ func
+ task_1 = Shell(
+ name="task_1",
+ command="echo hello pydolphinscheduler",
+ input_params={
+ "value_VARCHAR": "abc",
+ "value_INTEGER": 123,
+ "value_FLOAT": 0.1,
+ "value_BOOLEAN": True,
+ },
+ output_params={
+ "value_EMPTY": None,
+ },
+ )
+
+ # Add in task instance
Review Comment:
```suggestion
# Add parameter via task instance's method
```
##########
src/pydolphinscheduler/examples/local_parameter_example.py:
##########
@@ -0,0 +1,81 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# [start workflow_declare]
+r"""
+A tutorial example set local parameter in pydolphinscheduler.
+
+Method 1:
+ task = Shell(..., input_params={"input":"a"}, output_params={"output": "b"})
+
+Method 2:
+ task = Shell(...)
+ task.add_in("input", "a")
+ task.add_out("output", "b")
+"""
+
+from pydolphinscheduler.core.parameter import ParameterType
+from pydolphinscheduler.core.workflow import Workflow
+from pydolphinscheduler.tasks.shell import Shell
+
+with Workflow(name="local_parameter_example", release_state="offline") as workflow:
+
+ # Add in task __init__ func
Review Comment:
```suggestion
# Add parameter via task arguments
```
##########
tests/core/test_task.py:
##########
@@ -509,3 +510,66 @@ def test_python_resource_list(
resource_list=resources,
)
assert task.resource_list == expect
+
+
+@patch(
+ "pydolphinscheduler.core.task.Task.gen_code_and_version",
+ return_value=(123, 1),
+)
+def test_local_parameter(m_code_version):
+ """Test task local_params."""
+ base_local_params = [
+ {"prop": "base", "direct": "IN", "type": "VARCHAR", "value": "2022"},
+ ]
+
+ task = Task(name="test", task_type="task_type", local_params=base_local_params)
Review Comment:
BTW, good test cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org