You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/12/14 09:28:13 UTC
[GitHub] [dolphinscheduler] zhongjiajie opened a new pull request #7405: [python] Add task dependent
zhongjiajie opened a new pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405
fix: #6926
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-993344077
@devosend Do you have time to take a look at this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a change in pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#discussion_r769651652
##########
File path: dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/dependent.py
##########
@@ -0,0 +1,276 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Task dependent."""
+
+from typing import Dict, Optional, Tuple
+
+from pydolphinscheduler.constants import TaskType
+from pydolphinscheduler.core.base import Base
+from pydolphinscheduler.core.task import Task
+from pydolphinscheduler.exceptions import PyDSJavaGatewayException, PyDSParamException
+from pydolphinscheduler.java_gateway import launch_gateway
+
+DEPENDENT_ALL_TASK_IN_WORKFLOW = "0"
+
+
+class DependentDate(str):
+ """Constant of Dependent date value.
+
+ These values set according to Java server side, if you want to add and change it,
+ please change Java server side first.
+ """
+
+ # TODO Maybe we should add parent level to DependentDate for easy to use, such as
+ # DependentDate.MONTH.THIS_MONTH
+
+ # Hour
+ CURRENT_HOUR = "currentHour"
+ LAST_ONE_HOUR = "last1Hour"
+ LAST_TWO_HOURS = "last2Hours"
+ LAST_THREE_HOURS = "last3Hours"
+ LAST_TWENTY_FOUR_HOURS = "last24Hours"
+
+ # Day
+ TODAY = "today"
+ LAST_ONE_DAYS = "last1Days"
+ LAST_TWO_DAYS = "last2Days"
+ LAST_THREE_DAYS = "last3Days"
+ LAST_SEVEN_DAYS = "last7Days"
+
+ # Week
+ THIS_WEEK = "thisWeek"
+ LAST_WEEK = "lastWeek"
+ LAST_MONDAY = "lastMonday"
+ LAST_TUESDAY = "lastTuesday"
+ LAST_WEDNESDAY = "lastWednesday"
+ LAST_THURSDAY = "lastThursday"
+ LAST_FRIDAY = "lastFriday"
+ LAST_SATURDAY = "lastSaturday"
+ LAST_SUNDAY = "lastSunday"
+
+ # Month
+ THIS_MONTH = "thisMonth"
+ LAST_MONTH = "lastMonth"
+ LAST_MONTH_BEGIN = "lastMonthBegin"
+ LAST_MONTH_END = "lastMonthEnd"
+
+
+class DependentItem(Base):
+ """Dependent item object, minimal unit for task dependent.
+
+ It declare which project, process_definition, task are dependent to this task.
+ """
+
+ _DEFINE_ATTR = {
+ "project_code",
+ "definition_code",
+ "dep_task_code",
+ "cycle",
+ "date_value",
+ }
+
+ # TODO maybe we should conside overwrite operator `and` and `or` for DependentItem to
+ # support more easy way to set relation
+ def __init__(
+ self,
+ project_name: str,
+ process_definition_name: str,
+ dependent_task_name: Optional[str] = DEPENDENT_ALL_TASK_IN_WORKFLOW,
+ dependent_date: Optional[DependentDate] = DependentDate.TODAY,
+ ):
+ obj_name = f"{project_name}.{process_definition_name}.{dependent_task_name}.{dependent_date}"
+ super().__init__(obj_name)
+ self.project_name = project_name
+ self.process_definition_name = process_definition_name
+ self.dependent_task_name = dependent_task_name
+ if dependent_date is None:
+ raise PyDSParamException(
+ "Parameter dependent_date must provider by got None."
+ )
+ else:
+ self.dependent_date = dependent_date
+ self._code = {}
+
+ def __repr__(self) -> str:
+ return "depend_item_list"
+
+ @property
+ def project_code(self) -> str:
+ """Get dependent project code."""
+ return self.get_code_from_gateway().get("projectCode")
+
+ @property
+ def definition_code(self) -> str:
+ """Get dependent definition code."""
+ return self.get_code_from_gateway().get("processDefinitionCode")
+
+ @property
+ def dep_task_code(self) -> str:
+ """Get dependent tasks code list."""
+ if self.is_all_task:
+ return DEPENDENT_ALL_TASK_IN_WORKFLOW
+ else:
+ return self.get_code_from_gateway().get("taskDefinitionCode")
+
+ # TODO Maybe we should get cycle from dependent date class.
+ @property
+ def cycle(self) -> str:
+ """Get dependent cycle."""
+ if "Hour" in self.dependent_date:
+ return "hour"
+ elif self.dependent_date == "today" or "Days" in self.dependent_date:
+ return "day"
+ elif "Month" in self.dependent_date:
+ return "month"
+ else:
+ return "week"
+
+ @property
+ def date_value(self) -> str:
+ """Get dependent date."""
+ return self.dependent_date
+
+ @property
+ def is_all_task(self) -> bool:
+ """Check whether dependent all tasks or not."""
+ return self.dependent_task_name == DEPENDENT_ALL_TASK_IN_WORKFLOW
+
+ @property
+ def code_parameter(self) -> Tuple:
+ """Get name info parameter to query code."""
+ param = (
+ self.project_name,
+ self.process_definition_name,
+ self.dependent_task_name if not self.is_all_task else None,
+ )
+ return param
+
+ def get_code_from_gateway(self) -> Dict:
+ """Get project, definition, task code from given parameter."""
+ if self._code:
+ return self._code
+ else:
+ gateway = launch_gateway()
+ try:
+ return gateway.entry_point.getDependentInfo(*self.code_parameter)
Review comment:
Here we found out another situation, as we discuss before, we did not test the interface from python to gateway, maybe we should find a way to do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a change in pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#discussion_r769647078
##########
File path: dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/dependent.py
##########
@@ -0,0 +1,276 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Task dependent."""
+
+from typing import Dict, Optional, Tuple
+
+from pydolphinscheduler.constants import TaskType
+from pydolphinscheduler.core.base import Base
+from pydolphinscheduler.core.task import Task
+from pydolphinscheduler.exceptions import PyDSJavaGatewayException, PyDSParamException
+from pydolphinscheduler.java_gateway import launch_gateway
+
+DEPENDENT_ALL_TASK_IN_WORKFLOW = "0"
+
+
+class DependentDate(str):
+ """Constant of Dependent date value.
+
+ These values set according to Java server side, if you want to add and change it,
+ please change Java server side first.
+ """
+
+ # TODO Maybe we should add parent level to DependentDate for easy to use, such as
+ # DependentDate.MONTH.THIS_MONTH
+
+ # Hour
+ CURRENT_HOUR = "currentHour"
+ LAST_ONE_HOUR = "last1Hour"
+ LAST_TWO_HOURS = "last2Hours"
+ LAST_THREE_HOURS = "last3Hours"
+ LAST_TWENTY_FOUR_HOURS = "last24Hours"
+
+ # Day
+ TODAY = "today"
+ LAST_ONE_DAYS = "last1Days"
+ LAST_TWO_DAYS = "last2Days"
+ LAST_THREE_DAYS = "last3Days"
+ LAST_SEVEN_DAYS = "last7Days"
+
+ # Week
+ THIS_WEEK = "thisWeek"
+ LAST_WEEK = "lastWeek"
+ LAST_MONDAY = "lastMonday"
+ LAST_TUESDAY = "lastTuesday"
+ LAST_WEDNESDAY = "lastWednesday"
+ LAST_THURSDAY = "lastThursday"
+ LAST_FRIDAY = "lastFriday"
+ LAST_SATURDAY = "lastSaturday"
+ LAST_SUNDAY = "lastSunday"
+
+ # Month
+ THIS_MONTH = "thisMonth"
+ LAST_MONTH = "lastMonth"
+ LAST_MONTH_BEGIN = "lastMonthBegin"
+ LAST_MONTH_END = "lastMonthEnd"
+
+
+class DependentItem(Base):
+ """Dependent item object, minimal unit for task dependent.
+
+ It declare which project, process_definition, task are dependent to this task.
+ """
+
+ _DEFINE_ATTR = {
+ "project_code",
+ "definition_code",
+ "dep_task_code",
+ "cycle",
+ "date_value",
+ }
+
+ # TODO maybe we should conside overwrite operator `and` and `or` for DependentItem to
+ # support more easy way to set relation
+ def __init__(
+ self,
+ project_name: str,
+ process_definition_name: str,
+ dependent_task_name: Optional[str] = DEPENDENT_ALL_TASK_IN_WORKFLOW,
+ dependent_date: Optional[DependentDate] = DependentDate.TODAY,
+ ):
+ obj_name = f"{project_name}.{process_definition_name}.{dependent_task_name}.{dependent_date}"
+ super().__init__(obj_name)
+ self.project_name = project_name
+ self.process_definition_name = process_definition_name
+ self.dependent_task_name = dependent_task_name
+ if dependent_date is None:
+ raise PyDSParamException(
+ "Parameter dependent_date must provider by got None."
+ )
+ else:
+ self.dependent_date = dependent_date
+ self._code = {}
+
+ def __repr__(self) -> str:
+ return "depend_item_list"
+
+ @property
+ def project_code(self) -> str:
+ """Get dependent project code."""
+ return self.get_code_from_gateway().get("projectCode")
+
+ @property
+ def definition_code(self) -> str:
+ """Get dependent definition code."""
+ return self.get_code_from_gateway().get("processDefinitionCode")
+
+ @property
+ def dep_task_code(self) -> str:
+ """Get dependent tasks code list."""
+ if self.is_all_task:
+ return DEPENDENT_ALL_TASK_IN_WORKFLOW
+ else:
+ return self.get_code_from_gateway().get("taskDefinitionCode")
+
+ # TODO Maybe we should get cycle from dependent date class.
+ @property
+ def cycle(self) -> str:
+ """Get dependent cycle."""
+ if "Hour" in self.dependent_date:
+ return "hour"
+ elif self.dependent_date == "today" or "Days" in self.dependent_date:
+ return "day"
+ elif "Month" in self.dependent_date:
+ return "month"
+ else:
+ return "week"
+
+ @property
+ def date_value(self) -> str:
+ """Get dependent date."""
+ return self.dependent_date
+
+ @property
+ def is_all_task(self) -> bool:
+ """Check whether dependent all tasks or not."""
+ return self.dependent_task_name == DEPENDENT_ALL_TASK_IN_WORKFLOW
+
+ @property
+ def code_parameter(self) -> Tuple:
+ """Get name info parameter to query code."""
+ param = (
+ self.project_name,
+ self.process_definition_name,
+ self.dependent_task_name if not self.is_all_task else None,
+ )
+ return param
+
+ def get_code_from_gateway(self) -> Dict:
+ """Get project, definition, task code from given parameter."""
+ if self._code:
+ return self._code
+ else:
+ gateway = launch_gateway()
+ try:
+ return gateway.entry_point.getDependentInfo(*self.code_parameter)
Review comment:
I would cause we query s java gateway many time for duplicate condition, thank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-994833596
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-993363074
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a change in pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on a change in pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#discussion_r769646046
##########
File path: dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/dependent.py
##########
@@ -0,0 +1,276 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Task dependent."""
+
+from typing import Dict, Optional, Tuple
+
+from pydolphinscheduler.constants import TaskType
+from pydolphinscheduler.core.base import Base
+from pydolphinscheduler.core.task import Task
+from pydolphinscheduler.exceptions import PyDSJavaGatewayException, PyDSParamException
+from pydolphinscheduler.java_gateway import launch_gateway
+
+DEPENDENT_ALL_TASK_IN_WORKFLOW = "0"
+
+
+class DependentDate(str):
+ """Constant of Dependent date value.
+
+ These values set according to Java server side, if you want to add and change it,
+ please change Java server side first.
+ """
+
+ # TODO Maybe we should add parent level to DependentDate for easy to use, such as
+ # DependentDate.MONTH.THIS_MONTH
+
+ # Hour
+ CURRENT_HOUR = "currentHour"
+ LAST_ONE_HOUR = "last1Hour"
+ LAST_TWO_HOURS = "last2Hours"
+ LAST_THREE_HOURS = "last3Hours"
+ LAST_TWENTY_FOUR_HOURS = "last24Hours"
+
+ # Day
+ TODAY = "today"
+ LAST_ONE_DAYS = "last1Days"
+ LAST_TWO_DAYS = "last2Days"
+ LAST_THREE_DAYS = "last3Days"
+ LAST_SEVEN_DAYS = "last7Days"
+
+ # Week
+ THIS_WEEK = "thisWeek"
+ LAST_WEEK = "lastWeek"
+ LAST_MONDAY = "lastMonday"
+ LAST_TUESDAY = "lastTuesday"
+ LAST_WEDNESDAY = "lastWednesday"
+ LAST_THURSDAY = "lastThursday"
+ LAST_FRIDAY = "lastFriday"
+ LAST_SATURDAY = "lastSaturday"
+ LAST_SUNDAY = "lastSunday"
+
+ # Month
+ THIS_MONTH = "thisMonth"
+ LAST_MONTH = "lastMonth"
+ LAST_MONTH_BEGIN = "lastMonthBegin"
+ LAST_MONTH_END = "lastMonthEnd"
+
+
+class DependentItem(Base):
+ """Dependent item object, minimal unit for task dependent.
+
+ It declare which project, process_definition, task are dependent to this task.
+ """
+
+ _DEFINE_ATTR = {
+ "project_code",
+ "definition_code",
+ "dep_task_code",
+ "cycle",
+ "date_value",
+ }
+
+ # TODO maybe we should conside overwrite operator `and` and `or` for DependentItem to
+ # support more easy way to set relation
+ def __init__(
+ self,
+ project_name: str,
+ process_definition_name: str,
+ dependent_task_name: Optional[str] = DEPENDENT_ALL_TASK_IN_WORKFLOW,
+ dependent_date: Optional[DependentDate] = DependentDate.TODAY,
+ ):
+ obj_name = f"{project_name}.{process_definition_name}.{dependent_task_name}.{dependent_date}"
+ super().__init__(obj_name)
+ self.project_name = project_name
+ self.process_definition_name = process_definition_name
+ self.dependent_task_name = dependent_task_name
+ if dependent_date is None:
+ raise PyDSParamException(
+ "Parameter dependent_date must provider by got None."
+ )
+ else:
+ self.dependent_date = dependent_date
+ self._code = {}
+
+ def __repr__(self) -> str:
+ return "depend_item_list"
+
+ @property
+ def project_code(self) -> str:
+ """Get dependent project code."""
+ return self.get_code_from_gateway().get("projectCode")
+
+ @property
+ def definition_code(self) -> str:
+ """Get dependent definition code."""
+ return self.get_code_from_gateway().get("processDefinitionCode")
+
+ @property
+ def dep_task_code(self) -> str:
+ """Get dependent tasks code list."""
+ if self.is_all_task:
+ return DEPENDENT_ALL_TASK_IN_WORKFLOW
+ else:
+ return self.get_code_from_gateway().get("taskDefinitionCode")
+
+ # TODO Maybe we should get cycle from dependent date class.
+ @property
+ def cycle(self) -> str:
+ """Get dependent cycle."""
+ if "Hour" in self.dependent_date:
+ return "hour"
+ elif self.dependent_date == "today" or "Days" in self.dependent_date:
+ return "day"
+ elif "Month" in self.dependent_date:
+ return "month"
+ else:
+ return "week"
+
+ @property
+ def date_value(self) -> str:
+ """Get dependent date."""
+ return self.dependent_date
+
+ @property
+ def is_all_task(self) -> bool:
+ """Check whether dependent all tasks or not."""
+ return self.dependent_task_name == DEPENDENT_ALL_TASK_IN_WORKFLOW
+
+ @property
+ def code_parameter(self) -> Tuple:
+ """Get name info parameter to query code."""
+ param = (
+ self.project_name,
+ self.process_definition_name,
+ self.dependent_task_name if not self.is_all_task else None,
+ )
+ return param
+
+ def get_code_from_gateway(self) -> Dict:
+ """Get project, definition, task code from given parameter."""
+ if self._code:
+ return self._code
+ else:
+ gateway = launch_gateway()
+ try:
+ return gateway.entry_point.getDependentInfo(*self.code_parameter)
Review comment:
Good catch! I miss it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-994833596
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-995013477
workflow e2e failed, I rebase and force restart CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-995368428
CI pass now, PTAL again @kezhenxu94
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-994828285
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7405](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (eae60b4) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/8e62dd8246997c5519176098cbc3f9075bb60265?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8e62dd8) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #7405 +/- ##
============================================
+ Coverage 41.24% 41.26% +0.01%
- Complexity 3661 3662 +1
============================================
Files 634 634
Lines 26399 26405 +6
Branches 2958 2959 +1
============================================
+ Hits 10889 10896 +7
- Misses 14483 14484 +1
+ Partials 1027 1025 -2
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...uler/api/service/impl/TaskInstanceServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9UYXNrSW5zdGFuY2VTZXJ2aWNlSW1wbC5qYXZh) | `88.52% <0.00%> (-3.01%)` | :arrow_down: |
| [...i/service/impl/AlertPluginInstanceServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9BbGVydFBsdWdpbkluc3RhbmNlU2VydmljZUltcGwuamF2YQ==) | `65.59% <0.00%> (-2.19%)` | :arrow_down: |
| [.../api/controller/AlertPluginInstanceController.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2NvbnRyb2xsZXIvQWxlcnRQbHVnaW5JbnN0YW5jZUNvbnRyb2xsZXIuamF2YQ==) | `7.69% <0.00%> (-0.31%)` | :arrow_down: |
| [...r/plugin/datasource/hive/HiveDataSourceClient.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2UtaGl2ZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZG9scGhpbnNjaGVkdWxlci9wbHVnaW4vZGF0YXNvdXJjZS9oaXZlL0hpdmVEYXRhU291cmNlQ2xpZW50LmphdmE=) | `1.51% <0.00%> (-0.03%)` | :arrow_down: |
| [...cheduler/api/controller/AccessTokenController.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2NvbnRyb2xsZXIvQWNjZXNzVG9rZW5Db250cm9sbGVyLmphdmE=) | `88.23% <0.00%> (ø)` | |
| [...urce/api/datasource/db2/Db2DatasourceParamDTO.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2UtYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi9kYXRhc291cmNlL2FwaS9kYXRhc291cmNlL2RiMi9EYjJEYXRhc291cmNlUGFyYW1EVE8uamF2YQ==) | | |
| [...atasource/sqlserver/SqlserverDataSourceClient.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2Utc3Fsc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi9kYXRhc291cmNlL3NxbHNlcnZlci9TcWxzZXJ2ZXJEYXRhU291cmNlQ2xpZW50LmphdmE=) | | |
| [.../api/datasource/mysql/MysqlDatasourceParamDTO.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2UtYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi9kYXRhc291cmNlL2FwaS9kYXRhc291cmNlL215c3FsL015c3FsRGF0YXNvdXJjZVBhcmFtRFRPLmphdmE=) | | |
| [...ource/postgresql/PostgreSqlDatasourceParamDTO.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2UtYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi9kYXRhc291cmNlL2FwaS9kYXRhc291cmNlL3Bvc3RncmVzcWwvUG9zdGdyZVNxbERhdGFzb3VyY2VQYXJhbURUTy5qYXZh) | | |
| [...api/datasource/spark/SparkDatasourceProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2UtYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi9kYXRhc291cmNlL2FwaS9kYXRhc291cmNlL3NwYXJrL1NwYXJrRGF0YXNvdXJjZVByb2Nlc3Nvci5qYXZh) | | |
| ... and [76 more](https://codecov.io/gh/apache/dolphinscheduler/pull/7405/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8e62dd8...eae60b4](https://codecov.io/gh/apache/dolphinscheduler/pull/7405?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-993363074
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#issuecomment-995027346
SonarCloud Quality Gate failed. 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=7405&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=7405&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] devosend commented on a change in pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
devosend commented on a change in pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405#discussion_r768735419
##########
File path: dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/dependent.py
##########
@@ -0,0 +1,276 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Task dependent."""
+
+from typing import Dict, Optional, Tuple
+
+from pydolphinscheduler.constants import TaskType
+from pydolphinscheduler.core.base import Base
+from pydolphinscheduler.core.task import Task
+from pydolphinscheduler.exceptions import PyDSJavaGatewayException, PyDSParamException
+from pydolphinscheduler.java_gateway import launch_gateway
+
+DEPENDENT_ALL_TASK_IN_WORKFLOW = "0"
+
+
+class DependentDate(str):
+ """Constant of Dependent date value.
+
+ These values set according to Java server side, if you want to add and change it,
+ please change Java server side first.
+ """
+
+ # TODO Maybe we should add parent level to DependentDate for easy to use, such as
+ # DependentDate.MONTH.THIS_MONTH
+
+ # Hour
+ CURRENT_HOUR = "currentHour"
+ LAST_ONE_HOUR = "last1Hour"
+ LAST_TWO_HOURS = "last2Hours"
+ LAST_THREE_HOURS = "last3Hours"
+ LAST_TWENTY_FOUR_HOURS = "last24Hours"
+
+ # Day
+ TODAY = "today"
+ LAST_ONE_DAYS = "last1Days"
+ LAST_TWO_DAYS = "last2Days"
+ LAST_THREE_DAYS = "last3Days"
+ LAST_SEVEN_DAYS = "last7Days"
+
+ # Week
+ THIS_WEEK = "thisWeek"
+ LAST_WEEK = "lastWeek"
+ LAST_MONDAY = "lastMonday"
+ LAST_TUESDAY = "lastTuesday"
+ LAST_WEDNESDAY = "lastWednesday"
+ LAST_THURSDAY = "lastThursday"
+ LAST_FRIDAY = "lastFriday"
+ LAST_SATURDAY = "lastSaturday"
+ LAST_SUNDAY = "lastSunday"
+
+ # Month
+ THIS_MONTH = "thisMonth"
+ LAST_MONTH = "lastMonth"
+ LAST_MONTH_BEGIN = "lastMonthBegin"
+ LAST_MONTH_END = "lastMonthEnd"
+
+
+class DependentItem(Base):
+ """Dependent item object, minimal unit for task dependent.
+
+ It declare which project, process_definition, task are dependent to this task.
+ """
+
+ _DEFINE_ATTR = {
+ "project_code",
+ "definition_code",
+ "dep_task_code",
+ "cycle",
+ "date_value",
+ }
+
+ # TODO maybe we should conside overwrite operator `and` and `or` for DependentItem to
+ # support more easy way to set relation
+ def __init__(
+ self,
+ project_name: str,
+ process_definition_name: str,
+ dependent_task_name: Optional[str] = DEPENDENT_ALL_TASK_IN_WORKFLOW,
+ dependent_date: Optional[DependentDate] = DependentDate.TODAY,
+ ):
+ obj_name = f"{project_name}.{process_definition_name}.{dependent_task_name}.{dependent_date}"
+ super().__init__(obj_name)
+ self.project_name = project_name
+ self.process_definition_name = process_definition_name
+ self.dependent_task_name = dependent_task_name
+ if dependent_date is None:
+ raise PyDSParamException(
+ "Parameter dependent_date must provider by got None."
+ )
+ else:
+ self.dependent_date = dependent_date
+ self._code = {}
+
+ def __repr__(self) -> str:
+ return "depend_item_list"
+
+ @property
+ def project_code(self) -> str:
+ """Get dependent project code."""
+ return self.get_code_from_gateway().get("projectCode")
+
+ @property
+ def definition_code(self) -> str:
+ """Get dependent definition code."""
+ return self.get_code_from_gateway().get("processDefinitionCode")
+
+ @property
+ def dep_task_code(self) -> str:
+ """Get dependent tasks code list."""
+ if self.is_all_task:
+ return DEPENDENT_ALL_TASK_IN_WORKFLOW
+ else:
+ return self.get_code_from_gateway().get("taskDefinitionCode")
+
+ # TODO Maybe we should get cycle from dependent date class.
+ @property
+ def cycle(self) -> str:
+ """Get dependent cycle."""
+ if "Hour" in self.dependent_date:
+ return "hour"
+ elif self.dependent_date == "today" or "Days" in self.dependent_date:
+ return "day"
+ elif "Month" in self.dependent_date:
+ return "month"
+ else:
+ return "week"
+
+ @property
+ def date_value(self) -> str:
+ """Get dependent date."""
+ return self.dependent_date
+
+ @property
+ def is_all_task(self) -> bool:
+ """Check whether dependent all tasks or not."""
+ return self.dependent_task_name == DEPENDENT_ALL_TASK_IN_WORKFLOW
+
+ @property
+ def code_parameter(self) -> Tuple:
+ """Get name info parameter to query code."""
+ param = (
+ self.project_name,
+ self.process_definition_name,
+ self.dependent_task_name if not self.is_all_task else None,
+ )
+ return param
+
+ def get_code_from_gateway(self) -> Dict:
+ """Get project, definition, task code from given parameter."""
+ if self._code:
+ return self._code
+ else:
+ gateway = launch_gateway()
+ try:
+ return gateway.entry_point.getDependentInfo(*self.code_parameter)
Review comment:
The return value of `getDependentInfo ` was not assigned to `self.code`. @zhongjiajie
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie merged pull request #7405: [python] Add task dependent
Posted by GitBox <gi...@apache.org>.
zhongjiajie merged pull request #7405:
URL: https://github.com/apache/dolphinscheduler/pull/7405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org