You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by li...@apache.org on 2022/06/18 13:53:37 UTC
[dolphinscheduler] branch 3.0.0-beta-2-prepare updated (3259efc673 -> ac5179be73)
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a change to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
from 3259efc673 Add dolphinscheduler-task-all in api module (#10350)
new ab7a843201 issues-10354: upgrade commons-io to fix CVE (#10355)
new 67f6c003f7 issue-10356: upgrade logback to fix cve (#10357)
new 7a240fb807 Upgrade log4j in E2E (#10359)
new 478463fe6a [Docs] Upgrade license. (#10365)
new f7b18fd59d [fix][deploy]fix run on kubernetes bug (#10379)
new ed02db213e [Bug][Tools] Add datasource url for mysql profile in tools application.yaml (#10399)
new 248a85daab [Doc] Add how to set the alarm mailbox in faq (#10401)
new df0735273b [Fix-10412] [Standalone] Failed to start standalone server according to the official document (#10414)
new deb1c594e5 Update chart.yaml dependencies version (#10420)
new ba2f2ba64d [fix-10386]LDAP user create with status enable (#10426)
new 476f2395dc [fix][Python] Support same task name in project defferent process definition (#10428)
new 55e39eedfa [Fix-10425]Recovery LDAP Config (#10429)
new 4d608a7d8f [DolphinScheduler] Fix wrong class name in doc (#10438)
new 788e334329 [Fix][UI] Add loading before file modification data is returned. (#10450)
new 6edb3412a9 [fix][python] Correct variable type of SQL type (#10464)
new ac5179be73 fix rename form not init bug in file manange (#10471)
The 16 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
deploy/kubernetes/dolphinscheduler/Chart.yaml | 4 +-
.../dolphinscheduler/templates/_helpers.tpl | 2 +-
deploy/kubernetes/dolphinscheduler/values.yaml | 2 +
docs/docs/en/architecture/configuration.md | 8 +-
.../development/development-environment-setup.md | 2 +-
docs/docs/en/faq.md | 3 +-
docs/docs/en/guide/alert/email.md | 6 +
docs/docs/zh/architecture/configuration.md | 9 +-
.../development/development-environment-setup.md | 2 +-
docs/docs/zh/faq.md | 3 +-
docs/docs/zh/guide/alert/email.md | 6 +
docs/img/alert/email-alter-setup1-en.png | Bin 0 -> 27742 bytes
docs/img/alert/email-alter-setup2-en.png | Bin 0 -> 47734 bytes
docs/img/alert/email-alter-setup3-en.png | Bin 0 -> 43983 bytes
.../dolphinscheduler/api/python/PythonGateway.java | 14 ++-
.../api/security/impl/ldap/LdapService.java | 12 +-
.../api/service/TaskDefinitionService.java | 2 +
.../service/impl/TaskDefinitionServiceImpl.java | 5 +-
.../api/service/impl/UsersServiceImpl.java | 1 +
.../src/main/resources/application.yaml | 16 +++
.../api/python/PythonGatewayTest.java | 123 +++++++++++++++++++++
.../security/impl/ldap/LdapAuthenticatorTest.java | 26 ++---
.../api/security/impl/ldap/LdapServiceTest.java | 12 +-
.../api/service/TaskDefinitionServiceImplTest.java | 6 +-
.../dao/mapper/TaskDefinitionMapper.java | 2 +
.../dao/mapper/TaskDefinitionMapper.xml | 20 +++-
.../dao/mapper/TaskDefinitionMapperTest.java | 25 ++++-
dolphinscheduler-dist/release-docs/LICENSE | 42 +++----
dolphinscheduler-e2e/pom.xml | 2 +-
.../src/pydolphinscheduler/core/task.py | 2 +-
.../src/pydolphinscheduler/tasks/sql.py | 8 +-
.../pydolphinscheduler/tests/tasks/test_sql.py | 4 +-
.../src/main/bin/start.sh | 1 +
.../src/main/dist-bin/start.sh | 1 +
.../src/main/resources/application.yaml | 21 ++++
.../src/main/resources/application.yaml | 1 +
.../src/views/resource/file/edit/index.tsx | 83 +++++++-------
.../src/views/resource/file/edit/use-edit.ts | 3 +-
.../src/views/resource/file/rename/index.tsx | 2 +-
pom.xml | 6 +-
tools/dependencies/known-dependencies.txt | 8 +-
41 files changed, 370 insertions(+), 125 deletions(-)
create mode 100644 docs/docs/en/guide/alert/email.md
create mode 100644 docs/docs/zh/guide/alert/email.md
create mode 100644 docs/img/alert/email-alter-setup1-en.png
create mode 100644 docs/img/alert/email-alter-setup2-en.png
create mode 100644 docs/img/alert/email-alter-setup3-en.png
create mode 100644 dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/python/PythonGatewayTest.java
[dolphinscheduler] 15/16: [fix][python] Correct variable type of SQL type (#10464)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 6edb3412a974d8bdddca351c70c1bc8455958006
Author: 陈家名 <13...@163.com>
AuthorDate: Thu Jun 16 09:39:53 2022 +0800
[fix][python] Correct variable type of SQL type (#10464)
(cherry picked from commit f512e3c31f4850c7272e868b291193a58c661d57)
---
.../pydolphinscheduler/src/pydolphinscheduler/tasks/sql.py | 8 ++++----
.../pydolphinscheduler/tests/tasks/test_sql.py | 4 ++--
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/sql.py b/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/sql.py
index c15170c448..a1259824f1 100644
--- a/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/sql.py
+++ b/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/tasks/sql.py
@@ -31,8 +31,8 @@ log = logging.getLogger(__file__)
class SqlType:
"""SQL type, for now it just contain `SELECT` and `NO_SELECT`."""
- SELECT = 0
- NOT_SELECT = 1
+ SELECT = "0"
+ NOT_SELECT = "1"
class Sql(Task):
@@ -64,7 +64,7 @@ class Sql(Task):
name: str,
datasource_name: str,
sql: str,
- sql_type: Optional[int] = None,
+ sql_type: Optional[str] = None,
pre_statements: Optional[str] = None,
post_statements: Optional[str] = None,
display_rows: Optional[int] = 10,
@@ -80,7 +80,7 @@ class Sql(Task):
self.display_rows = display_rows
@property
- def sql_type(self) -> int:
+ def sql_type(self) -> str:
"""Judgement sql type, it will return the SQL type for type `SELECT` or `NOT_SELECT`.
If `param_sql_type` dot not specific, will use regexp to check
diff --git a/dolphinscheduler-python/pydolphinscheduler/tests/tasks/test_sql.py b/dolphinscheduler-python/pydolphinscheduler/tests/tasks/test_sql.py
index 74ac0f67ea..ee0acc442e 100644
--- a/dolphinscheduler-python/pydolphinscheduler/tests/tasks/test_sql.py
+++ b/dolphinscheduler-python/pydolphinscheduler/tests/tasks/test_sql.py
@@ -89,7 +89,7 @@ def test_get_sql_type(
"sql": "select 1",
"type": "MYSQL",
"datasource": 1,
- "sqlType": SqlType.SELECT,
+ "sqlType": "0",
"preStatements": [],
"postStatements": [],
"displayRows": 10,
@@ -138,7 +138,7 @@ def test_sql_get_define(mock_datasource):
"type": "MYSQL",
"datasource": 1,
"sql": command,
- "sqlType": SqlType.SELECT,
+ "sqlType": "0",
"displayRows": 10,
"preStatements": [],
"postStatements": [],
[dolphinscheduler] 11/16: [fix][Python] Support same task name in project defferent process definition (#10428)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 476f2395dc2841f2f148cb40de0dbfe10e22536a
Author: 陈家名 <13...@163.com>
AuthorDate: Tue Jun 14 13:53:18 2022 +0800
[fix][Python] Support same task name in project defferent process definition (#10428)
close: #10431
(cherry picked from commit b86dc53ad10cb5c4d76f5e85c38e5a5659a349fd)
---
.../dolphinscheduler/api/python/PythonGateway.java | 14 ++-
.../api/service/TaskDefinitionService.java | 2 +
.../service/impl/TaskDefinitionServiceImpl.java | 5 +-
.../api/python/PythonGatewayTest.java | 123 +++++++++++++++++++++
.../api/service/TaskDefinitionServiceImplTest.java | 6 +-
.../dao/mapper/TaskDefinitionMapper.java | 2 +
.../dao/mapper/TaskDefinitionMapper.xml | 20 +++-
.../dao/mapper/TaskDefinitionMapperTest.java | 25 ++++-
.../src/pydolphinscheduler/core/task.py | 2 +-
9 files changed, 185 insertions(+), 14 deletions(-)
diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/python/PythonGateway.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/python/PythonGateway.java
index 817f411854..54962b523d 100644
--- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/python/PythonGateway.java
+++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/python/PythonGateway.java
@@ -166,7 +166,7 @@ public class PythonGateway {
return taskDefinitionService.genTaskCodeList(genNum);
}
- public Map<String, Long> getCodeAndVersion(String projectName, String taskName) throws CodeGenerateUtils.CodeGenerateException {
+ public Map<String, Long> getCodeAndVersion(String projectName, String processDefinitionName, String taskName) throws CodeGenerateUtils.CodeGenerateException {
Project project = projectMapper.queryByName(projectName);
Map<String, Long> result = new HashMap<>();
// project do not exists, mean task not exists too, so we should directly return init value
@@ -175,7 +175,15 @@ public class PythonGateway {
result.put("version", 0L);
return result;
}
- TaskDefinition taskDefinition = taskDefinitionMapper.queryByName(project.getCode(), taskName);
+
+ ProcessDefinition processDefinition = processDefinitionMapper.queryByDefineName(project.getCode(), processDefinitionName);
+ if (processDefinition == null) {
+ String msg = String.format("Can not find valid process definition by name %s", processDefinitionName);
+ logger.error(msg);
+ throw new IllegalArgumentException(msg);
+ }
+
+ TaskDefinition taskDefinition = taskDefinitionMapper.queryByName(project.getCode(), processDefinition.getCode(), taskName);
if (taskDefinition == null) {
result.put("code", CodeGenerateUtils.getInstance().genCode());
result.put("version", 0L);
@@ -520,7 +528,7 @@ public class PythonGateway {
result.put("processDefinitionCode", processDefinition.getCode());
if (taskName != null) {
- TaskDefinition taskDefinition = taskDefinitionMapper.queryByName(projectCode, taskName);
+ TaskDefinition taskDefinition = taskDefinitionMapper.queryByName(projectCode, processDefinition.getCode(), taskName);
result.put("taskDefinitionCode", taskDefinition.getCode());
}
return result;
diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskDefinitionService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskDefinitionService.java
index 5d2ba85b02..a715b68e97 100644
--- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskDefinitionService.java
+++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/TaskDefinitionService.java
@@ -60,10 +60,12 @@ public interface TaskDefinitionService {
*
* @param loginUser login user
* @param projectCode project code
+ * @param processCode process code
* @param taskName task name
*/
Map<String, Object> queryTaskDefinitionByName(User loginUser,
long projectCode,
+ long processCode,
String taskName);
/**
diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
index cd617958e0..15105acfe8 100644
--- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
+++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/TaskDefinitionServiceImpl.java
@@ -270,10 +270,11 @@ public class TaskDefinitionServiceImpl extends BaseServiceImpl implements TaskDe
*
* @param loginUser login user
* @param projectCode project code
+ * @param processCode process code
* @param taskName task name
*/
@Override
- public Map<String, Object> queryTaskDefinitionByName(User loginUser, long projectCode, String taskName) {
+ public Map<String, Object> queryTaskDefinitionByName(User loginUser, long projectCode, long processCode, String taskName) {
Project project = projectMapper.queryByCode(projectCode);
//check user access for project
Map<String, Object> result = projectService.checkProjectAndAuth(loginUser, project, projectCode);
@@ -281,7 +282,7 @@ public class TaskDefinitionServiceImpl extends BaseServiceImpl implements TaskDe
return result;
}
- TaskDefinition taskDefinition = taskDefinitionMapper.queryByName(project.getCode(), taskName);
+ TaskDefinition taskDefinition = taskDefinitionMapper.queryByName(project.getCode(), processCode, taskName);
if (taskDefinition == null) {
putMsg(result, Status.TASK_DEFINE_NOT_EXIST, taskName);
} else {
diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/python/PythonGatewayTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/python/PythonGatewayTest.java
new file mode 100644
index 0000000000..7d8b6efabc
--- /dev/null
+++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/python/PythonGatewayTest.java
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.python;
+
+import org.apache.dolphinscheduler.common.utils.CodeGenerateUtils;
+import org.apache.dolphinscheduler.dao.entity.ProcessDefinition;
+import org.apache.dolphinscheduler.dao.entity.Project;
+import org.apache.dolphinscheduler.dao.entity.TaskDefinition;
+import org.apache.dolphinscheduler.dao.mapper.ProcessDefinitionMapper;
+import org.apache.dolphinscheduler.dao.mapper.ProjectMapper;
+import org.apache.dolphinscheduler.dao.mapper.TaskDefinitionMapper;
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.junit.MockitoJUnitRunner;
+
+import java.util.Date;
+import java.util.Map;
+
+/**
+ * python gate test
+ */
+@RunWith(MockitoJUnitRunner.class)
+public class PythonGatewayTest {
+
+ @InjectMocks
+ private PythonGateway pythonGateway;
+
+ @Mock
+ private ProjectMapper projectMapper;
+
+ @Mock
+ private ProcessDefinitionMapper processDefinitionMapper;
+
+ @Mock
+ private TaskDefinitionMapper taskDefinitionMapper;
+
+ @Test
+ public void testGetCodeAndVersion() throws CodeGenerateUtils.CodeGenerateException {
+ Project project = getTestProject();
+ Mockito.when(projectMapper.queryByName(project.getName())).thenReturn(project);
+
+ ProcessDefinition processDefinition = getTestProcessDefinition();
+ Mockito.when(processDefinitionMapper.queryByDefineName(project.getCode(), processDefinition.getName())).thenReturn(processDefinition);
+
+ TaskDefinition taskDefinition = getTestTaskDefinition();
+ Mockito.when(taskDefinitionMapper.queryByName(project.getCode(), processDefinition.getCode(), taskDefinition.getName())).thenReturn(taskDefinition);
+
+ Map<String, Long> result = pythonGateway.getCodeAndVersion(project.getName(), processDefinition.getName(), taskDefinition.getName());
+ Assert.assertEquals(result.get("code").longValue(), taskDefinition.getCode());
+ }
+
+ @Test
+ public void testGetDependentInfo() {
+ Project project = getTestProject();
+ Mockito.when(projectMapper.queryByName(project.getName())).thenReturn(project);
+
+ ProcessDefinition processDefinition = getTestProcessDefinition();
+ Mockito.when(processDefinitionMapper.queryByDefineName(project.getCode(), processDefinition.getName())).thenReturn(processDefinition);
+
+ TaskDefinition taskDefinition = getTestTaskDefinition();
+ Mockito.when(taskDefinitionMapper.queryByName(project.getCode(), processDefinition.getCode(), taskDefinition.getName())).thenReturn(taskDefinition);
+
+ Map<String, Object> result = pythonGateway.getDependentInfo(project.getName(), processDefinition.getName(), taskDefinition.getName());
+ Assert.assertEquals((long) result.get("taskDefinitionCode"), taskDefinition.getCode());
+ }
+
+ private Project getTestProject() {
+ Project project = new Project();
+ project.setName("ut-project");
+ project.setUserId(111);
+ project.setCode(1L);
+ project.setCreateTime(new Date());
+ project.setUpdateTime(new Date());
+ return project;
+ }
+
+ private ProcessDefinition getTestProcessDefinition() {
+ ProcessDefinition processDefinition = new ProcessDefinition();
+ processDefinition.setCode(1L);
+ processDefinition.setName("ut-process-definition");
+ processDefinition.setProjectCode(1L);
+ processDefinition.setUserId(111);
+ processDefinition.setUpdateTime(new Date());
+ processDefinition.setCreateTime(new Date());
+ return processDefinition;
+ }
+
+ private TaskDefinition getTestTaskDefinition() {
+ TaskDefinition taskDefinition = new TaskDefinition();
+ taskDefinition.setCode(888888L);
+ taskDefinition.setName("ut-task-definition");
+ taskDefinition.setProjectCode(1L);
+ taskDefinition.setTaskType("SHELL");
+ taskDefinition.setUserId(111);
+ taskDefinition.setResourceIds("1");
+ taskDefinition.setWorkerGroup("default");
+ taskDefinition.setEnvironmentCode(1L);
+ taskDefinition.setVersion(1);
+ taskDefinition.setCreateTime(new Date());
+ taskDefinition.setUpdateTime(new Date());
+ return taskDefinition;
+ }
+
+}
diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/TaskDefinitionServiceImplTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/TaskDefinitionServiceImplTest.java
index 9aade5555c..676688a3ec 100644
--- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/TaskDefinitionServiceImplTest.java
+++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/service/TaskDefinitionServiceImplTest.java
@@ -145,7 +145,7 @@ public class TaskDefinitionServiceImplTest {
public void queryTaskDefinitionByName() {
String taskName = "task";
long projectCode = 1L;
-
+ long processCode = 1L;
Project project = getProject(projectCode);
Mockito.when(projectMapper.queryByCode(projectCode)).thenReturn(project);
@@ -157,11 +157,11 @@ public class TaskDefinitionServiceImplTest {
putMsg(result, Status.SUCCESS, projectCode);
Mockito.when(projectService.checkProjectAndAuth(loginUser, project, projectCode)).thenReturn(result);
- Mockito.when(taskDefinitionMapper.queryByName(project.getCode(), taskName))
+ Mockito.when(taskDefinitionMapper.queryByName(project.getCode(), processCode, taskName))
.thenReturn(new TaskDefinition());
Map<String, Object> relation = taskDefinitionService
- .queryTaskDefinitionByName(loginUser, projectCode, taskName);
+ .queryTaskDefinitionByName(loginUser, projectCode, processCode, taskName);
Assert.assertEquals(Status.SUCCESS, relation.get(Constants.STATUS));
}
diff --git a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.java b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.java
index 304e623a0a..c426da6b04 100644
--- a/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.java
+++ b/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.java
@@ -41,10 +41,12 @@ public interface TaskDefinitionMapper extends BaseMapper<TaskDefinition> {
* query task definition by name
*
* @param projectCode projectCode
+ * @param processCode processCode
* @param name name
* @return task definition
*/
TaskDefinition queryByName(@Param("projectCode") long projectCode,
+ @Param("processCode") long processCode,
@Param("name") String name);
/**
diff --git a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml
index 5c889d1b07..255be38b2f 100644
--- a/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml
+++ b/dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.xml
@@ -23,12 +23,24 @@
worker_group, environment_code, fail_retry_times, fail_retry_interval, timeout_flag, timeout_notify_strategy, timeout, delay_time,
resource_ids, create_time, update_time, task_group_id,task_group_priority
</sql>
+ <sql id="baseSqlV2">
+ ${alias}.id, ${alias}.code, ${alias}.name, ${alias}.version, ${alias}.description, ${alias}.project_code, ${alias}.user_id,
+ ${alias}.task_type, ${alias}.task_params, ${alias}.flag, ${alias}.task_priority, ${alias}.worker_group, ${alias}.environment_code,
+ ${alias}.fail_retry_times, ${alias}.fail_retry_interval, ${alias}.timeout_flag, ${alias}.timeout_notify_strategy, ${alias}.timeout,
+ ${alias}.delay_time, ${alias}.resource_ids, ${alias}.create_time, ${alias}.update_time, ${alias}.task_group_id,
+ ${alias}.task_group_priority, ${alias}.cpu_quota, ${alias}.memory_max
+ </sql>
<select id="queryByName" resultType="org.apache.dolphinscheduler.dao.entity.TaskDefinition">
select
- <include refid="baseSql"/>
- from t_ds_task_definition
- WHERE project_code = #{projectCode}
- and name = #{name}
+ <include refid="baseSqlV2">
+ <property name="alias" value="td"/>
+ </include>
+ from t_ds_task_definition td
+ join t_ds_process_task_relation ptr on ptr.project_code = td.project_code
+ where td.project_code = #{projectCode}
+ and td.name = #{name}
+ and ptr.process_definition_code = #{processCode}
+ and td.code = ptr.post_task_code
</select>
<select id="queryAllDefinitionList" resultType="org.apache.dolphinscheduler.dao.entity.TaskDefinition">
select
diff --git a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapperTest.java b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapperTest.java
index 757f751dd2..3d16e2e44d 100644
--- a/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapperTest.java
+++ b/dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapperTest.java
@@ -20,6 +20,7 @@ package org.apache.dolphinscheduler.dao.mapper;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.dao.BaseDaoTest;
import org.apache.dolphinscheduler.dao.entity.DefinitionGroupByUser;
+import org.apache.dolphinscheduler.dao.entity.ProcessTaskRelation;
import org.apache.dolphinscheduler.dao.entity.TaskDefinition;
import org.apache.dolphinscheduler.dao.entity.User;
@@ -36,6 +37,9 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
@Autowired
private TaskDefinitionMapper taskDefinitionMapper;
+ @Autowired
+ private ProcessTaskRelationMapper processTaskRelationMapper;
+
@Autowired
private UserMapper userMapper;
@@ -60,6 +64,24 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
return taskDefinition;
}
+ /**
+ * insert
+ *
+ * @return ProcessDefinition
+ */
+ private ProcessTaskRelation insertTaskRelation(long postTaskCode) {
+ ProcessTaskRelation processTaskRelation = new ProcessTaskRelation();
+ processTaskRelation.setName("def 1");
+ processTaskRelation.setProjectCode(1L);
+ processTaskRelation.setProcessDefinitionCode(1L);
+ processTaskRelation.setPostTaskCode(postTaskCode);
+ processTaskRelation.setPreTaskCode(0L);
+ processTaskRelation.setUpdateTime(new Date());
+ processTaskRelation.setCreateTime(new Date());
+ processTaskRelationMapper.insert(processTaskRelation);
+ return processTaskRelation;
+ }
+
@Test
public void testInsert() {
TaskDefinition taskDefinition = insertOne();
@@ -69,7 +91,8 @@ public class TaskDefinitionMapperTest extends BaseDaoTest {
@Test
public void testQueryByDefinitionName() {
TaskDefinition taskDefinition = insertOne();
- TaskDefinition result = taskDefinitionMapper.queryByName(taskDefinition.getProjectCode()
+ ProcessTaskRelation processTaskRelation = insertTaskRelation(taskDefinition.getCode());
+ TaskDefinition result = taskDefinitionMapper.queryByName(taskDefinition.getProjectCode(), processTaskRelation.getProcessDefinitionCode()
, taskDefinition.getName());
Assert.assertNotNull(result);
diff --git a/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/core/task.py b/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/core/task.py
index 599b979369..08a1cdf7e8 100644
--- a/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/core/task.py
+++ b/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/core/task.py
@@ -273,7 +273,7 @@ class Task(Base):
# TODO get code from specific project process definition and task name
gateway = launch_gateway()
result = gateway.entry_point.getCodeAndVersion(
- self.process_definition._project, self.name
+ self.process_definition._project, self.process_definition.name, self.name
)
# result = gateway.entry_point.genTaskCodeList(DefaultTaskCodeNum.DEFAULT)
# gateway_result_checker(result)
[dolphinscheduler] 04/16: [Docs] Upgrade license. (#10365)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 478463fe6af5986ffe4ffc6dc2e3cdf28304c16e
Author: songjianet <17...@qq.com>
AuthorDate: Mon Jun 6 11:16:06 2022 +0800
[Docs] Upgrade license. (#10365)
(cherry picked from commit 36bba3fcdff58176a27b71155db6539244496106)
---
dolphinscheduler-dist/release-docs/LICENSE | 34 +++++++++++++++---------------
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/dolphinscheduler-dist/release-docs/LICENSE b/dolphinscheduler-dist/release-docs/LICENSE
index ff489f4c7e..fe4d1253a6 100644
--- a/dolphinscheduler-dist/release-docs/LICENSE
+++ b/dolphinscheduler-dist/release-docs/LICENSE
@@ -565,40 +565,40 @@ The text of each license is also included at licenses/ui-licenses/LICENSE-[proje
========================================
MIT licenses
========================================
- @antv/layout 0.2.2: https://github.com/antvis/layout MIT
+ @antv/layout 0.2.3: https://github.com/antvis/layout MIT
@types/js-cookie 3.0.2: https://github.com/DefinitelyTyped/DefinitelyTyped MIT
- @antv/x6 1.31.5: https://github.com/antvis/x6 MIT
- @vueuse/core 8.4.2: https://github.com/vueuse/vueuse MIT
+ @antv/x6 1.32.3: https://github.com/antvis/x6 MIT
+ @vueuse/core 8.6.0: https://github.com/vueuse/vueuse MIT
axios 0.27.2: https://github.com/axios/axios MIT
date-fns 2.28.0: https://github.com/date-fns/date-fns MIT
lodash 4.17.21: https://github.com/lodash/lodash MIT
monaco-editor 0.33.0: https://github.com/microsoft/monaco-editor MIT
- naive-ui 2.28.2: https://github.com/TuSimple/naive-ui MIT
+ naive-ui 2.29.0: https://github.com/TuSimple/naive-ui MIT
nprogress 0.2.0: https://github.com/rstacruz/nprogress MIT
pinia 2.0.14: https://github.com/vuejs/pinia MIT
- pinia-plugin-persistedstate 1.5.1: https://github.com/prazdevs/pinia-plugin-persistedstate MIT
+ pinia-plugin-persistedstate 1.6.1: https://github.com/prazdevs/pinia-plugin-persistedstate MIT
vfonts 0.0.3: https://www.npmjs.com/package/vfonts MIT
- vue 3.2.33: https://github.com/vuejs/core MIT
+ vue 3.2.36: https://github.com/vuejs/core MIT
vue-i18n 9.1.10: https://github.com/kazupon/vue-i18n MIT
vue-router 4.0.15: https://github.com/vuejs/router MIT
- @types/node 17.0.31: https://github.com/DefinitelyTyped/DefinitelyTyped MIT
+ @types/node 17.0.38: https://github.com/DefinitelyTyped/DefinitelyTyped MIT
@types/nprogress 0.2.0: https://github.com/DefinitelyTyped/DefinitelyTyped MIT
@types/qs 6.9.7: https://github.com/DefinitelyTyped/DefinitelyTyped MIT
@vicons/antd 0.12.0: https://www.npmjs.com/package/@vicons/antd MIT
- @vitejs/plugin-vue 2.3.2: https://github.com/vitejs/vite MIT
+ @vitejs/plugin-vue 2.3.3: https://github.com/vitejs/vite MIT
@vitejs/plugin-vue-jsx 1.3.10: https://github.com/vitejs/vite MIT
dart-sass 1.25.0: https://github.com/sass/dart-sass MIT
- eslint 8.15.0: https://github.com/eslint/eslint MIT
+ eslint 8.16.0: https://github.com/eslint/eslint MIT
eslint-config-prettier 8.5.0: https://github.com/prettier/eslint-config-prettier MIT
eslint-plugin-prettier 4.0.0: https://github.com/prettier/eslint-plugin-prettier MIT
- eslint-plugin-vue 8.7.1: https://github.com/vuejs/eslint-plugin-vue MIT
+ eslint-plugin-vue 9.1.0: https://github.com/vuejs/eslint-plugin-vue MIT
prettier 2.6.2: https://github.com/prettier/prettier MIT
- sass 1.51.0: https://github.com/sass/dart-sass MIT
- sass-loader 12.6.0: https://github.com/webpack-contrib/sass-loader MIT
+ sass 1.52.1: https://github.com/sass/dart-sass MIT
+ sass-loader 13.0.0: https://github.com/webpack-contrib/sass-loader MIT
typescript-plugin-css-modules 3.4.0: https://github.com/mrmckeb/typescript-plugin-css-modules MIT
- vite 2.9.8: https://github.com/vitejs/vite MIT
+ vite 2.9.9: https://github.com/vitejs/vite MIT
vite-plugin-compression 0.5.1: https://github.com/vbenjs/vite-plugin-compression MIT
- vue-tsc 0.34.11: https://github.com/johnsoncodehk/volar MIT
+ vue-tsc 0.35.2: https://github.com/johnsoncodehk/volar MIT
date-fns-tz 1.3.4: https://github.com/marnusw/date-fns-tz MIT
screenfull 6.0.1: https://github.com/sindresorhus/screenfull MIT
@types/lodash 4.14.182: https://github.com/DefinitelyTyped/DefinitelyTyped MIT
@@ -607,7 +607,7 @@ MIT licenses
Apache 2.0 licenses
========================================
echarts 5.3.2: https://github.com/apache/echarts Apache-2.0
- typescript 4.6.4: https://github.com/Microsoft/TypeScript Apache-2.0
+ typescript 4.7.2: https://github.com/Microsoft/TypeScript Apache-2.0
open-sans: Apache-2.0
roboto: Apache-2.0
@@ -618,8 +618,8 @@ BSD licenses
The following components are provided under a BSD license. See project link for details.
The text of each license is also included at licenses/ui-licenses/LICENSE-[project].txt.
qs 6.10.3: https://github.com/ljharb/qs BSD 3-Clause
- @typescript-eslint/eslint-plugin 5.22.0: https://github.com/typescript-eslint/typescript-eslint BSD 2-clause
- @typescript-eslint/parser 5.22.0: https://github.com/typescript-eslint/typescript-eslint BSD 2-clause
+ @typescript-eslint/eslint-plugin 5.27.0: https://github.com/typescript-eslint/typescript-eslint BSD 2-clause
+ @typescript-eslint/parser 5.27.0: https://github.com/typescript-eslint/typescript-eslint BSD 2-clause
========================================
SIL licenses
[dolphinscheduler] 02/16: issue-10356: upgrade logback to fix cve (#10357)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 67f6c003f724632619ff0c98341b111ca80eca1a
Author: PJ Fanning <pj...@users.noreply.github.com>
AuthorDate: Fri Jun 3 12:21:40 2022 +0100
issue-10356: upgrade logback to fix cve (#10357)
(cherry picked from commit d044e0479deb88c694973d0e0c51d8b7cbcfac06)
---
dolphinscheduler-dist/release-docs/LICENSE | 4 ++--
pom.xml | 2 +-
tools/dependencies/known-dependencies.txt | 4 ++--
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/dolphinscheduler-dist/release-docs/LICENSE b/dolphinscheduler-dist/release-docs/LICENSE
index 0dd740217a..ff489f4c7e 100644
--- a/dolphinscheduler-dist/release-docs/LICENSE
+++ b/dolphinscheduler-dist/release-docs/LICENSE
@@ -506,8 +506,8 @@ EPL licenses
The following components are provided under the EPL License. See project link for details.
The text of each license is also included at licenses/LICENSE-[project].txt.
aspectjweaver 1.9.7:https://mvnrepository.com/artifact/org.aspectj/aspectjweaver/1.9.7, EPL 1.0
- logback-classic 1.2.3: https://mvnrepository.com/artifact/ch.qos.logback/logback-classic/1.2.3, EPL 1.0 and LGPL 2.1
- logback-core 1.2.3: https://mvnrepository.com/artifact/ch.qos.logback/logback-core/1.2.3, EPL 1.0 and LGPL 2.1
+ logback-classic 1.2.11: https://mvnrepository.com/artifact/ch.qos.logback/logback-classic/1.2.11, EPL 1.0 and LGPL 2.1
+ logback-core 1.2.11: https://mvnrepository.com/artifact/ch.qos.logback/logback-core/1.2.11, EPL 1.0 and LGPL 2.1
h2-1.4.200 https://github.com/h2database/h2database/blob/master/LICENSE.txt, MPL 2.0 or EPL 1.0
========================================================================
diff --git a/pom.xml b/pom.xml
index ada4533989..c0ef327c74 100644
--- a/pom.xml
+++ b/pom.xml
@@ -55,7 +55,7 @@
<spring.version>5.3.12</spring.version>
<spring.boot.version>2.5.6</spring.boot.version>
<java.version>1.8</java.version>
- <logback.version>1.2.3</logback.version>
+ <logback.version>1.2.11</logback.version>
<hadoop.version>2.7.3</hadoop.version>
<quartz.version>2.3.2</quartz.version>
<jackson.version>2.10.5</jackson.version>
diff --git a/tools/dependencies/known-dependencies.txt b/tools/dependencies/known-dependencies.txt
index b89bc00d84..8262bc32c8 100755
--- a/tools/dependencies/known-dependencies.txt
+++ b/tools/dependencies/known-dependencies.txt
@@ -151,8 +151,8 @@ libfb303-0.9.3.jar
libthrift-0.9.3.jar
log4j-1.2-api-2.14.1.jar
log4j-1.2.17.jar
-logback-classic-1.2.3.jar
-logback-core-1.2.3.jar
+logback-classic-1.2.11.jar
+logback-core-1.2.11.jar
lz4-1.3.0.jar
mapstruct-1.2.0.Final.jar
micrometer-core-1.7.5.jar
[dolphinscheduler] 01/16: issues-10354: upgrade commons-io to fix CVE (#10355)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit ab7a84320181102e276a6725fd84e44af9c2416a
Author: PJ Fanning <pj...@users.noreply.github.com>
AuthorDate: Fri Jun 3 10:17:52 2022 +0100
issues-10354: upgrade commons-io to fix CVE (#10355)
(cherry picked from commit b6350280e66f604968e249919dc4a13a04eecee4)
---
dolphinscheduler-dist/release-docs/LICENSE | 4 ++--
pom.xml | 4 ++--
tools/dependencies/known-dependencies.txt | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/dolphinscheduler-dist/release-docs/LICENSE b/dolphinscheduler-dist/release-docs/LICENSE
index 0ef564565e..0dd740217a 100644
--- a/dolphinscheduler-dist/release-docs/LICENSE
+++ b/dolphinscheduler-dist/release-docs/LICENSE
@@ -238,7 +238,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
commons-dbcp 1.4: https://github.com/apache/commons-dbcp, Apache 2.0
commons-email 1.5: https://github.com/apache/commons-email, Apache 2.0
commons-httpclient 3.0.1: https://mvnrepository.com/artifact/commons-httpclient/commons-httpclient/3.0.1, Apache 2.0
- commons-io 2.4: https://github.com/apache/commons-io, Apache 2.0
+ commons-io 2.11.0: https://github.com/apache/commons-io, Apache 2.0
commons-lang 2.6: https://github.com/apache/commons-lang, Apache 2.0
commons-logging 1.1.1: https://github.com/apache/commons-logging, Apache 2.0
commons-math3 3.1.1: https://mvnrepository.com/artifact/org.apache.commons/commons-math3/3.1.1, Apache 2.0
@@ -286,7 +286,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
hive-storage-api 2.1.0: https://mvnrepository.com/artifact/org.apache.hive/hive-storage-api/2.1.0, Apache 2.0
htrace-core 3.1.0-incubating: https://mvnrepository.com/artifact/org.apache.htrace/htrace-core/3.1.0-incubating, Apache 2.0
httpclient 4.5.13: https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient/4.5.13, Apache 2.0
- httpcore 4.4.1: https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore/4.4.1, Apache 2.0
+ httpcore 4.4.15: https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore/4.4.15, Apache 2.0
httpmime 4.5.13: https://mvnrepository.com/artifact/org.apache.httpcomponents/httpmime/4.5.13, Apache 2.0
jackson-annotations 2.10.5: https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations/2.10.5, Apache 2.0
jackson-core 2.10.5: https://github.com/FasterXML/jackson-core, Apache 2.0
diff --git a/pom.xml b/pom.xml
index d370d57fa4..ada4533989 100644
--- a/pom.xml
+++ b/pom.xml
@@ -67,7 +67,7 @@
<commons.codec.version>1.11</commons.codec.version>
<commons.logging.version>1.1.1</commons.logging.version>
<httpclient.version>4.5.13</httpclient.version>
- <httpcore.version>4.4.1</httpcore.version>
+ <httpcore.version>4.4.15</httpcore.version>
<junit.version>4.12</junit.version>
<mysql.connector.version>8.0.16</mysql.connector.version>
<slf4j.api.version>1.7.5</slf4j.api.version>
@@ -84,7 +84,7 @@
<guava.version>24.1-jre</guava.version>
<postgresql.version>42.3.4</postgresql.version>
<hive.jdbc.version>2.1.0</hive.jdbc.version>
- <commons.io.version>2.4</commons.io.version>
+ <commons.io.version>2.11.0</commons.io.version>
<oshi.core.version>6.1.1</oshi.core.version>
<clickhouse.jdbc.version>0.1.52</clickhouse.jdbc.version>
<mssql.jdbc.version>6.1.0.jre8</mssql.jdbc.version>
diff --git a/tools/dependencies/known-dependencies.txt b/tools/dependencies/known-dependencies.txt
index 65a370772a..b89bc00d84 100755
--- a/tools/dependencies/known-dependencies.txt
+++ b/tools/dependencies/known-dependencies.txt
@@ -33,7 +33,7 @@ commons-daemon-1.0.13.jar
commons-beanutils-1.9.4.jar
commons-dbcp-1.4.jar
commons-httpclient-3.0.1.jar
-commons-io-2.4.jar
+commons-io-2.11.0.jar
commons-lang-2.6.jar
commons-logging-1.1.1.jar
commons-math3-3.1.1.jar
@@ -85,7 +85,7 @@ hive-service-rpc-2.1.0.jar
hive-storage-api-2.1.0.jar
htrace-core-3.1.0-incubating.jar
httpclient-4.5.13.jar
-httpcore-4.4.1.jar
+httpcore-4.4.15.jar
httpmime-4.5.13.jar
j2objc-annotations-1.1.jar
jackson-annotations-2.10.5.jar
[dolphinscheduler] 16/16: fix rename form not init bug in file manange (#10471)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit ac5179be73feeb75908d9751a294a1753d6f5856
Author: Devosend <de...@gmail.com>
AuthorDate: Thu Jun 16 13:51:17 2022 +0800
fix rename form not init bug in file manange (#10471)
(cherry picked from commit d1cae6025b7a8c6427120ecfeb8a2c8b1e4bbb72)
---
dolphinscheduler-ui/src/views/resource/file/rename/index.tsx | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dolphinscheduler-ui/src/views/resource/file/rename/index.tsx b/dolphinscheduler-ui/src/views/resource/file/rename/index.tsx
index 34578835f5..d57302c362 100644
--- a/dolphinscheduler-ui/src/views/resource/file/rename/index.tsx
+++ b/dolphinscheduler-ui/src/views/resource/file/rename/index.tsx
@@ -57,7 +57,7 @@ export default defineComponent({
}
watch(
- () => props.name,
+ () => props.show,
() => {
state.renameForm.id = props.id
state.renameForm.name = props.name
[dolphinscheduler] 08/16: [Fix-10412] [Standalone] Failed to start standalone server according to the official document (#10414)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit df0735273b58c4040580682a8ad619025a51bf78
Author: xiangzihao <46...@qq.com>
AuthorDate: Sun Jun 12 16:16:57 2022 +0800
[Fix-10412] [Standalone] Failed to start standalone server according to the official document (#10414)
* fix bug_10412
* fix bug_10412
(cherry picked from commit f4dad689d493dcd0d64026c6df3350ca0f9ba805)
---
dolphinscheduler-standalone-server/src/main/bin/start.sh | 1 +
dolphinscheduler-standalone-server/src/main/dist-bin/start.sh | 1 +
.../src/main/resources/application.yaml | 5 +++++
3 files changed, 7 insertions(+)
diff --git a/dolphinscheduler-standalone-server/src/main/bin/start.sh b/dolphinscheduler-standalone-server/src/main/bin/start.sh
index 63079e948b..c76887a265 100755
--- a/dolphinscheduler-standalone-server/src/main/bin/start.sh
+++ b/dolphinscheduler-standalone-server/src/main/bin/start.sh
@@ -19,6 +19,7 @@
BIN_DIR=$(dirname $0)
DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}
+export DATABASE=${DATABASE:-h2}
source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh"
JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}
diff --git a/dolphinscheduler-standalone-server/src/main/dist-bin/start.sh b/dolphinscheduler-standalone-server/src/main/dist-bin/start.sh
index 584e5f6f08..693dbe4faa 100755
--- a/dolphinscheduler-standalone-server/src/main/dist-bin/start.sh
+++ b/dolphinscheduler-standalone-server/src/main/dist-bin/start.sh
@@ -19,6 +19,7 @@
BIN_DIR=$(dirname $0)
DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/..; pwd)}
+export DATABASE=${DATABASE:-h2}
source "$DOLPHINSCHEDULER_HOME/conf/dolphinscheduler_env.sh"
JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}
diff --git a/dolphinscheduler-standalone-server/src/main/resources/application.yaml b/dolphinscheduler-standalone-server/src/main/resources/application.yaml
index 980a088267..3ea980e67b 100644
--- a/dolphinscheduler-standalone-server/src/main/resources/application.yaml
+++ b/dolphinscheduler-standalone-server/src/main/resources/application.yaml
@@ -195,6 +195,11 @@ spring:
quartz:
properties:
org.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
+ datasource:
+ driver-class-name: org.postgresql.Driver
+ url: jdbc:postgresql://127.0.0.1:5432/dolphinscheduler
+ username: root
+ password: root
---
spring:
[dolphinscheduler] 09/16: Update chart.yaml dependencies version (#10420)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit deb1c594e54c61f2a80f36688750819ef1e7a328
Author: Mr.An <42...@users.noreply.github.com>
AuthorDate: Mon Jun 13 10:27:34 2022 +0800
Update chart.yaml dependencies version (#10420)
(cherry picked from commit a7e20798cbef28ca9d5d2a1a5fc4e5436a8a4c6d)
---
deploy/kubernetes/dolphinscheduler/Chart.yaml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/deploy/kubernetes/dolphinscheduler/Chart.yaml b/deploy/kubernetes/dolphinscheduler/Chart.yaml
index d3f2b9644c..cf93992d6d 100644
--- a/deploy/kubernetes/dolphinscheduler/Chart.yaml
+++ b/deploy/kubernetes/dolphinscheduler/Chart.yaml
@@ -43,10 +43,10 @@ appVersion: 3.0.0-beta-1
dependencies:
- name: postgresql
- version: 10.3.18
+ version: 11.6.6
repository: https://charts.bitnami.com/bitnami
condition: postgresql.enabled
- name: zookeeper
- version: 6.5.3
+ version: 9.2.5
repository: https://charts.bitnami.com/bitnami
condition: zookeeper.enabled
[dolphinscheduler] 12/16: [Fix-10425]Recovery LDAP Config (#10429)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 55e39eedfa5339177f125ee074e778e114e380dd
Author: 旺阳 <qi...@cisco.com>
AuthorDate: Wed Jun 15 09:52:37 2022 +0800
[Fix-10425]Recovery LDAP Config (#10429)
* recovery ladp code
* add ldap config in doc
(cherry picked from commit 0efcd5c6317c9e12c172103a4ed6df0cdcda027d)
---
docs/docs/en/architecture/configuration.md | 8 +++++++-
docs/docs/zh/architecture/configuration.md | 9 ++++++++-
.../api/security/impl/ldap/LdapService.java | 12 ++++++------
dolphinscheduler-api/src/main/resources/application.yaml | 16 ++++++++++++++++
.../api/security/impl/ldap/LdapAuthenticatorTest.java | 12 ++++++------
.../api/security/impl/ldap/LdapServiceTest.java | 12 ++++++------
.../src/main/resources/application.yaml | 16 ++++++++++++++++
7 files changed, 65 insertions(+), 20 deletions(-)
diff --git a/docs/docs/en/architecture/configuration.md b/docs/docs/en/architecture/configuration.md
index 0f1df00372..85a45d0ab4 100644
--- a/docs/docs/en/architecture/configuration.md
+++ b/docs/docs/en/architecture/configuration.md
@@ -218,7 +218,13 @@ spring.messages.encoding|UTF-8| message encoding
spring.jackson.time-zone|GMT+8| time zone
spring.messages.basename|i18n/messages| i18n config
security.authentication.type|PASSWORD| authentication type
-
+security.authentication.ldap.user.admin|read-only-admin|admin user account when you log-in with LDAP
+security.authentication.ldap.urls|ldap://ldap.forumsys.com:389/|LDAP urls
+security.authentication.ldap.base.dn|dc=example,dc=com|LDAP base dn
+security.authentication.ldap.username|cn=read-only-admin,dc=example,dc=com|LDAP username
+security.authentication.ldap.password|password|LDAP password
+security.authentication.ldap.user.identity.attribute|uid|LDAP user identity attribute
+security.authentication.ldap.user.email.attribute|mail|LDAP user email attribute
### master.properties [master-service log config]
diff --git a/docs/docs/zh/architecture/configuration.md b/docs/docs/zh/architecture/configuration.md
index 14f79f1344..7bcf697f34 100644
--- a/docs/docs/zh/architecture/configuration.md
+++ b/docs/docs/zh/architecture/configuration.md
@@ -209,6 +209,13 @@ spring.messages.encoding|UTF-8|请求编码
spring.jackson.time-zone|GMT+8|设置时区
spring.messages.basename|i18n/messages|i18n配置
security.authentication.type|PASSWORD|权限校验类型
+security.authentication.ldap.user.admin|read-only-admin|LDAP登陆时，系统管理员账号
+security.authentication.ldap.urls|ldap://ldap.forumsys.com:389/|LDAP urls
+security.authentication.ldap.base.dn|dc=example,dc=com|LDAP base dn
+security.authentication.ldap.username|cn=read-only-admin,dc=example,dc=com|LDAP账号
+security.authentication.ldap.password|password|LDAP密码
+security.authentication.ldap.user.identity.attribute|uid|LDAP用户身份标识字段名
+security.authentication.ldap.user.email.attribute|mail|LDAP邮箱字段名
## 6.master.properties [Master服务配置]
@@ -380,7 +387,7 @@ singleYarnIp="yarnIp1"
resourceUploadPath="/dolphinscheduler"
-# HDFS/S3 操作用户
+# HDFS/S3 操作用户
hdfsRootUser="hdfs"
# 以下为 kerberos 配置
diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapService.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapService.java
index 9f4fd1f0ae..35abb11479 100644
--- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapService.java
+++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapService.java
@@ -45,22 +45,22 @@ public class LdapService {
@Value("${security.authentication.ldap.user.admin:null}")
private String adminUserId;
- @Value("${ldap.urls:null}")
+ @Value("${security.authentication.ldap.urls:null}")
private String ldapUrls;
- @Value("${ldap.base.dn:null}")
+ @Value("${security.authentication.ldap.base.dn:null}")
private String ldapBaseDn;
- @Value("${ldap.username:null}")
+ @Value("${security.authentication.ldap.username:null}")
private String ldapSecurityPrincipal;
- @Value("${ldap.password:null}")
+ @Value("${security.authentication.ldap.password:null}")
private String ldapPrincipalPassword;
- @Value("${ldap.user.identity.attribute:null}")
+ @Value("${security.authentication.ldap.user.identity.attribute:null}")
private String ldapUserIdentifyingAttribute;
- @Value("${ldap.user.email.attribute:null}")
+ @Value("${security.authentication.ldap.user.email.attribute:null}")
private String ldapEmailAttribute;
/***
diff --git a/dolphinscheduler-api/src/main/resources/application.yaml b/dolphinscheduler-api/src/main/resources/application.yaml
index 59e28b5064..366457f504 100644
--- a/dolphinscheduler-api/src/main/resources/application.yaml
+++ b/dolphinscheduler-api/src/main/resources/application.yaml
@@ -128,6 +128,22 @@ python-gateway:
# (0 = infinite), and socket server would never close even though no requests accept
read-timeout: 0
+security:
+ authentication:
+ # Authentication types (supported types: PASSWORD,LDAP)
+ type: PASSWORD
+ # IF you set type `LDAP`, below config will be effective
+ ldap:
+ # admin userId
+ user.admin: read-only-admin
+ # ldap server config
+ urls: ldap://ldap.forumsys.com:389/
+ base.dn: dc=example,dc=com
+ username: cn=read-only-admin,dc=example,dc=com
+ password: password
+ user.identity.attribute: uid
+ user.email.attribute: mail
+
# Override by profile
---
diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java
index d427eaf2a9..ba0f72a039 100644
--- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java
+++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java
@@ -50,12 +50,12 @@ import org.springframework.test.context.TestPropertySource;
properties = {
"security.authentication.type=LDAP",
"security.authentication.ldap.user.admin=read-only-admin",
- "ldap.urls=ldap://ldap.forumsys.com:389/",
- "ldap.base.dn=dc=example,dc=com",
- "ldap.username=cn=read-only-admin,dc=example,dc=com",
- "ldap.password=password",
- "ldap.user.identity.attribute=uid",
- "ldap.user.email.attribute=mail",
+ "security.authentication.ldap.urls=ldap://ldap.forumsys.com:389/",
+ "security.authentication.ldap.base.dn=dc=example,dc=com",
+ "security.authentication.ldap.username=cn=read-only-admin,dc=example,dc=com",
+ "security.authentication.ldap.password=password",
+ "security.authentication.ldap.user.identity.attribute=uid",
+ "security.authentication.ldap.user.email.attribute=mail",
})
public class LdapAuthenticatorTest extends AbstractControllerTest {
private static Logger logger = LoggerFactory.getLogger(LdapAuthenticatorTest.class);
diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapServiceTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapServiceTest.java
index 4cfdb0e50a..54e25f4f0e 100644
--- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapServiceTest.java
+++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapServiceTest.java
@@ -41,12 +41,12 @@ import org.springframework.test.context.junit4.SpringRunner;
properties = {
"security.authentication.type=LDAP",
"security.authentication.ldap.user.admin=read-only-admin",
- "ldap.urls=ldap://ldap.forumsys.com:389/",
- "ldap.base.dn=dc=example,dc=com",
- "ldap.username=cn=read-only-admin,dc=example,dc=com",
- "ldap.password=password",
- "ldap.user.identity.attribute=uid",
- "ldap.user.email.attribute=mail",
+ "security.authentication.ldap.urls=ldap://ldap.forumsys.com:389/",
+ "security.authentication.ldap.base.dn=dc=example,dc=com",
+ "security.authentication.ldap.username=cn=read-only-admin,dc=example,dc=com",
+ "security.authentication.ldap.password=password",
+ "security.authentication.ldap.user.identity.attribute=uid",
+ "security.authentication.ldap.user.email.attribute=mail",
})
public class LdapServiceTest {
@Autowired
diff --git a/dolphinscheduler-standalone-server/src/main/resources/application.yaml b/dolphinscheduler-standalone-server/src/main/resources/application.yaml
index 3ea980e67b..4640ce71ba 100644
--- a/dolphinscheduler-standalone-server/src/main/resources/application.yaml
+++ b/dolphinscheduler-standalone-server/src/main/resources/application.yaml
@@ -86,6 +86,22 @@ registry:
block-until-connected: 600ms
digest: ~
+security:
+ authentication:
+ # Authentication types (supported types: PASSWORD,LDAP)
+ type: PASSWORD
+ # IF you set type `LDAP`, below config will be effective
+ ldap:
+ # admin userId
+ user.admin: read-only-admin
+ # ldap server config
+ urls: ldap://ldap.forumsys.com:389/
+ base.dn: dc=example,dc=com
+ username: cn=read-only-admin,dc=example,dc=com
+ password: password
+ user.identity.attribute: uid
+ user.email.attribute: mail
+
master:
listen-port: 5678
# master fetch command num
[dolphinscheduler] 14/16: [Fix][UI] Add loading before file modification data is returned. (#10450)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 788e3343291b7a4b2d080f7050f6ed574c249802
Author: Amy0104 <97...@users.noreply.github.com>
AuthorDate: Wed Jun 15 15:06:06 2022 +0800
[Fix][UI] Add loading before file modification data is returned. (#10450)
(cherry picked from commit d516369e8076166ec85227f4adf3d1e9085285a6)
---
.../src/views/resource/file/edit/index.tsx | 83 ++++++++++++----------
.../src/views/resource/file/edit/use-edit.ts | 3 +-
2 files changed, 46 insertions(+), 40 deletions(-)
diff --git a/dolphinscheduler-ui/src/views/resource/file/edit/index.tsx b/dolphinscheduler-ui/src/views/resource/file/edit/index.tsx
index 9845bdbff4..73686ba053 100644
--- a/dolphinscheduler-ui/src/views/resource/file/edit/index.tsx
+++ b/dolphinscheduler-ui/src/views/resource/file/edit/index.tsx
@@ -17,7 +17,7 @@
import { useRoute, useRouter } from 'vue-router'
import { defineComponent, toRefs, watch } from 'vue'
-import { NButton, NForm, NFormItem, NSpace } from 'naive-ui'
+import { NButton, NForm, NFormItem, NSpace, NSpin } from 'naive-ui'
import { useI18n } from 'vue-i18n'
import { useForm } from './use-form'
import { useEdit } from './use-edit'
@@ -38,7 +38,7 @@ export default defineComponent({
const { getResourceView, handleUpdateContent } = useEdit(state)
const handleFileContent = () => {
- state.fileForm.content = resourceViewRef.value.content
+ state.fileForm.content = resourceViewRef.state.value.content
handleUpdateContent(fileId)
}
@@ -47,10 +47,9 @@ export default defineComponent({
}
const resourceViewRef = getResourceView(fileId)
-
watch(
- () => resourceViewRef.value.content,
- () => (state.fileForm.content = resourceViewRef.value.content)
+ () => resourceViewRef.state.value.content,
+ () => (state.fileForm.content = resourceViewRef.state.value.content)
)
return {
@@ -65,44 +64,52 @@ export default defineComponent({
const { t } = useI18n()
return (
<Card title={t('resource.file.file_details')}>
- <div class={styles['file-edit-content']}>
- <h2>
- <span>{this.resourceViewRef.alias}</span>
- </h2>
- <NForm
- rules={this.rules}
- ref='fileFormRef'
- class={styles['form-content']}
- disabled={this.componentName !== 'resource-file-edit'}
- >
- <NFormItem path='content'>
- <MonacoEditor v-model={[this.resourceViewRef.content, 'value']} />
- </NFormItem>
- <NSpace>
- <NButton
- type='info'
- size='small'
- text
- style={{ marginRight: '15px' }}
- onClick={this.handleReturn}
- class='btn-cancel'
- >
- {t('resource.file.return')}
- </NButton>
- {this.componentName === 'resource-file-edit' && (
+ {this.resourceViewRef.isReady.value ? (
+ <div class={styles['file-edit-content']}>
+ <h2>
+ <span>{this.resourceViewRef.state.value.alias}</span>
+ </h2>
+ <NForm
+ rules={this.rules}
+ ref='fileFormRef'
+ class={styles['form-content']}
+ disabled={this.componentName !== 'resource-file-edit'}
+ >
+ <NFormItem path='content'>
+ <MonacoEditor
+ v-model={[this.resourceViewRef.state.value.content, 'value']}
+ />
+ </NFormItem>
+ <NSpace>
<NButton
type='info'
size='small'
- round
- onClick={() => this.handleFileContent()}
- class='btn-submit'
+ text
+ style={{ marginRight: '15px' }}
+ onClick={this.handleReturn}
+ class='btn-cancel'
>
- {t('resource.file.save')}
+ {t('resource.file.return')}
</NButton>
- )}
- </NSpace>
- </NForm>
- </div>
+ {this.componentName === 'resource-file-edit' && (
+ <NButton
+ type='info'
+ size='small'
+ round
+ onClick={() => this.handleFileContent()}
+ class='btn-submit'
+ >
+ {t('resource.file.save')}
+ </NButton>
+ )}
+ </NSpace>
+ </NForm>
+ </div>
+ ) : (
+ <NSpace justify='center'>
+ <NSpin show={true} />
+ </NSpace>
+ )}
</Card>
)
}
diff --git a/dolphinscheduler-ui/src/views/resource/file/edit/use-edit.ts b/dolphinscheduler-ui/src/views/resource/file/edit/use-edit.ts
index 7dac246bb3..d2edaa9665 100644
--- a/dolphinscheduler-ui/src/views/resource/file/edit/use-edit.ts
+++ b/dolphinscheduler-ui/src/views/resource/file/edit/use-edit.ts
@@ -33,11 +33,10 @@ export function useEdit(state: any) {
skipLineNum: 0,
limit: 3000
}
- const { state } = useAsyncState(viewResource(params, id), {
+ return useAsyncState(viewResource(params, id), {
alias: '',
content: ''
})
- return state
}
const handleUpdateContent = (id: number) => {
[dolphinscheduler] 05/16: [fix][deploy]fix run on kubernetes bug (#10379)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit f7b18fd59d19db8a380626902cd03e18e7200ef4
Author: Hwting <83...@qq.com>
AuthorDate: Thu Jun 9 15:57:20 2022 +0800
[fix][deploy]fix run on kubernetes bug (#10379)
* [fix][kubernetes]fix run on kubernetes bug
Fix the problem that the "REGISTRY_ZOOKEEPER_CONNECT_STRING" variable zookeeper port number is "nil"
* [fix][kubernetes]fix run on kubernetes bug
Fix the problem that the "REGISTRY_ZOOKEEPER_CONNECT_STRING" variable zookeeper port number is "nil"
(cherry picked from commit 5803d5c07cf36a88f2e83dc757f1c576854142e0)
---
deploy/kubernetes/dolphinscheduler/templates/_helpers.tpl | 2 +-
deploy/kubernetes/dolphinscheduler/values.yaml | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/deploy/kubernetes/dolphinscheduler/templates/_helpers.tpl b/deploy/kubernetes/dolphinscheduler/templates/_helpers.tpl
index c2684e029a..8507f4669c 100644
--- a/deploy/kubernetes/dolphinscheduler/templates/_helpers.tpl
+++ b/deploy/kubernetes/dolphinscheduler/templates/_helpers.tpl
@@ -112,7 +112,7 @@ We truncate at 63 chars because some Kubernetes name fields are limited to this
Create a default fully qualified zookkeeper quorum.
*/}}
{{- define "dolphinscheduler.zookeeper.quorum" -}}
-{{- $port := default "2181" (.Values.zookeeper.service.port | toString) -}}
+{{- $port := default "2181" .Values.zookeeper.service.port | toString -}}
{{- printf "%s:%s" (include "dolphinscheduler.zookeeper.fullname" .) $port -}}
{{- end -}}
diff --git a/deploy/kubernetes/dolphinscheduler/values.yaml b/deploy/kubernetes/dolphinscheduler/values.yaml
index 93685fb454..2e9432a0ee 100644
--- a/deploy/kubernetes/dolphinscheduler/values.yaml
+++ b/deploy/kubernetes/dolphinscheduler/values.yaml
@@ -52,6 +52,8 @@ externalDatabase:
## If not exists external registry, the zookeeper registry will be used by default.
zookeeper:
enabled: true
+ service:
+ port: 2181
fourlwCommandsWhitelist: "srvr,ruok,wchs,cons"
persistence:
enabled: false
[dolphinscheduler] 13/16: [DolphinScheduler] Fix wrong class name in doc (#10438)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 4d608a7d8fafb2327c9b8018c2782c133343ae42
Author: 阿福Chris <yu...@126.com>
AuthorDate: Tue Jun 14 19:53:33 2022 +0800
[DolphinScheduler] Fix wrong class name in doc (#10438)
(cherry picked from commit 8205444e0a9265b7f76a498ae9d0e7091800aac5)
---
docs/docs/en/development/development-environment-setup.md | 2 +-
docs/docs/zh/development/development-environment-setup.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/docs/en/development/development-environment-setup.md b/docs/docs/en/development/development-environment-setup.md
index 518bfb010e..0940682d6e 100644
--- a/docs/docs/en/development/development-environment-setup.md
+++ b/docs/docs/en/development/development-environment-setup.md
@@ -103,7 +103,7 @@ Use different Git branch to develop different codes
### Start backend server
-Find the class `org.apache.dolphinscheduler.server.StandaloneServer` in Intellij IDEA and clikc run main function to startup.
+Find the class `org.apache.dolphinscheduler.StandaloneServer` in Intellij IDEA and clikc run main function to startup.
### Start frontend server
diff --git a/docs/docs/zh/development/development-environment-setup.md b/docs/docs/zh/development/development-environment-setup.md
index cd7cdd1721..5b2dbe0595 100644
--- a/docs/docs/zh/development/development-environment-setup.md
+++ b/docs/docs/zh/development/development-environment-setup.md
@@ -99,7 +99,7 @@ DolphinScheduler 开发环境配置有两个方式，分别是standalone模式
### 启动后端
-在 Intellij IDEA 找到并启动类 `org.apache.dolphinscheduler.server.StandaloneServer` 即可完成后端启动
+在 Intellij IDEA 找到并启动类 `org.apache.dolphinscheduler.StandaloneServer` 即可完成后端启动
### 启动前端
[dolphinscheduler] 03/16: Upgrade log4j in E2E (#10359)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 7a240fb807c9a082aacc22cc81380985931a81db
Author: PJ Fanning <pj...@users.noreply.github.com>
AuthorDate: Thu Jun 9 09:09:46 2022 +0100
Upgrade log4j in E2E (#10359)
(cherry picked from commit f90f0f8490d8f4cd87d2368cd0264a4837918c7d)
---
dolphinscheduler-e2e/pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dolphinscheduler-e2e/pom.xml b/dolphinscheduler-e2e/pom.xml
index 23d99c2d02..e404bf6b02 100644
--- a/dolphinscheduler-e2e/pom.xml
+++ b/dolphinscheduler-e2e/pom.xml
@@ -42,7 +42,7 @@
<awaitility.version>4.1.0</awaitility.version>
<kotlin.version>1.5.30</kotlin.version>
<slf4j-api.version>1.7.32</slf4j-api.version>
- <log4j-slf4j-impl.version>2.14.1</log4j-slf4j-impl.version>
+ <log4j-slf4j-impl.version>2.17.2</log4j-slf4j-impl.version>
<guava.version>31.0.1-jre</guava.version>
</properties>
[dolphinscheduler] 06/16: [Bug][Tools] Add datasource url for mysql profile in tools application.yaml (#10399)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit ed02db213e26786cc7bd21f4bb853df8ea835b1b
Author: Eric Gao <er...@gmail.com>
AuthorDate: Tue Jun 14 20:20:54 2022 +0800
[Bug][Tools] Add datasource url for mysql profile in tools application.yaml (#10399)
(cherry picked from commit 42d4aba347ce0b92152a8c4b025d8699c0e5d256)
---
dolphinscheduler-tools/src/main/resources/application.yaml | 1 +
1 file changed, 1 insertion(+)
diff --git a/dolphinscheduler-tools/src/main/resources/application.yaml b/dolphinscheduler-tools/src/main/resources/application.yaml
index 8d36a365f4..84a89097d8 100644
--- a/dolphinscheduler-tools/src/main/resources/application.yaml
+++ b/dolphinscheduler-tools/src/main/resources/application.yaml
@@ -43,6 +43,7 @@ spring:
on-profile: mysql
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
+ url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
---
spring:
[dolphinscheduler] 10/16: [fix-10386]LDAP user create with status enable (#10426)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit ba2f2ba64db603c94001bcd2274a5880835fcd63
Author: 旺阳 <qi...@cisco.com>
AuthorDate: Wed Jun 15 10:19:27 2022 +0800
[fix-10386]LDAP user create with status enable (#10426)
* LDAP user create with status enable
* currect unit test
(cherry picked from commit 50846760e5ab5f68cb708edc72725b235e933a34)
---
.../api/service/impl/UsersServiceImpl.java | 1 +
.../api/security/impl/ldap/LdapAuthenticatorTest.java | 14 ++++++--------
2 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UsersServiceImpl.java b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UsersServiceImpl.java
index cfe6fb4e9f..deb57d6f4b 100644
--- a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UsersServiceImpl.java
+++ b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/UsersServiceImpl.java
@@ -228,6 +228,7 @@ public class UsersServiceImpl extends BaseServiceImpl implements UsersService {
user.setCreateTime(now);
user.setUpdateTime(now);
user.setQueue("");
+ user.setState(Flag.YES.getCode());
// save user
userMapper.insert(user);
diff --git a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java
index e75cb2e9a8..d427eaf2a9 100644
--- a/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java
+++ b/dolphinscheduler-api/src/test/java/org/apache/dolphinscheduler/api/security/impl/ldap/LdapAuthenticatorTest.java
@@ -38,6 +38,7 @@ import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Mockito;
+import org.mockito.Spy;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
@@ -64,7 +65,7 @@ public class LdapAuthenticatorTest extends AbstractControllerTest {
private LdapService ldapService;
@MockBean(name = "sessionServiceImpl")
private SessionService sessionService;
- @MockBean(name = "usersServiceImpl")
+ @Spy
private UsersService usersService;
private LdapAuthenticator ldapAuthenticator;
@@ -102,22 +103,19 @@ public class LdapAuthenticatorTest extends AbstractControllerTest {
@Test
public void testAuthenticate() {
- when(usersService.createUser(userType, ldapUid, ldapEmail)).thenReturn(mockUser);
- when(usersService.getUserByUserName(ldapUid)).thenReturn(mockUser);
- when(sessionService.createSession(mockUser, ip)).thenReturn(mockSession.getId());
-
+ when(sessionService.createSession(Mockito.any(User.class), Mockito.eq(ip))).thenReturn(mockSession.getId());
when(ldapService.ldapLogin(ldapUid, ldapUserPwd)).thenReturn(ldapEmail);
Result result = ldapAuthenticator.authenticate(ldapUid, ldapUserPwd, ip);
Assert.assertEquals(Status.SUCCESS.getCode(), (int) result.getCode());
logger.info(result.toString());
- when(sessionService.createSession(mockUser, ip)).thenReturn(null);
+ when(sessionService.createSession(Mockito.any(User.class), Mockito.eq(ip))).thenReturn(null);
+
result = ldapAuthenticator.authenticate(ldapUid, ldapUserPwd, ip);
Assert.assertEquals(Status.LOGIN_SESSION_FAILED.getCode(), (int) result.getCode());
- when(sessionService.createSession(mockUser, ip)).thenReturn(mockSession.getId());
- when(usersService.getUserByUserName(ldapUid)).thenReturn(null);
+ when(ldapService.ldapLogin(ldapUid, ldapUserPwd)).thenReturn(null);
result = ldapAuthenticator.authenticate(ldapUid, ldapUserPwd, ip);
Assert.assertEquals(Status.USER_NAME_PASSWD_ERROR.getCode(), (int) result.getCode());
}
[dolphinscheduler] 07/16: [Doc] Add how to set the alarm mailbox in faq (#10401)
Posted by li...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
liudongkai pushed a commit to branch 3.0.0-beta-2-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 248a85daab8c619a3b8a2fd827facfab635a80bc
Author: liubo1990 <au...@163.com>
AuthorDate: Fri Jun 17 21:16:38 2022 +0800
[Doc] Add how to set the alarm mailbox in faq (#10401)
(cherry picked from commit b8d5e78994f43d8febf6ff6ae565dab45f2c8c2f)
---
docs/docs/en/faq.md | 3 ++-
docs/docs/en/guide/alert/email.md | 6 ++++++
docs/docs/zh/faq.md | 3 ++-
docs/docs/zh/guide/alert/email.md | 6 ++++++
docs/img/alert/email-alter-setup1-en.png | Bin 0 -> 27742 bytes
docs/img/alert/email-alter-setup2-en.png | Bin 0 -> 47734 bytes
docs/img/alert/email-alter-setup3-en.png | Bin 0 -> 43983 bytes
7 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/docs/docs/en/faq.md b/docs/docs/en/faq.md
index 31b7ef47a9..fd19a8d142 100644
--- a/docs/docs/en/faq.md
+++ b/docs/docs/en/faq.md
@@ -23,7 +23,8 @@ Note：**Due to the large number of services, it is recommended that the single-
## Q: Which mailboxes does the system support?
-A: Support most mailboxes, qq, 163, 126, 139, outlook, aliyun, etc. are supported. Support TLS and SSL protocols, optionally configured in alert.properties
+A: Support most mailboxes, qq, 163, 126, 139, outlook, aliyun, etc. are supported. Support TLS and SSL protocols, these can be configured in the ui of dolphinscheduler:
+[How to setup email alarm](../en/guide/alert/email.md).
---
diff --git a/docs/docs/en/guide/alert/email.md b/docs/docs/en/guide/alert/email.md
new file mode 100644
index 0000000000..aa3cf2ba73
--- /dev/null
+++ b/docs/docs/en/guide/alert/email.md
@@ -0,0 +1,6 @@
+# Email
+If you need to use `Email` for alerting, create an alert instance in the alert instance management and select the Email plugin.
+The following shows the `Email` configuration example:
+
+
+
\ No newline at end of file
diff --git a/docs/docs/zh/faq.md b/docs/docs/zh/faq.md
index f16701fae8..9bdb55e8e4 100644
--- a/docs/docs/zh/faq.md
+++ b/docs/docs/zh/faq.md
@@ -24,7 +24,8 @@ A：DolphinScheduler 由 5 个服务组成，MasterServer、WorkerServer、ApiSe
## Q：系统支持哪些邮箱？
-A：支持绝大多数邮箱，qq、163、126、139、outlook、aliyun 等皆支持。支持 **TLS 和 SSL** 协议，可以在 alert.properties 中选择性配置
+A：支持绝大多数邮箱，qq、163、126、139、outlook、aliyun 等皆支持。支持 **TLS 和 SSL** 协议，可以在dolphinscheduler的ui中进行配置：
+[如何配置邮箱告警](../zh/guide/alert/email.md)。
---
diff --git a/docs/docs/zh/guide/alert/email.md b/docs/docs/zh/guide/alert/email.md
new file mode 100644
index 0000000000..b636b03af8
--- /dev/null
+++ b/docs/docs/zh/guide/alert/email.md
@@ -0,0 +1,6 @@
+# Email
+如果需要使用`Email`进行告警，请在告警实例管理中创建告警实例，并选择Email插件。
+下面显示了 `Email` 配置示例：:
+
+
+
\ No newline at end of file
diff --git a/docs/img/alert/email-alter-setup1-en.png b/docs/img/alert/email-alter-setup1-en.png
new file mode 100644
index 0000000000..d09253de12
Binary files /dev/null and b/docs/img/alert/email-alter-setup1-en.png differ
diff --git a/docs/img/alert/email-alter-setup2-en.png b/docs/img/alert/email-alter-setup2-en.png
new file mode 100644
index 0000000000..ac44c012be
Binary files /dev/null and b/docs/img/alert/email-alter-setup2-en.png differ
diff --git a/docs/img/alert/email-alter-setup3-en.png b/docs/img/alert/email-alter-setup3-en.png
new file mode 100644
index 0000000000..1673d5d96c
Binary files /dev/null and b/docs/img/alert/email-alter-setup3-en.png differ