You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/12/13 06:23:02 UTC
[GitHub] [dolphinscheduler] JinyLeeChina commented on a change in pull request #7214: [DS-7016][feat] Auto create workflow while import sql script with specific hint
JinyLeeChina commented on a change in pull request #7214:
URL: https://github.com/apache/dolphinscheduler/pull/7214#discussion_r767436122
##########
File path: dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/DataSourceMapper.xml
##########
@@ -98,4 +98,13 @@
</foreach>
</if>
</select>
+
+ <select id="queryDataSourceByNameAndUserId" resultType="org.apache.dolphinscheduler.dao.entity.DataSource">
+ select ds.* from t_ds_datasource ds
Review comment:
Avoid using `*` as far as possible.
##########
File path: dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ProcessDefinitionServiceImpl.java
##########
@@ -865,6 +882,149 @@ public DagDataSchedule exportProcessDagData(ProcessDefinition processDefinition)
return result;
}
+ @Override
+ public Map<String, Object> importSqlProcessDefinition(User loginUser, long projectCode, MultipartFile file) {
+ Map<String, Object> result = new HashMap<>();
+ String processDefinitionName = file.getOriginalFilename() == null ? file.getName() : file.getOriginalFilename();
+ int index = processDefinitionName.lastIndexOf(".");
+ if (index > 0) {
+ processDefinitionName = processDefinitionName.substring(0, index);
+ }
+ // build process definition
+ Date now = new Date();
+ ProcessDefinition processDefinition = new ProcessDefinition();
+ processDefinition.setName(processDefinitionName);
+ processDefinition.setCreateTime(now);
+ processDefinition.setUpdateTime(now);
+ processDefinition.setFlag(Flag.YES);
+ processDefinition.setTenantId(-1);
+ processDefinition.setGlobalParamList(Collections.emptyList());
+
+ DagDataSchedule dagDataSchedule = new DagDataSchedule();
+ dagDataSchedule.setProcessDefinition(processDefinition);
+ List<TaskDefinition> taskDefinitionList = new ArrayList<>();
+ dagDataSchedule.setTaskDefinitionList(taskDefinitionList);
+ List<ProcessTaskRelation> processTaskRelationList = new ArrayList<>();
+ dagDataSchedule.setProcessTaskRelationList(processTaskRelationList);
+
+ // In most cases, there will be only one data source
+ Map<String, DataSource> dataSourceCache = new HashMap<>(1);
+ Map<String, Long> taskNameToCode = new HashMap<>(16);
+ Map<String, List<String>> taskNameToUpstream = new HashMap<>(16);
+ try (ZipInputStream zIn = new ZipInputStream(file.getInputStream());
+ BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(zIn))) {
+ ZipEntry entry;
+ while ((entry = zIn.getNextEntry()) != null) {
+ if (!entry.isDirectory()) {
+ StringBuilder sql = new StringBuilder();
+ String taskName = null;
+ String datasourceName = null;
+ List<String> upstreams = Collections.emptyList();
+ String line;
+ while ((line = bufferedReader.readLine()) != null) {
+ int commentIndex = line.indexOf("--");
Review comment:
I think SQL regular may also contain these symbols, and you should exclude this possibility here. You can refer to
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org