[GitHub] [dolphinscheduler] jieguangzhou commented on a diff in pull request #12552: [Feature][Task] Transfer files between tasks

[GitBox <gi...@apache.org>]

jieguangzhou commented on code in PR #12552:
URL: https://github.com/apache/dolphinscheduler/pull/12552#discussion_r1006353575


##########
dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/runner/WorkerTaskExecuteRunnable.java:
##########
@@ -216,15 +217,17 @@ protected void beforeExecute() {
         TaskExecutionCheckerUtils.downloadResourcesIfNeeded(storageOperate, taskExecutionContext, logger);
         logger.info("Resources:{} check success", taskExecutionContext.getResources());
 
+        TaskFilesTransferUtils.downloadUpstreamFiles(taskExecutionContext, storageOperate);

Review Comment:
   Good idea, but I think this should be implemented in A in the resource center interface because the resource center can connect HDFS, S3, and OSS. . Maybe we can create a new issue to do this, a unified interface can be provided later. WDYT? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org