You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/06/30 07:52:10 UTC

[GitHub] [incubator-dolphinscheduler] primoshen opened a new issue #3092: [Feature]upload file to resource one file at a time very inefficient, can upload more at a time?

primoshen opened a new issue #3092:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3092


   **Describe the solution you'd like**
   upload file to resource one file at a time very inefficient, can upload more at a time?
   资源上传，一次只能上传一个文件，效率太低了，如果文件多，耗时就会比较久，此外如果文件上传到错误的文件夹，则又需要重新上传；
   
   在此也可以通过后台把文件一次上传到hdfs，然后文件信息插入到表t_ds_resources中，但效率还是有点...
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] yangyichao-mango commented on issue #3092: [Feature]upload file to resource one file at a time very inefficient, can upload more at a time?

Posted by GitBox <gi...@apache.org>.
yangyichao-mango commented on issue #3092:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3092#issuecomment-658094374


   Hi,
   Good idea, would it be better to include the following two functions:
   1. DS directly binds a certain HDFS path, the files can be uploaded from the background to HDFS by user.
   2. The front end uploads multiple files at a time.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] liuxuedongcn commented on issue #3092: [Feature]upload file to resource one file at a time very inefficient, can upload more at a time?

Posted by GitBox <gi...@apache.org>.
liuxuedongcn commented on issue #3092:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3092#issuecomment-759183949


   > Hi,
   > Good idea, would it be better to include the following two functions:
   > 
   > 1. DS directly binds a certain HDFS path, the files can be uploaded from the background to HDFS by user.
   > 2. The front end uploads multiple files at a time.
   
   A node type can be extended to upload local files to hdfs, which can be run in multiple threads, I can complete this


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #3092: [Feature]upload file to resource one file at a time very inefficient, can upload more at a time?

Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #3092:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3092#issuecomment-651618102


   good idea


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #3092: [Feature]upload file to resource one file at a time very inefficient, can upload more at a time?

Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #3092:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3092#issuecomment-817116593


   Related issue: #5201


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org