You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@doris.apache.org by GitBox <gi...@apache.org> on 2019/07/01 07:57:18 UTC

[GitHub] [incubator-doris] zttaotao2013 opened a new issue #1416: ETL_SUBMIT_FAIL problem

zttaotao2013 opened a new issue #1416: ETL_SUBMIT_FAIL problem
URL: https://github.com/apache/incubator-doris/issues/1416
 
 
   我搭建了3fe + 3be + 3broker的doris测试环境，计划对存储于hdfs目录中的数据使用Broker进行批量入库，我定义了一个任务：
   load label data_load_1
   data infile
   (
     "hdfs://192.168.0.2:8020/doris/data/*.nb"
      into table test
   )
   ....
   在/doris/data/目录下，有300个数据文件，每个文件1G左右，当我提交一个入库任务时，报 ETL_SUBMIT_FAIL; msg:create job request fail. scan bytes per broker scanner limit: 3221225472.
   当单个文件导入时能够顺利导入。doris对这种场景的批量导入有数据量限制？如果是这样，感觉批量入库不太友好，300个文件我需要设置300个Load任务。

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@doris.apache.org
For additional commands, e-mail: dev-help@doris.apache.org