You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/07/08 10:28:54 UTC

[GitHub] [doris] wangsha0 opened a new issue, #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris

wangsha0 opened a new issue, #10703:
URL: https://github.com/apache/doris/issues/10703

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Description
   
   hdfs file has used EC police
   BrokerLoad Error:
   
   ```
            JobId: 31095590
            Label: xx
            State: CANCELLED
         Progress: ETL:N/A; LOAD:N/A
             Type: BROKER
          EtlInfo: NULL
         TaskInfo: cluster:N/A; timeout(s):14400; max_filter_ratio:0.01
         ErrorMsg: type:LOAD_RUN_FAIL; msg:ParseError : Bad read of hdfs://nsxx
       CreateTime: 2022-07-08 15:19:28
     EtlStartTime: 2022-07-08 15:19:30
    EtlFinishTime: 2022-07-08 15:19:30
    LoadStartTime: 2022-07-08 15:19:30
   LoadFinishTime: 2022-07-08 15:20:41
              URL: NULL
       JobDetails: {xx
   ```
   
   after:
   
   `hdfs ec -getPolicy -path hdfs://ns10xx`
   
   then:
   `RS-6-3-1024k`
   
   ### Use case
   
   RS-6-3-1024k
   
   ### Related issues
   
   RS-6-3-1024k
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [I] [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman closed issue #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris
URL: https://github.com/apache/doris/issues/10703


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [doris] whutpencil commented on issue #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris

Posted by GitBox <gi...@apache.org>.
whutpencil commented on issue #10703:
URL: https://github.com/apache/doris/issues/10703#issuecomment-1179088487

   you can use seatunnel or spark connetor to load ec file into doris.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [doris] wangsha0 commented on issue #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris

Posted by GitBox <gi...@apache.org>.
wangsha0 commented on issue #10703:
URL: https://github.com/apache/doris/issues/10703#issuecomment-1180379223

   Is it possible to support by using BrokerLoad?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org