You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/07/08 10:28:54 UTC
[GitHub] [doris] wangsha0 opened a new issue, #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris
wangsha0 opened a new issue, #10703:
URL: https://github.com/apache/doris/issues/10703
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Description
hdfs file has used EC police
BrokerLoad Error:
```
JobId: 31095590
Label: xx
State: CANCELLED
Progress: ETL:N/A; LOAD:N/A
Type: BROKER
EtlInfo: NULL
TaskInfo: cluster:N/A; timeout(s):14400; max_filter_ratio:0.01
ErrorMsg: type:LOAD_RUN_FAIL; msg:ParseError : Bad read of hdfs://nsxx
CreateTime: 2022-07-08 15:19:28
EtlStartTime: 2022-07-08 15:19:30
EtlFinishTime: 2022-07-08 15:19:30
LoadStartTime: 2022-07-08 15:19:30
LoadFinishTime: 2022-07-08 15:20:41
URL: NULL
JobDetails: {xx
```
after:
`hdfs ec -getPolicy -path hdfs://ns10xx`
then:
`RS-6-3-1024k`
### Use case
RS-6-3-1024k
### Related issues
RS-6-3-1024k
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman closed issue #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris
URL: https://github.com/apache/doris/issues/10703
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] whutpencil commented on issue #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris
Posted by GitBox <gi...@apache.org>.
whutpencil commented on issue #10703:
URL: https://github.com/apache/doris/issues/10703#issuecomment-1179088487
you can use seatunnel or spark connetor to load ec file into doris.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] wangsha0 commented on issue #10703: [Feature] About supporting the format file of EC when using BrokerLoad to sync data from hdfs to doris
Posted by GitBox <gi...@apache.org>.
wangsha0 commented on issue #10703:
URL: https://github.com/apache/doris/issues/10703#issuecomment-1180379223
Is it possible to support by using BrokerLoad?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org