You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/03/27 17:52:33 UTC
[GitHub] [druid] ccaominh opened a new pull request #9574: Fix HDFS input
source split
ccaominh opened a new pull request #9574: Fix HDFS input source split
URL: https://github.com/apache/druid/pull/9574
### Description
Fixes an issue where splitting an HDFS input source for use in native parallel batch ingestion would cause the subtasks to get a split with an invalid HDFS path.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added unit tests or modified existing tests to cover new code paths.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] maytasm3 commented on issue #9574: Fix HDFS input source
split
Posted by GitBox <gi...@apache.org>.
maytasm3 commented on issue #9574: Fix HDFS input source split
URL: https://github.com/apache/druid/pull/9574#issuecomment-605311458
Have you tried to manually verify the fix? Or better yet, you can try running ITHdfsToHdfsSimpleIndexTest (changing INDEX_TASK from /indexer/wikipedia_cloud_simple_index_task.json to /indexer/wikipedia_cloud_index_task.json) Currently, that IT has to do Simple batch (no parallel) because of this bug. If this fix works, then that test should be able to use the Parallel batch ingestionSpec and pass.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] fjy merged pull request #9574: Fix HDFS input source split
Posted by GitBox <gi...@apache.org>.
fjy merged pull request #9574: Fix HDFS input source split
URL: https://github.com/apache/druid/pull/9574
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org