[GitHub] [druid] jihoonson commented on pull request #10419: Automatically determine numShards for parallel ingestion hash partitioning

[GitBox <gi...@apache.org>]

jihoonson commented on pull request #10419:
URL: https://github.com/apache/druid/pull/10419#issuecomment-698508270


   One thing to note: after this PR, the parallel task can compute the number of partitions automatically, but the same number will be applied to all intervals. I think it will be better to compute and apply different numbers of partitions per interval to handle potential data skew between intervals which is what the simple task (`IndexTask`) does. But I'm OK with doing this improvement as a follow-up.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org