You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/06/11 22:30:40 UTC
[GitHub] [druid] jihoonson opened a new pull request #10025: Set the core partition set size properly for hash-partitioned segments in native batch ingestion
jihoonson opened a new pull request #10025:
URL: https://github.com/apache/druid/pull/10025
### Description
Similar to #10012, but for hash partitioning. This PR is implemented on top of #10012.
Tagged "Release Notes" since it 1) changes the behavior in segment loading and 2) adds a new field in `HashBasedNumberedShardSpec`.
<hr>
This PR has:
- [x] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [x] added documentation for new or modified features or behaviors.
- [x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/licenses.yaml)
- [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson commented on pull request #10025: Set the core partition set size properly for hash-partitioned segments in native batch ingestion
Posted by GitBox <gi...@apache.org>.
jihoonson commented on pull request #10025:
URL: https://github.com/apache/druid/pull/10025#issuecomment-643055501
This PR is incomplete, tagged WIP.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson merged pull request #10025: Create packed core partitions for hash/range-partitioned segments in native batch ingestion
Posted by GitBox <gi...@apache.org>.
jihoonson merged pull request #10025:
URL: https://github.com/apache/druid/pull/10025
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson commented on pull request #10025: Create packed core partitions for hash/range-partitioned segments in native batch ingestion
Posted by GitBox <gi...@apache.org>.
jihoonson commented on pull request #10025:
URL: https://github.com/apache/druid/pull/10025#issuecomment-646388044
Thanks @clintropolis and @maytasm for review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson commented on pull request #10025: Create packed core partitions for hash/range-partitioned segments in native batch ingestion
Posted by GitBox <gi...@apache.org>.
jihoonson commented on pull request #10025:
URL: https://github.com/apache/druid/pull/10025#issuecomment-643700333
This PR is ready for review now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson commented on pull request #10025: Create packed core partitions for hash/range-partitioned segments in native batch ingestion
Posted by GitBox <gi...@apache.org>.
jihoonson commented on pull request #10025:
URL: https://github.com/apache/druid/pull/10025#issuecomment-646391029
> LGTM. The huge number of impl of shardSpec makes it's a little confusing since they are not the same thing and have different purpose. Maybe the interface can be refactor so that you have the topmost parent contains a minimum with different stage of ShardSpec a child classes. And different stage of ingestion can takes in different subtype of the ShardSpec corresponding to the phase/stage.
I totally agree. The approach I took in this PR was to reduce the size of the PR, but it is not the best. We can clean up them later someday.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org