You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/04/03 00:09:57 UTC
[GitHub] [druid] samarthjain opened a new pull request #9603: Add new round
robin strategy for loading segments.
samarthjain opened a new pull request #9603: Add new round robin strategy for loading segments.
URL: https://github.com/apache/druid/pull/9603
This strategy aims to address situations where datasources that had older
data restated often wait a long time before their segments are loaded on the cluster.
This behavior happens because Druid by default prefers loading newer data. With this
new strategy segments are loaded by selecting datasources in a round-robin fashion.
For each datasource, the strategy ensures that the newer segments are loaded before the older ones.
This PR has:
- [X] been self-reviewed.
- [X] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [X] added documentation for new or modified features or behaviors.
- [X] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [X] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [X] added unit tests or modified existing tests to cover new code paths.
- [X] been tested in a test Druid cluster.
* `DruidCoordinatorRuntimeParams.java`
* `ImmutableDruidDataSource.java`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] surekhasaharan commented on a change in pull request #9603:
Add new round robin strategy for loading segments.
Posted by GitBox <gi...@apache.org>.
surekhasaharan commented on a change in pull request #9603: Add new round robin strategy for loading segments.
URL: https://github.com/apache/druid/pull/9603#discussion_r407111092
##########
File path: server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinatorConfig.java
##########
@@ -92,4 +92,10 @@ public int getHttpLoadQueuePeonBatchSize()
{
return 1;
}
+
+ @Config("druid.coordinator.load.segment.strategy")
+ public String getLoadSegmentStrategy()
+ {
+ return "newFirst";
Review comment:
shouldn't this be "roundRobin"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] samarthjain commented on a change in pull request #9603:
Add new round robin strategy for loading segments.
Posted by GitBox <gi...@apache.org>.
samarthjain commented on a change in pull request #9603: Add new round robin strategy for loading segments.
URL: https://github.com/apache/druid/pull/9603#discussion_r407625056
##########
File path: server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinatorConfig.java
##########
@@ -92,4 +92,10 @@ public int getHttpLoadQueuePeonBatchSize()
{
return 1;
}
+
+ @Config("druid.coordinator.load.segment.strategy")
+ public String getLoadSegmentStrategy()
+ {
+ return "newFirst";
Review comment:
I named the current strategy of loading newer segments first as `newFirst`. It is also the default strategy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org