You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/07/21 16:49:12 UTC
[GitHub] [druid] petermarshallio opened a new pull request #11480: MiddleManager Affinity "strong" definition
petermarshallio opened a new pull request #11480:
URL: https://github.com/apache/druid/pull/11480
Reworded "strong" to emphasise meaning and consequences - OTBO https://the-asf.slack.com/archives/CJ8D1JTB8/p1609558156092800
### Description
Clarify that "strong" is required for full affinity of task assignment.
This PR has:
- [x] been self-reviewed.
- [ ] been tested in a test Druid cluster.
cc: @sthetland @techdocsmith @sixtus
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] asdf2014 commented on a change in pull request #11480: Docs - MiddleManager Affinity "strong" definition
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on a change in pull request #11480:
URL: https://github.com/apache/druid/pull/11480#discussion_r676002946
##########
File path: docs/configuration/index.md
##########
@@ -1399,7 +1398,7 @@ then the value from the configuration below is used:
|--------|-----------|-------|
|`druid.worker.version`|Version identifier for the Indexer.|0|
|`druid.worker.capacity`|Maximum number of tasks the Indexer can accept.|Number of available processors - 1|
-|`druid.worker.globalIngestionHeapLimitBytes`|Total amount of heap available for ingestion processing. This is applied by automatically setting the `maxBytesInMemory` property on tasks.|60% of configured JVM heap|
+|`druid.worker.globalIngestionHeapLimitBytes`|Total amount of heap available for ingestion processing. This is applied by automatically setting the `maxBytesInMemory` property on tasks.|60% of configured JVM heap|
Review comment:
nit: the blank character at the end of line is unnecessary :sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s merged pull request #11480: Docs - MiddleManager Affinity "strong" definition
Posted by GitBox <gi...@apache.org>.
suneet-s merged pull request #11480:
URL: https://github.com/apache/druid/pull/11480
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] petermarshallio commented on pull request #11480: Docs - MiddleManager Affinity "strong" definition
Posted by GitBox <gi...@apache.org>.
petermarshallio commented on pull request #11480:
URL: https://github.com/apache/druid/pull/11480#issuecomment-896982483
Travis CI failure seems unrelated to docs?
```
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/m/maven/libmaven3-core-java_3.3.9-3_all.deb Unable to connect to archive.ubuntu.com:http: [IP: 91.189.88.142 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/m/maven/maven_3.3.9-3_all.deb Unable to connect to archive.ubuntu.com:http: [IP: 91.189.88.142 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
apt-get.diagnostics
apt-get install failed
The command "sudo -E apt-get -yq --no-install-suggests --no-install-recommends $(travis_apt_get_options) install maven python3" failed and exited with 100 during .
Your build has been stopped.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] techdocsmith commented on a change in pull request #11480: Docs - MiddleManager Affinity "strong" definition
Posted by GitBox <gi...@apache.org>.
techdocsmith commented on a change in pull request #11480:
URL: https://github.com/apache/druid/pull/11480#discussion_r685623217
##########
File path: docs/configuration/index.md
##########
@@ -1202,13 +1202,12 @@ Example: a function that sends batch_index_task to workers 10.0.0.1 and 10.0.0.2
###### Affinity
-Affinity configs can be provided to the _equalDistribution_ and _fillCapacity_ strategies using the "affinityConfig"
-field. If not provided, the default is to not use affinity at all.
+Affinity configs can be provided to the _equalDistribution_ and _fillCapacity_ strategies using the "affinityConfig" field. If not provided, the default is to not use affinity at all.
Review comment:
```suggestion
Use the `affinityConfig` field to pass affinity configuration to the _equalDistribution_ and _fillCapacity_ strategies. If not provided, the default is to not use affinity at all.
```
##########
File path: docs/configuration/index.md
##########
@@ -1202,13 +1202,12 @@ Example: a function that sends batch_index_task to workers 10.0.0.1 and 10.0.0.2
###### Affinity
-Affinity configs can be provided to the _equalDistribution_ and _fillCapacity_ strategies using the "affinityConfig"
-field. If not provided, the default is to not use affinity at all.
+Affinity configs can be provided to the _equalDistribution_ and _fillCapacity_ strategies using the "affinityConfig" field. If not provided, the default is to not use affinity at all.
|Property|Description|Default|
|--------|-----------|-------|
|`affinity`|JSON object mapping a datasource String name to a list of indexing service MiddleManager host:port String values. Druid doesn't perform DNS resolution, so the 'host' value must match what is configured on the MiddleManager and what the MiddleManager announces itself as (examine the Overlord logs to see what your MiddleManager announces itself as).|{}|
-|`strong`|With weak affinity (the default), tasks for a dataSource may be assigned to other MiddleManagers if their affinity-mapped MiddleManagers are not able to run all pending tasks in the queue for that dataSource. With strong affinity, tasks for a dataSource will only ever be assigned to their affinity-mapped MiddleManagers, and will wait in the pending queue if necessary.|false|
+|`strong`|A boolean determining whether tasks for a datasource must be assigned to affinity-mapped MiddleManagers, causing tasks to remain queued until a slot is available. If `false`, tasks for a datasource may be assigned to other MiddleManagers whenever affinity-mapped MiddleManagers are not available to run queued tasks.|false|
Review comment:
```suggestion
|`strong`|When `true` tasks for a datasource must be assigned to affinity-mapped MiddleManagers. Tasks remain queued until a slot becomes available. When `false`, Druid may assign tasks for a datasource to other MiddleManagers when affinity-mapped MiddleManagers are unavailable to run queued tasks.|false|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s commented on pull request #11480: Docs - MiddleManager Affinity "strong" definition
Posted by GitBox <gi...@apache.org>.
suneet-s commented on pull request #11480:
URL: https://github.com/apache/druid/pull/11480#issuecomment-896985892
I've re-triggered Travis for you @petermarshallio
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org