You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "mjsax (via GitHub)" <gi...@apache.org> on 2023/02/09 20:17:09 UTC
[GitHub] [kafka] mjsax commented on a diff in pull request #13082: MINOR: Clarify docs for Streams config max.warmup.replicas.
mjsax commented on code in PR #13082:
URL: https://github.com/apache/kafka/pull/13082#discussion_r1101930903
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
Review Comment:
```suggestion
The maximum number of warmup replicas (extra standbys beyond the configured <code>num.standbys</code>) that can be assigned at once for the purpose of keeping
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
Review Comment:
```suggestion
Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup task can only be promoted to an active task during
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
+ a rebalance (normally a Probing Rebalance, which occur at a frequency specified by the
Review Comment:
```suggestion
a rebalance (normally a co-called probing rebalance, which occur at a frequency specified by the
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
+ a rebalance (normally a Probing Rebalance, which occur at a frequency specified by the
+ <code class="docutils literal"><span class="pre">probing.rebalance.interval.ms</span></code> config). This means that the
+ maximum rate at which Stream Tasks can be migrated from over-burdened Streams Instances to fresher Streams Instances can be determined by
+ (<code class="docutils literal"><span class="pre">max.warmup.replicas</span></code> /
+ <code class="docutils literal"><span class="pre">probing.rebalance.interval.ms</span></code>). If it takes longer than the probing rebalance interval
+ for the data for a Task to be migrated, then that rate will be lower.
Review Comment:
> If it takes longer than the probing rebalance interval for the data for a Task to be migrated, then that rate will be lower.
Not sure if I can follow?
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
Review Comment:
"Stream Task" sound a little bit like "active task" (at least, we use `StreamTask.java` in the code...)
Maybe:
```
Note: in contrast to <code>num.standbys</code> that works similar to topic <code>replication.factor</code> config, <code>max-warmup-replicas</code> is the <it>absolute</it> numbers of warmup tasks.
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
Review Comment:
It's not you...
Cf https://cwiki.apache.org/confluence/display/KAFKA/Contributing+Website+Documentation+Changes and https://cwiki.apache.org/confluence/display/KAFKA/Setup+Kafka+Website+on+Local+Apache+Server
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
+ a rebalance (normally a Probing Rebalance, which occur at a frequency specified by the
+ <code class="docutils literal"><span class="pre">probing.rebalance.interval.ms</span></code> config). This means that the
+ maximum rate at which Stream Tasks can be migrated from over-burdened Streams Instances to fresher Streams Instances can be determined by
Review Comment:
```suggestion
maximum rate at which an active tasks can be migrated from one Kafka Streams instances to another instance can be determined by
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -778,10 +778,21 @@ <h4><a class="toc-backref" href="#id34">rack.aware.assignment.tags</a><a class="
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
- The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
- the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
- traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
- for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ <p>
+ The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
+ the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
+ traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
+ for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
+ </p>
+ <p>
+ Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active Task during
Review Comment:
It's not you...
Cf https://cwiki.apache.org/confluence/display/KAFKA/Contributing+Website+Documentation+Changes and https://cwiki.apache.org/confluence/display/KAFKA/Setup+Kafka+Website+on+Local+Apache+Server
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org