You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ezhou413 (via GitHub)" <gi...@apache.org> on 2025/11/26 17:39:09 UTC
[PR] MINOR: Streams developer guide improvement [kafka]
ezhou413 opened a new pull request, #21000:
URL: https://github.com/apache/kafka/pull/21000
This PR adds a section about how to prevent crashes and failures to the Kafka Streams developer guide.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Streams developer guide improvement [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21000:
URL: https://github.com/apache/kafka/pull/21000#discussion_r2566046038
##########
docs/streams/developer-guide/running-app.html:
##########
@@ -134,6 +135,22 @@ <h3><a class="toc-backref" href="#id8">Determining how many application instance
<li>Data should be equally distributed across topic partitions. For example, if two topic partitions each have 1 million messages, this is better than a single partition with 2 million messages and none in the other.</li>
<li>Processing workload should be equally distributed across topic partitions. For example, if the time to process messages varies widely, then it is better to spread the processing-intensive messages across partitions rather than storing these messages within the same partition.</li>
</ul>
+ </div>
+ <div class="section" id="handling-crashes-and-failures">
+ <span id="streams-developer-guide-execution-scaling"></span><h2><a class="toc-backref" href="#id9">Handling crashes and failures</a><a class="headerlink" href="#handling-crashes-and-failures" title="Permalink to this headline"></a></h2>
+ <p>There are a few things you can do to reduce the likelihood of crashes and failures of your Kafka Streams application.
+ <ul class="simple">
+ <li>Kafka Streams has a few configurations that can help with resilience in the face of broker failures. They can be found here: <a class="reference internal" href="config-streams.html#recommended-configuration-parameters-for-resiliency">Recommended configuration parameters for resiliency</a></li>
Review Comment:
> They can be found here
It is bad writing style to use "here". Text should be written as if it was a book, with no hyperlinks, and the sentence should still make sense w/o any hyperlinks.
So something like:
> They can be found in the configuration guide.
and just make "configuration guide" the link.
##########
docs/streams/developer-guide/running-app.html:
##########
@@ -134,6 +135,22 @@ <h3><a class="toc-backref" href="#id8">Determining how many application instance
<li>Data should be equally distributed across topic partitions. For example, if two topic partitions each have 1 million messages, this is better than a single partition with 2 million messages and none in the other.</li>
<li>Processing workload should be equally distributed across topic partitions. For example, if the time to process messages varies widely, then it is better to spread the processing-intensive messages across partitions rather than storing these messages within the same partition.</li>
</ul>
+ </div>
+ <div class="section" id="handling-crashes-and-failures">
+ <span id="streams-developer-guide-execution-scaling"></span><h2><a class="toc-backref" href="#id9">Handling crashes and failures</a><a class="headerlink" href="#handling-crashes-and-failures" title="Permalink to this headline"></a></h2>
+ <p>There are a few things you can do to reduce the likelihood of crashes and failures of your Kafka Streams application.
+ <ul class="simple">
+ <li>Kafka Streams has a few configurations that can help with resilience in the face of broker failures. They can be found here: <a class="reference internal" href="config-streams.html#recommended-configuration-parameters-for-resiliency">Recommended configuration parameters for resiliency</a></li>
+ <li>Ensure that your application is able to handle errors and failures. This includes things like
+ configuring the correct exception handlers to handle errors such as authorization and deserialization errors,
+ and using strategies such as dead letter queues to handle "poison pill" records. </li>
+ </ul>
+ </p>
+ <p>If your Kafka Streams application does crash or fail, it will first enter the <code class="docutils literal"><span class="pre">PENDING_ERROR</span></code> state
+ to gracefully close all of its existing resources, and then transition into the <code class="docutils literal"><span class="pre">ERROR</span></code> state.
+ It is important to note that the <code class="docutils literal"><span class="pre">PENDING_ERROR</span></code> state is not recoverable, and only a
+ restart will get the application back to the RUNNING state.
Review Comment:
Maybe add a sentence about monitoring both states?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Streams developer guide improvement [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on PR #21000:
URL: https://github.com/apache/kafka/pull/21000#issuecomment-3599626107
Thanks for the PR. Merged to `trunk` and cherry-picked to `4.2` branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Streams developer guide improvement [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax merged PR #21000:
URL: https://github.com/apache/kafka/pull/21000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org