[GitHub] [kafka] guozhangwang commented on a change in pull request #10941: KAFKA-10847: Remove internal config for enabling the fix

[GitBox <gi...@apache.org>]

guozhangwang commented on a change in pull request #10941:
URL: https://github.com/apache/kafka/pull/10941#discussion_r670119642



##########
File path: docs/streams/upgrade-guide.html
##########
@@ -117,6 +117,21 @@ <h3><a id="streams_api_changes_300" href="#streams_api_changes_300">Streams API
     <p>
         We removed the default implementation of <code>RocksDBConfigSetter#close()</code>.
     </p>
+    <p>
+        We dropped the default 24 hours grace period for windowed operations such as Window or Session aggregates, or stream-stream joins.
+        This period determines how long after a window ends any late arrived records will still be processed.
+        Records coming in after the grace period has elapsed will be dropped from those windows.
+        With a long grace period, though Kafka Streams can handle out-of-order data up to that amount of time, it will also incur a high and confusing latency for users,
+        e.g. suppression operators with the default won't emit results up for 24 hours, while lso in practice out-of-order data usually has a much smaller time-skewness.
+        Instead of abstracting this config from users with a long default value, we introduced new constructs such as <code>TimeWindows#ofSizeWithNoGrace</code> to let callers always set it upon constructing the windows;

Review comment:
       `TimeDiffernce` is only in `JoinWindows`.

##########
File path: docs/streams/upgrade-guide.html
##########
@@ -117,6 +117,21 @@ <h3><a id="streams_api_changes_300" href="#streams_api_changes_300">Streams API
     <p>
         We removed the default implementation of <code>RocksDBConfigSetter#close()</code>.
     </p>
+    <p>
+        We dropped the default 24 hours grace period for windowed operations such as Window or Session aggregates, or stream-stream joins.

Review comment:
       Since we are piggy-backing the fix on KIP-663 now, I want to incorporate the change along with this PR.

##########
File path: docs/streams/upgrade-guide.html
##########
@@ -117,6 +117,21 @@ <h3><a id="streams_api_changes_300" href="#streams_api_changes_300">Streams API
     <p>
         We removed the default implementation of <code>RocksDBConfigSetter#close()</code>.
     </p>
+    <p>
+        We dropped the default 24 hours grace period for windowed operations such as Window or Session aggregates, or stream-stream joins.
+        This period determines how long after a window ends any late arrived records will still be processed.
+        Records coming in after the grace period has elapsed will be dropped from those windows.
+        With a long grace period, though Kafka Streams can handle out-of-order data up to that amount of time, it will also incur a high and confusing latency for users,
+        e.g. suppression operators with the default won't emit results up for 24 hours, while lso in practice out-of-order data usually has a much smaller time-skewness.
+        Instead of abstracting this config from users with a long default value, we introduced new constructs such as <code>TimeWindows#ofSizeWithNoGrace</code> to let callers always set it upon constructing the windows;
+        the other setters such as <code>TimeWindows#grace</code> are deprecated and will be removed in the future.
+        Also when the new construct API are used for left/outer stream-stream joins, Kafka Streams would fix emitting spurious join results which may have an impact on the throughput.

Review comment:
       Ack!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org