You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Jeetu Bethina (JIRA)" <ji...@apache.org> on 2017/06/18 15:20:00 UTC
[jira] [Updated] (KAFKA-5466) Unable to fetch elements from
WindowStore when timestamp is before the first inserted Elem timestamp
[ https://issues.apache.org/jira/browse/KAFKA-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeetu Bethina updated KAFKA-5466:
---------------------------------
Description:
I am unable to fetch elements that are inserted with a timestamp before the timestamp of the first element inserted into a WindowStore.
This affects newly created WindowStores or one that is restored from a changelog topic.
The method I use to insert into the Window Store is
http://docs.confluent.io/3.2.1/streams/javadocs/org/apache/kafka/streams/state/WindowStore.html#put-K-V-long-
Looking into the code, the issue might be with how the minSegmentId is being calculated here.
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/Segments.java#L88
Something like the below fixes the issue with regards to the WindowStore but I am not sure what the implications would be for Streams as a whole.
if (minSegmentId == Long.MAX_VALUE) {
minSegmentId = maxSegmentId;
} else {
minSegmentId = segmentId < minSegmentId ? segmentId : minSegmentId;
}
was:
I am unable to fetch elements that are inserted with a timestamp before the timestamp of the first element inserted into the WindowStore.
This affects a newly created WindowStore or one that is restored from a changelog topic.
The method I use to insert into the Window Store is
http://docs.confluent.io/3.2.1/streams/javadocs/org/apache/kafka/streams/state/WindowStore.html#put-K-V-long-
Looking into the code, the issue might be with how the minSegmentId is being calculated here.
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/Segments.java#L88
Something like the below fixes the issue with regards to the WindowStore but I am not sure what the implications would be for Streams as a whole.
if (minSegmentId == Long.MAX_VALUE) {
minSegmentId = maxSegmentId;
} else {
minSegmentId = segmentId < minSegmentId ? segmentId : minSegmentId;
}
> Unable to fetch elements from WindowStore when timestamp is before the first inserted Elem timestamp
> ----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-5466
> URL: https://issues.apache.org/jira/browse/KAFKA-5466
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.1
> Reporter: Jeetu Bethina
>
> I am unable to fetch elements that are inserted with a timestamp before the timestamp of the first element inserted into a WindowStore.
> This affects newly created WindowStores or one that is restored from a changelog topic.
> The method I use to insert into the Window Store is
> http://docs.confluent.io/3.2.1/streams/javadocs/org/apache/kafka/streams/state/WindowStore.html#put-K-V-long-
> Looking into the code, the issue might be with how the minSegmentId is being calculated here.
> https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/Segments.java#L88
> Something like the below fixes the issue with regards to the WindowStore but I am not sure what the implications would be for Streams as a whole.
> if (minSegmentId == Long.MAX_VALUE) {
> minSegmentId = maxSegmentId;
> } else {
> minSegmentId = segmentId < minSegmentId ? segmentId : minSegmentId;
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)