You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by James Cheng <wu...@gmail.com> on 2022/05/19 19:49:02 UTC

Are timestamps available for records stored in Kafka Streams state stores?

Hi,

I'm trying to see if timestamps are available for records that are stored in Kafka Streams state stores. 

I saw "KIP-258: Allow to Store Record Timestamps in RocksDB"
https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB <https://cwiki.apache.org/confluence/display/KAFKA/KIP-258:+Allow+to+Store+Record+Timestamps+in+RocksDB>

But I am not sure if it has been fully implemented. The vote thread went through, but it looks like the implementation is still in progress.
https://issues.apache.org/jira/browse/KAFKA-8382 <https://issues.apache.org/jira/browse/KAFKA-8382>

The KIP page says "2.3.0 (partially implemented, inactive)"

Can someone share what the current thoughts are, around this KIP?

Thanks!

-James

Re: Are timestamps available for records stored in Kafka Streams state stores?

Posted by "Matthias J. Sax" <mj...@apache.org>.

Added. Feel free to update wiki pages with smaller details like this 
directly.


-Matthias

On 5/19/22 2:27 PM, James Cheng wrote:
> Thanks Guozhang! Based on your comment, I searched through the repo and found the associated pull requests and JIRAs.
> 
> It looks like most of the support was added in https://issues.apache.org/jira/browse/KAFKA-6521 <https://issues.apache.org/jira/browse/KAFKA-6521>
> 
> Can you add that to the KIP page for KIP-258? It would make it easier for other people to find when/where the timestamp support was added.
> 
> Thanks!
> -James
> 
>> On May 19, 2022, at 1:24 PM, Guozhang Wang <wa...@gmail.com> wrote:
>>
>> Hi James,
>>
>> For kv / time-window stores, they have been geared with timestamps and you
>> can access via the TimestampedKeyValueStore/TimstampedWindowStore.
>>
>> What's not implemented yet are timestamped session stores.
>>
>> Guozhang
>>
>> On Thu, May 19, 2022 at 12:49 PM James Cheng <wu...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to see if timestamps are available for records that are stored
>>> in Kafka Streams state stores.
>>>
>>> I saw "KIP-258: Allow to Store Record Timestamps in RocksDB"
>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB
>>> <
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258:+Allow+to+Store+Record+Timestamps+in+RocksDB
>>>>
>>>
>>> But I am not sure if it has been fully implemented. The vote thread went
>>> through, but it looks like the implementation is still in progress.
>>> https://issues.apache.org/jira/browse/KAFKA-8382 <
>>> https://issues.apache.org/jira/browse/KAFKA-8382>
>>>
>>> The KIP page says "2.3.0 (partially implemented, inactive)"
>>>
>>> Can someone share what the current thoughts are, around this KIP?
>>>
>>> Thanks!
>>>
>>> -James
>>
>>
>>
>> -- 
>> -- Guozhang
> 
>

Re: Are timestamps available for records stored in Kafka Streams state stores?

Posted by James Cheng <wu...@gmail.com>.

Thanks Guozhang! Based on your comment, I searched through the repo and found the associated pull requests and JIRAs.

It looks like most of the support was added in https://issues.apache.org/jira/browse/KAFKA-6521 <https://issues.apache.org/jira/browse/KAFKA-6521>

Can you add that to the KIP page for KIP-258? It would make it easier for other people to find when/where the timestamp support was added.

Thanks!
-James

> On May 19, 2022, at 1:24 PM, Guozhang Wang <wa...@gmail.com> wrote:
> 
> Hi James,
> 
> For kv / time-window stores, they have been geared with timestamps and you
> can access via the TimestampedKeyValueStore/TimstampedWindowStore.
> 
> What's not implemented yet are timestamped session stores.
> 
> Guozhang
> 
> On Thu, May 19, 2022 at 12:49 PM James Cheng <wu...@gmail.com> wrote:
> 
>> Hi,
>> 
>> I'm trying to see if timestamps are available for records that are stored
>> in Kafka Streams state stores.
>> 
>> I saw "KIP-258: Allow to Store Record Timestamps in RocksDB"
>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB
>> <
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258:+Allow+to+Store+Record+Timestamps+in+RocksDB
>>> 
>> 
>> But I am not sure if it has been fully implemented. The vote thread went
>> through, but it looks like the implementation is still in progress.
>> https://issues.apache.org/jira/browse/KAFKA-8382 <
>> https://issues.apache.org/jira/browse/KAFKA-8382>
>> 
>> The KIP page says "2.3.0 (partially implemented, inactive)"
>> 
>> Can someone share what the current thoughts are, around this KIP?
>> 
>> Thanks!
>> 
>> -James
> 
> 
> 
> -- 
> -- Guozhang

Re: Are timestamps available for records stored in Kafka Streams state stores?

Posted by Guozhang Wang <wa...@gmail.com>.

Hi James,

For kv / time-window stores, they have been geared with timestamps and you
can access via the TimestampedKeyValueStore/TimstampedWindowStore.

What's not implemented yet are timestamped session stores.

Guozhang

On Thu, May 19, 2022 at 12:49 PM James Cheng <wu...@gmail.com> wrote:

> Hi,
>
> I'm trying to see if timestamps are available for records that are stored
> in Kafka Streams state stores.
>
> I saw "KIP-258: Allow to Store Record Timestamps in RocksDB"
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-258:+Allow+to+Store+Record+Timestamps+in+RocksDB
> >
>
> But I am not sure if it has been fully implemented. The vote thread went
> through, but it looks like the implementation is still in progress.
> https://issues.apache.org/jira/browse/KAFKA-8382 <
> https://issues.apache.org/jira/browse/KAFKA-8382>
>
> The KIP page says "2.3.0 (partially implemented, inactive)"
>
> Can someone share what the current thoughts are, around this KIP?
>
> Thanks!
>
> -James



-- 
-- Guozhang