You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Alexandre Dupriez (Jira)" <ji...@apache.org> on 2023/10/24 17:11:00 UTC
[jira] [Updated] (KAFKA-15301) [Tiered Storage] Historically compacted topics send request to remote for active segment during consume
[ https://issues.apache.org/jira/browse/KAFKA-15301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexandre Dupriez updated KAFKA-15301:
--------------------------------------
Description:
I have a use case where tiered storage plugin received requests for active segments. The topics for which it happened were historically compacted topics for which compaction was disabled and tiering was enabled.
Create topic with compact cleanup policy -> Produce data with few repeat keys and create multiple segments -> let compaction happen -> change cleanup policy to delete -> produce some more data for segment rollover -> enable tiering on topic -> wait for segments to be uploaded to remote storage and cleanup from local (active segment would remain), consume from beginning -> Observe logs.
was:
In AWS MSK (Kafka 2.8) tiered storage a case surfaced where tiered storage plugin received requests for active segments. The topics for which it happened were historically compacted topics for which compaction was disabled and tiering was enabled.
Create topic with compact cleanup policy -> Produce data with few repeat keys and create multiple segments -> let compaction happen -> change cleanup policy to delete -> produce some more data for segment rollover -> enable tiering on topic -> wait for segments to be uploaded to remote storage and cleanup from local (active segment would remain), consume from beginning -> Observe logs.
> [Tiered Storage] Historically compacted topics send request to remote for active segment during consume
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-15301
> URL: https://issues.apache.org/jira/browse/KAFKA-15301
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 3.6.0
> Reporter: Mital Awachat
> Assignee: Jimmy Wang
> Priority: Major
> Fix For: 3.7.0
>
>
> I have a use case where tiered storage plugin received requests for active segments. The topics for which it happened were historically compacted topics for which compaction was disabled and tiering was enabled.
> Create topic with compact cleanup policy -> Produce data with few repeat keys and create multiple segments -> let compaction happen -> change cleanup policy to delete -> produce some more data for segment rollover -> enable tiering on topic -> wait for segments to be uploaded to remote storage and cleanup from local (active segment would remain), consume from beginning -> Observe logs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)