You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Mark Wimpory (Jira)" <ji...@apache.org> on 2024/07/10 07:18:00 UTC
[jira] [Commented] (CAMEL-20958) Kinesis consumer loops on closed shards
[ https://issues.apache.org/jira/browse/CAMEL-20958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864511#comment-17864511 ]
Mark Wimpory commented on CAMEL-20958:
--------------------------------------
Readme.md contains additional steps to reproduce as it requires aws command
> Kinesis consumer loops on closed shards
> ---------------------------------------
>
> Key: CAMEL-20958
> URL: https://issues.apache.org/jira/browse/CAMEL-20958
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-kinesis
> Affects Versions: 4.6.0
> Environment: Camel 4.6.0
> Localstack 3.0.2
> Reproducer attached.
> Reporter: Mark Wimpory
> Priority: Major
> Attachments: camel-reproducer.zip
>
>
> When shards are merged the consumer reads any remaining messages from those shards and then notes that the shard has closed. However it continuously loops as in the log exert below. This continues on a restart.
> Note this occurs no matter what type of failure option (silent, fail etc)
> I have not tested so far what happens when the shard reaches its expired state (due to a localstack bug)
>
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] WARN o.a.c.c.a.k.Kinesis2Consumer - The shard with id=shardId-000000000000 on stream ingestion-death-stream reached CLOSE status
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] WARN o.a.c.c.a.k.Kinesis2Consumer - The shard with id=shardId-000000000001 on stream ingestion-death-stream reached CLOSE status
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] INFO o.a.c.c.a.k.c.KinesisDefaultResumeAdapter - Resuming from offset 49653759514412492265585802359002135113225509352150925314 for key ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] INFO o.a.c.c.a.k.c.KinesisDefaultResumeAdapter - Resuming from offset 49653759514434793010784332982141252979858928386588016658 for key ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-4] DEBUG o.a.c.c.a.k.Kinesis2Consumer - Obtained new ShardIterator AAAAAAAAAAFhGdwQ2Ac96sw4ScfsOEaTn+V+4vvB9RE9cnHLqD+s4rLMSSioedJLXVbHf47GfSV8ktk53sJmQXMFb+EZOOA/PaByEh66RTC7iBK+8BCZrrL5wb4wNHV7M1qtua1Cu7NrsqEZli3+HgagoG4V7bopK5xPcezzfDN9EWclJRCnlq3Z9nJRSmB4JCmIweWsX8VrM7SHnaVabQ3Kd8l9zoj7 for shard shardId-000000000000 on stream ingestion-death-stream
> 2024-07-09 12:01:43 [ForkJoinPool.commonPool-worker-3] DEBUG o.a.c.c.a.k.Kinesis2Consumer - Obtained new ShardIterator AAAAAAAAAAEGg3GnGpu+nfbhufxJ7/V1iSjlD7G24d03xeQATtF9pV1dK21qLxxKpdS8g/CLPdV3MZaJaZoXkthM0J4xNvLOAkCwPpEsl9ympInzr/3QgNqFHqhGopNBlGKMyCdQUYi0UErBuUJ8dPKCq+IYRNVQvhSGg4fF4zXAlDi7jsXC7HZ2X9Xo3Q4XF/E7yEhi1YnFKAmMOmzstDkUtJd+U3MO for shard shardId-000000000001 on stream ingestion-death-stream
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)