You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "hudeqi (Jira)" <ji...@apache.org> on 2023/10/19 09:29:00 UTC
[jira] [Updated] (KAFKA-15607) Possible NPE is thrown in MirrorCheckpointTask
[ https://issues.apache.org/jira/browse/KAFKA-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
hudeqi updated KAFKA-15607:
---------------------------
Description:
In the `syncGroupOffset` method, if `targetConsumerOffset.get(topicPartition)` gets null, then the calculation of `latestDownstreamOffset` will throw NPE. This usually occurs in this situation: a group consumed a topic in the target cluster previously. Later, the group offset of some partitions was reset to -1, the `OffsetAndMetadata` of these partitions was null.
It is possible that when reset offsets are performed in the java kafka client, the reset to -1 will be intercepted. However, there are some other types of clients such as sarama, which can magically reset the group offset to -1, so MM2 will trigger an NPE exception in this scenario. Therefore, a defensive measure to avoid NPE is needed here.
was:In the `syncGroupOffset` method, if `targetConsumerOffset.get(topicPartition)` gets null, then the calculation of `latestDownstreamOffset` will throw NPE. This usually occurs in this situation: a group consumed a topic in the target cluster previously. Later, the group offset of some partitions was reset to -1, the `OffsetAndMetadata` of these partitions was null.
> Possible NPE is thrown in MirrorCheckpointTask
> ----------------------------------------------
>
> Key: KAFKA-15607
> URL: https://issues.apache.org/jira/browse/KAFKA-15607
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 2.8.1
> Reporter: hudeqi
> Assignee: hudeqi
> Priority: Blocker
>
> In the `syncGroupOffset` method, if `targetConsumerOffset.get(topicPartition)` gets null, then the calculation of `latestDownstreamOffset` will throw NPE. This usually occurs in this situation: a group consumed a topic in the target cluster previously. Later, the group offset of some partitions was reset to -1, the `OffsetAndMetadata` of these partitions was null.
> It is possible that when reset offsets are performed in the java kafka client, the reset to -1 will be intercepted. However, there are some other types of clients such as sarama, which can magically reset the group offset to -1, so MM2 will trigger an NPE exception in this scenario. Therefore, a defensive measure to avoid NPE is needed here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)