You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Kim Chew <kc...@gmail.com> on 2018/04/24 00:19:12 UTC
Topic partitions logSize are way behind the offsets.
The following is the output from "kafka-consumer-offset-checker.sh",
Group Topic Pid Offset
logSize Lag Owner
secorV2_backup auth-data 0 13433
2 -13431 secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup auth-data 1 17466
17 -17449 secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup auth-data 2 15284
11 -15273 secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup auth-data 3 10174
14 -10160 secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup auth-data 4 24228
44 -24184 secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup auth-data 5 5239
1 -5238 secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup auth-data 6 653137
10921 -642216 secorV2_backup_ip-172-30-13-99_3448_243-0
...............
It happened after the kafka cluster has restarted.
I am trying to understand why there are such a huge "Lag" values for some
of the partitions and the logSize are so small. What are some of the
reasons behind it? And what would be the good ways to fix it? Because just
waiting kafka to "catch up" with the offsets, where in this case, may be
weeks.
TIA.
Kim