Fetch Purgatory Request Size

[Zakee <kz...@netzero.net>]

Looking for ideas from those who have been using kafka for some time.

Should I be concerned about the fetch purgatory size increasing to high numbers and consistently remaining there while the producing data rates vary between 210k to 270k per sec (60M to 82M per sec)?


There are 5 brokers with following set of properties, currently hosting 35 topics (total 400 partitions), some of which high volume; others quite low in throughputs. There is no consumer running yet, I guess, once consumers start, purgatory size will further increase. 

auto.leader.rebalance.enable=true
leader.imbalance.check.interval.seconds=600
leader.imbalance.per.broker.percentage=10
default.replication.factor=3
log.cleaner.enable=true
log.cleaner.threads=5
log.cleanup.policy=delete
log.flush.scheduler.interval.ms=3000
log.retention.minutes=1440
log.segment.bytes=1073741824
message.max.bytes=100000000
num.io.threads=14
num.network.threads=14
num.replica.fetchers=4
queued.max.requests=500
replica.fetch.max.bytes=200000000
replica.fetch.min.bytes=51200
replica.lag.max.messages=5000
replica.lag.time.max.ms=30000
replica.fetch.wait.max.ms=1000
fetch.purgatory.purge.interval.requests=5000
producer.purgatory.purge.interval.requests=5000


Thanks
Zakee



____________________________________________________________
Skin Tightening For Men
Reduce The Look of Saggy Skin and Wrinkles, without Leaving Home
http://thirdpartyoffers.netzero.net/TGL3231/54f5083a31f938396843st04vuc