You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Gene Gao <10...@qq.com> on 2015/10/20 17:29:19 UTC

Failed to read complete buffer while using simpleApi fetch message from broker

hi there
     we're running kafka cluster with 10 brokers and two topics, each topics has 500 partitions(kafka version is 0.8.2.1), when we start a hadoop job to fetch message from cluster(one hadoop map for one partition), 499/500 were successed, only one task fail. And the error on that broker is like this(I found it in server.log):


[2015-10-20 19:00:02,996] ERROR [Replica Manager on Broker 10]: Error when processing fetch request for partition [ualog,54] offset 109455388 from consumer with correlation id 0. Possible cause: Failed to read complete buffer for targetOffset 110982779 startPosition 2147469523 in /disk1/data/ualog-54/00000000000106895524.log (kafka.server.ReplicaManager)



    My code will keeping try to fetch the specificed offset message until hadoop task timeout , and of course the broker keep give me this error log but nothing changes. I have no idea what's that means.
So I hope you guys can help me out here. 
thanks