You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "radai rosenblatt (JIRA)" <ji...@apache.org> on 2016/10/11 22:09:20 UTC
[jira] [Created] (KAFKA-4293) ByteBufferMessageSet.deepIterator
burns CPU catching EOFExceptions
radai rosenblatt created KAFKA-4293:
---------------------------------------
Summary: ByteBufferMessageSet.deepIterator burns CPU catching EOFExceptions
Key: KAFKA-4293
URL: https://issues.apache.org/jira/browse/KAFKA-4293
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.10.0.1
Reporter: radai rosenblatt
around line 110:
{noformat}
try {
while (true)
innerMessageAndOffsets.add(readMessageFromStream(compressed))
} catch {
case eofe: EOFException =>
// we don't do anything at all here, because the finally
// will close the compressed input stream, and we simply
// want to return the innerMessageAndOffsets
{noformat}
the only indication the code has that the end of the oteration was reached is by catching EOFException (which will be thrown inside readMessageFromStream()).
profiling runs performed at linkedIn show 10% of the total broker CPU time taken up by Throwable.fillInStack() because of this behaviour.
unfortunately InputStream.available() cannot be relied upon (concrete example - GZipInputStream will not correctly return 0) so the fix would probably be a wire format change to also encode the number of messages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)