[jira] [Work started] (KAFKA-1065) IO exception on windows when high throughput of messages

["Jun Rao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on KAFKA-1065 started by Jun Rao.

> IO exception on windows when high throughput of messages
> --------------------------------------------------------
>
>                 Key: KAFKA-1065
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1065
>             Project: Kafka
>          Issue Type: Bug
>          Components: log
>    Affects Versions: 0.8
>         Environment: Windows 7 64bit
>            Reporter: Carl Austin
>            Assignee: Jun Rao
>
> When a large number of messages are sent per second to a broker on Windows a memory mapping exception occurs and kills Kafka. The exception follows : 
> {code}kafka.common.KafkaStorageException: I/O exception in append to log 'test-0'
> 	at kafka.log.Log.append(Log.scala:349)
> 	at kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:340)
> 	at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:236)
> 	at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:228)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> 	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> 	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> 	at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> 	at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
> 	at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
> 	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> 	at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
> 	at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
> 	at scala.collection.mutable.HashMap.map(HashMap.scala:39)
> 	at kafka.server.KafkaApis.appendToLocalLog(KafkaApis.scala:228)
> 	at kafka.server.KafkaApis.handleProducerRequest(KafkaApis.scala:162)
> 	at kafka.server.KafkaApis.handle(KafkaApis.scala:66)
> 	at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
> 	at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: The requested operation cannot be performed on a file with a user-mapped section open
> 	at java.io.RandomAccessFile.setLength(Native Method)
> 	at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
> 	at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
> 	at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
> 	at kafka.log.Log.rollToOffset(Log.scala:518)
> 	at kafka.log.Log.roll(Log.scala:502)
> 	at kafka.log.Log.maybeRoll(Log.scala:484)
> 	at kafka.log.Log.append(Log.scala:297)
> 	... 19 more{code}
> This seems to have been mentioned in the past on the Kafka mail list and is an issue related to http://bugs.sun.com/view_bug.do?bug_id=4724038.
> Unfortunately this means that we cannot use Kafka on Windows.



--
This message was sent by Atlassian JIRA
(v6.1#6144)