You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "翟玉勇 (JIRA)" <ji...@apache.org> on 2018/09/26 07:15:00 UTC
[jira] [Updated] (KAFKA-7442) forceUnmap mmap on linux when index
resize
[ https://issues.apache.org/jira/browse/KAFKA-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
翟玉勇 updated KAFKA-7442:
-----------------------
Summary: forceUnmap mmap on linux when index resize (was: forceUnmap mmap on linux)
> forceUnmap mmap on linux when index resize
> ------------------------------------------
>
> Key: KAFKA-7442
> URL: https://issues.apache.org/jira/browse/KAFKA-7442
> Project: Kafka
> Issue Type: Improvement
> Components: log
> Affects Versions: 0.10.1.1
> Reporter: 翟玉勇
> Priority: Minor
>
> when resize OffsetIndex or TimeIndex,We should force unmap mmap for linux platform. Rather than waiting fullgc to unmap MappedByteBuffer object
> ##before full gc
> {code}
> {"request":{"mbean":"java.nio:name=mapped,type=BufferPool","type":"read"},"value":{"TotalCapacity":2434496968,"MemoryUsed":2434496968,"Count":5392,"Name":"mapped","ObjectName":{"objectName":"java.nio:name=mapped,type=BufferPool"}},"timestamp":1537945759,"status":200}
> S0 S1 E O M CCS YGC YGCT FGC FGCT GCT
> 0.00 100.00 28.88 4.93 97.64 94.72 24 0.176 0 0.000 0.176
> 0.00 100.00 31.37 4.93 97.64 94.72 24 0.176 0 0.000 0.176
> {code}
> {code}
> jmap -histo:live kafka_pid
> {code}
>
> ###after full gc
> {code}
> S0 S1 E O M CCS YGC YGCT FGC FGCT GCT
> 0.00 0.00 23.22 5.03 97.92 94.93 24 0.176 1 0.617 0.793
> 0.00 0.00 25.70 5.03 97.92 94.93 24 0.176 1 0.617 0.793
> 0.00 0.00 27.86 5.03 97.92 94.93 24 0.176 1 0.617 0.793
> {"request":{"mbean":"java.nio:name=mapped,type=BufferPool","type":"read"},"value":{"TotalCapacity":1868266036,"MemoryUsed":1868266036,"Count":5338,"Name":"mapped","ObjectName":{"objectName":"java.nio:name=mapped,type=BufferPool"}},"timestamp":1537945860,"status":200}
> {code}
> {code}
> def resize(newSize: Int) {
> inLock(lock) {
> val raf = new RandomAccessFile(_file, "rw")
> val roundedNewSize = roundDownToExactMultiple(newSize, entrySize)
> val position = mmap.position
> /* Windows won't let us modify the file length while the file is mmapped :-( */
> if(Os.isWindows)
> forceUnmap(mmap)
> try {
> raf.setLength(roundedNewSize)
> mmap = raf.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, roundedNewSize)
> _maxEntries = mmap.limit / entrySize
> mmap.position(position)
> } finally {
> CoreUtils.swallow(raf.close())
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)