You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2015/10/26 19:27:27 UTC
[jira] [Updated] (HBASE-14697) Random Read Improvements
[ https://issues.apache.org/jira/browse/HBASE-14697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-14697:
--------------------------
Attachment: flamegraph-32667.svg
Random reads flame graph over a 60 second perf sample.
Done with https://github.com/jrudolph/perf-map-agent and jdk8u60+ https://github.com/jrudolph/perf-map-agent has nice scripts now that make it easy doingh top and flamegraph production.
> Random Read Improvements
> ------------------------
>
> Key: HBASE-14697
> URL: https://issues.apache.org/jira/browse/HBASE-14697
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Attachments: flamegraph-32667.svg
>
>
> I was looking at something else and then got drawn into graphs of where we are spending time doing random reads. Interesting is that most CPU -- ~5% is in metrics adding -- and bloomfilter look ups cost.
> This is an umbrella background issue to host random read improvements as subtasks.
> Here is what perf top shows doing YCSB workloadc (all random read) against a server that is up on a 5-node HDFS cluster hosting 7 regions loaded by YCSB.
> {code}
> 4.91% perf-32667.map [.] Lorg/apache/hadoop/metrics2/lib/MutableHistogram;.add
> 2.27% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Call;.setResponse
> 2.10% perf-32667.map [.] Lorg/apache/hadoop/hbase/io/hfile/CompoundBloomFilter;.checkContains
> 2.06% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/HRegion$RegionScannerImpl;.<init>
> 1.94% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/CallRunner;.run
> 1.89% perf-32667.map [.] Lorg/apache/hadoop/hbase/io/hfile/CombinedBlockCache;.getBlock
> 1.82% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/RSRpcServices;.get
> 1.66% perf-32667.map [.] Lorg/apache/hadoop/hbase/util/Counter;.add
> 1.66% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.getScannersForStoreFiles
> 1.41% perf-32667.map [.] Lorg/apache/hadoop/hbase/util/MurmurHash;.hash
> 1.40% [kernel] [k] _spin_lock
> 1.34% perf-32667.map [.] Lsun/nio/ch/SocketChannelImpl;.read
> 1.32% perf-32667.map [.] Ljava/util/concurrent/locks/ReentrantReadWriteLock$ReadLock;.unlock
> 1.28% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.shouldUseScanner
> 1.27% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFile$Reader;.checkGeneralBloomFilter
> 1.17% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Connection;.processRequest
> 1.15% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Listener$Reader;.processing
> 1.14% perf-32667.map [.] Lsun/nio/ch/SocketChannelImpl;.write
> 1.13% [vdso] [.] 0x00007fffbedff8d4
> 1.11% [kernel] [k] copy_user_generic_string
> 1.11% perf-32667.map [.] Lsun/nio/ch/IOUtil;.write
> 1.05% [nx_nic] [.] 0x000000005f49014a
> 0.96% perf-32667.map [.] Ljava/util/concurrent/ConcurrentHashMap;.replaceNode
> 0.88% perf-32667.map [.] Ljava/util/concurrent/locks/ReentrantReadWriteLock$Sync;.tryAcquireShared
> 0.86% perf-32667.map [.] Ljava/util/concurrent/ConcurrentHashMap;.putVal
> 0.82% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFile$Reader;.passesKeyRangeFilter
> 0.81% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer;.call
> 0.77% perf-32667.map [.] Lorg/apache/hadoop/hbase/protobuf/ProtobufUtil;.toGet
> 0.76% perf-32667.map [.] Lsun/nio/ch/Util;.getTemporaryDirectBuffer
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)