You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2015/06/03 13:28:49 UTC
[jira] [Resolved] (TEZ-2514) Refactor PipelinedSorter::collect for
inlining
[ https://issues.apache.org/jira/browse/TEZ-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajesh Balamohan resolved TEZ-2514.
-----------------------------------
Resolution: Not A Problem
Microbenchmarked with JMH after refactoring. No significant gains observed. Marking as "Not a problem".
> Refactor PipelinedSorter::collect for inlining
> ----------------------------------------------
>
> Key: TEZ-2514
> URL: https://issues.apache.org/jira/browse/TEZ-2514
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
>
> Used JDK 1.8 and ran a sample job which exercised PipelinedSorter code path. Need to refactor collect() call (which is in hot path) so that it can be inlined.
> {noformat}
> @ 143 org.apache.hadoop.hive.ql.exec.tez.TezProcessor$TezKVOutputCollector::collect (10 bytes) inline (hot)
> \-> TypeProfile (6604/6604 counts) = org/apache/hadoop/hive/ql/exec/tez/TezProcessor$TezKVOutputCollector
> @ 6 org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput$1::write (13 bytes) inline (hot)
> @ 9 org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter::write (22 bytes) inline (hot)
> \-> TypeProfile (6656/6656 counts) = org/apache/tez/runtime/library/common/sort/impl/PipelinedSorter
> @ 13 org.apache.tez.mapreduce.partition.MRPartitioner::getPartition (31 bytes) inline (hot)
> \-> TypeProfile (6656/6656 counts) = org/apache/tez/mapreduce/partition/MRPartitioner
> @ 25 org.apache.hadoop.mapred.lib.HashPartitioner::getPartition (10 bytes) inline (hot)
> \-> TypeProfile (6656/6656 counts) = org/apache/hadoop/hive/ql/io/DefaultHivePartitioner
> @ 1 org.apache.hadoop.hive.ql.io.HiveKey::hashCode (40 bytes) inline (hot)
> \-> TypeProfile (6656/6656 counts) = org/apache/hadoop/hive/ql/io/HiveKey
> @ 18 org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter::collect (406 bytes) hot method too big
> @ 5 org.apache.tez.runtime.library.common.comparator.TezBytesComparator::getProxy (99 bytes) inline (hot)
> @ 3 org.apache.hadoop.io.BytesWritable::getLength (5 bytes) accessor
> @ 8 org.apache.hadoop.io.BytesWritable::getBytes (5 bytes) accessor
> @ 3 org.apache.hadoop.io.BytesWritable::getLength (5 bytes) accessor
> @ 8 org.apache.hadoop.io.BytesWritable::getBytes (5 bytes) accessor
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)