You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Jesus Camacho Rodriguez (Jira)" <ji...@apache.org> on 2020/03/03 22:38:00 UTC
[jira] [Comment Edited] (HIVE-22962) Reuse HiveRelFieldTrimmer
instance across queries
[ https://issues.apache.org/jira/browse/HIVE-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050612#comment-17050612 ]
Jesus Camacho Rodriguez edited comment on HIVE-22962 at 3/3/20 10:37 PM:
-------------------------------------------------------------------------
Latest patch includes a very simple benchmark {{FieldTrimmerBench}} that applies the trimmer on a 10 operator plan repeatedly.
{noformat}
Benchmark Mode Cnt Score Error Units
FieldTrimmerBench.baseRelFieldTrimmer thrpt 10 0.088 ± 0.002 ops/us
FieldTrimmerBench.modBaseRelFieldTrimmer thrpt 10 8.292 ± 0.138 ops/us
FieldTrimmerBench.hiveRelFieldTrimmer thrpt 10 10.182 ± 0.117 ops/us
FieldTrimmerBench.baseRelFieldTrimmer avgt 10 10.548 ± 0.148 us/op
FieldTrimmerBench.modBaseRelFieldTrimmer avgt 10 0.116 ± 0.002 us/op
FieldTrimmerBench.hiveRelFieldTrimmer avgt 10 0.109 ± 0.001 us/op
{noformat}
- baseRelFieldTrimmer : Calcite implementation that instantiates trimmer on every call and thus does not exploit method resolution caching. It uses reflection.
- modBaseRelFieldTrimmer : Modified implementation that can use the same trimmer instance and thus exploits method resolution caching. It uses reflection.
- hiveRelFieldTrimmer : Modified implementation that can use the same trimmer instance and thus exploits method resolution caching. It uses lambda metafactory.
was (Author: jcamachorodriguez):
Latest patch includes a very simple benchmark {{FieldTrimmerBench}} that applies the trimmer on a 10 operator plan repeatedly.
{noformat}
Benchmark Mode Cnt Score Error Units
FieldTrimmerBench.baseRelFieldTrimmer thrpt 10 0.088 ± 0.002 ops/us
FieldTrimmerBench.hiveRelFieldTrimmer thrpt 10 10.182 ± 0.117 ops/us
FieldTrimmerBench.modBaseRelFieldTrimmer thrpt 10 8.292 ± 0.138 ops/us
FieldTrimmerBench.baseRelFieldTrimmer avgt 10 10.548 ± 0.148 us/op
FieldTrimmerBench.hiveRelFieldTrimmer avgt 10 0.109 ± 0.001 us/op
FieldTrimmerBench.modBaseRelFieldTrimmer avgt 10 0.116 ± 0.002 us/op
{noformat}
- baseRelFieldTrimmer : Calcite implementation that instantiates trimmer on every call and thus does not exploit method resolution caching. It uses reflection.
- modBaseRelFieldTrimmer : Modified implementation that can use the same trimmer instance and thus exploits method resolution caching. It uses reflection.
- hiveRelFieldTrimmer : Modified implementation that can use the same trimmer instance and thus exploits method resolution caching. It uses lambda metafactory.
> Reuse HiveRelFieldTrimmer instance across queries
> -------------------------------------------------
>
> Key: HIVE-22962
> URL: https://issues.apache.org/jira/browse/HIVE-22962
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
> Attachments: HIVE-22962.01.patch, HIVE-22962.02.patch, HIVE-22962.patch
>
>
> Currently we create multiple {{HiveRelFieldTrimmer}} instances per query. {{HiveRelFieldTrimmer}} uses a method dispatcher that has a built-in caching mechanism: given a certain object, it stores the method that was called for the object class. However, by instantiating the trimmer multiple times per query and across queries, we create a new dispatcher with each instantiation, thus effectively removing the caching mechanism that is built within the dispatcher.
> This issue is to reutilize the same {{HiveRelFieldTrimmer}} instance within a single query and across queries.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)