[jira] Updated: (HIVE-1751) Optimize ColumnarStructObjectInspector.getStructFieldData()

["Siying Dong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siying Dong updated HIVE-1751:
------------------------------

    Attachment: HIVE-1751.1.patch

ExprNodeColumnEvaluator.evaluate() is very heavily used function. For most queries, it is called multiple time per row. In the group-by query in the benchmark, it is even 10 times per row.

This function call sometimes takes 17%-20% CPU time. Usually ExprNodeColumnEvaluator.evaluate() itself takes 2%-3%, UnionStructObjectInspector.getStructFieldData() itself takes 2%-3%, ColumnarStruct.uncheckedGetField() itself takes 3%.

It's hard to come up with a general solution that reduce the costs in a structual way. I tried to did several small code rewriting and hope we can get slight improvements:

1. nullSequence is not passed in for every call but from constructor
2. Restructure ColumnarStruct a little bit.
3. In ExprNodeColumnEvaluator, makes the single level special case, which in most of the time is the common case when referring a column.

When trying to optimize functions which already only take 3%, it's hard to verify the performance enhancement since experiments anyway have slight variation eveyr time.

For 1 and 2, I think they anyway make code better readable. I ran many times, and consistently see about 1% improvement too.
3 might make code less readable, but I see about 5% improvement from some simple group-by query.


> Optimize ColumnarStructObjectInspector.getStructFieldData()
> -----------------------------------------------------------
>
>                 Key: HIVE-1751
>                 URL: https://issues.apache.org/jira/browse/HIVE-1751
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Siying Dong
>         Attachments: HIVE-1751.1.patch
>
>
> ColumnarStructObjectInspector.getStructFieldData() is a heavy used function and is expensive.
> By optimizing this function, including ColumnarStruct.uncheckedGetField() called by it, most queries can benefit from it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.