[jira] [Resolved] (IMPALA-301) Row batches should automatically switch to a compact representation

["Tim Armstrong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-301.
----------------------------------
    Resolution: Won't Fix

No longer relevant with the partitioned hash join node.

> Row batches should automatically switch to a compact representation
> -------------------------------------------------------------------
>
>                 Key: IMPALA-301
>                 URL: https://issues.apache.org/jira/browse/IMPALA-301
>             Project: IMPALA
>          Issue Type: Task
>          Components: Backend
>    Affects Versions: Impala 0.7
>            Reporter: Nong Li
>            Priority: Minor
>
> Row batches are (should) be sized to maximize cache coherence.  We don't want to pass batches that are too small (function call overhead) nor too big (or too sparse) which causes bad cache performance.  For streaming operators (e.g. scans or the probe side of a hash join), the output row batch accumulates memory by inheriting memory from the input batch.  If the rows are skipped due to failed conjuncts, this can result in a very sparse representation in the output row batch.  We should, instead, compact the row batch (copy the sparse memory into a dense one) when things are "sparse enough".



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)