You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Denys Kuzmenko (Jira)" <ji...@apache.org> on 2026/10/15 18:17:00 UTC

[jira] [Updated] (HIVE-30182) Parquet: a leaf without its runtime value drops the whole

     [ https://issues.apache.org/jira/browse/HIVE-30182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Denys Kuzmenko updated HIVE-30182:
----------------------------------
    Component/s: Parquet
                 Query Processor
                     (was: llap)
                     (was: Query Planning)

> Parquet: a leaf without its runtime value drops the whole
> ---------------------------------------------------------
>
>                 Key: HIVE-30182
>                 URL: https://issues.apache.org/jira/browse/HIVE-30182
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Parquet, Query Processor
>            Reporter: Denys Kuzmenko
>            Priority: Major
>
> LlapDecider raises the minimum reducers of an auto-reduced vertex to hive.tez.llap.min.reducer.per.executor (0.33) x
> the cluster's executors (adjustAutoParallelism), so auto-reduce does not shrink a reducer to a handful of tasks on a
> large cluster. It applies the floor only to a ReduceWork: a reduce-side merge join is a MergeJoinWork, whose main and
> merged reduce works feed one vertex through their edges, and is skipped.
> TPC-DS q51 at 10 TB on 24 LLAP daemons (288 executors): the FULL OUTER merge join vertex ran 13 tasks while its input
> reducers and the PTF after it ran 253; the floor would have kept it at 96 or more.
> Fix: apply the floor to the merge join's main work and each merged reduce work (their edge properties are the ones
> the vertex's edges use). TestLlapDecider covers it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)