[jira] [Work started] (IMPALA-12192) Scan fragment parallelism is limited by its ScanNode cost.

["Riza Suminto (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on IMPALA-12192 started by Riza Suminto.
---------------------------------------------
> Scan fragment parallelism is limited by its ScanNode cost.
> ----------------------------------------------------------
>
>                 Key: IMPALA-12192
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12192
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>            Reporter: Riza Suminto
>            Assignee: Riza Suminto
>            Priority: Major
>
> IMPALA-12091 implement processing cost for ScanNode. Scan fragment parallelism is capped by this ScanNode cost, originally intended to prevent scanner over-parallelization (small files problem).
> However, there is a bug in the implementation where scan fragment parallelism will always be limited solely by the ScanNode cost. If ScanNode is colocated with other query node operator that have higher processing cost, Planner will not scale it up beyond what is allowed by the ScanNode cost. Scan fragment should be allowed to scale up, as long as it is within total processing cost of the fragment and number of effective scan ranges.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org