[jira] [Created] (IMPALA-12192) Scan fragment parallelism is limited by its ScanNode cost.

["Riza Suminto (Jira)" <ji...@apache.org>]

Riza Suminto created IMPALA-12192:
-------------------------------------

             Summary: Scan fragment parallelism is limited by its ScanNode cost.
                 Key: IMPALA-12192
                 URL: https://issues.apache.org/jira/browse/IMPALA-12192
             Project: IMPALA
          Issue Type: Bug
          Components: Frontend
            Reporter: Riza Suminto
            Assignee: Riza Suminto


IMPALA-12091 implement processing cost for ScanNode. Scan fragment parallelism is capped by this ScanNode cost, originally intended to prevent scanner over-parallelization (small files problem).

However, there is a bug in the implementation where scan fragment parallelism will always be limited solely by the ScanNode cost. If ScanNode is colocated with other query node operator that have higher processing cost, Planner will not scale it up beyond what is allowed by the ScanNode cost. Scan fragment should be allowed to scale up, as long as it is within total processing cost of the fragment and number of effective scan ranges.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org