[jira] [Created] (IMPALA-15380) Estimate UNNEST node cardinality using sampled collection sizes

["Abhishek Rawat (Jira)" <ji...@apache.org>]

Abhishek Rawat created IMPALA-15380:
---------------------------------------

             Summary: Estimate UNNEST node cardinality using sampled collection sizes
                 Key: IMPALA-15380
                 URL: https://issues.apache.org/jira/browse/IMPALA-15380
             Project: IMPALA
          Issue Type: Task
            Reporter: Abhishek Rawat
            Assignee: Xuebin Su


{{PlannerContext.java uses a constant AVG_COLLECTION_SIZE which can result in over/under estimating cardinality. }}
{code:java}
  // Assumed average number of items in a nested collection, since we currently have no
  // statistics on nested fields. The motivation for this constant is to avoid
  // pathological plan choices that could result from a SubplanNode having an unknown
  // cardinality (due to UnnestNodes not knowing their cardinality), or from a ScanNode
  // significantly underestimating its output cardinality because intermediate collections
  // are not accounted for at all. For example, we will place a table ref plan with a
  // SubplanNode on the build side of a join due to an unknown cardinality if the other
  // input is a base table scan with stats.
  // The constant value was chosen arbitrarily to not be "too high" or "too low".
  // TODO: Compute stats for nested types and pick them up here.
  public static final long AVG_COLLECTION_SIZE = 10; {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org