You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Manu Zhang (Jira)" <ji...@apache.org> on 2022/05/31 05:12:00 UTC

[jira] [Created] (SPARK-39344) Only disable bucketing when autoBucketedScan is enabled if bucket columns are not in scan output

Manu Zhang created SPARK-39344:
----------------------------------

             Summary: Only disable bucketing when autoBucketedScan is enabled if bucket columns are not in scan output
                 Key: SPARK-39344
                 URL: https://issues.apache.org/jira/browse/SPARK-39344
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.3.0
            Reporter: Manu Zhang


Currently, bucketing was disabled when bucket columns are not in scan output after https://github.com/apache/spark/pull/27924. It break existing applications whose input size is huge by creating too many FilePartitions and causing driver hang. And it cannot be switched off. This is to propose merging the rule into DisableUnnecessaryBucketedScan.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org