You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Norio Akagi (Jira)" <ji...@apache.org> on 2026/05/26 02:16:00 UTC

[jira] [Created] (SPARK-57064) Bucketing rules should match on FileSourceScanLike trait instead of FileSourceScanExec

Norio Akagi created SPARK-57064:
-----------------------------------

             Summary: Bucketing rules should match on FileSourceScanLike trait instead of FileSourceScanExec
                 Key: SPARK-57064
                 URL: https://issues.apache.org/jira/browse/SPARK-57064
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 5.0.0
            Reporter: Norio Akagi


**Summary:** `Bucketing rules should match on FileSourceScanLike trait instead of FileSourceScanExec`

**Description:**

### What changes were proposed in this pull request?

`DisableUnnecessaryBucketedScan` and `CoalesceBucketsInJoin` pattern-match on the concrete class `FileSourceScanExec` in several read-only match sites where only trait-level fields (`bucketedScan`, `relation`, `optionalNumCoalescedBuckets`) are accessed. The `FileSourceScanLike` trait already declares all of these fields, so the matches can safely be widened.

This PR changes 3 match sites from `FileSourceScanExec` to `FileSourceScanLike`:

- `DisableUnnecessaryBucketedScan.apply` — the `hasBucketedScan` existence check
- `ExtractJoinWithBuckets.hasScanOperation` — the bucket spec existence check
- `ExtractJoinWithBuckets.getBucketSpec` — the bucket spec extraction

Two match sites that call `.copy()` (a case-class-specific method) are intentionally left on `FileSourceScanExec`.

### Why are the changes needed?

Third-party columnar execution plugins (Gluten, Comet, RAPIDS) replace `FileSourceScanExec` with their own scan operators that extend `FileSourceScanLike`. With the current concrete-class matches, these plugins' scan operators are invisible to the bucketing rules — `DisableUnnecessaryBucketedScan` never finds them and `ExtractJoinWithBuckets` never extracts their bucket specs.

This is the same class of issue addressed by SPARK-32332 and SPARK-32430 (AQE hardcoding concrete classes instead of traits), but in the bucketing physical rules which were not covered by those fixes.

### Does this PR introduce _any_ user-facing change?

No. `FileSourceScanExec` already extends `FileSourceScanLike`, so behavior is unchanged for vanilla Spark. Plugins that extend `FileSourceScanLike` will now be recognized by the bucketing rules.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org