[PR] fix(rust/sedona-datasource): merge a pushed-down projection with the existing one [sedona-db]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu opened a new pull request, #1454:
URL: https://github.com/apache/sedona-db/pull/1454

   Closes #1405.
   
   ## What changes
   
   Reading through an external format such as `read_pyogrio()`, a query that both selects a subset of columns and renames one, such as `SELECT v, geom AS geometry FROM t`, failed in the `ProjectionPushdown` optimizer rule with a schema mismatch (selecting the subset without renaming, or renaming with every column, worked).
   
   Column pruning first narrows the scan to the selected columns; the renaming projection is then pushed down on top of it, and its column indices refer to the narrowed output. `ExternalFileSource::try_pushdown_projection` split that projection against the file schema, so it read the wrong columns. It now merges the new projection with the one already pushed down (`ProjectionExprs::try_merge`), as DataFusion's own file sources do.
   
   In `sedonadb-geopandas`, this lets `read_file()` rename the geometry column to `geometry` as GeoPandas does (a follow-up once a release includes it).
   
   ## Testing
   
   - Rust: a scan of the crate's test format selecting two columns not starting at the first, one of them renamed. It failed before (the projected column `filter_count` was matched against the file's first column `src`) and now returns the expected names and types.
   - Python: the reproduction from #1405 through `read_pyogrio()` on a GeoPackage, by SQL and by `rename()` plus a column subset.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(rust/sedona-datasource): merge a pushed-down projection with the existing one [sedona-db]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu merged PR #1454:
URL: https://github.com/apache/sedona-db/pull/1454


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org