[I] Support full-data-file delete Iceberg changelog tasks in native scan [auron]

["weimingdiit (via GitHub)" <gi...@apache.org>]

weimingdiit opened a new issue, #2434:
URL: https://github.com/apache/auron/issues/2434

   **Is your feature request related to a problem? Please describe.**
   
   Auron native Iceberg changelog scan currently supports insert-only changelog tasks.
   
   When a changelog range contains full-data-file delete tasks, Auron falls back to Spark even though this case does not require row-level delete-file handling. The deleted data file can be scanned directly and changelog metadata such as `_change_type = DELETE` can be materialized from the Iceberg changelog task.
   
   **Describe the solution you'd like**
   
   Support native execution for Iceberg `DeletedDataFileScanTask` when:
   
   - the changelog operation is `DELETE`
   - `existingDeletes()` is empty
   
   Keep the support boundary conservative:
   
   - continue supporting `AddedRowsScanTask` with `INSERT` and empty deletes
   - support only full-data-file delete changelog tasks
   - keep falling back for row-level deletes, position deletes, equality deletes, update changelog tasks, mixed file formats, and delete tasks with existing delete files
   
   **Describe alternatives you've considered**
   
   The alternative is to keep falling back to Spark for all delete changelog tasks. That is safe, but it leaves a native execution gap for simple full-data-file delete changelog scans.
   
   **Additional context**
   
   This change does not attempt to support position deletes, equality deletes, or update changelog operations. Those require separate handling and should be implemented independently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Support full-data-file delete Iceberg changelog tasks in native scan [auron]

["slfan1989 (via GitHub)" <gi...@apache.org>]

slfan1989 closed issue #2434: Support full-data-file delete Iceberg changelog tasks in native scan
URL: https://github.com/apache/auron/issues/2434


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org