You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Heatao (via GitHub)" <gi...@apache.org> on 2026/01/16 07:48:22 UTC
[I] DELETE operations don't apply custom snapshot properties from session config [iceberg]
Heatao opened a new issue, #15060:
URL: https://github.com/apache/iceberg/issues/15060
### Apache Iceberg version
main (development)
### Query engine
Spark
### Please describe the bug 🐞
### Description
Custom snapshot properties set via Spark session config (e.g., `spark.sql.iceberg.snapshot-property.custom-key`) are not applied to DELETE operations, while they work correctly for INSERT, UPDATE, and MERGE operations.
### Expected Behavior
DELETE should respect custom snapshot properties from session config, consistent with other DML operations.
### Actual Behavior
DELETE operations silently ignore custom snapshot properties.
### Steps to Reproduce
```java
spark.conf().set("spark.sql.iceberg.snapshot-property.my-key", "my-value");
// INSERT works - snapshot has my-key ✅
sql("INSERT INTO table VALUES (1, 'a')");
// DELETE doesn't work - snapshot missing my-key ❌
sql("DELETE FROM table WHERE id = 1");
```
### Root Cause
In SparkTable.deleteFromRowFilter(), the code doesn't call SparkWriteConf.extraSnapshotMetadata() to apply session properties, unlike insert/update paths.
### Impact
Users relying on custom snapshot properties for tracking (e.g., audit trails, job metadata) will have incomplete information for DELETE operations.
### Related
- Feature introduced in #14545
- Affects Spark 4.0 and 4.1
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DELETE operations don't apply custom snapshot properties from session config [iceberg]
Posted by "Heatao (via GitHub)" <gi...@apache.org>.
Heatao commented on issue #15060:
URL: https://github.com/apache/iceberg/issues/15060#issuecomment-3817596716
> +1, thanks for flagging.
>
> To clarify: it's only `DELETE` commands that match entire files (and so lead to `delete` Iceberg operation type snapshots) that don't apply the properties. `DELETE FROM` commands resulting in row level operations and `overwrite` snapshots do respect the custom snapshot properties.
+1 Thanks for the detailed clarification!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DELETE operations don't apply custom snapshot properties from session config [iceberg]
Posted by "Heatao (via GitHub)" <gi...@apache.org>.
Heatao commented on issue #15060:
URL: https://github.com/apache/iceberg/issues/15060#issuecomment-3817592916
+1 Thanks for the detailed clarification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DELETE operations don't apply custom snapshot properties from session config [iceberg]
Posted by "yadavay-amzn (via GitHub)" <gi...@apache.org>.
yadavay-amzn commented on issue #15060:
URL: https://github.com/apache/iceberg/issues/15060#issuecomment-4812736631
I would like to pick this up. Plan: propagate the configured snapshot properties (SparkWriteConf.extraSnapshotMetadata()) into the row-level DELETE path in SparkTable.deleteWhere, mirroring how SparkWrite already does it, so file-level DELETEs carry the same custom snapshot metadata as writes. This follows the approach approved on the earlier attempt (#15061). Will put up a PR shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] DELETE operations don't apply custom snapshot properties from session config [iceberg]
Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on issue #15060:
URL: https://github.com/apache/iceberg/issues/15060#issuecomment-3780499603
+1, thanks for flagging.
To clarify: it's only `DELETE` commands that match entire files (and so lead to `delete` Iceberg operation type snapshots) that don't apply the properties. `DELETE FROM` commands resulting in row level operations and `overwrite` snapshots do respect the custom snapshot properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org