You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "steveloughran (via GitHub)" <gi...@apache.org> on 2026/01/20 17:50:34 UTC
[I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
steveloughran opened a new issue, #15093:
URL: https://github.com/apache/iceberg/issues/15093
### Feature Request / Improvement
Core: HadoopFileIO to take list of filesystem schemas to enable trash for
#14501 uses trash policy when the target path resolves to local or hdfs, which it does by looking at the classname of the FS instance.
That
* Breaks distributions which don't have hdfs on the classpath (for example Azure HD/Insight)
* Adds the overhead of instantiating Trash policies on every single delete
* Doesn't allow trash to be applied to other filesystems, or disapplied to localfs (which doesn't have any remote cleaner in the background).
Proposed:
* add option `iceberg.hadoop.trash.schemas` to take a list of filesystems, (defaults "hdfs" and "viewfs") and only apply if there's a match
* add test which makes file the schema, verifies it can be enabled/disabled.
* document. (where?). Maybe also update SupportsBulkOperations javadocs to mention deletion may be replaced by trash.
### Query engine
None
### Willingness to contribute
- [ ] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue closed issue #15093: Core: HadoopFileIO to take list of filesystem schemas to enable trash for
URL: https://github.com/apache/iceberg/issues/15093
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #15093:
URL: https://github.com/apache/iceberg/issues/15093#issuecomment-3782009399
You may add one section in https://iceberg.apache.org/docs/nightly/fileio/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on issue #15093:
URL: https://github.com/apache/iceberg/issues/15093#issuecomment-3774206019
I can do the code and test, just need reviews and a suggestion as to where the documentation should go. there is no obvious place in the docs that I can see to just add a new entry
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on issue #15093:
URL: https://github.com/apache/iceberg/issues/15093#issuecomment-3947524139
The original commit was just reverted in https://github.com/apache/iceberg/pull/15386, so I'm going to close this one. Thanks for the fix and for joining the discussion on the dev list, @steveloughran!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on issue #15093:
URL: https://github.com/apache/iceberg/issues/15093#issuecomment-3800646578
@manuzhang done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Core: HadoopFileIO to take list of filesystem schemas to enable trash for [iceberg]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on issue #15093:
URL: https://github.com/apache/iceberg/issues/15093#issuecomment-3950899449
thanks for the rollback, it was only adding complexity
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org