You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ZhendongBai (via GitHub)" <gi...@apache.org> on 2023/05/22 06:26:48 UTC
[GitHub] [iceberg] ZhendongBai opened a new issue, #7677: Is there any way to generate an empty partition without any data or delete files?
ZhendongBai opened a new issue, #7677:
URL: https://github.com/apache/iceberg/issues/7677
### Query engine
spark
### Question
Is there any way to generate an empty partition without any data or delete files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ZhendongBai commented on issue #7677: Is there any way to generate an empty partition without any data or delete files?
Posted by "ZhendongBai (via GitHub)" <gi...@apache.org>.
ZhendongBai commented on issue #7677:
URL: https://github.com/apache/iceberg/issues/7677#issuecomment-1558413651
@RussellSpitzer I saw the Iceberg implementation as you said, thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on issue #7677: Is there any way to generate an empty partition without any data or delete files?
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #7677:
URL: https://github.com/apache/iceberg/issues/7677#issuecomment-1557020619
@ZhendongBai I don't think this is possible today. Technically it is possible, but it doesn't make much sense (you would have a partition in the manifest-list, that would point to a manifest without any entries). What would be the use case of this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ZhendongBai commented on issue #7677: Is there any way to generate an empty partition without any data or delete files?
Posted by "ZhendongBai (via GitHub)" <gi...@apache.org>.
ZhendongBai commented on issue #7677:
URL: https://github.com/apache/iceberg/issues/7677#issuecomment-1557057218
> @ZhendongBai I don't think this is possible today. Technically it is possible, but it doesn't make much sense (you would have a partition in the manifest-list, that would point to a manifest without any entries). What would be the use case of this?
@Fokko In some of our user use cases, the execution of downstream tasks depends on the existence of upstream task result table partitions, without caring about whether the upstream tasks are completed. This is feasible in hive, but migrating to iceberg doesn't work, but we do not want to change user usage habits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ZhendongBai closed issue #7677: Is there any way to generate an empty partition without any data or delete files?
Posted by "ZhendongBai (via GitHub)" <gi...@apache.org>.
ZhendongBai closed issue #7677: Is there any way to generate an empty partition without any data or delete files?
URL: https://github.com/apache/iceberg/issues/7677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #7677: Is there any way to generate an empty partition without any data or delete files?
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #7677:
URL: https://github.com/apache/iceberg/issues/7677#issuecomment-1557357237
In iceberg that doesn't happen @ZhendongBai, because it's an atomic system either the partition exists or it doesn't. More so, partitions only exist within metadata in the context of a committed datafiles. An Iceberg table by definition cannot have a partition without files. A particular partition value only exists as a field in the listing of a data file's metadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org