You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "puchengy (via GitHub)" <gi...@apache.org> on 2023/09/22 04:06:49 UTC
[GitHub] [iceberg] puchengy opened a new issue, #8619: Python: support get partition table with filter
puchengy opened a new issue, #8619:
URL: https://github.com/apache/iceberg/issues/8619
### Feature Request / Improvement
Support get partition tables, especially with filter.
This is especially helpful for airflow sensor job.
### Query engine
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on issue #8619: Python: support get partition table with filter
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #8619:
URL: https://github.com/apache/iceberg/issues/8619#issuecomment-1730859760
@puchengy Can you provide more context? Would you like to block a partition using an Airflow sensor? I've been thinking about this for a while. For the CDC use case we have a property that we block on: https://github.com/apache/airflow/pull/32786 We could also check if a particular partition exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Python: support get partition table with filter [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #8619: Python: support get partition table with filter
URL: https://github.com/apache/iceberg/issues/8619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Python: support get partition table with filter [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #8619:
URL: https://github.com/apache/iceberg/issues/8619#issuecomment-1742780206
@puchengy Could you maybe elaborate on the use case in https://github.com/apache/iceberg-python/issues/24? We've moved PyIceberg to a separate repository.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] puchengy commented on issue #8619: Python: support get partition table with filter
Posted by "puchengy (via GitHub)" <gi...@apache.org>.
puchengy commented on issue #8619:
URL: https://github.com/apache/iceberg/issues/8619#issuecomment-1730777580
@Fokko Hi, may I know if we have all the building blocks to implement this feature? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] puchengy commented on issue #8619: Python: support get partition table with filter
Posted by "puchengy (via GitHub)" <gi...@apache.org>.
puchengy commented on issue #8619:
URL: https://github.com/apache/iceberg/issues/8619#issuecomment-1731641358
@puchengy Hi Fokko, so you are saying instead of doing partition compute, we want to store what partitions is produced by a given snapshot in the snapshot summary metadata? If so, how do we deal with snapshot expiration cases (this might also be a question to your Airflow PR as well).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org