You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rambleraptor (via GitHub)" <gi...@apache.org> on 2025/07/09 22:51:43 UTC

[PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

rambleraptor opened a new pull request, #2192:
URL: https://github.com/apache/iceberg-python/pull/2192

   <!--
   Thanks for opening a pull request!
   -->
   
   <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
   Closes #2191 
   
   # Rationale for this change
   apache/iceberg supports these two event types. We should do the same to match the Java implementation and allow users to alter their partition statistics files.
   
   # Are these changes tested?
   Unit test included.
   
   # Are there any user-facing changes?
   - Adds `RemovePartitionStatisticsUpdate` and `SetPartitionStatisticsUpdate` actions.
   
   <!-- In the case of user-facing changes, please add the changelog label. -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "rambleraptor (via GitHub)" <gi...@apache.org>.
rambleraptor commented on code in PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198416689


##########
pyiceberg/table/statistics.py:
##########
@@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id(
     reject_snapshot_id: int,
 ) -> List[StatisticsFile]:
     return [stat for stat in statistics if stat.snapshot_id != reject_snapshot_id]
+
+
+def filter_partition_statistics_by_snapshot_id(

Review Comment:
   I gave this a try and couldn't get it to work without very strange looking type errors / wonky annotations (this could also be my Python-foo failing me). I'm going to keep it as-is if that's alright.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198333647


##########
pyiceberg/table/statistics.py:
##########
@@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id(
     reject_snapshot_id: int,
 ) -> List[StatisticsFile]:
     return [stat for stat in statistics if stat.snapshot_id != reject_snapshot_id]
+
+
+def filter_partition_statistics_by_snapshot_id(

Review Comment:
   nit: wdyt about combing this with the function above?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu merged PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#issuecomment-3071705304

   Thanks for the PR @rambleraptor and @smaheshwar-pltr for the review! :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on code in PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198543820


##########
pyiceberg/table/statistics.py:
##########
@@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id(
     reject_snapshot_id: int,
 ) -> List[StatisticsFile]:
     return [stat for stat in statistics if stat.snapshot_id != reject_snapshot_id]
+
+
+def filter_partition_statistics_by_snapshot_id(

Review Comment:
   FWIW don't think we've released https://github.com/apache/iceberg-python/pull/2146 yet so can maybe rename `StatisticsCommonFields` to `BaseStatisticsFile` but unsure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on code in PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198547493


##########
pyiceberg/table/statistics.py:
##########
@@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id(
     reject_snapshot_id: int,
 ) -> List[StatisticsFile]:
     return [stat for stat in statistics if stat.snapshot_id != reject_snapshot_id]
+
+
+def filter_partition_statistics_by_snapshot_id(

Review Comment:
   Then or otherwise I'd have thought `S = TypeVar("S", bound=BaseStatisticsFile)` may work? But not tested it 😄



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on code in PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#discussion_r2198547493


##########
pyiceberg/table/statistics.py:
##########
@@ -52,3 +52,10 @@ def filter_statistics_by_snapshot_id(
     reject_snapshot_id: int,
 ) -> List[StatisticsFile]:
     return [stat for stat in statistics if stat.snapshot_id != reject_snapshot_id]
+
+
+def filter_partition_statistics_by_snapshot_id(

Review Comment:
   Then or otherwise I'd have thought `S = TypeVar("S", bound=BaseStatisticsFile)` (or `StatisticsCommonFields` without rename) may work? But not tested it 😄



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Add RemovePartitionStatisticsUpdate and SetPartitionStatisticsUpdate [iceberg-python]

Posted by "rambleraptor (via GitHub)" <gi...@apache.org>.
rambleraptor commented on PR #2192:
URL: https://github.com/apache/iceberg-python/pull/2192#issuecomment-3058503363

   @kevinjqliu please merge whenever you're ready!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org