You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "stevenzwu (via GitHub)" <gi...@apache.org> on 2026/10/22 18:09:01 UTC
[PR] Core: Read total bytes in MapBackedContentStats [iceberg]
stevenzwu opened a new pull request, #18614:
URL: https://github.com/apache/iceberg/pull/18614
MapBackedContentStats still read ContentFile.avgValueSizes() after #18310 replaced stored averages with total_bytes. That method derives the average and drops the total, so a data file wrapped as a v4 TrackedFile lost total_bytes. The wrapper now stores file.totalBytes(), and FieldStats reports hasTotalBytes() and totalBytes(). The deprecated average stays on the FieldStats default.
Follow-up to https://github.com/apache/iceberg/pull/16936#discussion_r4347091655
Test plan:
- `./gradlew :iceberg-core:test --tests org.apache.iceberg.TestMapBackedContentStats --tests org.apache.iceberg.TestTrackedFileAdapters`
---
**AI Disclosure**
- Model: Grok 4.7
- Platform/Tool: Cursor
- Human Oversight: partially reviewed
- Prompt Summary: Update MapBackedContentStats to store total bytes instead of the deprecated average, following Eduard's review comment on PR 16936.
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Read total bytes in MapBackedContentStats [iceberg]
Posted by "amogh-jahagirdar (via GitHub)" <gi...@apache.org>.
amogh-jahagirdar commented on code in PR #18614:
URL: https://github.com/apache/iceberg/pull/18614#discussion_r4364223413
##########
core/src/test/java/org/apache/iceberg/TestMapBackedContentStats.java:
##########
@@ -216,6 +216,8 @@ void countsAndTightBounds() {
assertThatThrownBy(name::nanValueCount)
.isInstanceOf(NullPointerException.class)
.hasMessageContaining("Long.longValue()");
+ assertThat(name.hasTotalBytes()).isTrue();
+ assertThat(name.totalBytes()).isEqualTo(240L);
Review Comment:
Good call, I agree with that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Read total bytes in MapBackedContentStats [iceberg]
Posted by "stevenzwu (via GitHub)" <gi...@apache.org>.
stevenzwu commented on code in PR #18614:
URL: https://github.com/apache/iceberg/pull/18614#discussion_r4353279149
##########
core/src/test/java/org/apache/iceberg/TestMapBackedContentStats.java:
##########
@@ -216,6 +216,8 @@ void countsAndTightBounds() {
assertThatThrownBy(name::nanValueCount)
.isInstanceOf(NullPointerException.class)
.hasMessageContaining("Long.longValue()");
+ assertThat(name.hasTotalBytes()).isTrue();
+ assertThat(name.totalBytes()).isEqualTo(240L);
Review Comment:
it is probably good to verify `hasTotalBytes()` API behavior. so I would keep the line above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Read total bytes in MapBackedContentStats [iceberg]
Posted by "stevenzwu (via GitHub)" <gi...@apache.org>.
stevenzwu commented on PR #18614:
URL: https://github.com/apache/iceberg/pull/18614#issuecomment-6374260437
thanks @amogh-jahagirdar @huaxingao @nastra for the review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Read total bytes in MapBackedContentStats [iceberg]
Posted by "stevenzwu (via GitHub)" <gi...@apache.org>.
stevenzwu merged PR #18614:
URL: https://github.com/apache/iceberg/pull/18614
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Read total bytes in MapBackedContentStats [iceberg]
Posted by "amogh-jahagirdar (via GitHub)" <gi...@apache.org>.
amogh-jahagirdar commented on code in PR #18614:
URL: https://github.com/apache/iceberg/pull/18614#discussion_r4353062804
##########
core/src/test/java/org/apache/iceberg/TestMapBackedContentStats.java:
##########
@@ -216,6 +216,8 @@ void countsAndTightBounds() {
assertThatThrownBy(name::nanValueCount)
.isInstanceOf(NullPointerException.class)
.hasMessageContaining("Long.longValue()");
+ assertThat(name.hasTotalBytes()).isTrue();
+ assertThat(name.totalBytes()).isEqualTo(240L);
Review Comment:
minor: Wouldn't `assertThat(name.totalBytes()).isEqualTo(240L);` just cover it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org