You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "khjoshi94 (via GitHub)" <gi...@apache.org> on 2026/02/28 10:23:46 UTC
[PR] docs: document preserved whitespace behavior in string partition values [iceberg]
khjoshi94 opened a new pull request, #15474:
URL: https://github.com/apache/iceberg/pull/15474
### Overview
This PR adds a new section to the partitioning documentation that explains how Iceberg preserves string partition values exactly as written by the engine, including any leading or trailing whitespace. Because engines rely on strict string equality for partition pruning, whitespace differences can cause partitions to be skipped unexpectedly. This behavior is correct but previously undocumented.
### What’s Included
The new section covers:
- How Iceberg stores string partition values without trimming or normalization
- How strict equality impacts pruning when whitespace differs
- Examples illustrating mismatches between query filters and stored values
- Implications for ingestion pipelines and query behavior
- Best practices for avoiding unintended whitespace
- Troubleshooting steps for diagnosing unexpected empty results
The section is added immediately after “Iceberg’s hidden partitioning” in `docs/docs/partitioning.md`.
### Motivation
Users occasionally encounter unexpected query results due to trailing or leading whitespace in partition values. While this behavior aligns with Iceberg’s design principles, it is not currently documented. Adding this section improves clarity, helps users avoid subtle ingestion issues, and provides guidance for debugging.
### Documentation Impact
This change updates the nightly documentation and will appear under:
```
User Experience → Partitioning
```
No navigation updates are required.
Closes #15427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] docs: document preserved whitespace behavior in string partition values [iceberg]
Posted by "khjoshi94 (via GitHub)" <gi...@apache.org>.
khjoshi94 commented on PR #15474:
URL: https://github.com/apache/iceberg/pull/15474#issuecomment-3991534188
@RussellSpitzer Thanks for your response again.
I understand the concern about adding a full standalone section. My main goal was to surface this behavior because it may be a source of confusion when users hit unexpected pruning results due to whitespace in partition values.
I’m happy to reduce this to a short note or warning callout within the existing partitioning page instead of a full section if that fits better with the style of the docs. Something like:
> "Note: Iceberg preserves string partition values exactly as written, including leading or trailing whitespace. Engines use strict equality for pruning, so ingestion pipelines should canonicalize values to avoid unexpected pruning."
And if you still feel it doesn’t belong in the main docs even in a smaller form, I can close this out and publish it as a blog post instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] docs: document preserved whitespace behavior in string partition values [iceberg]
Posted by "khjoshi94 (via GitHub)" <gi...@apache.org>.
khjoshi94 commented on PR #15474:
URL: https://github.com/apache/iceberg/pull/15474#issuecomment-3984223027
@RussellSpitzer if you please could take a look whenever possible and let me know if you have suggestions based on our conversation on https://github.com/apache/iceberg/issues/15427 . Please do let me know the community recommended guidelines on requesting reviews for PRs if direct tagging is not preferred. Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] docs: document preserved whitespace behavior in string partition values [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #15474:
URL: https://github.com/apache/iceberg/pull/15474#issuecomment-4052619080
I think it's probably better as a blog post about your experience and troubleshooting it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] docs: document preserved whitespace behavior in string partition values [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #15474:
URL: https://github.com/apache/iceberg/pull/15474#issuecomment-3985227430
I'm not sure we really need this as a doc section, it's probably good content as a personal blog post though. I may be biased here but seems very expected to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org