You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Tishj (via GitHub)" <gi...@apache.org> on 2026/05/30 21:50:07 UTC
[I] Spec: Avro key-value metadata "schema" key is not matching implementations [iceberg]
Tishj opened a new issue, #16624:
URL: https://github.com/apache/iceberg/issues/16624
In the code, this key is `"iceberg.schema"`
See https://github.com/apache/iceberg/blame/f6740364c46afb811fd57c8ca965555600f644bc/core/src/main/java/org/apache/iceberg/avro/Avro.java#L203
The spec mentions the `"schema"` key should be written:
See https://iceberg.apache.org/spec/#manifests
> JSON representation of the table schema at the time the manifest was written
But it looks like in reality, this key should be `"iceberg.schema"`
I propose the spec gets changed to fix this inaccuracy
As this line of code in the Avro.java file is 8 years old at this point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Spec: Avro key-value metadata "iceberg.schema" key is missing [iceberg]
Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj closed issue #16624: Spec: Avro key-value metadata "iceberg.schema" key is missing
URL: https://github.com/apache/iceberg/issues/16624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Spec: Avro key-value metadata "iceberg.schema" key is missing [iceberg]
Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #16624:
URL: https://github.com/apache/iceberg/issues/16624#issuecomment-4586734894
Sorry for the noise, I think we can close this issue
Some more context for those interested.
When first implementing the Avro writing portion in duckdb-iceberg, we looked at produced Avro files by other writers.
To the best of my knowledge (it's been a while), these contained `iceberg.schema` but *not* `schema`.
This resulted in us interpreting the spec's:
> JSON representation of the table schema at the time the manifest was written
To mean: the schema of the manifest file.
So we implemented `schema` wrong because of this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Spec: Avro key-value metadata "schema" key is not matching implementations [iceberg]
Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #16624:
URL: https://github.com/apache/iceberg/issues/16624#issuecomment-4585002192
So it seems that *both* are written
But `iceberg.schema"` isn't listed in the spec
Then the new proposal is to add it to the spec, because in reality many engines will require this field, as they are modeled based on Spark behavior, not the spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Spec: Avro key-value metadata "schema" key is not matching implementations [iceberg]
Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #16624:
URL: https://github.com/apache/iceberg/issues/16624#issuecomment-4584994345
Nvm I'm confused again, after finding this: https://github.com/apache/iceberg/blob/f6740364c46afb811fd57c8ca965555600f644bc/core/src/main/java/org/apache/iceberg/ManifestWriter.java#L297
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org