You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Tishj (via GitHub)" <gi...@apache.org> on 2026/05/30 21:50:07 UTC

[I] Spec: Avro key-value metadata "schema" key is not matching implementations [iceberg]

Tishj opened a new issue, #16624:
URL: https://github.com/apache/iceberg/issues/16624

   In the code, this key is `"iceberg.schema"`
   See https://github.com/apache/iceberg/blame/f6740364c46afb811fd57c8ca965555600f644bc/core/src/main/java/org/apache/iceberg/avro/Avro.java#L203
   
   The spec mentions the `"schema"` key should be written:
   See https://iceberg.apache.org/spec/#manifests
   
   > JSON representation of the table schema at the time the manifest was written
   
   But it looks like in reality, this key should be `"iceberg.schema"`
   
   I propose the spec gets changed to fix this inaccuracy
   As this line of code in the Avro.java file is 8 years old at this point


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] Spec: Avro key-value metadata "iceberg.schema" key is missing [iceberg]

Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj closed issue #16624: Spec: Avro key-value metadata "iceberg.schema" key is missing
URL: https://github.com/apache/iceberg/issues/16624


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] Spec: Avro key-value metadata "iceberg.schema" key is missing [iceberg]

Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #16624:
URL: https://github.com/apache/iceberg/issues/16624#issuecomment-4586734894

   Sorry for the noise, I think we can close this issue
   
   Some more context for those interested.
   When first implementing the Avro writing portion in duckdb-iceberg, we looked at produced Avro files by other writers.
   To the best of my knowledge (it's been a while), these contained `iceberg.schema` but *not* `schema`.
   This resulted in us interpreting the spec's:
   > JSON representation of the table schema at the time the manifest was written
   
   To mean: the schema of the manifest file.
   
   So we implemented `schema` wrong because of this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] Spec: Avro key-value metadata "schema" key is not matching implementations [iceberg]

Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #16624:
URL: https://github.com/apache/iceberg/issues/16624#issuecomment-4585002192

   So it seems that *both* are written
   But `iceberg.schema"` isn't listed in the spec
   
   Then the new proposal is to add it to the spec, because in reality many engines will require this field, as they are modeled based on Spark behavior, not the spec.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] Spec: Avro key-value metadata "schema" key is not matching implementations [iceberg]

Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #16624:
URL: https://github.com/apache/iceberg/issues/16624#issuecomment-4584994345

   Nvm I'm confused again, after finding this: https://github.com/apache/iceberg/blob/f6740364c46afb811fd57c8ca965555600f644bc/core/src/main/java/org/apache/iceberg/ManifestWriter.java#L297


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org