You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rami-lv (via GitHub)" <gi...@apache.org> on 2025/10/01 17:45:40 UTC
[I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
rami-lv opened a new issue, #14231:
URL: https://github.com/apache/iceberg/issues/14231
### Query engine
pyiceberg
### Question
I altered my Iceberg table to change the Avro codec via table property:
```
ALTER TABLE mycat.db.table
SET TBLPROPERTIES ('write.avro.compression-codec'='snappy');
```
After this, I ran:
`
CALL mycat.system.rewrite_manifests('db.table');
`
But when I inspected the rewritten manifest files, they still had **deflate** as their codec.
### Expected behavior
I expected the table property write.avro.compression-codec to control the compression codec used for manifest files (just like it does for data files).
### Questions:
- Is it correct that write.avro.compression-codec does not apply to manifest files?
- If so, is there currently a supported way to configure the codec for manifest/metadata files?
- Are manifests always written with deflate by design?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
Posted by "rami-lv (via GitHub)" <gi...@apache.org>.
rami-lv commented on issue #14231:
URL: https://github.com/apache/iceberg/issues/14231#issuecomment-3361370251
@nastra It is not only related to pyiceberg, this can apply to any engine parsing the manifest files!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #14231: write.avro.compression-codec has no effect on manifest file compression
URL: https://github.com/apache/iceberg/issues/14231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #14231:
URL: https://github.com/apache/iceberg/issues/14231#issuecomment-4166591202
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #14231:
URL: https://github.com/apache/iceberg/issues/14231#issuecomment-4248281299
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on issue #14231:
URL: https://github.com/apache/iceberg/issues/14231#issuecomment-3361582150
@rami-lv my bad, I didn't really read past `pyiceberg` in the issue and just assumed you're reporting an issue for `pyiceberg`.
But it is true that we don't set any avro compression codec when writing manifests in https://github.com/apache/iceberg/blob/11f3dcb3011fe5e37a4ec57fbc7a0c859ad5f24e/core/src/main/java/org/apache/iceberg/ManifestWriter.java#L330-L341. I don't know the historical reason for that but right now we don't really pass any of the table properties down to the place where manifests are being written
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] write.avro.compression-codec has no effect on manifest file compression [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on issue #14231:
URL: https://github.com/apache/iceberg/issues/14231#issuecomment-3361341729
Since this is pyiceberg-related I'd suggest to move this to https://github.com/apache/iceberg-python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org