You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ZENOTME (via GitHub)" <gi...@apache.org> on 2023/08/20 15:00:35 UTC
[GitHub] [iceberg-rust] ZENOTME opened a new issue, #36: Read ManifestList, Manifest
ZENOTME opened a new issue, #36:
URL: https://github.com/apache/iceberg-rust/issues/36
For ManifestList, Manifest, I think #29 already as a good example to refer. We can define a unified view in memory and distinguish version1 and version2 in representation for serde.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-rust] Fokko commented on issue #36: Read ManifestList, Manifest
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1685912190
> There is one issue with the upper & lower bounds. The upper and lower bounds are stored as bytes, which can only be interpreted with the schema. So we will not be able to deserialize directly into the unified view. There needs to be a function which takes the schema into account.
In other languages, we keep this as bytes before deserialization. The metrics can become quite large and therefore become pretty expensive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-rust] JanKaul commented on issue #36: Read ManifestList, Manifest
Posted by "JanKaul (via GitHub)" <gi...@apache.org>.
JanKaul commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1685323755
Thanks for taking this on!
There is one issue with the upper & lower bounds. The upper and lower bounds are stored as bytes, which can only be interpreted with the schema. So we will not be able to deserialize directly into the unified view. There needs to be a function which takes the schema into account.
Alternatively we could store the bytes in the unified view, but I think that's not the way to go.
Serialization should be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Read ManifestList, Manifest [iceberg-rust]
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 closed issue #36: Read ManifestList, Manifest
URL: https://github.com/apache/iceberg-rust/issues/36
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-rust] liurenjie1024 commented on issue #36: Read ManifestList, Manifest
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1685604735
Maybe we should complete #37 first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-rust] ZENOTME commented on issue #36: Read ManifestList, Manifest
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1685638262
> There is one issue with the upper & lower bounds. The upper and lower bounds are stored as bytes, which can only be interpreted with the schema. So we will not be able to deserialize directly into the unified view. There needs to be a function which takes the schema into account.
> Alternatively we could store the bytes in the unified view, but I think that's not the way to go.
Yes, I agree with you. Maybe we can store the bytes in representation of serde first. And then take the schema to interpreted them to unified view in memory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Read ManifestList, Manifest [iceberg-rust]
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1855092587
Close by #79
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-rust] ZENOTME commented on issue #36: Read ManifestList, Manifest
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1685307049
I'm glad to prepare PR for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-rust] JanKaul commented on issue #36: Read ManifestList, Manifest
Posted by "JanKaul (via GitHub)" <gi...@apache.org>.
JanKaul commented on issue #36:
URL: https://github.com/apache/iceberg-rust/issues/36#issuecomment-1685652213
You could deserialize into `serde_bytes::ByteBuf` and then use [Literal::try_from_bytes](https://github.com/apache/iceberg-rust/blob/main/crates/iceberg/src/spec/values.rs#L267C12-L267C26).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org