You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "badalprasadsingh (via GitHub)" <gi...@apache.org> on 2026/08/18 21:02:09 UTC
[I] Manifest: Partition logical-type decoding panics and mis-scales milliseconds [iceberg-go]
badalprasadsingh opened a new issue, #1847:
URL: https://github.com/apache/iceberg-go/issues/1847
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
Right now, `dataFile.convertAvroValueToIcebergType` asserts `v.(int64)` unchecked on every time and timestamp arm. Per the Avro spec, `twmb/avro` ignores a logical type invalid for its underlying Avro type and decodes the raw primitive - but the parsed schema still reports the `logicalType`.
Reading such a manifest panics. `Partition()` returns no error, so callers cannot recover.
```bash
panic: interface conversion: interface {} is int32, not int64
iceberg-go.(*dataFile).convertAvroValueToIcebergType(manifest.go:2316)
iceberg-go.(*dataFile).Partition(manifest.go:2370)
```
Separately, `time-millis` and `timestamp-millis` are returned as milliseconds, though `iceberg.Time` and `iceberg.Timestamp` count microseconds : 1000× too small, silently corrupting partition filtering.
Reachable only via manifests from foreign writers.
### Status
Currently short on bandwidth to raise a PR for this fix. Contributions are welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Manifest: Partition logical-type decoding panics and mis-scales milliseconds [iceberg-go]
Posted by "badalprasadsingh (via GitHub)" <gi...@apache.org>.
badalprasadsingh commented on issue #1847:
URL: https://github.com/apache/iceberg-go/issues/1847#issuecomment-5359900334
Thanks @tanmayrauth
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Manifest: Partition logical-type decoding panics and mis-scales milliseconds [iceberg-go]
Posted by "badalprasadsingh (via GitHub) via issues" <is...@iceberg.apache.org>.
badalprasadsingh commented on issue #1847:
URL: https://github.com/apache/iceberg-go/issues/1847#issuecomment-6654410112
thanks @ShivamSharma43, you are right!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Manifest: Partition logical-type decoding panics and mis-scales milliseconds [iceberg-go]
Posted by "ShivamSharma43 (via GitHub) via issues" <is...@iceberg.apache.org>.
ShivamSharma43 commented on issue #1847:
URL: https://github.com/apache/iceberg-go/issues/1847#issuecomment-6577061840
It looks like this was fixed by #1861 (merged 2026-08-26). That PR's description says "Closes #1487", which I think was meant to be #1847. #1487 is an unrelated PR, so this issue never closed automatically.
I checked on current `main` (b354505):
- `convertAvroValueToIcebergType` reads the raw value through `avroIntAsInt64` on every time and timestamp arm, which accepts `int32` or `int64`. No unchecked `v.(int64)` assertion is left.
- `time-millis` and `timestamp-millis` now come back in microseconds: the raw value is multiplied by 1000, and decoded values go through `Duration.Microseconds()` / `UnixMicro()`.
- The two regression tests from #1861 in `manifest_time_partition_test.go` pass locally: `TestTimeLogicalPartitionScaling` (all four subtests) and `TestTimestampLogicalOnMismatchedTypeDoesNotPanic`.
Unless I've missed part of the report, I think this can be closed. cc @tanmayrauth @badalprasadsingh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Manifest: Partition logical-type decoding panics and mis-scales milliseconds [iceberg-go]
Posted by "badalprasadsingh (via GitHub) via issues" <is...@iceberg.apache.org>.
badalprasadsingh closed issue #1847: Manifest: Partition logical-type decoding panics and mis-scales milliseconds
URL: https://github.com/apache/iceberg-go/issues/1847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Manifest: Partition logical-type decoding panics and mis-scales milliseconds [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on issue #1847:
URL: https://github.com/apache/iceberg-go/issues/1847#issuecomment-5359532952
Will raise a PR for the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org