You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "mattfaltyn (via GitHub)" <gi...@apache.org> on 2026/08/19 08:55:37 UTC
[I] Hour transform rejects Arrow nanosecond timestamp arrays [iceberg-rust]
mattfaltyn opened a new issue, #3025:
URL: https://github.com/apache/iceberg-rust/issues/3025
### Apache Iceberg Rust version
`main` at `7d2dfce494c94e75f726fe52c6c96272b6cc5b32` and `v0.10.1`.
### Describe the bug
The Arrow implementation of the `hour` partition transform rejects nanosecond timestamp arrays, even though `Transform::Hour::result_type` accepts `timestamp_ns` and `timestamptz_ns`, and the literal implementation handles both types.
This reaches production write paths through `PartitionValueCalculator`: calculating partitions for a table with an `hour(event_time)` partition field fails when `event_time` is an Arrow `Timestamp(Nanosecond, ...)` column.
The transform currently handles only `TimestampMicrosecondArray` in `Hour::transform`. The existing `hour_timestamp_nano` helper is already used by `transform_literal`, so the array behavior is inconsistent with the public type contract and literal behavior.
### To reproduce
```rust
use std::sync::Arc;
use arrow_array::{RecordBatch, TimestampNanosecondArray};
use iceberg::arrow::{PartitionValueCalculator, schema_to_arrow_schema};
use iceberg::spec::{NestedField, PartitionSpec, PrimitiveType, Schema, Transform, Type};
let table_schema = Arc::new(
Schema::builder()
.with_fields(vec![NestedField::required(
1,
"event_time",
Type::Primitive(PrimitiveType::TimestampNs),
)
.into()])
.build()?,
);
let partition_spec = PartitionSpec::builder(table_schema.clone())
.add_partition_field("event_time", "event_hour", Transform::Hour)?
.build()?;
let calculator = PartitionValueCalculator::try_new(&partition_spec, &table_schema)?;
let batch = RecordBatch::try_new(
Arc::new(schema_to_arrow_schema(&table_schema)?),
vec![Arc::new(TimestampNanosecondArray::from(vec![
0,
3_600_000_000_000,
]))],
)?;
calculator.calculate(&batch)?;
```
Actual result:
```text
FeatureUnsupported => Unsupported data type for hour transform: Timestamp(Nanosecond, None)
```
The equivalent microsecond Arrow array succeeds and produces `[0, 1]`.
### Expected behavior
The nanosecond array should produce hour partition values `[0, 1]`, matching the equivalent microsecond array and the supported `timestamp_ns` literal transform.
### Willingness to contribute
I can contribute a focused fix and regression test independently.
### AI disclosure
Codex assisted with repository inspection, duplicate-history searches, reproduction scaffolding, and drafting this report. I reproduced the failure against a clean checkout of current upstream and reviewed the relevant transform and partition-calculation paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Hour transform rejects Arrow nanosecond timestamp arrays [iceberg-rust]
Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk closed issue #3025: Hour transform rejects Arrow nanosecond timestamp arrays
URL: https://github.com/apache/iceberg-rust/issues/3025
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org