[PR] [core] Align BIGINT and timestamp schema evolution with Java [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #1107:
URL: https://github.com/apache/paimon-rust/pull/1107

   ## What is the purpose of the change?
   
   Align the existing BIGINT/timestamp schema-evolution paths with Java casts. PyPaimon already reads these type changes, but Arrow's generic cast interprets timestamps as physical ticks: for example `42001ms` becomes BIGINT `42001`, while Java returns epoch seconds `42`.
   
   ## Brief change log
   
   - Convert TIMESTAMP and TIMESTAMP_LTZ to BIGINT using epoch seconds and floor division for pre-epoch values.
   - Match Java's LTZ Timestamp local-date-time round trip, including its int epoch day.
   - Move the existing numeric-to-ordinary-TIMESTAMP conversion into the shared evolution cast module and apply Java signed-long wrapping to the intermediate `seconds * 1000` milliseconds. Preserve lenient NULLs when Arrow's target unit cannot represent the result.
   - Reuse recursive evolution and predicate/statistics paths without adding a public API or dependency.
   - Cover Arrow units, slices, NULLs, Java boundary outputs, nullable ROW/ARRAY/MAP values, and mixed old/new files with predicates and projection across append, primary-key and data-evolution tables.
   
   ## Verifying this change
   
   - Reproduced the old timestamp-to-BIGINT unit error in both the core test and a REST Native integration test.
   - `cargo test -p paimon --lib arrow::`: 522 passed, 1 ignored; after adding the final nullable-container test, all 28 nested-evolution tests passed.
   - `cargo test -p paimon --test schema_evolution_cast_test`: 2 passed (each covers append, PK and data evolution).
   - `cargo +1.98.0 fmt --all -- --check`
   - `cargo +1.98.0 clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings`
   - Java `CastExecutorTest#testTimestampToNumeric+testTimestampToNumericPreEpoch`: 2 passed. Direct calls to Java CastExecutors also confirmed the long-wrapping and extreme-LTZ golden values.
   - Rebuilt the Python extension from this branch: 166 focused Python schema-evolution tests, 112 existing read/projection tests, and final MAP key projection checks passed.
   
   The accompanying PyPaimon PR aligns its existing Python casts with Java and adds Native REST parity coverage. This only addresses type-change paths already supported by Python.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] Align BIGINT and timestamp schema evolution with Java [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #1107:
URL: https://github.com/apache/paimon-rust/pull/1107


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org