[PR] perf: vectorize int, decimal, float and boolean casts to timestamp [datafusion-comet]

["Ersa-tech (via GitHub)" <gi...@apache.org>]

Ersa-tech opened a new pull request, #6969:
URL: https://github.com/apache/datafusion-comet/pull/6969

   ## Which issue does this PR close?
   
   Closes #4942.
   
   ## Rationale for this change
   
   `cast_int_to_timestamp`, `cast_decimal_to_timestamp`, `cast_float_to_timestamp`, and `cast_boolean_to_timestamp` still built the output one row at a time with `TimestampMicrosecondBuilder`. These are the remaining per-element casts to timestamp in #4942.
   
   ## What changes are included in this PR?
   
   - Int casts use `unary` and `saturating_mul`. The multiply is infallible on a null slot.
   - Boolean casts use `from_unary`. `true` is still 1 microsecond and `false` is 0.
   - Float casts use `try_unary` in ANSI mode and `unary_opt` in Legacy and Try. The error string is built only when ANSI fails.
   - Decimal casts use one `unary_opt` pass. The 256-bit division does not vectorize, and evaluating null slots was slower on every measured shape, including a batch with no nulls. Spark still truncates toward zero and does not raise.
   - A batch with no nulls still has `nulls() == None`.
   
   `unary` and `from_unary` transform the value stored under a null. The builder wrote 0 there. Arrow equality ignores those slots. A negative decimal scale still panics in `10_i128.pow`, same as before.
   
   This does not edit the float-to-int or decimal-to-int macros, or the test before `test_spark_cast_float_min_value_to_string` (open PR #6953).
   
   Prepared with Cursor Grok 4.7. The semantics, tests, and benchmarks were reviewed.
   
   ## How are these changes tested?
   
   Rust tests pin saturation, decimal truncation and the low 64 bits, the ANSI error variant and the first offending value, Legacy and Try agreeing, and a NaN hidden under a null. `cargo test -p datafusion-comet-spark-expr --lib conversion_funcs::` (119 tests). `cargo fmt`. `cargo clippy --all-targets --workspace -- -D warnings` from `native/`.
   
   `CometNativeCastSuite` tests whose names contain `to TimestampType` (20 tests, Spark 4.1, release `libcomet.so`) all passed, including boolean, byte, short, int, long, float, double, and decimal to timestamp.
   
   Criterion, 8192 rows, baseline `main` on the same machine (4 vCPU Intel Xeon, family 6 model 207, AVX-512, 15 GiB RAM, rustc 1.99.0). Bench profile: `opt-level=3`, `lto=thin`, `codegen-units=1`, `overflow-checks=false`. No shape regressed. Decimal with no nulls is under 10%.
   
   ```
   cd native
   export CARGO_PROFILE_BENCH_OPT_LEVEL=3 CARGO_PROFILE_BENCH_LTO=thin \
     CARGO_PROFILE_BENCH_CODEGEN_UNITS=1 CARGO_PROFILE_BENCH_OVERFLOW_CHECKS=false
   cargo bench -p datafusion-comet-spark-expr \
     --bench cast_int_to_timestamp --bench cast_non_int_numeric_timestamp -- --save-baseline main
   cargo bench -p datafusion-comet-spark-expr \
     --bench cast_int_to_timestamp --bench cast_non_int_numeric_timestamp -- --baseline main
   ```
   
   | Cast | No nulls | Sparse nulls | Dense nulls |
   | --- | --- | --- | --- |
   | i64 | 20.2 → 2.3 µs (−89%) | 27.6 → 2.4 µs (−91%) | 21.8 → 2.3 µs (−89%) |
   | f64 legacy | 42.3 → 34.4 µs (−19%) | 50.2 → 33.0 µs (−34%) | 23.2 → 5.5 µs (−76%) |
   | f64 ANSI | 42.2 → 35.5 µs (−14%) | 50.3 → 32.8 µs (−35%) | 23.4 → 5.6 µs (−76%) |
   | decimal(38, 6) | 92.8 → 84.2 µs (−9%) | 94.7 → 76.4 µs (−19%) | 31.2 → 11.6 µs (−63%) |
   | bool | 20.5 → 3.8 µs (−81%) | 27.6 → 4.0 µs (−86%) | 20.9 → 3.8 µs (−81%) |
   
   All-null batches: i64 −89%, f64 legacy −93%, f64 ANSI −94%, decimal −94%, bool −80%. A legacy f64 batch of NaN and infinities is −34%. f32 and the narrower integers use the same kernels. Decimal scales 0 and 18 with no nulls, measured on this same `unary_opt` kernel, were −9.6% and −9.8%.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [PR] perf: vectorize int, decimal, float and boolean casts to timestamp [datafusion-comet]

["andygrove (via GitHub)" <gi...@apache.org>]

andygrove merged PR #6969:
URL: https://github.com/apache/datafusion-comet/pull/6969


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org