You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "XuQianJin-Stars (via GitHub)" <gi...@apache.org> on 2026/07/04 05:19:33 UTC

[PR] [lake/lance] Add tests for Decimal and Timestamp(LTZ) conversions [fluss]

XuQianJin-Stars opened a new pull request, #3573:
URL: https://github.com/apache/fluss/pull/3573

   
   Extend Lance test coverage for Decimal and Timestamp/TimestampLtz mappings on both the Fluss RowType -> non-shaded Arrow schema layer (`LanceArrowUtils`) and the shaded -> non-shaded `VectorSchemaRoot` copy layer (`ArrowDataConverter`):
   
   - **`LanceArrowUtilsTest`**: assert Arrow type for `DECIMAL(9,2)` / `DECIMAL(20,4)` including precision and scale round-trip; assert `TimeUnit` selection for `TIMESTAMP_LTZ(0/3/6/9)` and `TIMESTAMP(0/3/6/9)`.
   - **`ArrowDataConverterTest`**: end-to-end conversion of `DECIMAL(20,4)` values through `ShadedArrowBatchWriter` into a non-shaded `DecimalVector`, and `TIMESTAMP_LTZ(3)` / `TIMESTAMP_LTZ(6)` columns into `TimeStampMilliVector` and `TimeStampMicroVector` respectively, including sub-millisecond precision truncation semantics.
   
   These types are common in financial and event-time workloads; adding explicit coverage guards against future regressions in the Lance tiering path.
   
   ## Purpose
   
   Lance tiering already handles `DECIMAL`, `TIMESTAMP` and `TIMESTAMP_LTZ` in production code, but the existing unit tests did not directly exercise:
   
   1. The precision/scale round-trip for `DECIMAL` on the shaded -> non-shaded schema mapping,
   2. The `TimeUnit` selection rules (SECOND / MILLISECOND / MICROSECOND / NANOSECOND) for `TIMESTAMP` and `TIMESTAMP_LTZ` at every supported precision (`0/3/6/9`),
   3. The end-to-end value conversion from Fluss `Decimal` / `TimestampLtz` through `ShadedArrowBatchWriter` into the corresponding non-shaded Arrow vectors (including sub-millisecond truncation for `TIMESTAMP_LTZ(3)` and microsecond preservation for `TIMESTAMP_LTZ(6)`).
   
   This PR closes that gap so that any future refactor of `LanceArrowUtils` or `ArrowDataConverter` will surface regressions on these two commonly-used-in-production types immediately.
   
   Linked issue: close #xxx
   
   ## Brief change log
   
   Test-only change. No production code is touched.
   
   - `LanceArrowUtilsTest`: add three tests (+65 lines)
     - `testToArrowSchemaWithDecimal` — asserts `ArrowType.Decimal(9,2)` and `ArrowType.Decimal(20,4)` are produced for the corresponding Fluss types.
     - `testToArrowSchemaWithTimestampLtz` — asserts `TIMESTAMP_LTZ(0/3/6/9)` maps to Arrow `Timestamp` with `SECOND` / `MILLISECOND` / `MICROSECOND` / `NANOSECOND`.
     - `testToArrowSchemaWithTimestampNtz` — same coverage for `TIMESTAMP(0/3/6/9)` (no time zone).
   - `ArrowDataConverterTest`: add three tests (+104 lines)
     - `testConvertDecimalColumn` — writes three `DECIMAL(20,4)` values (positive / negative / very small) via `ShadedArrowBatchWriter`, converts to a non-shaded `VectorSchemaRoot`, and verifies each value using `BigDecimal.compareTo` on `DecimalVector#getObject`.
     - `testConvertTimestampLtzMillisColumn` — writes three `TIMESTAMP_LTZ(3)` values built with `TimestampLtz.fromEpochMillis` and asserts round-trip via `TimeStampMilliVector#get`.
     - `testConvertTimestampLtzMicrosColumn` — writes `TIMESTAMP_LTZ(6)` values with sub-millisecond components (`Instant.ofEpochMilli(...).plusNanos(...)`) and asserts the expected microsecond value in `TimeStampMicroVector` (i.e. `millis * 1_000 + nanoOfMillis / 1_000`), locking in the truncation semantics.
   
   ## Tests
   
   New tests are the deliverable of this PR. Verified locally:
   
   - `./mvnw -pl fluss-lake/fluss-lake-lance test -Dspotless.check.skip` — all Lance tests pass, including the 6 new ones.
   - `./mvnw -pl fluss-lake/fluss-lake-lance checkstyle:check` — 0 violations.
   
   No production behavior change; existing tests are unchanged.
   
   ## API and Format
   
   None. This PR only adds tests under `src/test/java`. There are no changes to public Java API, table properties, wire format, or on-disk / Lance dataset format.
   
   ## Documentation
   
   None. Test-only change; no user-facing documentation impact.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [lake/lance] Add tests for Decimal and Timestamp(LTZ) conversions [fluss]

Posted by "github-actions[bot] (via GitHub) via issues" <is...@fluss.apache.org>.
github-actions[bot] commented on PR #3573:
URL: https://github.com/apache/fluss/pull/3573#issuecomment-6915119240

   This PR was closed because it has been stale for 10 days with no activity.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [lake/lance] Add tests for Decimal and Timestamp(LTZ) conversions [fluss]

Posted by "github-actions[bot] (via GitHub) via issues" <is...@fluss.apache.org>.
github-actions[bot] closed pull request #3573: [lake/lance] Add tests for Decimal and Timestamp(LTZ) conversions
URL: https://github.com/apache/fluss/pull/3573


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [lake/lance] Add tests for Decimal and Timestamp(LTZ) conversions [fluss]

Posted by "github-actions[bot] (via GitHub) via issues" <is...@fluss.apache.org>.
github-actions[bot] commented on PR #3573:
URL: https://github.com/apache/fluss/pull/3573#issuecomment-6654097323

   This PR has been marked as stale because it has had no activity for 60 days. It will be closed in 10 days unless it receives a new comment, commit or other update.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org