You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "isabst (via GitHub)" <gi...@apache.org> on 2025/08/26 09:04:11 UTC

[I] [Feature][Connector-hudi] seatunnel hudi sink config add timestamp_semantics for kinds of timezone scenarios [seatunnel]

isabst opened a new issue, #9771:
URL: https://github.com/apache/seatunnel/issues/9771

   ### Search before asking
   
   - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   This PR adds a new optional Hudi sink option, timestamp_semantics, to control how timestamp values are converted to epoch millis. Supported values: instant (default; alias utc), local (system default ZoneId), and zone:{ZoneId} (e.g., zone:Asia/Shanghai). The option is read in HudiSinkConfig, added to HudiSinkFactory option rules, and propagated via HudiSinkWriter → HudiRecordWriter → HudiRecordConverter into RowDataToAvroConverters, including nested array/row/map fields. When unset, behavior remains UTC for full backward compatibility. This enables precise, consistent timestamp interpretation across environments and time zones. 
   
   ### Usage Scenario
   
    - When upstream timestamps are intended as UTC (or you want to preserve previous behavior), use instant (default). This keeps conversions stable across environments.
   
   - Data produced in a specific local time zone (no TZ info in schema)
     
     - For systems emitting LocalDateTime in a known business time zone (e.g., Asia/Shanghai POS logs), use zone:Asia/Shanghai so 2024-01-01T00:00:00 is interpreted as local midnight and converted accordingly.
   
   - Multi-region deployments
     
     - Different jobs/tables representing different locales can specify different zones (e.g., zone:Europe/Berlin vs zone:America/New_York) to ensure consistent, region-specific temporal semantics.
     
     - Use local to bind LocalDateTime to the JVM’s system default (affected by TZ or -Duser.timezone). This is convenient but less reproducible than an explicit zone:.
     
     - If unset or set to instant (alias utc), behavior remains UTC, ensuring existing jobs are unaffected.
   
   ### Related issues
   
   not yet
   
   ### Are you willing to submit a PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][Connector-hudi] seatunnel hudi sink config add timestamp_semantics for kinds of timezone scenarios [seatunnel]

Posted by "isabst (via GitHub)" <gi...@apache.org>.
isabst closed issue #9771: [Feature][Connector-hudi] seatunnel hudi sink config add timestamp_semantics for kinds of timezone scenarios
URL: https://github.com/apache/seatunnel/issues/9771


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org