You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Julien Chanaud (Jira)" <ji...@apache.org> on 2021/12/07 14:13:00 UTC
[jira] [Updated] (KAFKA-13511) Update TimestampConverter SMT to support unix epoch as millis, micros, and seconds
[ https://issues.apache.org/jira/browse/KAFKA-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Chanaud updated KAFKA-13511:
-----------------------------------
Description:
Currently, the SMT TimestampConverter can convert Timestamp from either source String, Long or any target Date to String, Long or Date.
The problem is that Long source or target is required to be epoch in milliseconds.
In many cases, epoch is represented with different precisions. This leads to several Jira tickets :
* KAFKA-12364
* KAFKA-10561
I propose to add a new config to TimestampConverter called "epoch.precision" which defaults to "millis" so as to not impact existing code, and allows for more precisions : seconds, millis, micros.
{code:json}
"transforms": "TimestampConverter",
"transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.TimestampConverter.field": "event_date",
"transforms.TimestampConverter.epoch.precision": "micros",
"transforms.TimestampConverter.target.type": "Timestamp"
{code}
was:
Currently, the SMT TimestampConverter can convert Timestamp from either source String, Long or any target Date to String, Long or Date.
The problem is that Long source or target is required to be epoch in milliseconds.
In many cases, epoch is represented with different precisions. This leads to several Jira tickets :
* KAFKA-12364
* KAFKA-10561
I propose to add a new config to TimestampConverter called "epoch.precision" which defaults to "millis" so as to not impact existing code, and allows for more precisions : seconds, millis, micros.
"transforms": "TimestampConverter",
"transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value","transforms.TimestampConverter.field": "event_date"
*"transforms.TimestampConverter.epoch.precision": "micros"*
"transforms.TimestampConverter.target.type": "Timestamp"
> Update TimestampConverter SMT to support unix epoch as millis, micros, and seconds
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-13511
> URL: https://issues.apache.org/jira/browse/KAFKA-13511
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Julien Chanaud
> Priority: Minor
>
> Currently, the SMT TimestampConverter can convert Timestamp from either source String, Long or any target Date to String, Long or Date.
> The problem is that Long source or target is required to be epoch in milliseconds.
> In many cases, epoch is represented with different precisions. This leads to several Jira tickets :
> * KAFKA-12364
> * KAFKA-10561
> I propose to add a new config to TimestampConverter called "epoch.precision" which defaults to "millis" so as to not impact existing code, and allows for more precisions : seconds, millis, micros.
> {code:json}
> "transforms": "TimestampConverter",
> "transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
> "transforms.TimestampConverter.field": "event_date",
> "transforms.TimestampConverter.epoch.precision": "micros",
> "transforms.TimestampConverter.target.type": "Timestamp"
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)