You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "haiqingchen (Jira)" <ji...@apache.org> on 2024/09/19 03:25:00 UTC
[jira] [Created] (FLINK-36320) Use Protobuf Nested timestamp field as event time column and watermark
haiqingchen created FLINK-36320:
-----------------------------------
Summary: Use Protobuf Nested timestamp field as event time column and watermark
Key: FLINK-36320
URL: https://issues.apache.org/jira/browse/FLINK-36320
Project: Flink
Issue Type: Improvement
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Reporter: haiqingchen
Hi I'm using protobuf format to read from kafka and there's a message type Test in the proto which is also a nested field in another message type TestObject
{code:java}
message Test {
int64 startTime,
int32 id
}
message TestObject {
repeated Test tests = 3
}{code}
Here's how I define the ddl.
{code:java}
create table test_pb (
Test Array<Row<startTime bigint, id int>>
) with (
"connector"="kafka",
'format' = 'protobuf',
'protobuf.message-class-name'= 'xxxx.TestObject'
......
)
{code}
I would like to use startTime field in the nested Test object as event time column and define watermark for it , but as it is a nested field in an Array, I can't simply use it. I would like to flatten the TestObject column but it doesn't work for DDL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)