[GitHub] [hudi] hanjun996 edited a comment on issue #4698: [SUPPORT] RowDataToAvroConverters does not support data in flink timestamp_ltz (timestamp_with_local_time_zone) format.

[GitBox <gi...@apache.org>]

hanjun996 edited a comment on issue #4698:
URL: https://github.com/apache/hudi/issues/4698#issuecomment-1031041107


   
   > Environment Description
   Hudi version :0.10.0
   flink version :1.13.0
   
   Steps to reproduce the behavior:
   
   mysql timestamp type, such as: 1970-01-01T00:00:01Z, the corresponding type in flink is TIMESTAMP_WITH_LOCAL_TIME_ZONE, which is not supported by avro during conversion in hudi version 0.10.0
   
   code：
   
   public static final DataType ROW_DATA_TYPE = DataTypes.ROW(
       DataTypes.FIELD("id", DataTypes.STRING()),
       DataTypes.FIELD("timestamp", DataTypes.TIMESTAMP_WITH_LOCAL_TIME_ZONE()))
       .notNull();
   public static final RowType ROW_TYPE = (RowType) ROW_DATA_TYPE.getLogicalType();
   
   @Test
   public void test() {
       String inferredSchema = AvroSchemaConverter.convertToSchema(ROW_TYPE).toString();
       Configuration configuration = new Configuration();
       configuration.setString(FlinkOptions.PATH, "test");
       configuration.setString(FlinkOptions.TABLE_NAME, "test");
       configuration.setString(FlinkOptions.TABLE_TYPE, "MERGE_ON_READ");
       configuration.setString(FlinkOptions.RECORD_KEY_FIELD, "id");
       configuration.setString(FlinkOptions.SOURCE_AVRO_SCHEMA, inferredSchema);
   
       Map<String, String> record1 = new HashMap<>();
       record1.put("id", "1");
       record1.put("timestamp", "1970-01-01T00:00:01Z");
   
       StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
       DataStreamSource source = env.fromElements(record1);
       Pipelines.bootstrap(configuration, ROW_TYPE, 1, source).print();
   }
   ERROR:
   
   java.lang.UnsupportedOperationException: Unsupported to derive Schema for type: TIMESTAMP_LTZ(6)
   at org.apache.hudi.util.AvroSchemaConverter.convertToSchema(AvroSchemaConverter.java:292)
   at org.apache.hudi.util.AvroSchemaConverter.convertToSchema(AvroSchemaConverter.java:263)
   at org.apache.hudi.util.AvroSchemaConverter.convertToSchema(AvroSchemaConverter.java:169)
   
   Expected behavior:
   
   support TIMESTAMP_WITH_LOCAL_TIME_ZONE
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org