[GitHub] aljoscha commented on a change in pull request #6105: [FLINK-8500] Get the timestamp of the Kafka message from kafka consumer

[GitBox <gi...@apache.org>]

aljoscha commented on a change in pull request #6105: [FLINK-8500] Get the timestamp of the Kafka message from kafka consumer
URL: https://github.com/apache/flink/pull/6105#discussion_r212222385
 
 

 ##########
 File path: flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/util/serialization/KeyedDeserializationSchema.java
 ##########
 @@ -45,6 +45,22 @@
 	 */
 	T deserialize(byte[] messageKey, byte[] message, String topic, int partition, long offset) throws IOException;
 
+	/**
+	 * Deserializes the byte message.
+	 *
+	 * @param messageKey the key as a byte array (null if no key has been set).
+	 * @param message The message, as a byte array (null if the message was empty or deleted).
+	 * @param partition The partition the message has originated from.
+	 * @param offset the offset of the message in the original source (for example the Kafka offset).
+	 * @param timestamp the timestamp of the consumer record
+	 * @param timestampType The timestamp type, could be NO_TIMESTAMP, CREATE_TIME or INGEST_TIME.
+	 *
+	 * @return The deserialized message as an object (null if the message cannot be deserialized).
+	 */
+	default T deserialize(byte[] messageKey, byte[] message, String topic, int partition, long offset, long timestamp, TimestampType timestampType) throws IOException {
 
 Review comment:
   Yes, to unblock this I thing we can go with this approach, basically the schema becomes this:
   ```
   @PublicEvolving
   public interface KeyedDeserializationSchema<T> extends Serializable, ResultTypeQueryable<T> {
   
           @Deprecated
   	default T deserialize(byte[] messageKey, byte[] message, String topic, int partition, long offset) {
               throw new RuntimeException("blammo");
           }
   
           default T deserialize(byte[] messageKey, byte[] message, String topic, int partition, long offset, long timestamp) {
               return deserialize(/* call the other method */);
           } 
   	boolean isEndOfStream(T nextElement);
   }
   ```
   
   With this, if you have an existing implementation of `KeyedDeserializationSchema` it will continue to work without any changes. If you implement a new one you have to implement one of the methods, otherwise the exception is thrown. And all Flink code only calls the version that takes the timestamp.
   
   What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services