You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Mr-Gump (via GitHub)" <gi...@apache.org> on 2023/02/27 11:45:48 UTC

[GitHub] [flink] Mr-Gump opened a new pull request, #22037: [FLINK-31006][connector/kafka] Add writeTimestamp option to KafkaRecordSerializationSchemaBuilder

Mr-Gump opened a new pull request, #22037:
URL: https://github.com/apache/flink/pull/22037

   ## What is the purpose of the change
   
   *This PR offers users an option to avoid passing event timestamp to Kafka., for the reason that in some cases, directly passing event timestamp to Kafka may cause Kafka segment deletion unexpectedly. For Example, I use Kafka source and Kafka sink transfer recodes from one topic to another, however, the target topic receives the data and deletes them in a short period. The reason is that event timestamps are directly transferred from source topic to target topic, and the data is published to Kafka just a week before. So It leads an automatic deletion in Kafka segment.*
   
   
   ## Brief change log
   
   *Add writeTimestamp option in KafkaRecordSerializationSchemaBuilder*
     - *Add a bool variable writeTimestamp and its setter.*
     - *Add an additional check for whether timestamp will be set to null.*
   
   
   ## Verifying this change
   
   This change is already covered by existing tests, such as *[KafkaRecordSerializationSchemaBuilderTest](https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/sink/KafkaRecordSerializationSchemaBuilderTest.java)*.
   
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   
   ## Documentation
   
     - Does this pull request introduce a new feature? yes
     - If yes, how is the feature documented? not documented
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #22037: [FLINK-31006][connector/kafka] Add writeTimestamp option to KafkaRecordSerializationSchemaBuilder

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #22037:
URL: https://github.com/apache/flink/pull/22037#issuecomment-1446190207

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bd0feb096984bd89584d98551a9873e6125e0398",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "bd0feb096984bd89584d98551a9873e6125e0398",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bd0feb096984bd89584d98551a9873e6125e0398 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] MartijnVisser commented on pull request #22037: [hotfix][connector/kafka] Add writeTimestamp option to KafkaRecordSerializationSchemaBuilder

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser commented on PR #22037:
URL: https://github.com/apache/flink/pull/22037#issuecomment-1493873549

   The Kafka connector code is in feature freeze, meaning that no new code will be merged to `master` but all PRs should be redirect towards https://github.com/apache/flink-connector-kafka 
   
   This also shouldn't be a hotfix, but requires a Jira since this introduces a new feature. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] MartijnVisser closed pull request #22037: [hotfix][connector/kafka] Add writeTimestamp option to KafkaRecordSerializationSchemaBuilder

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser closed pull request #22037: [hotfix][connector/kafka] Add writeTimestamp option to KafkaRecordSerializationSchemaBuilder
URL: https://github.com/apache/flink/pull/22037


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org