You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Jark Wu (Jira)" <ji...@apache.org> on 2020/08/12 03:28:00 UTC

[jira] [Updated] (FLINK-18826) Support to emit and encode upsert messages to Kafka

     [ https://issues.apache.org/jira/browse/FLINK-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jark Wu updated FLINK-18826:
----------------------------
    Description: 
Currently, it is not possible to emit the results of a group by query into Kafka. We can provide a format (e.g. {{upsert-json}}??) to encode changelogs to make it possible to write into Kafka. Another idea is provide an {{update-mode=upsert}} property, in this way, we can support upsert behavior for all the append-only formats (avro, avro-confluent, json, csv) without changing the implementation of the formats. The update-mode came from kafka connector [1] before 1.10, but was dropped after that. 

[1]: 

  was:Currently, it is not possible to emit the results of a group by query into Kafka. We can provide a format (e.g. {{upsert-json}}??) to encode changelogs to make it possible to write into Kafka. 


> Support to emit and encode upsert messages to Kafka
> ---------------------------------------------------
>
>                 Key: FLINK-18826
>                 URL: https://issues.apache.org/jira/browse/FLINK-18826
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Ecosystem
>            Reporter: Jark Wu
>            Assignee: Shengkai Fang
>            Priority: Major
>
> Currently, it is not possible to emit the results of a group by query into Kafka. We can provide a format (e.g. {{upsert-json}}??) to encode changelogs to make it possible to write into Kafka. Another idea is provide an {{update-mode=upsert}} property, in this way, we can support upsert behavior for all the append-only formats (avro, avro-confluent, json, csv) without changing the implementation of the formats. The update-mode came from kafka connector [1] before 1.10, but was dropped after that. 
> [1]: 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)