You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "liwei li (Jira)" <ji...@apache.org> on 2021/11/11 02:46:00 UTC

[jira] [Commented] (FLINK-22541) add json format filter params

    [ https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442052#comment-17442052 ] 

liwei li commented on FLINK-22541:
----------------------------------

What is the issuse conclusion?Should we support this feature? 

In my humble opinion, I'm inclined to vote for this feature because it simplifies table statements and makes it easier to get the nested data you want.

cc [~lirui] [~jark] 

 

> add json format filter params 
> ------------------------------
>
>                 Key: FLINK-22541
>                 URL: https://issues.apache.org/jira/browse/FLINK-22541
>             Project: Flink
>          Issue Type: Improvement
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Ecosystem
>    Affects Versions: 1.11.0, 1.12.0
>            Reporter: sandy du
>            Priority: Minor
>         Attachments: image-2021-05-06-13-37-49-160.png, image-2021-05-06-13-38-37-284.png
>
>
> In my case,one kafka topic store multiple table data,for example:
>  \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>  \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>  \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>  \{"id":"124","source":"order","content":{"orderId":"100001","price":34,"addr":"addr1231"}}
>  \{"id":"125","source":"order","content":{"orderId":"100002","price":34,"addr":"addr1232"}}
>  \{"id":"126","source":"order","content":{"orderId":"100003","price":34,"addr":"addr1233"}}
>   
>  I  just want to consume data from  talbe order,flink sql ddl like this:
>  CREATE TABLE order (
>  orderId STRING,
>  age INT,
>  addr STRING
>  )
>  with (
>  'connector'='kafka',
>  'topic'='kafkatopic',
>  'properties.bootstrap.servers'='localhost:9092',
>  'properties.group.id'='testGroup',
>  'scan.startup.mode'='earliest-offset',
>  'format'='json',
>  'path-fliter'='$[?(@.source=="order")]',
>  'path-data'='$.content'
>  );
>   
>  path-fliter and path-data can use  JsonPath ([https://github.com/json-path/JsonPath])
>   



--
This message was sent by Atlassian Jira
(v8.20.1#820001)