You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by RS <ti...@163.com> on 2022/07/30 11:30:48 UTC

Flink SQL 如何描述 ES(ElasticSearch)的nested字段类型?

Hi,


flink sql如何写es的nested数组数据?


原始示例数据:
{
    "id": "123",
    "field1":[
        {
            "k1":1
        },
        {
            "k1":1,
            "k2":2
        },
        {
            "k3":"3"
        }
    ]
}


filed1是一个数组,里面的原始是字典,字典内的字段名是动态的,我不知道里面有多少个key


现在准备将数据从kafka消费入库到es中,
已经在es中定义字段 field1mapping,为nested类型, "field1":{"type":"nested"},


flink sql类似如下:
create table source_kafka (
`id` string,
`field1` ARRAY<string>)
with (...)


create table sink_es(
`id` string,
`field1` ARRAY<string>)
with (...)


select id, field1 from source_kafka;
这个可以查看到数据


insert into sink_es select field1 from source_kafka;
这样会报错,
Caused by: [CIRCULAR REFERENCE: ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=object mapping for [field1] tried to parse field [null] as object, but found a concrete value]]]


看错误提示感觉是因为null导致的,但是field1不存在null的数据,所以请教下各位这个问题是什么,如何解决?


Thanks