You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by RS <ti...@163.com> on 2022/07/30 11:30:48 UTC
Flink SQL 如何描述 ES(ElasticSearch)的nested字段类型?
Hi,
flink sql如何写es的nested数组数据?
原始示例数据:
{
"id": "123",
"field1":[
{
"k1":1
},
{
"k1":1,
"k2":2
},
{
"k3":"3"
}
]
}
filed1是一个数组,里面的原始是字典,字典内的字段名是动态的,我不知道里面有多少个key
现在准备将数据从kafka消费入库到es中,
已经在es中定义字段 field1mapping,为nested类型, "field1":{"type":"nested"},
flink sql类似如下:
create table source_kafka (
`id` string,
`field1` ARRAY<string>)
with (...)
create table sink_es(
`id` string,
`field1` ARRAY<string>)
with (...)
select id, field1 from source_kafka;
这个可以查看到数据
insert into sink_es select field1 from source_kafka;
这样会报错,
Caused by: [CIRCULAR REFERENCE: ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=object mapping for [field1] tried to parse field [null] as object, but found a concrete value]]]
看错误提示感觉是因为null导致的,但是field1不存在null的数据,所以请教下各位这个问题是什么,如何解决?
Thanks