You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Benchao Li (Jira)" <ji...@apache.org> on 2020/02/21 04:56:00 UTC
[jira] [Comment Edited] (FLINK-15400) elasticsearch sink support
dynamic index.
[ https://issues.apache.org/jira/browse/FLINK-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041552#comment-17041552 ]
Benchao Li edited comment on FLINK-15400 at 2/21/20 4:55 AM:
-------------------------------------------------------------
[~Leonard Xu] [~ouyangwuli] I'd like to raise another option here.
In our scenario, we need to defer index/version/routing infos from the record, and setting there into the Request, and remove these info from the final data.
We implemented it like `reserved column`, users can define `_index` / `_version` / `routing` in their sink DDL, and those columns are regular columns in SQL. only when we finally write it out, we'll remove these columns, and add these infos into the Request.
was (Author: libenchao):
[~Leonard Xu] [~ouyangwuli] I raise another option here.
In our scenario, we need to defer index/version/routing infos from the record, and setting there into the Request, and remove these info from the final data.
We implemented it like `reserved column`, users can define `_index` / `_version` / `routing` in their sink DDL, and those columns are regular columns in SQL. only when we finally write it out, we'll remove these columns, and add these infos into the Request.
> elasticsearch sink support dynamic index.
> -----------------------------------------
>
> Key: FLINK-15400
> URL: https://issues.apache.org/jira/browse/FLINK-15400
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / ElasticSearch, Table SQL / Ecosystem
> Affects Versions: 1.11.0
> Reporter: ouyangwulin
> Priority: Major
> Labels: usability
> Fix For: 1.11.0
>
>
> From user-zh@flink.apache.org([https://lists.apache.org/thread.html/ac4e0c068baeb3b070f0213a2e1314e6552b226b8132a4c49d667ecd%40%3Cuser-zh.flink.apache.org%3E]), Becuase the es 6/7 not support ttl. so User need clean the index by timestamp. Add dynamic index is a useful function. Add with properties 'dynamicIndex' as a switch for open dynamicIndex. Add with properties 'indexField' for the extract time field, Add properties 'indexInterval' for change cycle mode.
>
> ||With property||discribe||default||Required||
> |dynamicIndex|Dynamic or not|false(true/false)|false|
> |indexField|extract index field| none|dynamicIndex is true , then indexField is required,only supported type "timestamp","date","long" |
> |indexInterval|mode for cycle|d|ddynamicIndex is true , this field is required ，the value optional is：
> d：day
> m：mouth
> w：week|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)