[jira] [Commented] (FLINK-8101) Elasticsearch 6.x support

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389778#comment-16389778 ] 

ASF GitHub Bot commented on FLINK-8101:
---------------------------------------

Github user jdewald commented on the issue:

    https://github.com/apache/flink/pull/5374
  
    We were in the process of testing the Flink integration with the ES High Level Rest client in order to perform upgrades to ES6 and stumbled on some issues related to the bulk rejection retry handling. In short, swapping in the HLR backend will not perform retries correctly and it is not actually safe to simply call `add` on the `RequestIndexer` that is sent into the `ActionRequestFailureHander` from the Flink-side sink due to some synchronization issues. 
    
    I have created a bug report that has been accepted on the Elasticsearch side https://github.com/elastic/elasticsearch/issues/28885 to automatically perform retries.
    
    One additional note however is that in the case where the retries have been used up from the `BackoffPolicy`, using the `RetryRejectionFailureHandler` appears to cause a deadlock, so this may be something to fix on the ElasticsearchSink side. But it may be necessary to independently confirm that in case that is an issue particular to my testing. (TL;DR: The RetryRejectionFailureHandler is not safe to use with HLR and I am not certain it's safe to use with the `TransportClient` based one either in the case that it actually triggers). 


> Elasticsearch 6.x support
> -------------------------
>
>                 Key: FLINK-8101
>                 URL: https://issues.apache.org/jira/browse/FLINK-8101
>             Project: Flink
>          Issue Type: New Feature
>          Components: ElasticSearch Connector
>    Affects Versions: 1.4.0
>            Reporter: Hai Zhou UTC+8
>            Assignee: Flavio Pompermaier
>            Priority: Major
>             Fix For: 1.5.0
>
>
> Recently, elasticsearch 6.0.0 was released: https://www.elastic.co/blog/elasticsearch-6-0-0-released  
> The minimum version of ES6 compatible Elasticsearch Java Client is 5.6.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)