[jira] [Commented] (FLINK-7386) Flink Elasticsearch 5 connector is not compatible with Elasticsearch 5.2+ client

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481116#comment-16481116 ] 

ASF GitHub Bot commented on FLINK-7386:
---------------------------------------

GitHub user cjolif opened a pull request:

    https://github.com/apache/flink/pull/6043

    [FLINK-7386] evolve RequestIndexer API to make it working with Elastic 5.3+, evolve ElasticsearchApiCallBridge API to make it compatible with a possible RestHighLevelClient implementation 

    ## What is the purpose of the change
    
    *The purpose of this PR is to make sure current Elasticsearch implementation is  compatible with Elasticsearch 5.3+ fixing [FLINK-7386]  and is also open to a future HighLevelRestClient implementation that could be used to provide elasticsearch 6 compatibility [FLINK-8101]*
    
    ## Brief change log
    
    
    * add specific IndexRequest, UpdateRequest and DeleteRequest add method on RequestIndexer so that  it is compatible both with 5.2- and 5.3+ APIs (knowing that in 5.3+ Elasticsearch does not accept anymore ActionRequest in BulkProcessor).
    * make sure existing ActionRequest method on RequestIndexer is calling the new specific method based on actual type.
    * throw an exception for other types.
    * Change returned values of createClient method in ElasticsearchApiCallBridge. As TransportClient and HighLevelRestClient have only the AutoCloseable interface in common, this is what the method returns now.
     * Make ElasticsearchSinkBase agnostic to whether it is using a TransportClient or RestClient by adding a createBulkProcessorBuilder method on ElasticsearchApiCallBridge that the ElasticsearchSinkBase calls. Implement this method on all bridges. 
      
    ## Verifying this change
    
    This change added tests and can be verified as follows:
    * Elasticsearch test base has also been reworked a little bit to make it compatible with the changes.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): no
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`:  a `@PublicEvolving` interface is now an abstract class. However typically the user does not extend/implement it but just call methods on it.
      - The serializers:  no
      - The runtime per-record code paths (performance sensitive): no
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
      - The S3 file system connector: no
    ## Documentation
    
      - Does this pull request introduce a new feature? yes
      - If yes, how is the feature documented? docs & javadocs


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cjolif/flink es-5.3-apis

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/6043.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #6043
    
----
commit b1f2abc1d33b39c1fed4f370e5b21cbf477e0aa8
Author: Christophe Jolif <cj...@...>
Date:   2018-05-17T22:17:04Z

    [FLINK-7386] evolve RequestIndexer API to make it working with Elastic
    5.3+, evolve ElasticsearchApiCallBridge API to make it compatible
    with a possible RestHighLevelClient implementation.

----


> Flink Elasticsearch 5 connector is not compatible with Elasticsearch 5.2+ client
> --------------------------------------------------------------------------------
>
>                 Key: FLINK-7386
>                 URL: https://issues.apache.org/jira/browse/FLINK-7386
>             Project: Flink
>          Issue Type: Improvement
>          Components: ElasticSearch Connector
>            Reporter: Dawid Wysakowicz
>            Assignee: Fang Yong
>            Priority: Critical
>             Fix For: 1.5.0
>
>
> In Elasticsearch 5.2.0 client the class {{BulkProcessor}} was refactored and has no longer the method {{add(ActionRequest)}}.
> For more info see: https://github.com/elastic/elasticsearch/pull/20109



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)