[jira] [Commented] (FLINK-8362) Shade Elasticsearch dependencies away

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312771#comment-16312771 ] 

ASF GitHub Bot commented on FLINK-8362:
---------------------------------------

GitHub user NicoK opened a pull request:

    https://github.com/apache/flink/pull/5243

    [FLINK-8362][elasticsearch] shade all dependencies

    ## What is the purpose of the change
    
    The Elasticsearch connectors have some dependencies that need to be available and should not conflict with other user or system code. Similarly to the cassandra connector and the S3 file systems, it should thus shade all its dependencies and become self-contained.
    
    ## Brief change log
    
    - shade dependencies of `flink-connector-elasticsearch`, `flink-connector-elasticsearch2`, and `flink-connector-elasticsearch5` and relocate to our namespace
    - add a test into `travis_mvn_watchdog.sh` to verify classes are shaded (one gap though: since `flink-connector-elasticsearch` and `flink-connector-elasticsearch-base` use the same package, we cannot really test that the latter is shaded correctly there)
    - update documentation
    
    ## Verifying this change
    
    This can be verified as follows:
    
    In the shaded jar files,
    - check for non-shaded `service` files (also their contents)
    - check for non-shaded `.class` files (this test is automated in `travis_mvn_watchdog.sh`, but `flink-connector-elasticsearch` should be verified manually as well - see the note above)
    - check for unnecessary files in `META-INF/maven` (only `org.apache.flink/flink-connector-elasticsearch<variant>_<scala_version>/` should remain)
    - manually test the connectors and verify they are working without adding other dependencies to a user jar (TODO! @tzulitai since you have been working with the connector in the past, can you chime in here?)
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): **no**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
      - The serializers: **no**
      - The runtime per-record code paths (performance sensitive): **no**
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
      - The S3 file system connector: **no**
    
    ## Documentation
    
      - Does this pull request introduce a new feature? **no**
      - If yes, how is the feature documented? **docs**

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/NicoK/flink flink-8362

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5243.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5243
    
----
commit e0c219dadcda5a7771b20c9873006ef8ebc699b0
Author: Nico Kruber <ni...@...>
Date:   2018-01-04T19:45:04Z

    [build][hotfix] fix s3-fs shading checks not exiting for a non-exiting file
    
    + improve the output a bit

commit b1dcb6d06fed7096c0e2e8a88882037559b29923
Author: Nico Kruber <ni...@...>
Date:   2018-01-04T20:09:09Z

    [FLINK-8362][elasticsearch] shade all dependencies

----


> Shade Elasticsearch dependencies away
> -------------------------------------
>
>                 Key: FLINK-8362
>                 URL: https://issues.apache.org/jira/browse/FLINK-8362
>             Project: Flink
>          Issue Type: Improvement
>          Components: Build System, ElasticSearch Connector
>    Affects Versions: 1.4.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>
> It would be nice to make the Elasticsearch connectors self-contained just like the s3 file system implementations and the cassandra connector.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)