[jira] [Commented] (FLINK-9816) Support Netty SslEngine based on openSSL

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542748#comment-16542748 ] 

ASF GitHub Bot commented on FLINK-9816:
---------------------------------------

GitHub user NicoK opened a pull request:

    https://github.com/apache/flink/pull/6328

    [FLINK-9816][network] add option to configure SSL engine provider for TM communication

    ## What is the purpose of the change
    
    Netty has the ability to run with different `SSLEngine` implementations but with our current setup, we are fixed to the JDK implementation, although one based on OpenSSL is expected to be faster [1].
    We should make this configurable and ideally also provide everything needed to run with OpenSSL in the future (the last part is not part of this PR).
    
    [1] https://netty.io/wiki/requirements-for-4.x.html#benefits-of-using-openssl
    
    ## Brief change log
    
    - allow selecting the SSL engine provider via `security.ssl.provider`
    - set up Netty SSL handler with its `SslContextBuilder` (in `NettyConfig`) to have this flexibility
    
    ## Verifying this change
    
    This change can be verified as follows:
    
    - I verified by running an SSL setup with 2 TMs and submitting a job through the WebUI with the default `JDK` SSL engine and `OPENSSL` using a custom build using `netty-tcnative` with statically linked boringssl libraries from http://netty.io/wiki/forked-tomcat-native.html
    -  there is an end-to-end test in #6327 which is currently blocked on the CLI submission not working with SSL
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): **no**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
      - The serializers: **no**
      - The runtime per-record code paths (performance sensitive): **no**
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
      - The S3 file system connector: **no**
    
    ## Documentation
    
      - Does this pull request introduce a new feature? **no**
      - If yes, how is the feature documented? **docs, JavaDocs**


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/NicoK/flink flink-9816

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/6328.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #6328
    
----
commit d6fb90a0c4cb1b105c482d982f8ab84744a80dc8
Author: Nico Kruber <ni...@...>
Date:   2018-07-11T20:10:22Z

    [hotfix][typo] fix a deprecation message

commit 83c00b9cd5e1178a1152feac04bbc7a68213deb3
Author: Nico Kruber <ni...@...>
Date:   2018-07-11T21:06:04Z

    [hotfix][checkstyle] fix a warning in NettyConfig

commit af8486d587a5dbc553fec42d80180a1f6ecc1571
Author: Nico Kruber <ni...@...>
Date:   2018-07-11T21:05:01Z

    [FLINK-9816][network] add option to configure SSL engine provider for TM communication
    
    This prepares Flink to use OpenSSL for TM communication channels via netty.
    
    Currently, there is no easy way to provide the required native libraries,
    though. We'll either include these in a future version of flink-shaded or update
    instructions on how to include/build them manually.

----


> Support Netty SslEngine based on openSSL
> ----------------------------------------
>
>                 Key: FLINK-9816
>                 URL: https://issues.apache.org/jira/browse/FLINK-9816
>             Project: Flink
>          Issue Type: Improvement
>          Components: Network
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Major
>              Labels: pull-request-available
>
> Since a while now, Netty does not only support the JDK's {{SSLEngine}} but also implements one based on openSSL which, according to https://netty.io/wiki/requirements-for-4.x.html#wiki-h4-4 is significantly faster. We should add support for using that engine instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)