You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@livy.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2018/10/02 18:43:00 UTC

[jira] [Created] (LIVY-521) Shade netty in RSC

Marcelo Vanzin created LIVY-521:
-----------------------------------

             Summary: Shade netty in RSC
                 Key: LIVY-521
                 URL: https://issues.apache.org/jira/browse/LIVY-521
             Project: Livy
          Issue Type: Task
          Components: RSC
    Affects Versions: 0.6.0
            Reporter: Marcelo Vanzin


Livy's build currently uses a different version of netty depending on the Spark version. That is dangerous because at runtime, Livy will be using the configured Spark's version of netty, which may not match the one Livy was built with.

That can cause issues if the netty API changes in an incompatible way (as it has in some areas between 4.0 and 4.1, which are the versions used by Spark 2.2 and 2.3 respectively).

Livy currently also adds its own netty jar to the Spark application, but that basically has no effect unless you use Spark's "userClassPathFirst" configs.

Shading makes sure the Livy code is using its expected version of netty regardless of the version Spark is using.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)