[jira] [Commented] (SAMZA-305) Isolate user library dependencies from framework dependencies

["Jon Bringhurst (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661265#comment-14661265 ] 

Jon Bringhurst commented on SAMZA-305:
--------------------------------------

Possible duplicate of SAMZA-697

> Isolate user library dependencies from framework dependencies
> -------------------------------------------------------------
>
>                 Key: SAMZA-305
>                 URL: https://issues.apache.org/jira/browse/SAMZA-305
>             Project: Samza
>          Issue Type: New Feature
>    Affects Versions: 0.7.0
>            Reporter: Martin Kleppmann
>
> In a running container, Samza framework classes and user job classes are bundled into the same classpath. This can cause conflicts, especially with transitive dependencies. For example, samza-yarn depends on hadoop-common which depends on Guava, and the user job may depend on some database client library which depends on a different version of Guava. The ensuing version conflicts can be nasty.
> The same problem also plagues other frameworks that run user code, for example MapReduce and Storm. To my knowledge, neither of those have solved the problem. Tomcat *has* solved the problem by using a separate classloader for every webapp running in its container.
> Storm's ideas for classpath isolation are discussed [here|https://github.com/xumingming/storm/wiki/Storm-Dependency-Isolation] and [here|https://www.bountysource.com/issues/87805-keep-storm-dependencies-and-user-dependencies-independent]. It's tracked in STORM-129.
> It would be worth investigating this — if classpath isolation requires an intrusive change to Samza, it might be worth getting the pain out of the way while Samza is still young.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)