[jira] [Reopened] (STORM-1537) Upgrade to Kryo 3

["Abhishek Agarwal (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhishek Agarwal reopened STORM-1537:
-------------------------------------

Reopening so that changes are also merged into master branch. 

> Upgrade to Kryo 3
> -----------------
>
>                 Key: STORM-1537
>                 URL: https://issues.apache.org/jira/browse/STORM-1537
>             Project: Apache Storm
>          Issue Type: Improvement
>    Affects Versions: 1.0.0
>            Reporter: Oscar Boykin
>            Assignee: Abhishek Agarwal
>             Fix For: 1.0.0
>
>
> In storm, Kryo (2.21) is used for serialization:
> https://github.com/apache/storm/blob/02a44c7fc1b7b3a1571b326fde7bcae13e1b5c8d/pom.xml#L231
> The user must use the same version storm does, or there will be a java class error at runtime.
> Storm depends on a quasi-abandoned library: carbonite:
> https://github.com/apache/storm/blob/02a44c7fc1b7b3a1571b326fde7bcae13e1b5c8d/pom.xml#L210
> which depends on Kryo 2.21 and Twitter chill 0.3.6:
> https://github.com/sritchie/carbonite/blob/master/project.clj#L8
> Chill, currently on 0.7.3, would like to upgrade to Kryo 3.0.3:
> https://github.com/twitter/chill/pull/245
> because Spark, also depending on chill, would like to upgrade for performance improvements and bugfixes.
> https://issues.apache.org/jira/browse/SPARK-11416
> Unfortunately, summingbird depends on storm:
> https://github.com/twitter/summingbird/blob/develop/build.sbt#L34
> so, if chill is upgraded, and that gets on the classpath, summingbird will break at runtime.
> I propose:
> 1) copy the carbonite code into storm. It is likely the only consumer.
> 2) bump the storm kryo dependency after chill upgrades: recall that storm actually depends on chill-java. A dependency that could possibly be removed after you pull carbonite in.
> 3) once a new version of storm is published, summingbird (and scalding) can upgrade to the latest chill.
> Also, I hope for:
> 4) we as a JVM community get better about classpath isolation and versioning. Diamonds like this in one big classpath make large codebases very fragile.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)