[jira] [Commented] (STORM-634) Storm should support rolling upgrade/downgrade of storm cluster.

["Robert Joseph Evans (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287618#comment-14287618 ] 

Robert Joseph Evans commented on STORM-634:
-------------------------------------------

Yes the Java serialization is the biggest problem we have run into, although for the most part we have been doing rolling upgrades of our clusters from one minor version to another without any issues.  We just have to be careful about checking if a serialized java class has changed from one release to another.  I personally would rather use Thrift over JSON.  JSON is great in many cases, but this data does not need to be human readable and thrift provides a known set of guidelines to ensure both forwards and backwards compatibility, including default values that JSON does not.  In addition thrift is what Trident uses for storing transactional state already, and would allow us to transition some of the data out of ZK and into a heartbeat server where thrift could handle the RPC as well.   

> Storm should support rolling upgrade/downgrade of storm cluster.
> ----------------------------------------------------------------
>
>                 Key: STORM-634
>                 URL: https://issues.apache.org/jira/browse/STORM-634
>             Project: Apache Storm
>          Issue Type: Improvement
>            Reporter: Parth Brahmbhatt
>            Assignee: Parth Brahmbhatt
>
> Currently when a new version of storm is released in order to upgrade existing storm clusters users need to backup their existing topologies , kill all the topologies , perform the upgrade and resubmit all the topologies. This is painful and results in downtime which may not be acceptable for "Always alive"  production systems.
> Storm should support a rolling  upgrade/downgrade deployment process to avoid these downtimes and to make the transition to a different version effortless. 
> Based on my initial attempt the primary issue seem to be the java serialization used to serialize java classes like StormBase, Assignment, WorkerHeartbeat which is then stored in zookeeper. When deserializing if the serial versions do not match the deserialization fails resulting in processes just getting killed indefinitely. We need to change the Utils/serialize and Utils/deserialize so it can support non java serialization mechanism like json. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)