[jira] [Updated] (STORM-876) Dist Cache: Basic Functionality

["Rick Kellogg (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rick Kellogg updated STORM-876:
-------------------------------
    Component/s: storm-core

> Dist Cache: Basic Functionality
> -------------------------------
>
>                 Key: STORM-876
>                 URL: https://issues.apache.org/jira/browse/STORM-876
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-core
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>
> Basic functionality for the Dist Cache feature.
> As part of this a new API should be added to support uploading and downloading dist cache items.  storm-core.ser, storm-conf.ser and storm.jar should be written into the blob store instead of residing locally. We need a default implementation of the blob store that does essentially what nimbus currently does and does not need anything extra.  But having an HDFS backend too would be great for scalability and HA.
> The supervisor should provide a way to download and manage these blobs and provide a working directory for the worker process with symlinks to the blobs.  It should also allow the blobs to be updated and switch the symlink atomically to point to the new blob once it is downloaded.
> All of this is already done by code internal to Yahoo! we are in the process of getting it ready to push back to open source shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)