Introducing Weave - Apache YARN simplified.

[Nitin Motgi <ni...@continuuity.com>]

Hi All,

At Continuuity, we use Apache YARN as an integral part of our products because of its vision to support a diverse set of applications and processing patterns. One such product is BigFlow, our realtime distributed stream-processing engine. Apache YARN is used to run and manage BigFlow applications including lifecycle and runtime elasticity. During our journey with YARN we have come to the realization that it is extremely powerful but its full capability is challenging to leverage.  It is difficult to get started, hard to test and debug, and complex to build new kinds of non-MapReduce applications and frameworks on. 

We decided to build Weave and be part of the journey to take Apache YARN to the next level of usability and functionality. We have been using Weave extensively to support our products and have seen the benefit and power of Apache YARN and Weave combined.  We have decided to share Weave under the Apache 2.0 license in an effort to collaborate with members of the community, broaden the set of applications and patterns that Weave supports, and further the overall adoption of Apache YARN. 

Weave is NOT a replacement for Apache YARN.  It is instead a value-added framework that operates on top of Apache YARN. 

What is Weave ?
==============

Weave is a simple set of libraries that allows you to easily manage distributed applications through an abstraction layer built on Apache YARN. Weave allows you to use YARN’s distributed capabilities with a programming model that is similar to running threads. 

Features of Weave
===============

  - Simple API for specifying, running and managing application lifecycle
  - An easy way to communicate with an application or parts of an application
  - A generic Application Master to better support simple applications
  - Simplified archive management and local file transport
  - Improved control over application logs, metrics and errors
  - Discovery service
  - And many more...

Where can I get this coolness ?
=========================

The code is available on github at http://github.com/continuuity/weave under the Apache 2.0 License.

We will continue adding features to improve Weave, but would love to hear from people who are willing to be contributors to this project and help us make it better.  If you are not interested in contributing - no problem - we would  still love to hear your comments, questions, and concerns.

Thanks for your time and we look forward to hearing your thoughts on Weave.

The Continuuity Team
oss@continuuity.com