You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@reef.apache.org by we...@apache.org on 2015/02/05 01:58:54 UTC
[1/2] incubator-reef git commit: [REEF-90]: Migrate REEF website code
from a private repo to REEF repo
Repository: incubator-reef
Updated Branches:
refs/heads/master 9defe611d -> fa77cc63c
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/reef-examples.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/reef-examples.md b/website/src/site/markdown/reef-examples.md
new file mode 100644
index 0000000..74f32e0
--- /dev/null
+++ b/website/src/site/markdown/reef-examples.md
@@ -0,0 +1,135 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Further REEF Examples
+
+- [Running HelloREEF on YARN](#yarn)
+ - [Prerequisites](#yarn-prerequisites)
+ - [How to configure REEF on YARN](#yarn-configurations)
+ - [How to launch HelloReefYarn](#yarn-launch)
+- [Running a REEF Webserver: HelloREEFHttp](#http)
+ - [Prerequisites](#http-prerequisites)
+ - [HttpServerShellCmdtHandler](#http-server-shell)
+- [Task Scheduler: Retaining Evaluators](#task-scheduler)
+ - [Prerequisites](#task-scheduler-prerequisites)
+ - [REST API](#task-scheduler-rest-api)
+ - [Reusing the Evaluators](#task-scheduler-reusing-evaluators)
+
+
+###<a name="yarn"></a>Running HelloREEF on YARN
+
+REEF applications can be run on multiple runtime environments. Using `HelloReefYarn`, we will see how to configure and launch REEF applications on YARN.
+
+####<a name="yarn-prerequisites"></a>Prerequisites
+
+* [You have compiled REEF locally](tutorial.html#install)
+* [YARN](http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html)
+
+####<a name="yarn-configurations"></a>How to configure REEF on YARN
+
+The only difference between running a REEF application on YARN vs locally is the runtime configuration:
+
+```
+ final LauncherStatus status = DriverLauncher
+ .getLauncher(YarnClientConfiguration.CONF.build())
+ .run(getDriverConfiguration(), JOB_TIMEOUT);
+```
+
+####<a name="yarn-launch"></a>How to launch HelloReefYarn
+
+Running `HelloReefYarn` is very similar to running `HelloREEF`:
+
+ yarn jar reef-examples/target/reef-examples-{$REEF_VERSION}-shaded.jar org.apache.reef.examples.hello.HelloREEFYarn
+
+**Note**: *The path divider may be different for different OS (e.g. Windows uses \\ while Linux uses / for dividers) so change the code as needed.*
+
+You can see how REEF applications work on YARN environments in [Introduction to REEF](introduction.html).
+
+###<a name="http"></a>Running a REEF Webserver: HelloREEFHttp
+
+REEF also has a webserver interface to handle HTTP requests. This webserver can be utilized in many different manners such as in Interprocess Communcation or in conjuction with the REST API.
+
+To demonstrate a possible use for this interface, `HelloREEFHttp` serves as a simple webserver to execute shell commands requested from user input. The first thing we should do is register a handler to receive the HTTP requests.
+
+####<a name="http-prerequisites"></a>Prerequisites
+
+* [You have compiled REEF locally](tutorial.html#install)
+
+####<a name="http-server-shell"></a>HttpServerShellCmdtHandler
+
+`HttpServerShellCmdtHandler` implements `HttpHandler` but three methods must be overridden first: `getUriSpecification`, `setUriSpecification`, and `onHttpRequest`.
+
+- <a name="http-urispecification"></a>
+`UriSpecification` defines the URI specification for the handler. More than one handler can exist per application and thus each handler is distinguished using this specification. Since `HelloREEFHttp` defines `UriSpecification` as `Command`, an HTTP request looks like `http://{host_address}:{host_port}/Command/{request}`.
+
+- <a name="http-onhttprequest"></a>
+`onHttpRequest` defines a hook for when an HTTP request for this handler is invoked.
+
+###<a name="task-scheduler"></a>Retaining Evaluators: Task Scheduler
+
+Another example is Task scheduler. Getting commands from users using the REST API, it allocates multiple evaluators and submits the tasks.
+
+It is a basic Task Scheduler example using Reef-webserver. The application receives the task (shell command) list from user and execute the tasks in a FIFO order.
+
+####<a name="task-scheduler-prerequisites"></a>Prerequisites
+
+* [You have compiled REEF locally](tutorial.html#install)
+* [Running REEF Webserver : HelloREEFHttp](#http)
+
+####<a name="task-scheduler-rest-api"></a>REST API
+
+Users can send the HTTP request to the server via URL :
+
+ http://{address}:{port}/reef-example-scheduler/v1
+
+And the possible requests are as follows:
+
+* `/list`: lists all the tasks' statuses.
+* `/clear`: clears all the tasks waiting in the queue and returns how many tasks have been removed.
+* `/submit?cmd=COMMAND`: submits a task to execute COMMAND and returns the task id.
+* `/status?id=ID`: returns the status of the task with the id, "ID".
+* `/cancel?id=ID`: cancels the task with the id, "ID".
+* `/max-eval?num={num}`: sets the maximum number of evaluators.
+
+The result of each task is written in the log files - both in the driver's and the evaluators'.
+
+####<a name="task-scheduler-reusing-evaluators"></a>Reusing the Evaluators
+
+You can find the method `retainEvaluator()` in SchedulerDriver:
+
+```
+ /**
+ * Retain the complete evaluators submitting another task
+ * until there is no need to reuse them.
+ */
+ private synchronized void retainEvaluator(final ActiveContext context) {
+ if (scheduler.hasPendingTasks()) {
+ scheduler.submitTask(context);
+ } else if (nActiveEval > 1) {
+ nActiveEval--;
+ context.close();
+ } else {
+ state = State.READY;
+ waitForCommands(context);
+ }
+ }
+```
+
+When `Task` completes, `EventHandler` for `CompletedTask` event is invoked. An instance of `CompletedTask` is then passed using the parameter to get the `ActiveContext` object from the `CompletedTask`. We can reuse this `Evaluator` by submitting another `Task` to it if there is a task to launch.
+
+Using the `-retain false` argument disables this functionality and allocates a new evalutor for every task.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/tang.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/tang.md b/website/src/site/markdown/tang.md
new file mode 100644
index 0000000..a43cd29
--- /dev/null
+++ b/website/src/site/markdown/tang.md
@@ -0,0 +1,455 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Tang
+
+Tang is a configuration managment and checking framework that emphasizes explicit documentation and automatic checkability of configurations and applications instead of ad-hoc, application-specific configuration and bootstrapping logic. It supports distributed, multi-language applications, but gracefully handles simpler use cases as well.
+
+Tang makes use of dependency injection to automatically instantiate applications. Dependency injectors can be thought of as "make for objects" -- given a request for some type of object, and information that explains how dependencies between objects should be resolved, dependency injectors automatically instantiate the requested object and all of the objects it dependes upon. Tang makes use of a few simple wire formats to support remote and even cross-language dependency injection.
+
+Outline
+-------
+
+ * [Motivation](#motivation)
+ * [Design principles](#design-principles)
+ * [Tutorial: Getting started](#tutorial-getting-started)
+ * [Defining configuration parameters](#configuration-parameters)
+ * [Configuration Modules](#configuration-modules)
+ * [Injecting objects with getInstance()](#injecting-objects-with-getinstance)
+ * [Cyclic Injections](#cyclic-injections)
+ * [Alternative configuration sources](#alternative-configuration-sources)
+ * [Raw configuration API](#raw-configuration-api)
+ * [Looking under the hood](#looking-under-the-hood)
+ * [InjectionPlan](#injectionPlan)
+ * [ClassHierarchy](#classHierarchy)
+
+
+<a name="motivation"></a>Motivation
+============
+
+Distributed systems suffer from problems that arise due to complex compositions of software modules and configuration errors. These problems compound over time: best-practice object oriented design dictates that code be factored into independent reusable modules, and today's distributed applications are increasingly expected to run atop multiple runtime environments. This leads application developers to push complexity into configuration settings, to the point where misconfiguration is now a primary cause of unavailability in fault tolerant systems.
+
+Tang is our attempt to address these problems. It consists of a dependency injection framework and a set of configuration and debugging tools that automatically and transparently bootstrap applications. We have focused on providing a narrow set of primitives that support the full range of design patterns that arise in distributed system development, and that encourage application developers to build their systems in a maintainable and debuggable way.
+
+Tang leverages existing language type systems, allowing unmodified IDEs such as Eclipse or NetBeans to surface configuration information in tooltips, provide auto-complete of configuration parameters, and to detect a wide range of configuration problems as you edit your code. Since such functionality is surfaced in the tools you are already familiar with, there is no need to install (or learn) additional development software to get started with Tang. Furthermore, we provide a set of sophisticated build time and runtime tools that detect a wide range of common architectural problems and configuration errors.
+
+This documentation consists of tutorials that present prefered Tang design patterns. By structuring your application according to the patterns we suggest throughout the tutorials, you will allow our static analysis framework, Tint ("Tang Lint"), to detect problematic design patterns and high-level configuration problems as part of your build. These patterns provide the cornerstone for a number of more advanced features, such as interacting with legacy configuration systems, designing for cross-language applications, and multi-tenancy issues, such as secure injections of untrusted application code. To the best of our knowledge, implementing such tools and addressing these real-world implementation constraints would be difficult, or even impossible, atop competing frameworks.
+
+<a name="design-principles"></a>Design principles
+=================
+
+Tang encourages application developers to specify default implementations and constructor parameters in terms of code annotations and configuration modules. This avoids the need for a number of subtle (and often confusing) dependency injection software patterns, though it does lead to a different approach to dependency injection than other frameworks encourage.
+
+In the process of building complicated systems built atop Tang, we found that, as the length of configurations that are passed around at runtime increased, it rapidly became impossible to debug or maintain our higher-level applications. In an attempt to address this problem, traditional dependency injection systems actually compound this issue. They encourage the developers of each application-level component to implement hand-written "Modules" that are executed at runtime. Hand-written modules introspect on the current runtime configuration, augment and modify it, and then return a new configuration that takes the new application component into account.
+
+In other systems, developers interact with modules by invoking ad-hoc builder methods, and passing configurations (in the correct order) from module to module. Modules frequently delgate to each other, either via inheritance or wrappers. This makes it difficult for developers and end-users to figure out which value of a given parameter will be used, or even to figure out why it was (or was not) set.
+
+Tang provides an alternative called `ConfigurationModule`s:
+
+
+- `Configurations` and `ConfigurationModules` are "just data," and can be read and written in human readable formats.
+- Interfaces and configuration parameters are encouraged to specify defaults, significantly shortening the configurations generated at runtime, and making it easy to see what was "strange" about a given run of the application.
+- Tang's static analysis and documentation tools sanity check `ConfigurationModule`s, and document their behavior and any extra parameters they export.
+- Configuration options can be set at most once. This avoids (or at least detects) situations in which users and application-level code inadvertantly "fight" over the setting of a particular option.
+
+The last property comes from Tang's use of _monotonic_ set oriented primitives. This allows us to leverage recent theoretical results in commtative data types; particularly CRDTs, and the CALM theorem. Concretely:
+
+- A large subset of Tang's public API is commutative, which frees application-level configuration and bootstrapping logic from worrying about the order in which configuration sources are processed at runtime.
+- Tang can detect configuration and injection problems much earlier than is possible with other approaches. Also, upon detecting a conflict, Tang lists the configuration sources that contributed to the problem.
+
+
+Finally, Tang is divided into a set of "core" primtives, and higher-level configuration "formats". Tang's core focuses on dependency injection and static checking of configurations. The formats provide higher-level configuration languages primitives, such as distributed, cross-language injection, configuration files, and `ConfigurationModule`. Each Tang format imports and/or exports standard Tang `Configuration` objects, which can then be composed with other configuration data at runtime.
+
+Improvements to these formats are planned, such as command-line tab completion, and improved APIs for extremely complex applications that are built by composing multiple Tang configurations to inject arbitrary object graphs.
+Furthermore, Tang formats include documentation facilities, and automatic command line and configuration file parsing. From an end-user perspective, this takes a lot of the guesswork out of configuration file formats.
+
+Although Tang surfaces a text-based interface for end-users of the applications built atop it, all configuration options and their types are specified in terms of Java classes and annotations. As with the core Tang primitives, this allows the Java compiler to statically check Tang formats for problems such as inconsistent usage of configuration parameters, naming conflicts and so on. This eliminates broad classes of runtime errors. These checks can be run independently of the application's runtime environment, and can find problems both in the Java-level implementation of the system, and with user-provided configuration files. The tools that perform these checks are designed to run as a post-processing step of projects built atop Tang. Like the Java compiler checks, this prevents such errors from making it to production environments. It also prevents such errors from being exposed to application logic or end-users, greatly simplifying applications built atop Tang.
+
+Taken together, these properties greatly simplify dependency injection in distributed environments. We expect Tang to be used in environments that are dominated by "plugin"-style APIs with many alternative implementations. Tang cleanly separates concerns over configuration management, dependency injection and object implementations, which hides most of the complexity of dependency injection from plugin implementers. It also prevents plugin implementations from inadvertently conflicting with each other or their runtime environements. Such clean semantics are crucial in distributed, heterogeneous environments.
+
+<a name="tutorial-getting-started"></a>Tutorial: Getting started
+=========================
+
+This tutorial is geared toward people that would like to quickly get started with Tang, or that are modifying an existing Tang application.
+
+<a name="configuration-parameters"></a>Constructors, @Inject and @Parameter
+------------------------
+
+Suppose you are implementing a new class, and would like to automatically pass configuration parameters to it at runtime:
+
+ package com.example;
+
+ public class Timer {
+ private final int seconds;
+
+ public Timer(int seconds) {
+ if(seconds < 0) {
+ throw new IllegalArgumentException("Cannot sleep for negative time!");
+ }
+ this.seconds = seconds;
+ }
+
+ public void sleep() throws Exception {
+ java.lang.Thread.sleep(seconds * 1000);
+ }
+ }
+
+Tang encourages applications to use Plain Old Java Objects (POJOs), and emphasizes the use of immutable state for configuration parameters. This reduces boiler plate (there is no need for extra setter methods), and does not interfere with encapsulation (the fields and even the constructor can be private). Furthermore, it is trivial for well-written classes to ensure that all objects are completely and properly instantiated: They simply need to check constructor parameters as any other POJO would, except that Tang never passes `null` references into constructors, allowing their implementations to assume that all parameter values are non-null.
+
+Tang aims to provide end users with error messages as early as possible, and encourages developers to throw exceptions inside of constructors. This allows it to automatically provide additional information to end-users when things go wrong:
+
+ Exception in thread "main" org.apache.reef.tang.exceptions.InjectionException: Could not invoke constructor: new Timer(Integer Seconds = -1)
+ at org.apache.reef.tang.implementation.java.InjectorImpl.injectFromPlan(InjectorImpl.java:585)
+ at org.apache.reef.tang.implementation.java.InjectorImpl.getInstance(InjectorImpl.java:449)
+ at org.apache.reef.tang.implementation.java.InjectorImpl.getInstance(InjectorImpl.java:466)
+ at org.apache.reef.tang.examples.Timer.main(Timer.java:48)
+ Caused by: java.lang.IllegalArgumentException: Cannot sleep for negative time!
+ at org.apache.reef.tang.examples.Timer.<init>(Timer.java:25)
+ at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
+ at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
+ at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
+ at java.lang.reflect.Constructor.newInstance(Unknown Source)
+ at org.apache.reef.tang.implementation.java.InjectorImpl.injectFromPlan(InjectorImpl.java:569)
+ ... 3 more
+
+In order for Tang to instantiate an object, we need to annotate the constructor with an `@Inject` annotation. While we're at it, we'll define a configuration parameter, allowing us to specify seconds on the command line and in a config file:
+
+ package com.example;
+
+ import javax.inject.Inject;
+
+ import org.apache.reef.tang.annotations.Name;
+ import org.apache.reef.tang.annotations.NamedParameter;
+ import org.apache.reef.tang.annotations.Parameter;
+
+ public class Timer {
+ @NamedParameter(default_value="10",
+ doc="Number of seconds to sleep", short_name="sec")
+ class Seconds implements Name<Integer> {}
+ private final int seconds;
+
+ @Inject
+ public Timer(@Parameter(Seconds.class) int seconds) {
+ if(seconds < 0) {
+ throw new IllegalArgumentException("Cannot sleep for negative time!");
+ }
+ this.seconds = seconds;
+ }
+
+ public void sleep() throws Exception {
+ java.lang.Thread.sleep(seconds * 1000);
+ }
+ }
+
+A few things happened here. First, we create the new configuration parameter by declaring a dummy class that implements Tang's `Name` interface. `Name` is a generic type with a single mandatory parameter that specifies the type of object to be passed in. Since `Seconds` implements `Name<Integer>`, it is a parameter called `Seconds` that expects `Integer` values. More precisely, `Seconds` is actually named `com.example.Timer.Seconds`. This reliance on language types to define parameter names exposes parameters to the compiler and IDE. Concretely:
+
+ * `javac` maps from `Seconds` to the full class name in the usual way, preventing parameters with the same name, but in different packages from conflicting.
+ * The Java classloader ensures that classes are unique at runtime.
+ * Standard IDE features, such as code navigation, completion and refactoring work as they normally would for class names.
+
+
+All instances of `Name` must be annotated with `@NamedParameter`, which takes the following optional parameters:
+
+ * `default_value`: The default value of the constructor parameter, encoded as a string. Tang will parse this value (and ones in config files and on the command line), and pass it into the constructor. For convenience Tang includes a number of helper variants of default value. `default_class` takes a Class (instead of a String), while `default_values` and `default_classes` take sets of values.
+ * `short_name`: The name of the command line option associated with this parameter. If omitted, no command line option will be created. Short names must be registered by calling `registerShortName()` on the instance of `org.apache.reef.tang.formats.CommandLine` that will process the command line options.
+ * `doc` (optional): Human readable documentation that describes the purpose of the parameter.
+
+Tang only invokes constructors that have been annotated with `@Inject`. This allows injectable constructors to coexist with ones that should not be invoked via dependency injection (such as ones with destructive side effects, or that expect `null` references). Constructor parameters must not be ambiguous. If two parameters in the same constructor have the same type, then they must be annotated with `@Parameter`, which associates a named parameter with the argument. Furthermore, two parameters to the same constructor cannot have the same name. This allows Tang to safely invoke constructors without exposing low level details (such as parameter ordering) as configuration options.
+
+<a name="configuration-modules"></a>Configuration modules
+---------
+
+Configuration modules allow applications to perform most configuration generation and verification tasks at build time. This allows Tang to automatically generate rich configuration-related documentation, to detect problematic design patterns, and to report errors before the application even begins to run.
+
+In the example below, we extend the Timer API to include a second implementation that simply outputs the amount of
+time a real timer would have slept to stderr. In a real unit testing example, it would likely interact with a scheduler based on logical time. Of course, in isolation, having the ability to specify configuration parameters is not particularly useful; this example also adds a `main()` method that invokes Tang, and instantiates an object.
+
+The process of instantiting an object with Tang is called _injection_. As with configurations, Tang's injection process is designed to catch as many potential runtime errors as possible before application code begins to run. This simplifies debugging and eliminates many types of runtime error handling code, since many configurations can be caught before running (or examining) application-specific initialization code.
+
+ package org.apache.reef.tang.examples.timer;
+
+ import javax.inject.Inject;
+
+ import org.apache.reef.tang.Configuration;
+ import org.apache.reef.tang.Tang;
+
+ import org.apache.reef.tang.annotations.DefaultImplementation;
+ import org.apache.reef.tang.annotations.Name;
+ import org.apache.reef.tang.annotations.NamedParameter;
+ import org.apache.reef.tang.annotations.Parameter;
+
+ import org.apache.reef.tang.exceptions.BindException;
+ import org.apache.reef.tang.exceptions.InjectionException;
+
+ import org.apache.reef.tang.formats.ConfigurationModule;
+ import org.apache.reef.tang.formats.ConfigurationModuleBuilder;
+ import org.apache.reef.tang.formats.OptionalParameter;
+
+ @DefaultImplementation(TimerImpl.class)
+ public interface Timer {
+ @NamedParameter(default_value="10",
+ doc="Number of seconds to sleep", short_name="sec")
+ public static class Seconds implements Name<Integer> { }
+ public void sleep() throws Exception;
+ }
+
+ public class TimerImpl implements Timer {
+
+ private final int seconds;
+ @Inject
+ public TimerImpl(@Parameter(Timer.Seconds.class) int seconds) {
+ if(seconds < 0) {
+ throw new IllegalArgumentException("Cannot sleep for negative time!");
+ }
+ this.seconds = seconds;
+ }
+ @Override
+ public void sleep() throws Exception {
+ java.lang.Thread.sleep(seconds);
+ }
+
+ }
+
+ public class TimerMock implements Timer {
+
+ public static class TimerMockConf extends ConfigurationModuleBuilder {
+ public static final OptionalParameter<Integer> MOCK_SLEEP_TIME = new OptionalParameter<>();
+ }
+ public static final ConfigurationModule CONF = new TimerMockConf()
+ .bindImplementation(Timer.class, TimerMock.class)
+ .bindNamedParameter(Timer.Seconds.class, TimerMockConf.MOCK_SLEEP_TIME)
+ .build();
+
+ private final int seconds;
+
+ @Inject
+ TimerMock(@Parameter(Timer.Seconds.class) int seconds) {
+ if(seconds < 0) {
+ throw new IllegalArgumentException("Cannot sleep for negative time!");
+ }
+ this.seconds = seconds;
+ }
+ @Override
+ public void sleep() {
+ System.out.println("Would have slept for " + seconds + "sec.");
+ }
+
+ public static void main(String[] args) throws BindException, InjectionException, Exception {
+ Configuration c = TimerMock.CONF
+ .set(TimerMockConf.MOCK_SLEEP_TIME, 1)
+ .build();
+ Timer t = Tang.Factory.getTang().newInjector(c).getInstance(Timer.class);
+ System.out.println("Tick...");
+ t.sleep();
+ System.out.println("...tock.");
+ }
+ }
+
+Again, there are a few things going on here:
+
+ - First, we push the implementation of `Timer` into a new class, `TimerImpl`. The `@DefaultImplementation` tells Tang to use `TimerImpl` when no other implementation is explicitly provided.
+ - We leave the `Sleep` class in the Timer interface. This, plus the `@DefaultImplementation` annotation maintain backward compatibility with code that used Tang to inject the old `Timer` class.
+ - The `TimerMock` class includes a dummy implementation of Timer, along with a `ConfigurationModule` final static field called `CONF`.
+ - The main method uses `CONF` to generate a configuration. Rather than set `Timer.Sleep` directly, it sets `MOCK_SLEEP_TIME`. In a more complicated example, this would allow `CONF` to route the sleep time to testing infrastructure, or other classes that are specific to the testing environment or implemenation of `TimerMock`.
+
+`ConfigurationModule`s serve a number of purposes:
+
+ - They allow application and library developers to encapsulate the details surrounding their code's instantiation.
+ - They provide Java APIs that expose `OptionalParameter`, `RequiredParameter`, `OptionalImplementation`, `RequiredImpementation` fields. These fields tell users of the ConfigurationModule which subsystems of the application require which configuration parameters, and allow the author of the ConfigurationModule to use JavaDoc to document the parameters they export.
+ - Finally, because ConfigurationModule data structures are populated at class load time (before the application begins to run), they can be inspected by Tang's static analysis tools.
+
+These tools are provided by `org.apache.reef.tang.util.Tint`, which is included by default in all Tang builds. As long as Tang is on the classpath, invoking:
+
+ java org.apache.reef.tang.util.Tint --doc tangdoc.html
+
+will perform full static analysis of all classes the class path, and emit a nicely formatted HTML document. The documentation generated by Tint includes cross-references between configuration options, interfaces, classes, and the `ConfigurationModules` that use and set them.
+
+Here are some sample Tint errors. These (and others) can be run by passing `--tang-tests` into Tint, and ensuring that Tang's unit tests are on the class path.:
+
+ interface org.apache.reef.tang.MyEventHandlerIface declares its default implementation to be non-subclass class org.apache.reef.tang.MyEventHandler
+ class org.apache.reef.tang.WaterBottleName defines a default class org.apache.reef.tang.GasCan with a type that does not extend its target's type org.apache.reef.tang.Bottle<org.apache.reef.tang.Water>
+ Named parameters org.apache.reef.tang.examples.Timer$Seconds and org.apache.reef.tang.examples.TimerV1$Seconds have the same short name: sec
+ Named parameter org.apache.reef.tang.implementation.AnnotatedNameMultipleInterfaces implements multiple interfaces. It is only allowed to implement Name<T>
+ Found illegal @NamedParameter org.apache.reef.tang.implementation.AnnotatedNotName does not implement Name<?>
+ interface org.apache.reef.tang.implementation.BadIfaceDefault declares its default implementation to be non-subclass class java.lang.String
+ class org.apache.reef.tang.implementation.BadName defines a default class java.lang.Integer with a raw type that does not extend of its target's raw type class java.lang.String
+ Named parameter org.apache.reef.tang.implementation.BadParsableDefaultClass defines default implementation for parsable type java.lang.String
+ Class org.apache.reef.tang.implementation.DanglingUnit has an @Unit annotation, but no non-static inner classes. Such @Unit annotations would have no effect, and are therefore disallowed.
+ Cannot @Inject non-static member class unless the enclosing class an @Unit. Nested class is:org.apache.reef.tang.implementation.InjectNonStaticLocalType$NonStaticLocal
+ Named parameter org.apache.reef.tang.implementation.NameWithConstructor has a constructor. Named parameters must not declare any constructors.
+ Named parameter type mismatch. Constructor expects a java.lang.String but Foo is a java.lang.Integer
+ public org.apache.reef.tang.implementation.NonInjectableParam(int) is not injectable, but it has an @Parameter annotation.
+ Detected explicit constructor in class enclosed in @Unit org.apache.reef.tang.implementation.OuterUnitBad$InA Such constructors are disallowed.
+ Repeated constructor parameter detected. Cannot inject constructor org.apache.reef.tang.implementation.RepeatConstructorArg(int,int)
+ Named parameters org.apache.reef.tang.implementation.ShortNameFooA and org.apache.reef.tang.implementation.ShortNameFooB have the same short name: foo
+ Named parameter org.apache.reef.tang.implementation.UnannotatedName is missing its @NamedParameter annotation.
+ Field org.apache.reef.tang.formats.MyMissingBindConfigurationModule.BAD_CONF: Found declared options that were not used in binds: { FOO_NESS }
+
+<a name="injnecting-objects-with-getInstance"></a>Injecting objects with getInstance()
+--------------------------------------
+
+Above, we explain how to register constructors with Tang, and how to configure Tang to inject the desired objects at runtime. This section explains how Tang actually instantiates objects, and how the primitives it provides can be combined to support sophisticated application architectures.
+
+In order to instantiate objects with Tang, one must invoke Tang.Factory.getTang().newInjector(Configuration...). This returns a new "empty" injector that will honor the configuration options that were set in the provided configurations, and that will have access to a merged version of the classpath they refer to.
+
+In a given Tang injector, all classes are treated as singletons: at most one instance of each class may exist. Furthermore, Tang Configuration objects are designed to be built up from trees of related (but non-conflicting) configuration files, command line parameters, and so on. At first, this may seem to be overly restrictive, since it prevents applications from creating multiple instances of the same class (or even two classes that require different values of the same named parameter).
+
+Tang addresses this by providing the runtime environment more explicit control over object and configuration parameter scopes. Taken together, `forkInjector()`, `bindVolatile()` and `InjectionFuture<T>` allow Tang to inject arbitrary sets of objects (including ones with multiple instances of the same class).
+
+Other injection frameworks take a different approach, and allow class implementations to decide if they should be singletons across a given JVM (e.g., with an `@Singleton` annotation), user session (for web services), user connection, and so on. This approach has at least two problems:
+
+ * It is not general purpose: after all, it encodes deployment scenarios into the injection framework and application APIs!
+ * It trades one barrier to composability and reuse: _hard-coded constructor invocations_ with another: _hard-coded runtime environments_. The former prevents runtime environments from adapting to application-level changes, while the latter prevents application code from adapting to new runtimes.
+
+Tang's approach avoids both issues by giving the implementation of the runtime environment explicit control over object scopes and lifetimes.
+
+`forkInjector()` makes a copy of a given injector, including references to all the objects already instantiated by the original injector. This allows runtime environments to implement scopes. First, a root injector is created. In order to create a child scope, the runtime simply invokes `forkInjector()` on the root context, and optionally passes additional `Configuration` objects in. These additional configurations allow the runtime to specialize the root context.
+
+Although the forked injector will have access to any objects and configuration bindings that existed when `forkInjector()` was called, neither the original nor the forked injectors will reflect future changes to the other injector.
+
+The second primitive, `bindVolatile()`, provides an injector with an instance of a class or named parameter. The injector treats this instance as though it had injected the object directly. This:
+
+ * allows passing of information between child scopes
+ * makes it possible to create (for example) chains of objects of the same type
+ * and allows objects that cannot be instantiated via Tang to be passed into injectable constructors.
+
+###<a name="cyclic-injections"></a>Cyclic Injections
+
+Although the above primitives allow applications to inject arbitrary DAGs (directed acyclic graphs) of objects, they do not support cycles of objects. Tang provides the `InjectionFuture<T>` interfaces to support such _cyclic injections_.
+
+When Tang encounters a constructor parameter of type `InjectionFuture<T>`, it injects an object that provides a method `T get()` that returns an injected instance of `T`.
+
+
+This can be used to break cycles:
+
+ A(B b) {...}
+ B(InjectionFuture<A> a) {...}
+
+In order to inject an instance of `A`, Tang first injects an instance of `B` by passing it an `InjectionFuture<A>`. Tang then invoke's `A`'s constructor, passing in the instance of `B`. Once the constructor returns, the new instance of `A` is passed into `B`'s `InjectionFuture<A>`. At this point, it becomes safe for `B` to invoke `get()`, which establishes the circular reference.
+
+Therefore, along with `forkInjector()` and `bindVolatile()`, this allows Tang to inject arbitrary graphs of objects. This pattern avoids non-final fields (once set, all fields of all objects are constant), and it also avoids boiler plate error handling code that checks to see if `B`'s instance of `A` has been set.
+
+
+When `get()` is called after the application-level call to `getInstance()` returns, it is guranteed to return a non-null reference to an injected instance of the object. However, if `get()` is called _before_ the constructor it was passed to returns, then it is guaranteed to throw an exception. In between these two points in time, `get()`'s behavior is undefined, but, for the sake of race-detection and forward compatibility it makes a best-effort attempt to throw an exception.
+
+Following Tang's singleton semantics, the instance returned by `get()` will be the same instance the injector would pass into other constructors or return from `getInstance()`.
+
+<a name="alternative-configuration-sources"></a>Alternative configuration sources
+=================================
+
+Tang provides a number of so-called _formats_ that interface with external configuration data. `ConfigurationModule` is one such example (see above). These formats transform configuration data to and from Tang's raw configuration API. The raw API provides an implementation of ConfigurationBuilder, which implements most of Tang's configuration checks. It also provides a `JavaConfigurationBuilder` interface provides convenience methods that take Java Classes, and leverage Java's generic type system to push a range of static type checks to Java compilation time.
+
+<a name="raw-configuration-api"></a>Raw configuration API
+---------
+Tang also provides a lower level configurtion API for applications that need more dynamic control over their configurations:
+
+ ...
+ import org.apache.reef.tang.Tang;
+ import org.apache.reef.tang.ConfigurationBuilder;
+ import org.apache.reef.tang.Configuration;
+ import org.apache.reef.tang.Injector;
+ import org.apache.reef.tang.exceptions.BindException;
+ import org.apache.reef.tang.exceptions.InjectionException;
+
+ ...
+ public static void main(String[] args) throws BindException, InjectionException {
+ Tang tang = Tang.Factory.getTang();
+ ConfigurationBuilder cb = (ConfigurationBuilder)tang.newConfigurationBuilder();
+ cb.bindNamedParameter(Timer.Seconds.class, 5);
+ Configuration conf = cb.build();
+ Injector injector = tang.newInjector(conf);
+ if(!injector.isInjectable(Timer.class)) {
+ System.err.println("If isInjectable returns false, the next line will throw an exception");
+ }
+ Timer timer = injector.getInstance(Timer.class);
+
+ try {
+ System.out.println("Tick...");
+ timer.sleep();
+ System.out.println("Tock.");
+ } catch(InterruptedException e) {
+ e.printStackTrace();
+ }
+ }
+
+The first step in using Tang is to get a handle to a Tang object by calling "Tang.Factory.getTang()". Having obtained a handle, we run through each of the phases of a Tang injection:
+
+ * We use `ConfigurationBuilder` objects to tell Tang about the class hierarchy that it will be using to inject objects and (in later examples) to register the contents of configuration files, override default configuration values, and to set default implementations of classes. `ConfigurationBuilder` and `ConfigurationModuleBuider` export similar API's. The difference is that `ConfigurationBuilder` produces `Configuration` objects directly, and is designed to be used at runtime. `ConfigurationModuleBuilder` is desgined to produce data structures that will be generated and analyzed during the build, and at class load time.
+ * `bindNamedParameter()` overrides the default value of Timer.Sleep, setting it to 5. Tang inteprets the 5 as a string, but allows instances of Number to be passed in as syntactic sugar.
+ * We call `.build()` on the `ConfigurationBuilder`, creating an immutable `Configuration` object. At this point, Tang ensures that all of the classes it has encountered so far are consistent with each other, and that they are suitable for injection. When Tang encounters conflicting classes or configuration files, it throws a `BindException` to indicate that the problem is due to configuration issues. Note that `ConfigurationBuilder` and `Configuration` do not determine whether or not a particular injection will succeed; that is the business of the _Injector_.
+ * To obtain an instance of Injector, we pass our Configuration object into `tang.newInjector()`.
+ * `injector.isInjectable(Timer.class)` checks to see if Timer is injectable without actually performing an injection or running application code. (Note that, in this example, the Java classloader may have run application code. For more information, see the advanced tutorials on cross-language injections and securely building configurations for untrusted code.)
+ * Finally, we call `injector.getInstance(Timer.class)`. Internally, this method considers all possible injection plans for `Timer`. If there is exactly one such plan, it performs the injection. Otherwise, it throws an `InjectionException`.
+
+Tang configuration information can be divided into two categories. The first type, _parameters_, pass values such as strings and integers into constructors. Users of Tang encode configuration parameters as strings, allowing them to be stored in configuration files, and passed in on the command line.
+
+The second type of configuration option, _implementation bindings_, are used to tell Tang which implementation should be used when an instance of an interface is requested. Like configuration parameters, implementation bindings are expressible as strings: Tang configuration files simply contain the raw (without the generic parameters) name of the Java Classes to be bound together.
+
+New parameters are created and passed into constructors as in the examples above, by creating implementations of `Name<T>`, and adding `@NamedParameter`, `@Parameter` and `@Inject` annotations as necessary. Specifying implementations for interfaces is a bit more involved, as a number of subtle use cases arise.
+
+However, all configuration settings in Tang can be unambiguously represented as a `key=value` pair that can be interpreted either asan `interface=implementation` pair or a `configuration_parameter=value` pair. This maps well to Java-style properties files. For example:
+
+ com.examples.Interface=com.examples.Implementation
+
+
+tells Tang to create a new Implementation each time it wants to invoke a constructor that asks for an instance of Interface. In most circumstances, Implementation extends or implements Interface (`ExternalConstructors` are the exception -- see the next section). In such cases, Tang makes sure that Implementation contains at least one constructor with an `@Inject` annotation, and performs the binding.
+
+See the `ConfigurationFile` API for more information about processing configuration files in this format.
+
+
+
+<a name="looking-under-the-hood"></a>Looking under the hood
+----------------------
+
+###<a name="injectionPlan"></a>InjectionPlan
+
+InjectionPlan objects explain what Tang would do to instantiate a new object, but don't actually instantiate anything.
+Add the following lines to the Timer example;
+
+ import org.apache.reef.tang.implementation.InjectionPlan;
+ import org.apache.reef.tang.implementation.InjectorImpl;
+ ...
+ InjectorImpl injector = (InjectorImpl)tang.newInjector(conf);
+ InjectionPlan<Timer> ip = injector.getInjectionPlan(Timer.class);
+ System.out.println(ip.toPrettyString());
+ System.out.println("Number of plans:" + ip.getNumAlternatives());
+
+
+Running the program now produces a bit of additional output:
+
+ new Timer(Integer Seconds = 10)
+ Number of plans:1
+
+
+InjectionPlan objects can be serialized to protocol buffers. The following file documents their format:
+
+[https://github.com/apache/incubator-reef/blob/master/reef-tang/tang/src/main/proto/injection_plan.proto](https://github.com/apache/incubator-reef/blob/master/reef-tang/tang/src/main/proto/injection_plan.proto)
+
+###<a name="classHierarchy"></a>ClassHierarchy
+
+InjectionPlan explains what would happen if you asked Tang to take some action, but it doesn't provide much insight into Tang's view of the object hierarchy, parameter defaults and so on. ClassHierarchy objects encode the state that Tang gets from .class files, including class inheritance relationships, parameter annotations, and so on.
+
+Internally, in the example above, TypeHierarchy walks the class definition for Timer, looking for superclasses, interfaces, and classes referenced by its constructors.
+
+ClassHierarchy objects can be serialized to protocol buffers. The following file documents their format:
+
+[https://github.com/apache/incubator-reef/blob/master/reef-tang/tang/src/main/proto/class_hierarchy.proto](https://github.com/apache/incubator-reef/blob/master/reef-tang/tang/src/main/proto/class_hierarchy.proto)
+
+The java interfaces are available in this package:
+
+[https://github.com/apache/incubator-reef/tree/master/reef-tang/tang/src/main/java/org/apache/reef/tang/types](https://github.com/apache/incubator-reef/tree/master/reef-tang/tang/src/main/java/org/apache/reef/tang/types)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/tutorial.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/tutorial.md b/website/src/site/markdown/tutorial.md
new file mode 100644
index 0000000..a81729e
--- /dev/null
+++ b/website/src/site/markdown/tutorial.md
@@ -0,0 +1,137 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#REEF Tutorial
+
+- [Installing and Compiling REEF](#install)
+- [Running HelloREEF](#running-reef)
+ - [Local](#local)
+ - [HelloREEFNoClient](#helloREEFNoClient)
+ - [YARN](reef-examples.html#yarn)
+- [Further Examples](#further-examples)
+
+
+###<a name="install"></a>Installing and Compiling REEF
+
+
+####Requirements
+
+- [Java](http://www.oracle.com/technetwork/java/index.html) 7 Development Kit
+- [Maven 3](http://maven.apache.org/) or newer. Make sure that `mvn` is in your `PATH`.
+- [Protocol Buffers Compiler (protoc) 2.5](http://code.google.com/p/protobuf/) Make sure that protoc is in your PATH. **Note**: You need to install version 2.5. Newer versions won't work.
+
+With these requirements met, the instructions below should work regardless of OS choice and command line interpreter. On Windows, you might find [this](http://cs.markusweimer.com/2013/08/02/how-to-setup-powershell-for-github-maven-and-java-development/) tutorial helpful in setting up PowerShell with Maven, GitHub and Java. You will still have to install the [Protocol Buffers Compiler](https://code.google.com/p/protobuf/), though.
+
+####Cloning the repository
+#####Comitters
+ $ git clone https://git-wip-us.apache.org/repos/asf/incubator-reef.git
+
+#####Users
+ $ git clone git://git.apache.org/incubator-reef.git
+
+####Compiling the code
+REEF is built using Maven. Hence, a simple
+
+ $ mvn clean install
+
+should suffice. Note that we have quite a few integration tests in the default build. Hence, you might be better off using
+
+ $ mvn -TC1 -DskipTests clean install
+
+This runs one thread per core (`-TC1`) and skips the tests (`-DskipTests`)
+
+**Note**: You will see many exception printouts during the compilation of REEF with tests. Those are not, in fact, problems with the build: REEF guarantees that exceptions thrown on remote machines get serialized and shipped back to the Driver. We have extensive unit tests for that feature that produce the confusing printouts.
+
+### <a name="running-reef"></a>Running HelloREEF
+
+####Prerequisites
+
+[You have compiled REEF locally](#install).
+
+####Running your first REEF program: Hello, REEF!
+
+The module REEF Examples in the folder `reef-examples` contains several simple programs built on REEF to help you get started with development. As always, the simplest of those is our "Hello World": Hello REEF. Upon launch, it grabs a single Evaluator and submits a single Task to it. That Actvity, fittingly, prints 'Hello REEF!' to stdout. To launch it, navigate to `REEF_HOME` and use the following command:
+
+ java -cp reef-examples/target/reef-examples-{$REEF_VERSION}-incubating-shaded.jar org.apache.reef.examples.hello.HelloREEF
+
+**Note**: *The path divider may be different for different OS (e.g. Windows uses \\ while Linux uses / for dividers) and the version number of your version of REEF must be placed instead of the \* so change the code as needed.*
+
+This invokes the shaded jar within the target directory and launches HelloREEF on the local runtime of REEF. During the run, you will see something similar to this output:
+
+ Powered by
+ ___________ ______ ______ _______
+ / ______ / / ___/ / ___/ / ____/
+ / _____/ / /__ / /__ / /___
+ / /\ \ / ___/ / ___/ / ____/
+ / / \ \ / /__ / /__ / /
+ /__/ \__\ /_____/ /_____/ /__/
+
+ ...
+ INFO: REEF Version: 0.10.0-incubating
+ ...
+ INFO: The Job HelloREEF is running.
+ ...
+ INFO: The Job HelloREEF is done.
+ ...
+ INFO: REEF job completed: COMPLETED
+
+####Where's the output?
+
+The local runtime simulates a cluster of machines: It executes each Evaluator in a separate process on your machine. Hence, the Evaluator that printed "Hello, REEF" is not executed in the same process as the program you launched above. So, how do you get to the output of the Evaluator? The local runtime generates one folder per job it executes in `REEF_LOCAL_RUNTIME`:
+
+ > cd REEF_LOCAL_RUNTIME
+ > ls HelloREEF*
+ Mode LastWriteTime Length Name
+ ---- ------------- ------ ----
+ d---- 1/26/2015 11:21 AM HelloREEF-1422238904731
+
+The job folder names are comprised of the job's name (here, `HelloREEF`) and the time stamp of its submission (here, `1422238904731`). If you submit the same job multiple times, you will get multiple folders here. Let's move on:
+
+ > cd HelloREEF-1422238904731
+ > ls
+ Mode LastWriteTime Length Name
+ ---- ------------- ------ ----
+ d---- 1/26/2015 11:21 AM driver
+ d---- 1/26/2015 11:22 AM Node-1-1422238907421
+
+Inside of the job's folder, you will find one folder for the job's Driver (named `driver`) and one per Evaluator. Their name comprises of the virtual node simulated by the local runtime (here, `Node-1`) followed by the time stamp of when this Evaluator was allocated on that node, here `1422238907421`. As the HelloREEF example program only allocated one Evaluator, we only see one of these folders here. Let's peek inside:
+
+ > cd Node-1-1422238907421
+ > ls evaluator*
+ Mode LastWriteTime Length Name
+ ---- ------------- ------ ----
+ -a--- 1/26/2015 11:21 AM 1303 evaluator.stderr
+ -a--- 1/26/2015 11:21 AM 14 evaluator.stdout
+
+`evaluator.stderr` contains the output on stderr of this Evaluator, which mostly consists of logs helpful in debugging. `evaluator.stdout` contains the output on stdout. And, sure enough, this is where you find the "Hello, REEF!" message.
+
+####<a name="helloREEFNoClient"></a>The difference between HelloREEF and HelloREEFNoClient
+
+The HelloREEF application has multiple versions that all service different needs and one of these applications, `HelloREEFNoClient`, allows the creation of the Driver and Evaluators without the creation of a Client. In many scenarios involving a cluster of machines, one Client will access multiple Drivers so not every Driver needs to create a Client and that is where the `HelloREEFNoClient` application shines.
+
+Running `HelloREEFNoClient` is nearly identical to running `HelloREEF`:
+
+ java -cp reef-examples/target/reef-examples-{$REEF_VERSION}-incubating-shaded.jar org.apache.reef.examples.hello.HelloREEFNoClient
+
+**Note**: *The path divider may be different for different OS (e.g. Windows uses \\ while Linux uses / for dividers) and the version number of your version of REEF must be placed instead of the \* so change the code as needed.*
+
+and the output should be the same with `evaluator.stdout` containing the "Hello, REEF!" message.
+
+###<a name="further-examples"></a>Further Examples
+
+Further examples of using REEF can be found [here](reef-examples.html).
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/wake.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/wake.md b/website/src/site/markdown/wake.md
new file mode 100644
index 0000000..8ae279d
--- /dev/null
+++ b/website/src/site/markdown/wake.md
@@ -0,0 +1,109 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+Wake
+====
+
+Wake is an event-driven framework based on ideas from SEDA, Click, Akka and Rx. It is *general purpose* in the sense that it is designed to support computationally intensive applications as well as high performance networking, storage, and legacy I/O systems. We implemented Wake to support high-performance, scalable analytical processing systems ("big data" applications), and have used it to implement control plane logic (which requires high fanout and low latency) and the data plane (which requires high-throughput processing as well).
+
+
+Background
+----------
+Wake applications consist of asynchronous *event handlers* that run inside of *stages*. Stages provide scheduling primitives such as thread pool sizing and performance isolation between event handlers. In addition to event handler and stage APIs, Wake includes profiling tools and a rich standard library of primitives for system builders.
+
+Event driven processing frameworks improve upon the performance of threaded architectures in two ways: (1) Event handlers often have lower memory and context switching overhead than threaded solutions, and (2) event driven systems allow applications to allocate and monitor computational and I/O resources in an extremely fine-grained fashion. Modern threading packages have done much to address the first concern, and have significantly lowered concurrency control and other implementation overheads in recent years. However, fine grained resource allocation remains a challenge in threaded systems, and is Wake's primary advantage over threading.
+
+Early event driven systems such as SEDA executed each event handler in a dedicated thread pool called a stage. This isolated low-latency event handlers (such as cache lookups) from expensive high-latency operations, such as disk I/O. With a single thread pool, high-latency I/O operations can easily monopolize the thread pool, causing all of the CPUs to block on disk I/O, even when there is computation to be scheduled. With separate thread pools, the operating system schedules I/O requests and computation separately, guaranteeing that runnable computations will not block on I/O requests.
+
+This is in contrast to event-driven systems such as the Click modular router that were designed to maximize throughput for predictable, low latency event-handlers. When possible, Click aggressively chains event handlers together, reducing the cost of an event dispatch to that of a function call, and allowing the compiler to perform optimizations such as inlining and constant propagation across event handlers.
+
+Wake allows developers to trade off between these two extremes by explicitly partitioning their event handlers into stages. Within a stage, event handlers engage in *thread-sharing* by simply calling each other directly. When an event crosses a stage boundary, it is placed in a queue of similar events. The queue is then drained by the threads managed by the receiving stage.
+
+Although event handling systems improve upon threaded performance in theory, they are notoriously difficult to reason about. We kept this in mind while designing Wake, and have gone to great pains to ensure that its APIs are simple and easy to implement without sacrificing our performance goals.
+
+Other event driven systems provide support for so-called *push-based* and *pull-based* event handlers. In push-based systems, event sources invoke event handlers that are exposed by the events' destinations, while pull-based APIs have the destination code invoke iterators to obtain the next available event from the source.
+
+Wake is completely push based. This eliminates the need for push and pull based variants of event handling logic, and also allowed us to unify all error handling in Wake into a single API. It is always possible to convert between push and pull based APIs by inserting a queue and a thread boundary between the push and pull based code. Wake supports libraries and applications that use this trick, since operating systems and legacy code sometimes expose pull-based APIs.
+
+Systems such as Rx allow event handlers to be dynamically registered and torn down at runtime, allowing applications to evolve over time. This leads to complicated setup and teardown protocols, where event handlers need to reason about the state of upstream and downstream handlers, both during setup and teardown, but also when routing messages at runtime. It also encourages design patterns such as dynamic event dispatching that break standard compiler optimizations. In contrast, Wake applications consist of immutable graphs of event handlers that are built up from sink to source. This ensures that, once an event handler has been instantiated, all downstream handlers are ready to receive messages.
+
+Wake is designed to work with [Tang](tang.html), a dependency injection system that focuses on configuration and debuggability. This makes it extremely easy to wire up complicated graphs of event handling logic. In addition to making it easy to build up event-driven applications, Tang provides a range of static analysis tools and provides a simple aspect-style programming facility that supports Wake's latency and throughput profilers.
+
+
+Core API
+--------
+
+### Event Handlers
+
+Wake provides two APIs for event handler implementations. The first is the [EventHandler](https://github.com/apache/incubator-reef/blob/master/reef-wake/wake/src/main/java/org/apache/reef/wake/EventHandler.java) interface:
+
+ public interface EventHandler<T> {
+ void onNext(T value);
+ }
+
+Callers of `onNext()` should assume that it is asynchronous, and that it always succeeds. Unrecoverable errors should be reported by throwing a runtime exception (which should not be caught, and will instead take down the process). Recoverable errors are reported by invoking an event handler that contains the appropriate error handling logic.
+
+The latter approach can be implemented by registering separate event handlers for each type of error. However, for convenience, it is formalized in Wake's simplified version of the Rx [Observer](https://github.com/apache/incubator-reef/blob/master/reef-wake/wake/src/main/java/org/apache/reef/wake/rx/Observer.java) interface:
+
+ public interface Observer<T> {
+ void onNext(final T value);
+ void onError(final Exception error);
+ void onCompleted();
+ }
+
+The `Observer` is designed for stateful event handlers that need to be explicitly torn down at exit, or when errors occor. Such event handlers may maintain open network sockets, write to disk, buffer output, and so on. As with `onNext()`, neither `onError()` nor `onCompleted()` throw exceptions. Instead, callers should assume that they are asynchronously invoked.
+
+`EventHandler` and `Observer` implementations should be threadsafe and handle concurrent invocations of `onNext()`. However, it is illegal to call `onCompleted()` or `onError()` in race with any calls to `onNext()`, and the call to `onCompleted()` or `onError()` must be the last call made to the object. Therefore, implementations of `onCompleted()` and `onError()` can assume they have a lock on `this`, and that `this` has not been torn down and is still in a valid state.
+
+We chose these invariants because they are simple and easy to enforce. In most cases, application logic simply limits calls to `onCompleted()` and `onError()` to other implementations of `onError()` and `onCompleted()`, and relies upon Wake (and any intervening application logic) to obey the same protocol.
+
+### Stages
+
+Wake Stages are responsible for resource management. The base [Stage](https://github.com/apache/incubator-reef/blob/master/reef-wake/wake/src/main/java/org/apache/reef/wake/Stage.java) interface is fairly simple:
+
+ public interface Stage extends AutoCloseable { }
+
+The only method it contains is `close()` from auto-closable. This reflects the fact that Wake stages can either contain `EventHandler`s, as [EStage](https://github.com/apache/incubator-reef/blob/master/reef-wake/wake/src/main/java/org/apache/reef/wake/EStage.java) implementations do:
+
+ public interface EStage<T> extends EventHandler<T>, Stage { }
+
+or they can contain `Observable`s, as [RxStage](https://github.com/apache/incubator-reef/blob/master/reef-wake/wake/src/main/java/org/apache/reef/wake/rx/RxStage.java) implementations do:
+
+ public interface RxStage<T> extends Observer<T>, Stage { }
+
+In both cases, the stage simply exposes the same API as the event handler that it manages. This allows code that produces events to treat downstream stages and raw `EventHandlers` / `Observers` interchangebly. Recall that Wake implements thread sharing by allowing EventHandlers and Observers to directly invoke each other. Since Stages implement the same interface as raw EventHandlers and Observers, this pushes the placement of thread boundaries and other scheduling tradeoffs to the code that is instantiating the application. In turn, this simplifies testing and improves the reusability of code written on top of Wake.
+
+#### close() vs. onCompleted()
+
+It may seem strange that Wake RxStage exposes two shutdown methods: `close()` and `onCompleted()`. Since `onCompleted()` is part of the Observer API, it may be implemented in an asynchronous fashion. This makes it difficult for applications to cleanly shut down, since, even after `onCompleted()` has returned, resources may still be held by the downstream code.
+
+In contrast, `close()` is synchronous, and is not allowed to return until all queued events have been processed, and any resources held by the Stage implementation have been released. The upshot is that shutdown sequences in Wake work as follows: Once the upstream event sources are done calling `onNext()` (and all calls to `onNext()` have returned), `onCompleted()` or `onError()` is called exactly once per stage. After the `onCompleted()` or `onError()` call to a given stage has returned, `close()` must be called. Once `close()` returns, all resources have been released, and the JVM may safely exit, or the code that is invoking Wake may proceed under the assumption that no resources or memory have been leaked. Note that, depending on the implementation of the downstream Stage, there may be a delay between the return of calls such as `onNext()` or `onCompleted()` and their execution. Therefore, it is possible that the stage will continue to schedule `onNext()` calls after `clos
e()` has been invoked. It is illegal for stages to drop events on shutdown, so the stage will execute the requests in its queue before it releases resources and returns from `close()`.
+
+`Observer` implementations do not expose a `close()` method, and generally do not invoke `close()`. Instead, when `onCompleted()` is invoked, it should arrange for `onCompleted()` to be called on any `Observer` instances that `this` directly invokes, free any resources it is holding, and then return. Since the downstream `onCompleted()` calls are potentially asynchronous, it cannot assume that downstream cleanup completes before it returns.
+
+In a thread pool `Stage`, the final `close()` call will block until there are no more outstanding events queued in the stage. Once `close()` has been called (and returns) on each stage, no events are left in any queues, and no `Observer` or `EventHandler` objects are holding resources or scheduled on any cores, so shutdown is compelete.
+
+Helper libraries
+----------------
+
+Wake includes a number of standard library packages:
+
+ - `org.apache.reef.wake.time` allows events to be scheduled in the future, and notifies the application when it starts and when it is being torn down.
+ - `org.apache.reef.wake.remote` provides networking primitives, including hooks into netty (a high-performance event-based networking library for Java).
+ - `org.apache.reef.wake.metrics` provides implementations of standard latency and throughput instrumentation.
+ - `org.apache.reef.wake.profiler` provides a graphical profiler that automatically instruments Tang-based Wake applications.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/resources/ApacheIncubator.png
----------------------------------------------------------------------
diff --git a/website/src/site/resources/ApacheIncubator.png b/website/src/site/resources/ApacheIncubator.png
new file mode 100644
index 0000000..c04e70d
Binary files /dev/null and b/website/src/site/resources/ApacheIncubator.png differ
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/resources/REEFDiagram.png
----------------------------------------------------------------------
diff --git a/website/src/site/resources/REEFDiagram.png b/website/src/site/resources/REEFDiagram.png
new file mode 100644
index 0000000..d7625a9
Binary files /dev/null and b/website/src/site/resources/REEFDiagram.png differ
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/resources/REEFLogo.png
----------------------------------------------------------------------
diff --git a/website/src/site/resources/REEFLogo.png b/website/src/site/resources/REEFLogo.png
new file mode 100644
index 0000000..865dd0c
Binary files /dev/null and b/website/src/site/resources/REEFLogo.png differ
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/resources/reef-architecture.png
----------------------------------------------------------------------
diff --git a/website/src/site/resources/reef-architecture.png b/website/src/site/resources/reef-architecture.png
new file mode 100644
index 0000000..321aaae
Binary files /dev/null and b/website/src/site/resources/reef-architecture.png differ
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/resources/states-horizontal.png
----------------------------------------------------------------------
diff --git a/website/src/site/resources/states-horizontal.png b/website/src/site/resources/states-horizontal.png
new file mode 100644
index 0000000..6627815
Binary files /dev/null and b/website/src/site/resources/states-horizontal.png differ
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/site.xml
----------------------------------------------------------------------
diff --git a/website/src/site/site.xml b/website/src/site/site.xml
new file mode 100644
index 0000000..9e8d581
--- /dev/null
+++ b/website/src/site/site.xml
@@ -0,0 +1,122 @@
+<?xml version="1.0" encoding="ISO-8859-1"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project name="Apache REEF" xmlns="http://maven.apache.org/DECORATION/1.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/DECORATION/1.0.0 http://maven.apache.org/xsd/decoration-1.0.0.xsd">
+ <skin>
+ <groupId>org.apache.maven.skins</groupId>
+ <artifactId>maven-fluido-skin</artifactId>
+ <version>1.3.1</version>
+
+ </skin>
+ <custom>
+
+
+ <fluidoSkin>
+ <breadcrumbDivider>|</breadcrumbDivider>
+ <topBarEnabled>true</topBarEnabled>
+ <!-- Should always add up to 12 -->
+ <leftColumnClass>span2</leftColumnClass>
+ <bodyColumnClass>span10</bodyColumnClass>
+
+ <sideBarEnabled>true</sideBarEnabled>
+ <!-- Makes the top bar color = black
+ <navBarStyle>navbar-inverse</navBarStyle>
+ -->
+
+ </fluidoSkin>
+ </custom>
+
+ <bannerLeft>
+ <src>REEFLogo.png</src>
+ <href></href>
+ </bannerLeft>
+ <bannerRight>
+ <src>ApacheIncubator.png</src>
+ <href>http://incubator.apache.org/</href>
+ </bannerRight>
+
+ <version position="none"/>
+ <publishDate position="none"/>
+
+ <body>
+ <breadcrumbs>
+ <item name="Apache REEF" href="index.html"/>
+ </breadcrumbs>
+ <links>
+ <item name="Apache REEF GitHub" href="https://github.com/apache/incubator-reef"/>
+ <item name="Apache" href="http://www.apache.org"/>
+ <item name="Apache Incubator" href="http://incubator.apache.org/"/>
+ </links>
+ <menu name="Apache REEF">
+ <item name="Overview" href="index.html"/>
+ <item name="FAQ" href="faq.html"/>
+ <item name="License" href="license.html"/>
+ <item name="Downloads" href="downloads.html"/>
+ <item name="0.10.0-incubating API" href="apidocs/0.10.0-incubating/index.html"/>
+ </menu>
+
+ <menu name="Documentation">
+ <item name="Introduction to REEF" href="introduction.html"/>
+ <item name="REEF Tutorial" href="tutorial.html"/>
+ <item name="Further REEF Examples" href="reef-examples.html"/>
+ <item name="Glossary" href="glossary.html"/>
+ <item name="Tang" href="tang.html"/>
+ <item name="Wake" href="wake.html"/>
+ </menu>
+ <menu name="Contribution">
+ <item name="Contributing" href="contributing.html"/>
+ <item name="Committer Guide" href="committer-guide.html"/>
+ <item name="Coding Guidelines" href="coding-guideline.html"/>
+ </menu>
+ <menu name="Community">
+ <item name="Team" href="team.html"/>
+ <item name="Mailing List" href="mailing-list.html"/>
+ <item name="Issue Tracker" href="issue-tracker.html"/>
+ <item name="Powered By" href="powered-by.html"/>
+ </menu>
+ <menu name="ASF">
+ <item name="Apache Software Foundation" href="http://www.apache.org/foundation/"/>
+ <item name="How Apache Works" href="http://www.apache.org/foundation/how-it-works.html"/>
+ <item name="Apache Incubator" href="http://incubator.apache.org/"/>
+ <item name="Apache License" href="http://www.apache.org/licenses/LICENSE-2.0.html"/>
+ <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html"/>
+ <item name="Thanks" href="http://www.apache.org/foundation/thanks.html"/>
+ </menu>
+ <footer>
+ <div class="container-fluid">
+ <div class="row-fluid">
+ <a href="http://www.apache.org">Apache Software Foundation</a>.
+ All Rights Reserved.
+ </div>
+ </div>
+
+ <div class="row span12">
+ Apache REEF, REEF, Apache, the Apache feather logo, and the Apache REEF logo are trademarks
+ of The Apache Software Foundation. All other marks mentioned may be trademarks or registered
+ trademarks of their respective owners.
+ </div>
+ </footer>
+ <!--
+ this command gives a bunch of information about the Maven site creation project
+ <menu ref="reports"/>
+ -->
+ </body>
+</project>
\ No newline at end of file
[2/2] incubator-reef git commit: [REEF-90]: Migrate REEF website code
from a private repo to REEF repo
Posted by we...@apache.org.
[REEF-90]: Migrate REEF website code from a private repo to REEF repo
This addressed the issue by
* Modifying the pom.xml file of $REEF_HOME to exclude README.* files in all
directories when using `mvn apache-rat:check`
* Adding a directory labelled `website` which contains the source code for the
REEF website
* Adding license headers to all website/src/* files except README.md
* Adding the parent artifact and groupID and artifactID for REEF inside the
website project's pom.xml file
JIRA:
[REEF-90] https://issues.apache.org/jira/browse/REEF-90
Pull Request:
This closes #66
Project: http://git-wip-us.apache.org/repos/asf/incubator-reef/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-reef/commit/fa77cc63
Tree: http://git-wip-us.apache.org/repos/asf/incubator-reef/tree/fa77cc63
Diff: http://git-wip-us.apache.org/repos/asf/incubator-reef/diff/fa77cc63
Branch: refs/heads/master
Commit: fa77cc63c530d878da7ab07b5e55962c720f9a40
Parents: 9defe61
Author: Josh Seol <jo...@gmail.com>
Authored: Wed Feb 4 14:25:19 2015 +0900
Committer: Markus Weimer <we...@apache.org>
Committed: Wed Feb 4 16:54:53 2015 -0800
----------------------------------------------------------------------
pom.xml | 2 +-
website/README.md | 24 +
website/pom.xml | 57 +++
website/src/site/apt/team.apt | 88 ++++
website/src/site/markdown/api.md | 21 +
website/src/site/markdown/coding-guideline.md | 51 +++
website/src/site/markdown/committer-guide.md | 159 +++++++
website/src/site/markdown/contributing.md | 159 +++++++
website/src/site/markdown/downloads.md | 75 +++
website/src/site/markdown/faq.md | 61 +++
website/src/site/markdown/glossary.md | 120 +++++
website/src/site/markdown/index.md | 55 +++
website/src/site/markdown/introduction.md | 63 +++
website/src/site/markdown/issue-tracker.md | 21 +
website/src/site/markdown/license.md | 221 +++++++++
website/src/site/markdown/mailing-list.md | 40 ++
website/src/site/markdown/powered-by.md | 21 +
website/src/site/markdown/reef-examples.md | 135 ++++++
website/src/site/markdown/tang.md | 455 +++++++++++++++++++
website/src/site/markdown/tutorial.md | 137 ++++++
website/src/site/markdown/wake.md | 109 +++++
website/src/site/resources/ApacheIncubator.png | Bin 0 -> 8626 bytes
website/src/site/resources/REEFDiagram.png | Bin 0 -> 36456 bytes
website/src/site/resources/REEFLogo.png | Bin 0 -> 1599 bytes
.../src/site/resources/reef-architecture.png | Bin 0 -> 23411 bytes
.../src/site/resources/states-horizontal.png | Bin 0 -> 27111 bytes
website/src/site/site.xml | 122 +++++
27 files changed, 2195 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index d24ece6..a334f75 100644
--- a/pom.xml
+++ b/pom.xml
@@ -239,7 +239,7 @@ under the License.
<exclude>lang/java/.idea/**</exclude>
<exclude>**/*.iml</exclude>
<exclude>**/target/**</exclude>
- <exclude>README.*</exclude>
+ <exclude>**/README.*</exclude>
<!-- The below are sometimes created during tests -->
<exclude>REEF_LOCAL_RUNTIME/**</exclude>
<!-- The Visual Studio build files -->
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/README.md
----------------------------------------------------------------------
diff --git a/website/README.md b/website/README.md
new file mode 100644
index 0000000..52b4c43
--- /dev/null
+++ b/website/README.md
@@ -0,0 +1,24 @@
+Website
+===========
+
+##Instructions
+
+####Generating the HTML files
+
+Navigate to the {$REEF_HOME}/website directory then use the command:
+
+ mvn clean site:site
+
+This will generate a directory named "target" and inside, there will be a directory named "site". The site folder contains all files required for the website and index.html is the default page shown when the domain is first accessed.
+
+####Generating the Javadocs files for REEF
+
+Navigate to {$REEF_HOME} then use the command:
+
+ mvn javadoc:aggregate
+
+This will create a directory named "apidocs" which includes all the necessary Javadocs for REEF.
+
+####Combining the HTML files with Javadocs files
+
+The site.xml file automatically searches for the apidocs at `apidocs/{$REEF_VERSION}/index.html`. When copying the apidocs to the svn, it is crucial to use this exact path because multiple index.html's exist and they must be referenced without ambiguity.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/pom.xml
----------------------------------------------------------------------
diff --git a/website/pom.xml b/website/pom.xml
new file mode 100644
index 0000000..455ba52
--- /dev/null
+++ b/website/pom.xml
@@ -0,0 +1,57 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>website</artifactId>
+ <name>REEF Website</name>
+ <parent>
+ <groupId>org.apache.reef</groupId>
+ <artifactId>reef-project</artifactId>
+ <version>0.11.0-incubating-SNAPSHOT</version>
+ <relativePath>..</relativePath>
+ </parent>
+
+ <build>
+ <sourceDirectory>src</sourceDirectory>
+ <plugins>
+ <plugin>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.1</version>
+ <configuration>
+ <source>1.7</source>
+ <target>1.7</target>
+ </configuration>
+ </plugin>
+ <plugin>
+ <artifactId>maven-site-plugin</artifactId>
+ <version>3.3</version>
+ <configuration>
+ <skipDeploy>true</skipDeploy>
+ </configuration>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.maven.doxia</groupId>
+ <artifactId>doxia-module-markdown</artifactId>
+ <version>1.6</version>
+ </dependency>
+ </dependencies>
+ </plugin>
+ </plugins>
+ </build>
+</project>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/apt/team.apt
----------------------------------------------------------------------
diff --git a/website/src/site/apt/team.apt b/website/src/site/apt/team.apt
new file mode 100644
index 0000000..a6e99e2
--- /dev/null
+++ b/website/src/site/apt/team.apt
@@ -0,0 +1,88 @@
+ ------
+ Team
+ ------
+ ------
+
+~~ Licensed to the Apache Software Foundation (ASF) under one
+~~ or more contributor license agreements. See the NOTICE file
+~~ distributed with this work for additional information
+~~ regarding copyright ownership. The ASF licenses this file
+~~ to you under the Apache License, Version 2.0 (the
+~~ "License"); you may not use this file except in compliance
+~~ with the License. You may obtain a copy of the License at
+~~
+~~ http://www.apache.org/licenses/LICENSE-2.0
+
+~~ Unless required by applicable law or agreed to in writing,
+~~ software distributed under the License is distributed on an
+~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+~~ KIND, either express or implied. See the License for the
+~~ specific language governing permissions and limitations
+~~ under the License.
+
+Team
+
+ A successful project requires many people to play many roles. Some members write code or documentation, while others are valuable as testers, submitting patches and suggestions.
+
+ The team is comprised of Project Management Committee Members and Contributors. Members have direct access to the source of a project and actively evolve the code-base. Contributors improve the project through submission of patches and suggestions to the Members. The number of Contributors to the project is unbounded. Get involved today. All contributions to the project are greatly appreciated.
+
+* Mentors
+
+*------------*-------------------------*
+<<ID>> | <<Full Name>>
+*------------*-------------------------*
+ cdouglas | Chris Douglas
+*------------*-------------------------*
+ mattmann | Chris Mattmann
+*------------*-------------------------*
+ rgardler | Ross Gardler
+*------------*-------------------------*
+ omalley | Owen O'Malley
+*------------*-------------------------*
+
+
+*Committers
+
+*------------*--------------------------*---------------------------*
+ <<ID>> || <<Full Name>> || <<Affiliation(s)>> |
+*------------*--------------------------*---------------------------*
+ weimer | Markus Weimer | Microsoft |
+*------------*--------------------------*---------------------------*
+ motus | Sergiy Matusevych | Microsoft |
+*------------*--------------------------*---------------------------*
+ juliaw | Julia Wang | Microsoft |
+*------------*--------------------------*---------------------------*
+ shravan | Shravan M Narayanamurthy | Microsoft |
+*------------*--------------------------*---------------------------*
+ yingdachen | Yingda Chen | Microsoft |
+*------------*--------------------------*---------------------------*
+ tmajest | Tony Majestro | Microsoft |
+*------------*--------------------------*---------------------------*
+ sezgin | Beysim Sezgin | Microsoft |
+*------------*--------------------------*---------------------------*
+ shulmanb | Boris Shulman | Microsoft |
+*------------*--------------------------*---------------------------*
+ sears | Russell Sears | Purestorage |
+*------------*--------------------------*---------------------------*
+ jrlee | Jung Ryong Lee | SK Telecom |
+*------------*--------------------------*---------------------------*
+ jerryjung | You Sun Jung | SK Telecom |
+*------------*--------------------------*---------------------------*
+ dongjoon | Dong Joon Hyun | SK Telecom |
+*------------*--------------------------*---------------------------*
+ joshrosen | Josh Rosen | UC Berekely |
+*------------*--------------------------*---------------------------*
+ tcondie | Tyson Condie | UCLA |
+*------------*--------------------------*---------------------------*
+ bmyers | Brandon Myers | University of Washington |
+*------------*--------------------------*---------------------------*
+ yunseong | Yunseong Lee | Seoul National University |
+*------------*--------------------------*---------------------------*
+ taegeonum | Taegeon Um | Seoul National University |
+*------------*--------------------------*---------------------------*
+ johnyangk | Youngseok Yang | Seoul National University |
+*------------*--------------------------*---------------------------*
+ chobrian | Brian Cho | Seoul National University |
+*------------*--------------------------*---------------------------*
+ bgchun | Byung-Gon Chun | Seoul National University |
+*------------*--------------------------*---------------------------*
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/api.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/api.md b/website/src/site/markdown/api.md
new file mode 100644
index 0000000..6b0f182
--- /dev/null
+++ b/website/src/site/markdown/api.md
@@ -0,0 +1,21 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#API
+
+- [0.10.0-incubating](apidocs/0.10.0-incubating/index.html)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/coding-guideline.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/coding-guideline.md b/website/src/site/markdown/coding-guideline.md
new file mode 100644
index 0000000..55a6d02
--- /dev/null
+++ b/website/src/site/markdown/coding-guideline.md
@@ -0,0 +1,51 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Coding Guidelines
+
+###The Basics
+
+We largely follow [Hadoop coding guidelines](http://wiki.apache.org/hadoop/CodeReviewChecklist):
+
+- Use 2 (TWO) spaces to indent code.
+- Use LF (Unix style) line endings.
+- Do **not** use the `@author` Javadoc tag. (Be modest ! :-))
+- For the rest, follow the original [Sun Java coding conventions](http://www.oracle.com/technetwork/java/codeconvtoc-136057.html).
+- If some file has different formatting, try to match the existing style.
+- Use [CheckStyle](http://checkstyle.sourceforge.net/) to verify your coding style.
+- Avoid mixing both style and functionality changes in one commit.
+- Make sure all files have the needed [license header](http://www.apache.org/legal/src-headers.html).
+
+###Comments
+
+We require committers to add comments to generate proper javadoc pages. In addition to this requirement, we encourage committers to add comments (if needed) to make code easier to understand.
+
+- Class
+- Interface
+- Public constructor
+- Public method
+- Important fields
+- Code part that captures complex logic
+
+###On immutability of why you should make everything final
+
+REEF favors immutable objects over mutable ones. When you browse the code base, you see that there is an enormous number of final and readonly modifiers used. The reason for this stems from the distributed and parallel nature of REEF: What cannot be changed doesn't have to be guarded with locks. Hence, immutability is an excellent tool to achieve both thread safety and performance. Following this line of thought, we arrive at the following guidelines:
+
+- Make all instance variables possible `final`.
+- Use the [Java Concurrency in Practice annotations](http://jcip.net.s3-website-us-east-1.amazonaws.com/) for the instance variables that aren't `final`.
+- When a class has more than 3 non-`final` instance variables, consider breaking the class in two to limit the lock complexity.
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/committer-guide.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/committer-guide.md b/website/src/site/markdown/committer-guide.md
new file mode 100644
index 0000000..ce094d9
--- /dev/null
+++ b/website/src/site/markdown/committer-guide.md
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Committer Guide
+
+This captures committer-specific information beyond what's covered in the [contributions guide](contributing.html).
+
+###Accepting Pull Requests on GitHub
+
+The ASF INFRA team maintains [a mirror of our git repository over on GitHub](https://github.com/apache/incubator-reef).That mirror is strictly one-way: changes from the Apache git get copied over, but not the other way around. Further, the mirror on GitHub is read-only for everyone. Which means we can neither accept nor close pull requests filled there via the GitHub web UI. However, we want to allow for contributions via GitHub Pull Requests. Here's how:
+
+####Add the ASF git repository
+
+If you have not done so already, add the ASF git repository. For example, to add as the "apache" remote repository:
+
+ $ git remote add apache https://git-wip-us.apache.org/repos/asf/incubator-reef.git
+
+As a result, you can refer to the ASF git as "apache". An example setup, with a the GitHub mirror at upstream and a forked GitHub origin at `{username}`:
+
+ $ git remote -v
+ apache https://git-wip-us.apache.org/repos/asf/incubator-reef.git (fetch)
+ apache https://git-wip-us.apache.org/repos/asf/incubator-reef.git (push)
+ origin https://github.com/{username}/incubator-reef.git (fetch)
+ origin https://github.com/{username}/incubator-reef.git (push)
+ upstream https://github.com/apache/incubator-reef.git (fetch)
+ upstream https://github.com/apache/incubator-reef.git (push)
+
+####Merge changes
+
+The next step is to merge all changes on the Pull Request as a single commit. There are two methods here: (A) pull the branch from the GitHub Pull Request and squash commits, or (B) get the `.diff` from GitHub and manually create a commit from this information. Each option has its pros and cons. With method A, git does some of the tedious work of preserving commit messages; but in some cases the squash may not apply cleanly, and the merger will have to carefully resolve conflicts. With method B, the `.diff` will apply cleanly (given that the branch is up-to-date, i.e. the GitHub GUI states that the "This pull request can be automatically merged by project collaborators"); but the merger will have to carefully copy over commit messages and edit author information.
+
+####Commit Message
+
+Whichever method you choose, the following should be included in the final commit message:
+
+- Pull request description and commit comments
+- A link to the JIRA this is addressing.
+- The text "closes #PRNUMBER", where PRNUMBER is the number of the pull request, e.g. "10"
+
+Following the last statement will close the GitHub pull request. It is important to close via the commit message, because we cannot close pull requests via the GitHub Web UI.
+
+#####Example Commit message (80 columns)
+
+ [REEF-33] Allow Tasks to initiate an Evaluator Heartbeat
+ This adds the class HeartBeatTriggerManager which can be used by a Task to
+ initiate an Evaluator Heartbeat. It is used by injecting it into the Task.
+
+ JIRA:
+ [REEF-33] https://issues.apache.org/jira/browse/REEF-33
+
+ Pull Request:
+ Closes #24
+
+ Author:
+ AuthorName AuthorEmail
+
+####Method A
+
+#####A-1. Create a branch on your local git repository
+
+You want to make sure that that branch is current with the the master branch on Apache git:
+
+ $ git checkout -b BRANCH_NAME
+ $ git pull apache master
+
+This assumes that you called the remote git repository at the ASF "apache". You can name the branch however you want, but it is customary to name them either after the pull request number or the JIRA id, e.g. REEF-24.
+
+#####A-2. Pull the contribution into that branch
+
+This is as simple as
+
+ $ git pull GIT_REPOSITORY_OF_THE_CONTRIBUTOR BRANCH_NAME_OF_THE_CONTRIBUTION
+
+However, the GitHub web ui makes it somewhat hard to get these two strings. The email from GitHub that informs us of Pull Requests makes this really obvious, though. Consider this quote from pull [request #10](https://github.com/apache/incubator-reef/pull/10):
+
+>**You can merge this Pull Request by running**
+>>`git pull https://github.com/jwang98052/incubator-reef juwang-logfactory`
+
+This copies the changes from the given remote branch into the one we just created locally.
+
+#####A-3. Check the pull request
+
+1. Make sure the code compiles and all tests pass.
+2. If the code touches code that you suspect might break on YARN or Mesos, please test on those environments. If you don't have access to a test environment, ask on the mailing list for help.
+3. Make sure the code adheres to our [coding guidelines](coding-guideline.html).
+4. Make sure that the additions don't create errors in a [Apache RAT](http://creadur.apache.org/rat/) check via `mvn apache-rat:check`
+
+#####A-4. Rebase the branch onto current apache master
+
+ # Make sure we have the latest bits
+ $ git fetch apache
+ # Rebase
+ $ git rebase -i apache/master
+
+In the rebase process, make sure that the contribution is squashed to a single commit. From the list of commits, "pick" the commit in the first line (the oldest), and "squash" the commits in the remaining lines:
+
+ pick 7387a49 Comment for first commit
+ squash 3371411 Comment for second commit
+ squash 9bf956d Comment for third commit
+
+[Chapter 3.6](http://www.git-scm.com/book/en/v2/Git-Branching-Rebasing) and [Chapter 7.6](http://git-scm.com/book/en/v2/Git-Tools-Rewriting-History) of the [Git Book](http://www.git-scm.com/book/en/v2) contains lots of information on what this means.
+
+Please make sure that the commit message contains the information given in "Commit Message" above. The latest commit can be changed at any time with the command:
+
+ $ git commit --amend
+
+#####A-5. Push the code into apache's git
+
+This is a good time to reflect back on this change and whether it is likely to break the build. If you are certain that it won't, go ahead and do:
+
+ $ git checkout master
+ $ git merge BRANCH_NAME
+ $ git push apache master
+
+This pushes the current branch into the master branch hosted on Apache's git repository. From there, it will be mirrored onto GitHub. And by virtue of the comment added above to the commit message, GitHub will now close the Pull Request.
+
+####Method B
+
+#####B-1. Update local master
+
+In this method, you will work directly on your local master branch. Make sure you have the latest changes on your local master branch, by running:
+
+ $ git pull apache master
+
+#####B-2. Download the .diff and apply it
+
+You can download the `.diff` file by appending `.diff` to the GitHub Pull Request url. This file will contain the exact changes that are shown in "Files changed" in the GitHub GUI. For example, for `https://github.com/apache/incubator-reef/pull/24` the `.diff` file is `https://github.com/apache/incubator-reef/pull/24.diff`. Using this url, run apply:
+
+ $ wget https://github.com/apache/incubator-reef/pull/24.diff
+ $ git apply 24.diff
+
+#####B-3. Commit and edit author information
+
+Commit all files, making sure to include all modified and **new** files. In the commit message, make sure to include all information given in "Commit Message" above. After committing you must also change the author information to reflect the original author (and not yourself):
+
+ $ git commit --amend --author "Original Author Name <em...@address.com>"
+
+#####B-4. Push the code into apache's git
+
+Now push your single commit to apache git:
+
+ $ git push apache master
+
+This pushes the current branch into the master branch hosted on Apache's git repository. From there, it will be mirrored onto GitHub. And by virtue of the comment added above to the commit message, GitHub will now close the Pull Request.
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/contributing.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/contributing.md b/website/src/site/markdown/contributing.md
new file mode 100644
index 0000000..8e26456
--- /dev/null
+++ b/website/src/site/markdown/contributing.md
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Contributing to REEF
+
+####First things first: Welcome!
+
+Arriving on this page means that you are interested in helping us out. For that: Welcome and thank you! REEF is a community driven project and we always welcome new people joining us. We are inviting contributions in many forms to all parts of the project, including:
+
+- Bug reports regarding the software, the documentation, the website, guides like this, etc.
+- Graphics (for instance, [we don't have a logo yet](https://issues.apache.org/jira/browse/REEF-14)).
+- Documentation updates, tutorials, examples.
+- Code: Patches, new features, etc
+
+####Getting started: Join the community.
+
+The first step is to **join the community** by joining our [mailing list](mailing-list.html). This is where all discussions regarding the project are happening, where we answer each other's questions and where you will find a friendly bunch to welcome you.
+
+If you want to work on the REEF code base, it is a good idea to [learn how to compile and test REEF](tutorial.html).
+
+####Finding things to work on
+
+At any given time, there is any number of [open, unassigned issues on REEF](https://issues.apache.org/jira/issues/?jql=project%20%3D%20REEF%20AND%20status%20%3D%20Open%20AND%20resolution%20%3D%20Unresolved%20AND%20assignee%20in%20\(EMPTY\)%20ORDER%20BY%20priority%20DESC). Note that that list doesn't only contain coding tasks, but also documentation, graphics work, the website and so on. We use JIRA to manage all open todos for the project, software or otherwise. However, some of the items on that list might since have become obsolete or such. Hence, it is always a good idea to get in touch with the rest of the community on the mailing list before getting started.
+
+####Code contribution process
+
+While contributing to REEF, you will encounter ASF infrastructure, GitHub infrastructure, and follow ASF and REEF-specific customs. The most important of those customs is to communicate throughout the whole process. That way, the REEF community has a chance to help you along the way.
+
+####Before you start the work
+
+An important part of the process comes before you start the contribution process. Most issues should first be brought up in the [dev@reef.incubator.apache.org](mailto:dev@reef.incubator.apache.org) mailing list. If you haven't done so yet, [subscribe to the list](mailing-list.html). After discussion, you or one of the other developers will create an Issue on [ASF JIRA](https://issues.apache.org/jira/browse/REEF/). Again, create an account if you don't have one. Write a succinct description of the Issue, making sure to incorporate any discussion from the mailing list.
+
+And once you are ready to make changes, have a look at the [coding guidelines](coding-guideline.html) to make sure your code agrees with the rest of REEF.
+
+####Creating the contribution as a GitHub Pull Request
+
+REEF uses [GitHub's Pull Requests](https://help.github.com/articles/using-pull-requests/) as the main method to accept changes: you fork REEF into your own repository, make changes, and then send a pull request to the GitHub repository. In more detail:
+
+1. Fork repository
+2. Create branch
+3. Make changes in your local machine
+4. Merge the master branch into your branch
+5. Push commits into the your remote repo (forked)
+6. Send a pull request.
+7. Participate in the code review.
+
+#####1. Fork repository
+
+First, you need to fork the REEF repository. Go to the [Github repository](https://github.com/apache/incubator-reef) mirrored from ASF, and click "Fork" button. Then you will have your own repository. Clone this repository to your local machine:
+
+ $ git clone https://github.com/{your_alias}/incubator-reef.git
+
+Then, add the Apache GitHub repository as upstream:
+
+ $ git remote add upstream https://github.com/apache/incubator-reef
+
+A correct git configuration should look similar to the following, with an origin and upstream:
+
+ $ git remote -v
+ origin https://github.com/{your_alias}/incubator-reef.git (fetch)
+ origin https://github.com/{your_alias}/incubator-reef.git (push)
+ upstream https://github.com/apache/incubator-reef.git (fetch)
+ upstream https://github.com/apache/incubator-reef.git (push)
+
+If you have an `apache.org` email address, now is the time to [configure git](https://git-wip-us.apache.org/) to use it:
+
+ $ git config user.name "My Name Here"
+ $ git config user.email myusername@apache.org
+
+#####2. Create a branch to work on
+Before making changes, you have to make sure the issue to resolve (e.g. fix a bug, implement a new feature, etc) is registered in the [REEF JIRA](https://issues.apache.org/jira/browse/REEF). Create a branch to address the issue on your own. The name of the branch should reference the issue, e.g., `REEF-{issue_number}`. You can take a look how others name their branches.
+
+#####3. Make changes in your local machine
+Write the code and make commits as usual. Make sure all new files contain the [ASF licensing header](https://github.com/apache/incubator-reef/blob/master/LICENSE_HEADER.txt).
+
+#####4. Merge the master branch into your branch
+Before sending a pull request, you should make sure that your branch includes the latest changes in the master branch. Please run the following:
+
+ $ git fetch upstream
+ $ git checkout {your_branch} # Skip this step if you are already on your branch.
+ $ git merge upstream/master
+
+Resolve the conflicts if they exist. Test with the merged code so that it does not break the system. Then, check that Apache headers are in place where needed, by running RAT:
+
+ $ mvn apache-rat:check
+
+Finally, as a courtesy to the merger, you can rebase to master and squash all the commits from your PR into one:
+
+ # Rebase
+ $ git rebase -i upstream/master
+
+In the rebase process, make sure that the contribution is squashed to a single commit. From the list of commits, "pick" the commit in the first line (the oldest), and "squash" the commits in the remaining lines:
+
+ pick 7387a49 Comment for first commit
+ squash 3371411 Comment for second commit
+ squash 9bf956d Comment for third commit
+
+[Chapter 3.6](http://www.git-scm.com/book/en/v2/Git-Branching-Rebasing) and [Chapter 7.6](http://git-scm.com/book/en/v2/Git-Tools-Rewriting-History) of the [Git Book](http://www.git-scm.com/book/en/v2) contains lots of information on what this means.
+
+In this process, git allows you to edit the commit message for the final, squashed commit. This commit message will serve as the description of the pull request and will in all likelihood appear in verbatim in the REEF history. In other words: Spend some time making it good. A common template for this commit message is:
+
+ [REEF-JIRA_ISSUE_NUMBER]: THE_TITLE_OF_THE_JIRA
+
+ This addressed the issue by
+ * INSERT_DESCRIPTION_OF_SOMETHING
+ * INSERT_DESCRIPTION_OF_SOMETHING_ELSE
+ * ...
+
+ JIRA: [REEF-JIRA_ISSUE_NUMBER](https://issues.apache.org/jira/browse/REEF-JIRA_ISSUE_NUMBER)
+
+You can get a good idea how other people write their commit messages by inspecting the output of `git log`.
+
+#####5. Push commits into the your remote repo (forked)
+
+You're almost done! Push your commits into your own repo.
+
+ $ git push origin HEAD
+
+#####6. Send a pull request
+
+It is time to send a pull request. If you do not know much about pull requests, you may want to read this [article](https://help.github.com/articles/using-pull-requests/).
+
+If you go to the repository at the Github website, you can notice that "Compare & Pull request" button appears. Click the button or move into "pull request" tab if you cannot find the button.
+
+When you create a pull request, you choose the branches to compare and merge. Choose the base as `apache:master` and the head `{your_alias}:{branch_name}`. The description will be the message of your one commit. Feel free to edit it if you aren't satisfied with your commit message.
+
+Please **update the JIRA issue with a link to the Pull Request**. This triggers an email to the mailing list, which will hopefully result in a committer stepping up for a code review.
+
+#####7. The code review
+
+REEF follows a review-then-commit (RTC) model. We perform all our code reviews on GitHub: Community members will leave comments on your pull request and suggest changes. You can have [a look at prior pull requests](https://github.com/apache/incubator-reef/pulls?q=is%3Apr+is%3Aclosed) to get an idea of what to expect. During this process, you may want to change your code. You can do that by pushing additional commits to your branch. Don't worry about squashing the commits into one at this stage. This will be done by the committer once your code is merged into REEF.
+
+When the code review concludes, one of Committers will merge your work into the REEF codebase. Good job!
+
+#####8. Merge the pull request (Committers only)
+
+If you are a committer, you can follow the steps in the [Committer Guide](committer-guide.html) to merge the pull request. Of course, you won't be merging your own pull requests. Nudge committers as needed to make sure everything gets merged correctly.
+
+####Other guides
+
+There are other pages in this area you might be interested in:
+
+- [Coding Guidelines](coding-guideline.html)
+- [REEF Tutorial](tutorial.html)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/downloads.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/downloads.md b/website/src/site/markdown/downloads.md
new file mode 100644
index 0000000..1a8cf59
--- /dev/null
+++ b/website/src/site/markdown/downloads.md
@@ -0,0 +1,75 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Downloads
+
+##Releases
+
+- [0.10.0-incubating](#0.10.0-incubating)
+
+###<a name="0.10.0-incubating"></a>0.10.0-incubating
+
+#####Downloads
+
+- [apache-reef-0.10.0-incubating.tar.gz](http://www.apache.org/dist/incubator/reef/0.10.0-incubating/apache-reef-0.10.0-incubating.tar.gz)
+- [apache-reef-0.10.0-incubating.tar.gz.asc](http://www.apache.org/dist/incubator/reef/0.10.0-incubating/apache-reef-0.10.0-incubating.tar.gz.asc)
+- [apache-reef-0.10.0-incubating.tar.gz.md5](http://www.apache.org/dist/incubator/reef/0.10.0-incubating/apache-reef-0.10.0-incubating.tar.gz.md5)
+- [apache-reef-0.10.0-incubating.tar.gz.sha1](http://www.apache.org/dist/incubator/reef/0.10.0-incubating/apache-reef-0.10.0-incubating.tar.gz.sha1)
+- [apache-reef-0.10.0-incubating.tar.gz.sha512](http://www.apache.org/dist/incubator/reef/0.10.0-incubating/apache-reef-0.10.0-incubating.tar.gz.sha512)
+
+##Development and Maintenance Branches
+
+If you are interested in working with the newest under-development code or contributing to REEF, you can also check out the master branch from Git:
+
+ $ git clone git://git.apache.org/incubator-reef.git
+
+##How to verify the integrity of the files
+
+It is essential that you verify the integrity of the downloaded files using the PGP or MD5 signatures. Please read [Verifying Apache HTTP Server Releases](http://www.apache.org/info/verification.html) for more information on why you should verify our releases.
+
+The PGP signatures can be verified using [PGP](http://www.pgpi.org/) or [GPG](https://www.gnupg.org/). First download the [KEYS](http://www.apache.org/dist/incubator/reef/KEYS) as well as the `*.asc` signature file for the relevant distribution. Make sure you get these files from the [main distribution directory](http://www.apache.org/dist/incubator/reef/) rather than from a mirror. Then verify the signatures using one of the following sets of commands:
+
+ % pgpk -a KEYS
+ % pgpv downloaded_file.asc
+
+or
+
+ % pgp -ka KEYS
+ % pgp downloaded_file.asc
+
+or
+
+ % gpg --import KEYS
+ % gpg --verify downloaded_file.asc
+
+Alternatively, you can verify the MD5 signature on the files. A Unix/Linux program called md5 or md5sum is included in most distributions. It is also available as part of [GNU Textutils](http://www.gnu.org/software/textutils/textutils.html). Windows users can get binary md5 programs from these (and likely other) places:
+
+
+- http://www.md5summer.org/
+- http://www.fourmilab.ch/md5/
+- http://www.pc-tools.net/win32/md5sums/
+
+##Maven Dependencies
+
+REEF artifacts are hosted in [Maven Central](http://search.maven.org/#search|ga|1|org.apache.reef) and can be added to your Maven project using the following format:
+
+ <dependency>
+ <groupId>org.apache.reef</groupId>
+ <artifactId>reef-project</artifactId>
+ <version>{$REEF_VERSION}</version>
+ </dependency>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/faq.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/faq.md b/website/src/site/markdown/faq.md
new file mode 100644
index 0000000..71b24a8
--- /dev/null
+++ b/website/src/site/markdown/faq.md
@@ -0,0 +1,61 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#FAQ
+
+1. [Who is REEF for?](#who)
+2. [Why did it come about?](#why)
+3. [What is Tang?](#tang)
+4. [What is Wake?](#wake)
+5. [How can I get started?](#how)
+
+###<a name="who"></a>Who is REEF for?
+
+REEF is for developers of data processing systems on cloud computing platforms that provide fine-grained resource allocations. REEF provides system authors with a centralized (pluggable) control flow that embeds a user-defined system controller called the Job Driver. The interfaces associated with the Job Driver are event driven; events signal resource allocations and failures, various states associated with task executions and communication channels, alarms based on wall-clock or logical time, and so on. REEF also aims to package a variety of data-processing libraries (e.g., high-bandwidth shuffle, relational operators, low-latency group communication, etc.) in a reusable form. Authors of big data systems and toolkits can leverage REEF to immediately begin development of their application specific data flow, while reusing packaged libraries where they make sense.
+
+___________________________________________________________________________
+
+###<a name="why"></a>Why did it come about?
+
+Traditional data-processing systems are built around a single programming model (like SQL or MapReduce) and a runtime (query) engine. These systems assume full ownership over the machine resources used to execute compiled queries. For example, Hadoop (version one) supports the MapReduce programming model, which is used to express jobs that execute a map step followed by an optional reduce step. Each step is carried out by some number of parallel tasks. The Hadoop runtime is built on a single master (the JobTracker) that schedules map and reduce tasks on a set of workers (TaskTrackers) that expose fixed-sized task "slots". This design leads to three key problems in Hadoop:
+
+
+1. The resources tied to a TaskTracker are provisioned for MapReduce only.
+2. Clients must speak some form of MapReduce in order to make use of cluster resources, and in turn, gain compute access to the data that lives there.
+3. Poor cluster utilization, especially in the case of idle resources (slots) due to straggler tasks.
+
+With YARN (Hadoop version two), resource management has been decoupled from the MapReduce programming model in Hadoop, freeing cluster resources from slotted formats, and opening the door to programming frameworks beyond MapReduce. It is well understood that while enticingly simple and fault-tolerant, the MapReduce model is not ideal for many applications, especially iterative or recursive workloads like machine learning and graph processing, and those that tremendously benefit from main memory (as opposed to disk based) computation. A variety of big data systems stem from this insight: Microsoft's Dryad, Apache Spark, Google's Pregel, CMU's GraphLab and UCI's AsterixDB, to name a few. Each of these systems add unique capabilities, but form islands around key functionalities, making it hard to share both data and compute resources between them. YARN, and related resource managers, move us one step closer toward a unified Big Data system stack. The goal of REEF is to provide the next
level of detail in this layering.
+
+______________________________________________________
+
+###<a name="tang"></a>What is Tang?
+
+Tang is a dependency injection Framework co-developed with REEF. It has extensive documentation which can be found [here](tang.html).
+
+________________________________________________
+
+###<a name="wake"></a>What is Wake?
+
+Please refer to [this](wake.html) section.
+
+________________________________________________
+
+
+###<a name="how"></a>How can I get started?
+
+Check out the [REEF Tutorial](tutorial.html) and the [Contributing](contributing.html) page and join the community!
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/glossary.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/glossary.md b/website/src/site/markdown/glossary.md
new file mode 100644
index 0000000..d424d71
--- /dev/null
+++ b/website/src/site/markdown/glossary.md
@@ -0,0 +1,120 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Glossary
+
+- [Context](#context)
+- [Driver](#driver)
+- [Evaluator](#evaluator)
+- [Task](#task)
+
+###<a name="context"></a>Context
+
+Contexts are a way to structure the state and Configuration of an Evaluator. A Context exists on one and only one individual Evaluator. Each Evaluator has at least one Context, which we refer to as the *root* Context. This root context is special, as it is *required* on all Evaluators and because closing it is synonymous to releasing the Evaluator. In many simple REEF programs, this is the only Context used and it is therefore convenient to think of it as synonymous to "the Evaluator": The Driver can submit Tasks to it, is notifified when they are done and can close the root context when it wants to dispose of the Evaluator.
+
+Contexts are formed by calls to `submitContext()` to the event types that allow this (`AllocatedEvaluator` and `ActiveContext`) Contexts are the main way for an Evaluator to be exposed and accessed. For instance, Tasks are submitted to an `ActiveContext` which represents the top Context on the Evaluator.
+
+Beyond this, a Driver can submit a Context to the root, or in fact any, Context, as long as the resulting structure is that of a stack: The root Context forms the bottom of the stack, the top-most Context is called *active*, hence the `ActiveContext` event. The two can be one and the same, and often are: The root Context is the subject of the first `ActiveContext` event on an Evaluator.
+
+Nomenclature: When Context B is submitted to an already existing Context A, we say that Context A is the parent Context of Context B. Also, Context B is the child of Context A.
+
+It is only the `ActiveContext` that allows the submission of Tasks or child Contexts. Hence, one can think of the whole Evaluator structure as that of a stack: the root Context at the bottom, layers of Contexts in the middle and either the current `ActiveContext` or the current Task at the top.
+
+####Objects and Configuration: What's in a Context?
+
+It is convenient to think of a Context as a `Configuration` that gets merged with the `Configuration` supplied for Tasks and child Contexts. While not entirely true (see below), this view allows us to show just *why* Contexts are a convenient construct.
+
+It is often the case that subsequent tasks that get executed on an Evaluator want access to the same Configuration variables and / or the same objects. Consider a simple `LinkedList` bound to a named parameter. If that linked list is part of the subsequent Task `Configurations` submited, each Task will get its very *own* `LinkedList`. If the named parameter is bound in the Context `Configuration`, all Tasks subsequently submitted to the Context will get the very *same* `LinkedList` instance.
+
+####Contexts are (Tang) Injectors
+
+This mechanism is implemented by using Tang's `Injector`s. On the Evaluator, a Task is launched by first *forking* the Context's `Injector` with the Task`Configuration` and then requesting an instance of the `Task` interface from that forked `Injector`. By this mechanism and the fact that objects are singletons with respect to an `Injector` in Tang, object sharing can be implemented. All objects already instantiated on the Context `Injector` will also be referenced by the Task`Injector`. Hence, the `LinkedList` in the example above would be shared amongst subsequent Task `Injectors` in the construction of the `Task` instance.
+
+###<a name="driver"></a>Driver
+
+REEF imposes a centralized control flow design on applications: All events are routed to the master node, called the Driver. REEF also prescribes event-driven programming for the Driver. In that sense, the application provided Driver is a collection of event handlers for the various events exposed in `DriverConfiguration`. While most of these deal with occurrences during the computation (Evaluator allocation, Task launch, ...), several stand out as life-cycle events of the Driver, and therefore the application:
+
+####ON_START
+
+This event is triggered by REEF when the Driver is ready to start executing. At this point communication with the Resource Manager has been established, all event handlers have been instantiated and the event graph in the Driver was deemed to be complete enough to start. In a typical application, this is when the Driver requests the first set of Evaluators.
+
+####ON_STOP
+
+This event is fired right before the Driver shuts down. REEF determines Driver shutdown by proof that no future events can happen:
+
+- The Clock has no outstanding alarms.
+- The resource manager has no outstanding requests.
+- The application has no Evaluators allocated anymore.
+
+Hence, the `ON_STOP` event can be used to prevent Driver shutdown, e.g. in applications that need to wait for external triggers (e.g. a UI) to proceed. To do so, one would schedule an alarm in the `ON_STOP` handler.
+
+###<a name="evaluator"></a>Evaluator
+
+####Evaluators and Tasks
+
+The Evaluator is the runtime environment for Tasks. On one Evaluator, there is either no or one Task executing at any given point in time. Different or multiple executions of the same Tasks can be executed in sequence on an Evaluator. The Evaluator and Task lifecycle are decoupled: Whenever a Task finishes, the Driver receives the CompletedTask event, which contains a reference to the Evaluator the Task executed on. It is then up to the Driver to decide whether to return the Evaluator to the resource manager or to make other use of it, e.g. by submitting another task.
+
+####Evaluators and Contexts
+
+Contexts are REEF's form of state management inside of the Evaluator. See the [Context](#context) section for more information.
+
+####Evaluators and the Resource Manager
+
+On typical resource managers, an Evaluator is a process executing inside a container. Depending on the resource manager, that process may or may not be guarded by a resource or security isolation layer.
+
+This also means that the Evaluator, not the Task, is the unit of resource consumption: while an Evaluator is occupying a Container, that Container is "used" from the perspective of the Resource Manager.That is true even if the Evaluator is idle from the perspective of the Driver, i.e. when no Task is running on it.
+
+###<a name="task"></a>Task
+
+####Definition
+
+A Task in REEF is a unit of work to be executed on an Evaluator. In its simplest form, a Task is merely an object implementing the Task interface which prescribes a single method:
+
+ public byte[] call(byte[] input);
+
+From REEF's perspective, a Task is therefore a single threaded method call. It starts when entering the call method. It is a `RunningTask` while it hasn't returned from it and is a `CompletedTask` when it has. Should there be an Exception thrown by `call()`, we call it a `FailedTask`.
+
+Task identity is established by a user-defined string set in `TaskConfiguration.IDENTIFIER`. All subsequent task-related events in the Driver will carry that ID. Note that REEF doesn't take any particular precautions to ensure unique Task identifiers. It is up to the application to do so. While technically feasible to assign the same identifier to multiple Tasks, this isn't advised as it makes error handling, debugging and logging unnecessarily hard.
+
+####Inputs and outputs of a Task
+
+The return value of the `call` method will be made available to the Driver as part of the `CompletedTask` event. Note that it isn't advised to return large values in this fashion, but merely small control flow or status information. Sending large data on this channel creates the risk of overloading the Driver at scale. The networking APIs provided by REEF IO are much better suited for data transmissions than this channel.
+
+The parameter given to the call method is also to be used in a similar fashion: The Driver passes its value as part of the Task submission. It is meant e.g. to convey a restart point for the task. Note that the same functionality can now be better provided by Tang and a constructor parameter.
+
+####Communicating between a Task and a Driver
+
+REEF provides some facilities to communicate between a Driver and a Task. These mostly stem from allowing application code to "free-ride" on REEF's control flow channels such as the heartbeat between the Evaluator and the Task.
+
+#####Sending a message from the Driver to a Task
+
+REEF maintains a heartbeat between any Evaluator and the Driver. There are two ways by which a heartbeat can be triggered.
+
+- Upon some schedule (which may also vary at runtime due to load conditions on the Driver), each Evaluator will report its current status to the Driver. This is used by the Driver to maintain health status and load statistics of the Evaluators.
+
+- Whenever the status of the Evaluator changes (e.g. when a Task completes), a Heartbeat is triggered immediately.
+
+Whenever the Evaluator performs a heartbeat, it will ask the Task whether it has any message to share with the Driver by inquiring the class registered in `TaskConfiguration.ON_SEND_MESSAGE`. It is wise for that message to be small, as we otherwise run the risk of overwhelming the Driver with heartbeat traffic at scale.
+
+####Multithreaded Tasks
+
+Just because REEF views a Task as a method call doesn't restrict the Task to be single threaded. A Task is free to spawn threads in the course of its execution. However, a Task that does so needs to take care of a few considerations:
+
+- All Threads spawned need to exit before the `Task.call()` method returns. Otherwise, you run the risk of resource leakage.
+
+- Exceptions on spawned Threads need to be caught and re-thrown by the `Thread.call()` method. Before that, all spawned threads need to be shut down, just like during a normal exit of `Task.call()`. If an exception from an another thread isn't caught, REEF's JVM level exception handler will catch it and declare a FailedEvaluator. This is inefficient, but not technically wrong: The Driver will then have to allocate another Evaluator and try again.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/index.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/index.md b/website/src/site/markdown/index.md
new file mode 100644
index 0000000..3c5a05e
--- /dev/null
+++ b/website/src/site/markdown/index.md
@@ -0,0 +1,55 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Apache REEF
+
+###What is REEF?
+
+**REEF**, the Retainable Evaluator Execution Framework, is our approach to simplify and unify the lower layers of big data systems on modern resource managers.
+
+For managers like Apache YARN, Apache Mesos, Google Omega, and Facebook Corona, REEF provides a centralized control plane abstraction that can be used to build a decentralized data plane for supporting big data systems. Special consideration is given to graph computation and machine learning applications, both of which require data *retention* on allocated resources to execute multiple passes over the data.
+
+More broadly, applications that run on YARN will have the need for a variety of data-processing tasks e.g., data shuffle, group communication, aggregation, checkpointing, and many more. Rather than reimplement these for each application, REEF aims to provide them in a library form, so that they can be reused by higher-level applications and tuned for a specific domain problem e.g., Machine Learning.
+
+In that sense, our long-term vision is that REEF will mature into a Big Data Application Server, that will host a variety of tool kits and applications, on modern resource managers.
+
+<div style="text-align:center" markdown="1">
+ <img>
+ <img src ="REEFDiagram.png"/>
+ </img>
+</div>
+
+###How can I get started?
+
+The official home for the REEF (and Tang and Wake) source code is at the Apache Software Foundation. You can check out the current code via:
+
+ $ git clone git://git.apache.org/incubator-reef.git
+
+or directly access its GitHub page [here](https://github.com/apache/incubator-reef).
+
+Detailed information about REEF and using it can be found in the [FAQ](faq.html) and the [Tutorial](tutorial.html).
+
+If you wish to contribute, start at the [Contributing](contributing.html) tutorial or the [Committer Guide](committer-guide.html)!
+
+###Further questions?
+
+Please visit our [Frequently Asked Questions](faq.html) page or use our [Mailing List](mailing-list.html) to send us a question!
+
+###Disclaimer
+
+Apache REEF is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/introduction.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/introduction.md b/website/src/site/markdown/introduction.md
new file mode 100644
index 0000000..37def4d
--- /dev/null
+++ b/website/src/site/markdown/introduction.md
@@ -0,0 +1,63 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Introduction to REEF
+
+It is instructive to first remind us of how MapReduce works: Hadoop Map-Reduce schedules compute tasks on containers for executing map and reduce functions on record data. The basic structure of a Map-Reduce job is as follows. For each input block, schedule a map task that passes each internal record to a user-defined map function and materializes the output in key-sorted order. Further, assign a user-defined number of reduce tasks to non-overlapping portions of the key-space from the map output, then shuffle it across the network to where the respective reduce task is scheduled. For each reduce task, perform a global key-based sort on the shuffled data, group it by key and call the reduce function on each record group; storing the output in a durable fashion (i.e., HDFS).
+
+From the perspective of the scheduler, a number of issues arise that must be appropriately handled in order to scale-out to massive datasets. First, each map task should be scheduled close to where the input block resides; ideally on the same machine or rack. Second, failures can occur at the task level at any step; requiring backup tasks to be scheduled or the job being aborted. Third, performance bottlenecks can cause an imbalance in the task-level progress. The scheduler must react to these stragglers by scheduling clones and incorporating the logical task that crosses the finish line first.
+
+Anyone of these issues can limit the scale-out degree of a Map-Reduce job. In what follows, we perscribe a scheduler framework that provides task life-cycle management mechanisms. Using this framework, we developed a complete version of the Map-Reduce runtime that addresses the above issues. Our framework is designed around three components.
+
+1. `Driver`: captures the client code embodying the scheduler
+2. `Evaluator`: provides a runtime environment on a container.
+3. `Task`: encapsulates the task-level client code to be executed in an Evaluator.
+
+Below, we describe the client facing interfaces to these components. The core REEF control flow design is based on the reactive extensions (Rx), which enforce asynchronous message-passing method signatures. In Rx terms, interfaces are based on an observer pattern, which expose methods that accept messages from a (possibly remote) asynchronous caller.
+
+<br></br>
+
+<div style="text-align:center" markdown="1">
+ <img src="reef-architecture.png"></img>
+</div>
+
+*A running REEF job with two `RunningEvaluators` and one `RunningTask`*
+
+<br></br>
+
+The above figure presents the REEF components in terms of a running application, which is written in terms of a `Driver` and task-specific `Task` modules. The application code is packaged and submitted to a REEF client API, which in turn submits a REEF-AM configuration to YARN. The REEF-AM contains a runtime for launching the `Driver` and client libraries for requesting Evaluators and launching `Activities`. When a request for `Evaluators` is made, the REEF-AM negotiates containers with the YARN-RM and launches an `Evaluator` runtime on the YARN-NM that hosts the allocated container. In turn, the `Driver` is given an `Evaluator` object reference, which it uses to submit an `Task`. The `Driver` is also given a `Task` object reference, which it may use to send messages to the `Task` running in the `Evaluator`. The REEF layer implements these communication channels and encodes the computational life-cycle as state transitions, which are surfaced to the `Driver` in the form of Rx messag
es.
+
+###Computational Life-Cycle
+
+<br></br>
+
+<div style="text-align:center" markdown="1">
+ <img src="states-horizontal.png"></img>
+</div>
+
+*States of `Evaluator`, `Contexts`, and `Activities`*
+
+<br></br>
+
+The Figure above describes the state transitions for (a) `Evaluator` and `Context` and (b) `Task` components. Each state transition is associated with an object reference that is surfaced to the `Driver` in an Rx-style interface. For instance, when the YARN-RM notifies the REEF-AM of an allocated container, the `Driver` is given an `AllocatedEvaluator` object; containing methods for adding configurations (i.e., for data services, see below) and file resources, and submit methods that bootstraps the `Evaluator` runtime on the YARN-NM. When an `Evaluator` bootstrap successfully completes, the `Driver` is given an `ActiveContext` object, which can be used to launch `Activities` or to initiate a close, which triggers a shutdown event at the `Evaluator` runtime and a subsequent container release at the YARN-RM. If at any point a failure occurs, the `Driver` is passed a `FailedEvaluator` object; containing an exception trace when possible.
+
+Recall that the `Driver` launches a `Task` on a submit method call from the `ActiveContext` reference. This state transition is denoted in the Figure above by the edge labeled submit; spanning the two state machines. The REEF-AM passes a `RunningActivity` object to the `Driver` after receiving confirmation of a successful `Task` start or resume. The `Driver` may use the `RunningActivity` reference to close or suspend the execution; triggering a `CompletedActivity` or `SuspendedActivity` object reference to the `Driver`. The `SuspendedActivity` object contains a memento used to resume the execution on some (possibly alternative) `ActiveContext`. Exceptions during the `Task` execution are surfaced to the `Driver` in the form of a `FailedActivity`, which contains the actual exception object.
+
+###Task Component
+
+A `Task` encapsulates the task work of a job. The client interface contains a single synchronous call method that takes an optional memento argument and returns a byte array, which will be packaged with the `CompletedActivity` object surfaced to the `Driver`. An exception may be thrown at any point during the call method; returning control back to the `Evaluator`, which packages the exception and sends it to the `Driver` where it is surfaced as a `FailedActivity`. The `Evaluator` periodically performs a heartbeat with the REEF-AM to convey its status information. A `Task` can optionally implement a method interface that, when called, returns a (bounded) byte array, which the `Evaluator` includes in its heartbeat to the REEF-AM and surfaced to the `Driver`.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/issue-tracker.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/issue-tracker.md b/website/src/site/markdown/issue-tracker.md
new file mode 100644
index 0000000..e5ce3ca
--- /dev/null
+++ b/website/src/site/markdown/issue-tracker.md
@@ -0,0 +1,21 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Issue Tracker
+
+Please visit the [ASF JIRA](https://issues.apache.org/jira/browse/REEF) website for the most updated issues.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/license.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/license.md b/website/src/site/markdown/license.md
new file mode 100644
index 0000000..7ebb4c4
--- /dev/null
+++ b/website/src/site/markdown/license.md
@@ -0,0 +1,221 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Project License
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/mailing-list.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/mailing-list.md b/website/src/site/markdown/mailing-list.md
new file mode 100644
index 0000000..da070ca
--- /dev/null
+++ b/website/src/site/markdown/mailing-list.md
@@ -0,0 +1,40 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Mailing List
+
+- [REEF Dev](#reef-dev)
+- [Commits](#commits)
+
+**Note:** These email addresses are automated. Just writing "Subscribe" or "Unsusbscribe" in both the subject line and the body will suffice and a confirmation email with further instructions will be sent as a reply to your email after some delay.
+
+###<a name="reef-dev"></a>REEF Dev Mailing List
+
+This gets traffic from both the JIRAs filed on REEF as well as the general development discussions
+
+- [Subscribe](mailto:dev-subscribe@reef.incubator.apache.org) (dev-subscribe@reef.incubator.apache.org)
+- [Unsusbscribe](mailto:dev-unsubscribe@reef.incubator.apache.org) (dev-unsubscribe@reef.incubator.apache.org)
+- [Archive](http://mail-archives.apache.org/mod_mbox/incubator-reef-dev/)
+
+###<a name="commits"></a>Commits Mailing List
+
+This email list contains gets all commits made to REEF
+
+- [Subscribe](mailto:commits-subscribe@reef.incubator.apache.org) (commits-subscribe@reef.incubator.apache.org)
+- [Unsusbscribe](mailto:commits-unsubscribe@reef.incubator.apache.org) (commits-unsubscribe@reef.incubator.apache.org)
+- [Archive](http://mail-archives.apache.org/mod_mbox/incubator-reef-commits/)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-reef/blob/fa77cc63/website/src/site/markdown/powered-by.md
----------------------------------------------------------------------
diff --git a/website/src/site/markdown/powered-by.md b/website/src/site/markdown/powered-by.md
new file mode 100644
index 0000000..d8d8bdf
--- /dev/null
+++ b/website/src/site/markdown/powered-by.md
@@ -0,0 +1,21 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+#Powered By
+
+- [Azure Stream Analytics](http://azure.microsoft.com/en-us/services/stream-analytics/)
\ No newline at end of file