You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by rm...@apache.org on 2014/07/04 11:57:04 UTC
[06/16] [documentation] fix internals linking
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/add_operator.md
----------------------------------------------------------------------
diff --git a/docs/internals/add_operator.md b/docs/internals/add_operator.md
deleted file mode 100644
index 8954682..0000000
--- a/docs/internals/add_operator.md
+++ /dev/null
@@ -1,215 +0,0 @@
----
-title: "How to add a new Operator"
----
-
-Operators in the Java API can be added in multiple different ways:
-
-1. On the DataSet, as a specialization/combination of existing operators
-2. As a custom extension operator
-3. As a new runtime operator
-
-The first two approaches are typically more lightweight and easier to implement. Sometimes,
-new functionality does require a new runtime operator, or it is much more efficient to
-
-
-## Implementing a new Operator on DataSet
-
-Many operators can be implemented as a specialization of another operator, or by means of a UDF.
-
-The simplest example are the `sum()`, `min()`, and `max()` functions on the [DataSet](https://github.com/apache/incubator-flink/blob/master/stratosphere-java/src/main/java/eu/stratosphere/api/java/DataSet.java). These functions simply call other operations
-with some pre-defined parameters:
-```
-public AggregateOperator<T> sum (int field) {
- return this.aggregate (Aggregations.SUM, field);
-}
-
-```
-
-Some operations can be implemented as compositions of multiple other operators. An example is to implement a
-*count()* function through a combination of *map* and *aggregate*.
-
-A simple way to do this is to define a function on the [DataSet](https://github.com/apache/incubator-flink/blob/master/stratosphere-java/src/main/java/eu/stratosphere/api/java/DataSet.java) that calls *map(...)* and *reduce(...)* in turn:
-```
-public DataSet<Long> count() {
- return this.map(new MapFunction<T, Long>() {
- public Long map(T value) {
- return 1L;
- }
- })
- .reduce(new ReduceFunction<Long>() {
- public Long reduce(Long val1, Long val1) {
- return val1 + val2;
- }
- });
-}
-```
-
-To define a new operator without altering the DataSet class is possible by putting the functions as static members
-into another class. The example of the *count()* operator would look the following way:
-```
-public static <T>DataSet<Long> count(DataSet<T> data) {
- return data.map(...).reduce(...);
-}
-```
-
-### More Complex Operators
-
-A more complex example of an operation via specialization is the [Aggregation Operation](https://github.com/apache/incubator-flink/blob/master/stratosphere-java/src/main/java/eu/stratosphere/api/java/operators/AggregateOperator.java) in the Java API. It is implemented by means of a *GroupReduce* UDF.
-
-The Aggregate Operation comes with its own operator in the *Java API*, but translates itself into a [GroupReduceOperatorBase](https://github.com/apache/incubator-flink/blob/master/stratosphere-core/src/main/java/eu/stratosphere/api/common/operators/base/GroupReduceOperatorBase.java) in the *Common API*. (see [Program Life Cycle](program_life_cycle.html) for details of how an operation from the *Java API* becomes an operation of the *Common API* and finally a runtime operation.)
-The Java API aggregation operator is only a builder that takes the types of aggregations and the field positions, and used that information to
-parameterize the GroupReduce UDF that performs the aggregations.
-
-Because the operation is translated into a GroupReduce operation, it appears as a GroupReduceOperator in the optimizer and runtime.
-
-
-
-## Implementing a Custom Extension Operator
-
-The DataSet offers a method for custom operators: `DataSet<X> runOperation(CustomUnaryOperation<T, X> operation)`.
-The *CustomUnaryOperation* interface defines operators by means of the two functions:
-``` java
-void setInput(DataSet<IN> inputData);
-
-DataSet<OUT> createResult();
-```
-
-The [VertexCentricIteration](https://github.com/apache/incubator-flink/blob/master/stratosphere-addons/spargel/src/main/java/eu/stratosphere/spargel/java/VertexCentricIteration.java) operator is implemented that way. Below is an example how to implement the *count()* operator that way.
-
-``` java
-public class Counter<T> implements CustomUnaryOperation<T, Long> {
-
- private DataSet<T> input;
-
- public void setInput(DataSet<IN> inputData) { this.input = inputData; }
-
- public DataSet<Long> createResult() {
- return input.map(...).reduce(...);
- }
-}
-```
-The CountOperator can be called in the following way:
-``` java
-DataSet<String> lines = ...;
-DataSet<Long> count = lines.runOperation(new Counter<String>());
-```
-
-
-## Implementing a new Runtime Operator
-
-Adding an new runtime operator requires changes throughout the entire stack, from the API to the runtime:
-
-- *Java API*
-- *Common API*
-- *Optimizer*
-- *Runtime*
-
-We start the description bottom up, at the example of the *mapPartition()* function, which is like a *map*
-function, but invoked only once per parallel partition.
-
-**Runtime**
-
-Runtime Operators are implemented using the [Driver](https://github.com/apache/incubator-flink/blob/master/stratosphere-runtime/src/main/java/eu/stratosphere/pact/runtime/task/PactDriver.java) interface. The interface defines the methods that describe the operator towards the runtime. The [MapDriver](https://github.com/apache/incubator-flink/blob/master/stratosphere-runtime/src/main/java/eu/stratosphere/pact/runtime/task/MapDriver.java) serves as a simple example of how those operators work.
-
-The runtime works with the `MutableObjectIterator`, which describes data streams with the ability to reuse objects, to reduce pressure on the garbage collector.
-
-An implementation of the central `run()` method for the *mapPartition* operator could look the following way:
-``` java
-public void run() throws Exception {
- final MutableObjectIterator<IN> input = this.taskContext.getInput(0);
- final MapPartitionFunction<IN, OUT> function = this.taskContext.getStub();
- final Collector<OUT> output = this.taskContext.getOutputCollector();
- final TypeSerializer<IN> serializer = this.taskContext.getInputSerializer(0);
-
- // we assume that the UDF takes a java.util.Iterator, so we wrap the MutableObjectIterator
- Iterator<IN> iterator = new MutableToRegularIteratorWrapper(input, serializer);
-
- function.mapPartition(iterator, output);
-}
-```
-
-To increase efficiency, it is often beneficial to implement a *chained* version of an operator. Chained
-operators run in the same thread as their preceding operator, and work with nested function calls.
-This is very efficient, because it saves serialization/deserialization overhead.
-
-To learn how to implement a chained operator, take a look at the [MapDriver](https://github.com/apache/incubator-flink/blob/master/stratosphere-runtime/src/main/java/eu/stratosphere/pact/runtime/task/MapDriver.java) (regular) and the
-[ChainedMapDriver](https://github.com/apache/incubator-flink/blob/master/stratosphere-runtime/src/main/java/eu/stratosphere/pact/runtime/task/chaining/ChainedMapDriver.java) (chained variant).
-
-
-**Optimizer/Compiler**
-
-This section does a minimal discussion of the important steps to add an operator. Please see the [Optimizer](optimizer.html) docs for more detail on how the optimizer works.
-To allow the optimizer to include a new operator in its planning, it needs a bit of information about it; in particular, the following information:
-
-- *[DriverStrategy](https://github.com/apache/incubator-flink/blob/master/stratosphere-runtime/src/main/java/eu/stratosphere/pact/runtime/task/DriverStrategy.java)*: The operation needs to be added to the Enum, to make it available to the optimizer. The parameters to the Enum entry define which class implements the runtime operator, its chained version, whether the operator accumulates records (and needs memory for that), and whether it requires a comparator (works on keys). For our example, we can add the entry
-``` java
-MAP_PARTITION(MapPartitionDriver.class, null /* or chained variant */, PIPELINED, false)
-```
-
-- *Cost function*: The class [CostEstimator](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/costs/CostEstimator.java) needs to know how expensive the operation is to the system. The costs here refer to the non-UDF part of the operator. Since the operator does essentially no work (it forwards the record stream to the UDF), the costs are zero. We change the `costOperator(...)` method by adding the *MAP_PARTITION* constant to the switch statement similar to the *MAP* constant such that no cost is accounted for it.
-
-- *OperatorDescriptor*: The operator descriptors define how an operation needs to be treated by the optimizer. They describe how the operation requires the input data to be (e.g., sorted or partitioned) and that way allows the optimizer to optimize the data movement, sorting, grouping in a global fashion. They do that by describing which [RequestedGlobalProperties](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/dataproperties/RequestedGlobalProperties.java) (partitioning, replication, etc) and which [RequestedLocalProperties](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/dataproperties/RequestedLocalProperties.java) (sorting, grouping, uniqueness) the operator has, as well as how the operator affects the existing [GlobalProperties](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/dat
aproperties/GlobalProperties.java) and [LocalProperties](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/dataproperties/LocalProperties.java). In addition, it defines a few utility methods, for example to instantiate an operator candidate.
-Since the *mapPartition()* function is very simple (no requirements on partitioning/grouping), the descriptor is very simple. Other operators have more complex requirements, for example the [GroupReduce](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/operators/GroupReduceProperties.java). Some operators, like *join* have multiple ways in which they can be executed and therefore have multiple descriptors ([Hash Join 1](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/operators/HashJoinBuildFirstProperties.java), [Hash Join 2](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/operators/HashJoinBuildSecondProperties.java), [SortMerge Join](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/operators/SortMergeJoinDescriptor.java
)).
-The code sample below explains (with comments) how to create a descriptor for the *MapPartitionOperator*
-``` java
- public DriverStrategy getStrategy() {
- return MAP_PARTITION;
- }
-
- // Instantiate the operator with the strategy over the input given in the form of the Channel
- public SingleInputPlanNode instantiate(Channel in, SingleInputNode node) {
- return new SingleInputPlanNode(node, "MapPartition", in, MAP_PARTITION);
- }
-
- // The operation accepts data with default global properties (arbitrary distribution)
- protected List<RequestedGlobalProperties> createPossibleGlobalProperties() {
- return Collections.singletonList(new RequestedGlobalProperties());
- }
-
- // The operation can accept data with any local properties. No grouping/sorting is necessary
- protected List<RequestedLocalProperties> createPossibleLocalProperties() {
- return Collections.singletonList(new RequestedLocalProperties());
- }
-
- // the operation itself does not affect the existing global properties.
- // The effect of the UDF's semantics// are evaluated separately (by interpreting the
- // semantic assertions)
- public GlobalProperties computeGlobalProperties(GlobalProperties gProps) {
- return gProps;
- }
-
- // since the operation can mess up all order, grouping, uniqueness, we cannot make any statements
- // about how local properties are preserved
- public LocalProperties computeLocalProperties(LocalProperties lProps) {
- return LocalProperties.EMPTY;
- }
-```
-
-- *OptimizerNode*: The optimizer node is the place where all comes together. It creates the list of *OperatorDescriptors*, implements the result data set size estimation, and assigns a name to the operation. It is a relatively small class and can be more or less copied again from the [MapNode](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/dag/MapNode.java).
-
-
-**Common API**
-
-To make the operation available to the higher-level APIs, it needs to be added to the Common API. The simplest way to do this is to add a
-base operator. Create a class `MapPartitionOperatorBase`, after the pattern of the [MapOperatorBase](https://github.com/apache/incubator-flink/blob/master/stratosphere-core/src/main/java/eu/stratosphere/api/common/operators/base/MapOperatorBase.java).
-
-In addition, the optimizer needs to know which OptimizerNode how to create an OptimizerNode from the OperatorBase. This happens in the class
-`GraphCreatingVisitor` in the [Optimizer](https://github.com/apache/incubator-flink/blob/master/stratosphere-compiler/src/main/java/eu/stratosphere/compiler/PactCompiler.java).
-
-*Note:* A pending idea is to allow to skip this step by unifying the OptimizerNode and the Common API operator. They essentially fulfill the
-same function. The Common API operator exists only in order for the `flink-java` and `flink-scala` packages to not have a dependency on the
-optimizer.
-
-
-**Java API**
-
-Create a Java API operator that is constructed in the same way as the [MapOperator](https://github.com/apache/incubator-flink/blob/master/stratosphere-java/src/main/java/eu/stratosphere/api/java/operators/MapOperator.java). The core method is the `translateToDataFlow(...)` method, which creates the Common API operator for the Java API operator.
-
-The final step is to add a function to the `DataSet` class:
-``` java
-public <R> DataSet<R> mapPartition(MapPartitionFunction<T, R> function) {
- return new MapPartitionOperator<T, R>(this, function);
-}
-```
-
-
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/distributed_runtime.md
----------------------------------------------------------------------
diff --git a/docs/internals/distributed_runtime.md b/docs/internals/distributed_runtime.md
deleted file mode 100644
index e6d7b83..0000000
--- a/docs/internals/distributed_runtime.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-title: "Distributed Runtime"
----
-
-Pending...
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/general_arch.md
----------------------------------------------------------------------
diff --git a/docs/internals/general_arch.md b/docs/internals/general_arch.md
deleted file mode 100644
index 70ac7be..0000000
--- a/docs/internals/general_arch.md
+++ /dev/null
@@ -1,67 +0,0 @@
----
-title: "General Architecture and Process Model"
----
-
-## The Processes
-
-When the Flink system is started, it bring up the *JobManager* and one or more *TaskManagers*. The JobManager
-is the coordinator of the Flink system, while the TaskManagers are the worksers that execute parts of the
-parallel programs. When starting the systen in *local* mode, a single JobManager and TaskManager are brought
-up within the same JVM.
-
-When a program is submitted, a client is created that performs the pre-processing and turns the program
-into the parallel data flow form that is executed by the JobManager and TaskManagers. The figure below
-illustrates the different actors in the system very coarsely.
-
-<div style="text-align: center;">
-<img src="ClientJmTm.svg" alt="The Interactions between Client, JobManager and TaskManager" height="400px" style="text-align: center;"/>
-</div>
-
-## Component Stack
-
-An alternative view on the system is given by the stack below. The different layers of the stack build on
-top of each other and raise the abstraction level of the program representations they accept:
-
-- The **runtime** layer receive a program in the form of a *JobGraph*. A JobGraph is a generic parallel
-data flow with arbitrary tasks that consume and produce data streams.
-
-- The **optimizer** and **common api** layer takes programs in the form of operator DAGs. The operators are
-specific (e.g., Map, Join, Filter, Reduce, ...), but are data type agnostic. The concrete types and their
-interaction with the runtime is specified by the higher layers.
-
-- The **API layer** implements multiple APIs that create operator DAGs for their programs. Each API needs
-to provide utilities (serializers, comparators) that describe the interaction between its data types and
-the runtime.
-
-<div style="text-align: center;">
-<img src="stack.svg" alt="The Flink component stack" width="800px" />
-</div>
-
-## Projects and Dependencies
-
-The Flink system code is divided into multiple sub-projects. The goal is to reduce the number of
-dependencies that a project implementing a Flink progam needs, as well as to faciltate easier testing
-of smaller sub-modules.
-
-The individual projects and their dependencies are shown in the figure below.
-
-<div style="text-align: center;">
-<img src="projects_dependencies.svg" alt="The Flink sub-projects and their dependencies" height="600px" style="text-align: center;"/>
-</div>
-
-In addition to the projects listed in the figure above, Flink currently contains the following sub-projects:
-
-- `flink-dist`: The *distribution* project. It defines how to assemble the compiled code, scripts, and other resources
-into the final folder structure that is ready to use.
-
-- `flink-addons`: A series of projects that are in an early version. Currently contains
-among other things projects for YARN support, JDBC data sources and sinks, hadoop compatibility,
-graph specific operators, and HBase connectors.
-
-- `flink-quickstart`: Scripts, maven archetypes, and example programs for the quickstarts and tutorials.
-
-
-
-
-
-
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/job_scheduling.md
----------------------------------------------------------------------
diff --git a/docs/internals/job_scheduling.md b/docs/internals/job_scheduling.md
deleted file mode 100644
index 5d40eb6..0000000
--- a/docs/internals/job_scheduling.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-title: "Jobs and Scheduling"
----
-
-
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/jobgraph_executiongraph.svg
----------------------------------------------------------------------
diff --git a/docs/internals/jobgraph_executiongraph.svg b/docs/internals/jobgraph_executiongraph.svg
deleted file mode 100644
index c74c3f4..0000000
--- a/docs/internals/jobgraph_executiongraph.svg
+++ /dev/null
@@ -1,453 +0,0 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
-<svg
- xmlns:dc="http://purl.org/dc/elements/1.1/"
- xmlns:cc="http://creativecommons.org/ns#"
- xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
- xmlns:svg="http://www.w3.org/2000/svg"
- xmlns="http://www.w3.org/2000/svg"
- xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
- xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
- version="1.0"
- width="329.56mm"
- height="259.01999mm"
- id="svg2"
- inkscape:version="0.48.4 r9939"
- sodipodi:docname="jobgraph_executiongraph.emf">
- <metadata
- id="metadata154">
- <rdf:RDF>
- <cc:Work
- rdf:about="">
- <dc:format>image/svg+xml</dc:format>
- <dc:type
- rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
- <dc:title></dc:title>
- </cc:Work>
- </rdf:RDF>
- </metadata>
- <sodipodi:namedview
- pagecolor="#ffffff"
- bordercolor="#666666"
- borderopacity="1"
- objecttolerance="10"
- gridtolerance="10"
- guidetolerance="10"
- inkscape:pageopacity="0"
- inkscape:pageshadow="2"
- inkscape:window-width="1600"
- inkscape:window-height="838"
- id="namedview152"
- showgrid="false"
- inkscape:zoom="0.25714017"
- inkscape:cx="583.86615"
- inkscape:cy="458.89368"
- inkscape:window-x="-8"
- inkscape:window-y="-8"
- inkscape:window-maximized="1"
- inkscape:current-layer="svg2" />
- <defs
- id="defs4" />
- <g
- id="g6"
- transform="translate(-50.556086,-241.11363)">
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 75.993932,278.1693 c 0,-10.35258 8.364584,-18.75467 18.754672,-18.75467 l 300.862456,0 c 10.35258,0 18.75467,8.40209 18.75467,18.75467 l 0,616.42857 c 0,10.35258 -8.40209,18.75467 -18.75467,18.75467 l -300.862456,0 c -10.390088,0 -18.754672,-8.40209 -18.754672,-18.75467 z"
- id="path8"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="174.01491"
- y="294.8385"
- style="font-size:30.0074749px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text10">JobGraph</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 446.54875,546.3236 47.74939,0 0,-42.49808 80.00743,62.49056 -80.00743,62.52808 0,-42.49809 -47.74939,0 z"
- id="path12"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 597.41133,324.7559 c 0,-36.08398 29.2948,-65.34127 65.37879,-65.34127 l 436.53378,0 c 36.1215,0 65.3788,29.25729 65.3788,65.34127 l 0,523.25536 c 0,36.1215 -29.2573,65.37879 -65.3788,65.37879 l -436.53378,0 c -36.08399,0 -65.37879,-29.25729 -65.37879,-65.37879 z"
- id="path14"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="761.92096"
- y="308.46613"
- style="font-size:30.0074749px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text16">ExecutionGraph</text>
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 281.73269,760.50196 0,-14.92872 -3.75094,0 0,14.92872 3.75094,0 z m 3.75093,-13.05325 -5.6264,-11.25281 -5.6264,11.25281 11.2528,0 z"
- id="path18"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 324.04323,778.24388 c 24.41858,24.41858 24.41858,64.02845 0,88.40953 -24.41859,24.41858 -63.99094,24.41858 -88.40953,0 -24.41858,-24.38108 -24.41858,-63.99095 0,-88.40953 24.41859,-24.38107 63.99094,-24.38107 88.40953,0"
- id="path20"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="238.73856"
- y="818.62347"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text22">JobInput</text>
- <text
- xml:space="preserve"
- x="249.24117"
- y="841.12909"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text24">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 324.04323,630.04446 c 24.41858,24.41858 24.41858,63.99094 0,88.40952 -24.41859,24.41859 -63.99094,24.41859 -88.40953,0 -24.41858,-24.41858 -24.41858,-63.99094 0,-88.40952 24.41859,-24.41858 63.99094,-24.41858 88.40953,0"
- id="path26"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="242.19254"
- y="670.40436"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text28">JobTask</text>
- <text
- xml:space="preserve"
- x="249.24428"
- y="692.90997"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text30">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.49437141px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 324.02447,333.58936 c 24.41859,24.41858 24.41859,63.99094 0,88.40952 -24.39983,24.41858 -63.99094,24.41858 -88.39077,0 -24.41858,-24.41858 -24.41858,-63.99094 0,-88.40952 24.39983,-24.41859 63.99094,-24.41859 88.39077,0"
- id="path32"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="231.68677"
- y="373.96625"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text34">JobOutput</text>
- <text
- xml:space="preserve"
- x="249.24113"
- y="396.47186"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text36">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.49437141px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 324.02447,481.80753 c 24.41859,24.41858 24.41859,64.0097 0,88.40953 -24.39983,24.41858 -63.99094,24.41858 -88.39077,0 -24.41858,-24.39983 -24.41858,-63.99095 0,-88.40953 24.39983,-24.39983 63.99094,-24.39983 88.39077,0"
- id="path38"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="242.19254"
- y="522.18524"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text40">JobTask</text>
- <text
- xml:space="preserve"
- x="249.24428"
- y="544.69086"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text42">Vertex</text>
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 281.73269,612.30254 0,-14.92872 -3.75094,0 0,14.92872 3.75094,0 z m 3.75093,-13.05325 -5.6264,-11.25281 -5.6264,11.25281 11.2528,0 z"
- id="path44"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.01875467px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 281.71393,464.06561 0,-14.92872 -3.75093,0 0,14.92872 3.75093,0 z m 3.75094,-13.05325 -5.62641,-11.2528 -5.6264,11.2528 11.25281,0 z"
- id="path46"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 949.62408,799.5867 c 21.98047,21.94296 21.98047,57.57684 0,79.55732 -21.98048,21.98047 -57.57685,21.98047 -79.55732,0 -21.98048,-21.98048 -21.98048,-57.61436 0,-79.55732 21.98047,-21.98048 57.57684,-21.98048 79.55732,0"
- id="path48"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="864.19208"
- y="835.50983"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text50">Execution</text>
- <text
- xml:space="preserve"
- x="879.1958"
- y="858.01538"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text52">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 802.09982,651.34977 c 21.98048,21.98047 21.98048,57.57684 0,79.55732 -21.98047,21.98047 -57.61435,21.98047 -79.55732,0 -21.98047,-21.98048 -21.98047,-57.57685 0,-79.55732 21.94297,-21.98048 57.57685,-21.98048 79.55732,0"
- id="path54"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="716.66034"
- y="687.29071"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text56">Execution</text>
- <text
- xml:space="preserve"
- x="731.66406"
- y="709.79626"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text58">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 949.62408,651.34977 c 21.98047,21.98047 21.98047,57.57684 0,79.55732 -21.98048,21.98047 -57.57685,21.98047 -79.55732,0 -21.98048,-21.98048 -21.98048,-57.57685 0,-79.55732 21.98047,-21.98048 57.57684,-21.98048 79.55732,0"
- id="path60"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="864.19208"
- y="687.29071"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text62">Execution</text>
- <text
- xml:space="preserve"
- x="879.1958"
- y="709.79626"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text64">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 1097.1483,651.34977 c 21.9805,21.98047 21.9805,57.57684 0,79.55732 -21.9429,21.98047 -57.5768,21.98047 -79.5573,0 -21.98047,-21.98048 -21.98047,-57.57685 0,-79.55732 21.9805,-21.98048 57.6144,-21.98048 79.5573,0"
- id="path66"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="1011.7239"
- y="687.29071"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text68">Execution</text>
- <text
- xml:space="preserve"
- x="1026.7275"
- y="709.79626"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text70">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 865.52812,501.91254 c 21.98048,21.94297 21.98048,57.57684 0,79.55732 -21.98047,21.98047 -57.61435,21.98047 -79.55731,0 -21.98048,-21.98048 -21.98048,-57.61435 0,-79.55732 21.94296,-21.98048 57.57684,-21.98048 79.55731,0"
- id="path72"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="780.09198"
- y="537.83459"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text74">Execution</text>
- <text
- xml:space="preserve"
- x="795.0957"
- y="560.34021"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text76">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 1032.0696,502.85027 c 21.943,21.94297 21.943,57.57685 0,79.55732 -21.9805,21.98048 -57.61434,21.98048 -79.59481,0 -21.94297,-21.98047 -21.94297,-57.61435 0,-79.55732 21.98047,-21.98047 57.61431,-21.98047 79.59481,0"
- id="path78"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="946.62244"
- y="538.78052"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text80">Execution</text>
- <text
- xml:space="preserve"
- x="961.62616"
- y="561.28613"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text82">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 865.52812,352.4378 c 21.98048,21.98048 21.98048,57.57684 0,79.55732 -21.98047,21.98048 -57.61435,21.98048 -79.55731,0 -21.98048,-21.98048 -21.98048,-57.57684 0,-79.55732 21.94296,-21.98047 57.57684,-21.98047 79.55731,0"
- id="path84"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="780.09198"
- y="388.37848"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text86">Execution</text>
- <text
- xml:space="preserve"
- x="795.0957"
- y="410.88409"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text88">Vertex</text>
- <path
- style="fill:none;stroke:#85888d;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 1032.0696,354.61334 c 21.943,21.98048 21.943,57.61436 0,79.55732 -21.9805,21.98048 -57.61434,21.98048 -79.59481,0 -21.94297,-21.94296 -21.94297,-57.57684 0,-79.55732 21.98047,-21.98047 57.61431,-21.98047 79.59481,0"
- id="path90"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="946.62244"
- y="390.56143"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text92">Execution</text>
- <text
- xml:space="preserve"
- x="961.62616"
- y="413.06705"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text94">Vertex</text>
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 827.60618,483.27039 0,-23.63088 -3.75094,0 0,23.63088 3.75094,0 z m 3.75093,-21.75542 -5.6264,-11.2528 -5.6264,11.2528 11.2528,0 z"
- id="path96"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 994.14767,483.27039 0,-23.63088 -3.75094,0 0,23.63088 3.75094,0 z m 3.75093,-21.75542 -5.6264,-11.2528 -5.6264,11.2528 11.2528,0 z"
- id="path98"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 911.73964,781.84478 0,-23.63089 -3.75094,0 0,23.63089 3.75094,0 z m 3.75093,-21.75542 -5.6264,-11.25281 -5.6264,11.25281 11.2528,0 z"
- id="path100"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 861.70217,806.56344 -86.27149,-49.32479 1.87547,-3.26332 86.23398,49.32479 -1.83796,3.26332 z m -86.49655,-45.16126 -6.97673,-10.4651 12.56563,0.71267 -5.5889,9.75243 z"
- id="path102"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 957.98866,806.56344 86.27154,-49.32479 -1.8755,-3.26332 -86.234,49.32479 1.83796,3.26332 z m 86.49654,-45.16126 6.9767,-10.4651 -12.5656,0.71267 5.5889,9.75243 z"
- id="path104"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 908.51383,636.6086 -40.47258,-39.87244 2.62565,-2.66316 40.5101,39.83492 -2.66317,2.70068 z m -41.78541,-35.85894 -4.08851,-11.92797 12.00299,3.90097 -7.91448,8.027 z"
- id="path106"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 914.17774,636.57109 40.73515,-40.99772 -2.66316,-2.66316 -40.73515,41.03522 2.66316,2.62566 z m 42.08549,-37.02173 3.90097,-11.96548 -11.89046,4.0135 7.98949,7.95198 z"
- id="path108"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 770.89205,636.57109 33.64588,-33.64589 -2.66316,-2.62565 -33.64588,33.64588 2.66316,2.62566 z m 34.95871,-29.63239 3.97599,-11.96548 -11.92797,3.97599 7.95198,7.98949 z"
- id="path110"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 1048.7988,636.57109 -33.6459,-33.64589 2.6632,-2.62565 33.6458,33.64588 -2.6631,2.62566 z m -34.9587,-29.63239 -3.976,-11.96548 11.928,3.97599 -7.952,7.98949 z"
- id="path112"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 770.02933,637.09622 178.43196,-46.51159 -0.93774,-3.63841 -178.43195,46.51159 0.93773,3.63841 z m 177.56924,-42.42307 9.45236,-8.28957 -12.30307,-2.58814 2.85071,10.87771 z"
- id="path114"
- inkscape:connector-curvature="0" />
- <path
- style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.03750934px;stroke-linecap:butt;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none"
- d="m 1049.624,636.68361 -159.11465,-45.87392 1.01275,-3.6009 159.1146,45.87393 -1.0127,3.60089 z m -158.36446,-41.7479 -9.26481,-8.55213 12.37808,-2.28807 -3.11327,10.8402 z"
- id="path116"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#861001;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 620.5546,357.31402 c 0,-17.40434 14.10351,-31.50785 31.50785,-31.50785 l 452.77525,0 c 17.4044,0 31.5079,14.10351 31.5079,31.50785 l 0,69.80489 c 0,17.40433 -14.1035,31.50785 -31.5079,31.50785 l -452.77525,0 c -17.40434,0 -31.50785,-14.10352 -31.50785,-31.50785 z"
- id="path118"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#861001;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 620.5546,506.75124 c 0,-17.36682 14.10351,-31.50784 31.50785,-31.50784 l 452.77525,0 c 17.4044,0 31.5079,14.14102 31.5079,31.50784 l 0,69.8424 c 0,17.40434 -14.1035,31.50785 -31.5079,31.50785 l -452.77525,0 c -17.40434,0 -31.50785,-14.10351 -31.50785,-31.50785 z"
- id="path120"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#861001;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 620.5546,655.62583 c 0,-17.40433 14.10351,-31.50785 31.50785,-31.50785 l 452.77525,0 c 17.4044,0 31.5079,14.10352 31.5079,31.50785 l 0,69.80489 c 0,17.40434 -14.1035,31.50785 -31.5079,31.50785 l -452.77525,0 c -17.40434,0 -31.50785,-14.10351 -31.50785,-31.50785 z"
- id="path122"
- inkscape:connector-curvature="0" />
- <path
- style="fill:none;stroke:#861001;stroke-width:2.51312613px;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none"
- d="m 620.5546,798.0113 c 0,-17.40433 14.10351,-31.50784 31.50785,-31.50784 l 452.77525,0 c 17.4044,0 31.5079,14.10351 31.5079,31.50784 l 0,69.80489 c 0,17.40434 -14.1035,31.50785 -31.5079,31.50785 l -452.77525,0 c -17.40434,0 -31.50785,-14.10351 -31.50785,-31.50785 z"
- id="path124"
- inkscape:connector-curvature="0" />
- <text
- xml:space="preserve"
- x="635.06665"
- y="361.29044"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text126">ExecutionGroup</text>
- <text
- xml:space="preserve"
- x="678.72754"
- y="383.79605"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text128">Vertex</text>
- <text
- xml:space="preserve"
- x="635.06665"
- y="510.74646"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text130">ExecutionGroup</text>
- <text
- xml:space="preserve"
- x="678.72754"
- y="533.25208"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text132">Vertex</text>
- <text
- xml:space="preserve"
- x="644.67828"
- y="796.58698"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text134">ExecutionGroup</text>
- <text
- xml:space="preserve"
- x="688.33917"
- y="819.09259"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text136">Vertex</text>
- <text
- xml:space="preserve"
- x="635.55035"
- y="659.14868"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text138">Execution</text>
- <text
- xml:space="preserve"
- x="652.50458"
- y="681.6543"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text140">Group</text>
- <text
- xml:space="preserve"
- x="650.55408"
- y="704.15991"
- style="font-size:18.754673px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text142">Vertex</text>
- <text
- xml:space="preserve"
- x="110.31272"
- y="834.37744"
- style="font-size:30.0074749px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text144">Source</text>
- <text
- xml:space="preserve"
- x="131.27104"
- y="686.15552"
- style="font-size:30.0074749px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text146">Map</text>
- <text
- xml:space="preserve"
- x="107.34327"
- y="537.93353"
- style="font-size:30.0074749px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text148">Reduce</text>
- <text
- xml:space="preserve"
- x="129.4863"
- y="389.71155"
- style="font-size:30.0074749px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Verdana"
- id="text150">Sink</text>
- </g>
-</svg>
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/operators_and_memory.md
----------------------------------------------------------------------
diff --git a/docs/internals/operators_and_memory.md b/docs/internals/operators_and_memory.md
deleted file mode 100644
index 5ff8584..0000000
--- a/docs/internals/operators_and_memory.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-title: "Runtime Algorithms and Memory Management"
----
-
-
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/optimizer.md
----------------------------------------------------------------------
diff --git a/docs/internals/optimizer.md b/docs/internals/optimizer.md
deleted file mode 100644
index 7fdf299..0000000
--- a/docs/internals/optimizer.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-title: "Optimizer"
----
-
-
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/overview.md
----------------------------------------------------------------------
diff --git a/docs/internals/overview.md b/docs/internals/overview.md
deleted file mode 100644
index 3e2b23f..0000000
--- a/docs/internals/overview.md
+++ /dev/null
@@ -1,41 +0,0 @@
----
-title: "Overview of Flink System Architecture & Internals"
----
-
-# Overview
-
-This documentation provides an overview of the architecture of the Flink system
-and its components. It is intended as guide to contributors, and people
-that are interested in the technology behind Flink.
-
-*This documentation is maintained by the contributors of the individual components.
-We kindly ask anyone that adds and changes components to eventually provide a patch
-or pull request that updates these documents as well.*
-
-
-### Architectures and Components
-
-- [General Architecture and Process Model](general_arch.html)
-
-<!--
-- [Life Cycle of a Program](program_life_cycle.html)
-
-- [Jobs and Scheduling](job_scheduling.html)
-
-- [Distributed Runtime](distributed_runtime.html)
-
-- [Runtime Algorithms and Memory Management](operators_and_memory.html)
-
-- [Program Optimizer](optimizer.html)
--->
-
-- [How-to: Adding a new Operator](add_operator.html)
-
-<!--
-- [Java API, Types, and Type Extraction](types.html)
--->
-
-<!--
-- [RPC and JobManager Communication](rpc_transfer.html)
--->
-
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/88e4dace/docs/internals/program_life_cycle.md
----------------------------------------------------------------------
diff --git a/docs/internals/program_life_cycle.md b/docs/internals/program_life_cycle.md
deleted file mode 100644
index c68232b..0000000
--- a/docs/internals/program_life_cycle.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-title: "Program Life Cycle"
----
-
-