You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by se...@apache.org on 2014/09/01 19:52:49 UTC
[3/4] git commit: [FLINK-1084] Fix broken links in "How to add an
operator"
[FLINK-1084] Fix broken links in "How to add an operator"
Project: http://git-wip-us.apache.org/repos/asf/incubator-flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-flink/commit/c9dd6038
Tree: http://git-wip-us.apache.org/repos/asf/incubator-flink/tree/c9dd6038
Diff: http://git-wip-us.apache.org/repos/asf/incubator-flink/diff/c9dd6038
Branch: refs/heads/master
Commit: c9dd60385e05f9723d01554b8674e0c620de49c2
Parents: 0a6e53d
Author: Stephan Ewen <se...@apache.org>
Authored: Mon Sep 1 19:14:24 2014 +0200
Committer: Stephan Ewen <se...@apache.org>
Committed: Mon Sep 1 19:15:17 2014 +0200
----------------------------------------------------------------------
docs/internal_add_operator.md | 12 ++++++------
docs/internal_program_life_cycle.md | 2 +-
2 files changed, 7 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/c9dd6038/docs/internal_add_operator.md
----------------------------------------------------------------------
diff --git a/docs/internal_add_operator.md b/docs/internal_add_operator.md
index 40b0a69..f3c9758 100644
--- a/docs/internal_add_operator.md
+++ b/docs/internal_add_operator.md
@@ -56,7 +56,7 @@ public static <T>DataSet<Long> count(DataSet<T> data) {
A more complex example of an operation via specialization is the {% gh_link /flink-java/src/main/java/org/apache/flink/api/java/operators/AggregateOperator.java "Aggregation Operation" %} in the Java API. It is implemented by means of a *GroupReduce* UDF.
-The Aggregate Operation comes with its own operator in the *Java API*, but translates itself into a {% gh_link /flink-core/src/main/java/org/apache/flink/api/common/operators/base/GroupReduceOperatorBase.java "GroupReduceOperatorBase" %} in the *Common API*. (see [Program Life Cycle](program_life_cycle.html) for details of how an operation from the *Java API* becomes an operation of the *Common API* and finally a runtime operation.)
+The Aggregate Operation comes with its own operator in the *Java API*, but translates itself into a {% gh_link /flink-core/src/main/java/org/apache/flink/api/common/operators/base/GroupReduceOperatorBase.java "GroupReduceOperatorBase" %} in the *Common API*. (see [Program Life Cycle](internal_program_life_cycle.html) for details of how an operation from the *Java API* becomes an operation of the *Common API* and finally a runtime operation.)
The Java API aggregation operator is only a builder that takes the types of aggregations and the field positions, and used that information to
parameterize the GroupReduce UDF that performs the aggregations.
@@ -109,7 +109,7 @@ function, but invoked only once per parallel partition.
**Runtime**
-Runtime Operators are implemented using the {% gh_link /flink-runtime/src/main/java/org/apache/flink/pact/runtime/task/PactDriver.java "Driver" %} interface. The interface defines the methods that describe the operator towards the runtime. The {% gh_link /flink-runtime/src/main/java/org/apache/flink/pact/runtime/task/MapDriver.java "MapDriver" %} serves as a simple example of how those operators work.
+Runtime Operators are implemented using the {% gh_link /flink-runtime/src/main/java/org/apache/flink/runtime/operators/PactDriver.java "Driver" %} interface. The interface defines the methods that describe the operator towards the runtime. The {% gh_link /flink-runtime/src/main/java/org/apache/flink/runtime/operators/MapDriver.java "MapDriver" %} serves as a simple example of how those operators work.
The runtime works with the `MutableObjectIterator`, which describes data streams with the ability to reuse objects, to reduce pressure on the garbage collector.
@@ -132,16 +132,16 @@ To increase efficiency, it is often beneficial to implement a *chained* version
operators run in the same thread as their preceding operator, and work with nested function calls.
This is very efficient, because it saves serialization/deserialization overhead.
-To learn how to implement a chained operator, take a look at the {% gh_link /flink-runtime/src/main/java/org/apache/flink/pact/runtime/task/MapDriver.java "MapDriver" %} (regular) and the
-{% gh_link /flink-runtime/src/main/java/org/apache/flink/pact/runtime/task/chaining/ChainedMapDriver.java "ChainedMapDriver" %} (chained variant).
+To learn how to implement a chained operator, take a look at the {% gh_link /flink-runtime/src/main/java/org/apache/flink/runtime/operators/MapDriver.java "MapDriver" %} (regular) and the
+{% gh_link /flink-runtime/src/main/java/org/apache/flink/runtime/operators/chaining/ChainedMapDriver.java "ChainedMapDriver" %} (chained variant).
**Optimizer/Compiler**
-This section does a minimal discussion of the important steps to add an operator. Please see the [Optimizer](optimizer.html) docs for more detail on how the optimizer works.
+This section does a minimal discussion of the important steps to add an operator. Please see the [Optimizer](internal_optimizer.html) docs for more detail on how the optimizer works.
To allow the optimizer to include a new operator in its planning, it needs a bit of information about it; in particular, the following information:
-- *{% gh_link /flink-runtime/src/main/java/org/apache/flink/pact/runtime/task/DriverStrategy.java "DriverStrategy" %}*: The operation needs to be added to the Enum, to make it available to the optimizer. The parameters to the Enum entry define which class implements the runtime operator, its chained version, whether the operator accumulates records (and needs memory for that), and whether it requires a comparator (works on keys). For our example, we can add the entry
+- *{% gh_link /flink-runtime/src/main/java/org/apache/flink/runtime/operators/DriverStrategy.java "DriverStrategy" %}*: The operation needs to be added to the Enum, to make it available to the optimizer. The parameters to the Enum entry define which class implements the runtime operator, its chained version, whether the operator accumulates records (and needs memory for that), and whether it requires a comparator (works on keys). For our example, we can add the entry
``` java
MAP_PARTITION(MapPartitionDriver.class, null /* or chained variant */, PIPELINED, false)
```
http://git-wip-us.apache.org/repos/asf/incubator-flink/blob/c9dd6038/docs/internal_program_life_cycle.md
----------------------------------------------------------------------
diff --git a/docs/internal_program_life_cycle.md b/docs/internal_program_life_cycle.md
index c68232b..90d31cc 100644
--- a/docs/internal_program_life_cycle.md
+++ b/docs/internal_program_life_cycle.md
@@ -2,4 +2,4 @@
title: "Program Life Cycle"
---
-
+To be done...