You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@apex.apache.org by th...@apache.org on 2016/03/05 01:27:25 UTC
incubator-apex-core git commit: Fixing broken links and removing
configuration packages references
Repository: incubator-apex-core
Updated Branches:
refs/heads/master 58633c2fb -> 735ec932e
Fixing broken links and removing configuration packages references
Project: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/commit/735ec932
Tree: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/tree/735ec932
Diff: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/diff/735ec932
Branch: refs/heads/master
Commit: 735ec932e6f8cce55994a69aff428c5c1de40d2d
Parents: 58633c2
Author: Sasha Parfenov <sa...@apache.org>
Authored: Fri Mar 4 16:14:08 2016 -0800
Committer: Sasha Parfenov <sa...@apache.org>
Committed: Fri Mar 4 16:14:08 2016 -0800
----------------------------------------------------------------------
docs/application_development.md | 49 +++++++++++++-----------------------
docs/application_packages.md | 2 +-
mkdocs.yml | 3 +--
3 files changed, 19 insertions(+), 35 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-apex-core/blob/735ec932/docs/application_development.md
----------------------------------------------------------------------
diff --git a/docs/application_development.md b/docs/application_development.md
index c5c9075..eb56e8e 100644
--- a/docs/application_development.md
+++ b/docs/application_development.md
@@ -28,7 +28,7 @@ Running A Test Application
If you are starting with the Apex platform for the first time,
it can be informative to launch an existing application and see it run.
-One of the simplest examples provided in [Apex Malhar](apex_malhar.md) is a Pi demo application,
+One of the simplest examples provided in [Apex-Malhar repository](https://github.com/apache/incubator-apex-malhar) is a Pi demo application,
which computes the value of PI using random numbers. After [setting up development environment](apex_development_setup.md)
Pi demo can be launched as follows:
@@ -611,7 +611,7 @@ STRAM in more detail in the next chapter.
The instructions below assume that the platform was installed in a
directory <INSTALL_DIR> and the command line interface (CLI) will
be used to launch the demo application. An application can be run in
-[local mode](#h.3dy6vkm)[ ](#h.3dy6vkm)(in IDE or from command line) or on a [Hadoop cluster](#h.1t3h5sf) [.](#h.1t3h5sf)
+local mode (in IDE or from command line) or on a Hadoop cluster.
@@ -796,11 +796,9 @@ for ease of use and rapid development of application. These
operators are open sourced to Apache Software Foundation under the
project name “Malhar” as part of our efforts to foster community
innovation. These operators can be used in a DAG as is, while others
-have [properties](#h.32hioqz)
-[ ](#h.32hioqz)that can be set to specify the
+have properties that can be set to specify the
desired computation. Those interested in details, should refer to
-[Apex Malhar Operator Library](apex_malhar.md)
-.
+[Apex-Malhar operator library](https://github.com/apache/incubator-apex-malhar).
@@ -876,22 +874,18 @@ responsibilities of STRAM include
3. Collecting Application System Statistics and Logs
4. Logging all application-wide decisions taken
5. Providing system data on the state of the application via a Web Service.
-6. Supporting [Fault Tolerance](#h.2nusc19)
+6. Supporting Fault Tolerance
a. Detecting a node outage
b. Requesting a replacement resource from the Resource Manager
and scheduling state restoration for the streaming operators
c. Saving state to Zookeeper
-7. Supporting [Dynamic
- Partitioning](#h.3hv69ve)[:](#h.3hv69ve) Periodically
- evaluating the SLA and modifying the physical plan if required
+7. Supporting Dynamic Partitioning: Periodically evaluating the SLA and modifying the physical plan if required
(logical plan does not change).
-8. Enabling [Security](#h.3q5sasy)[:](#h.3q5sasy) Distributing
- security tokens for distributed components of the execution engine
+8. Enabling Security: Distributing security tokens for distributed components of the execution engine
and securing web service requests.
-9. Enabling [Dynamic modification](#h.40ew0vw)[ ](#h.40ew0vw)of
- DAG: In the future, we intend to allow for user initiated
+9. Enabling Dynamic modification of DAG: In the future, we intend to allow for user initiated
modification of the logical plan to allow for changes to the
processing logic and functionality.
@@ -1123,8 +1117,8 @@ DAG. The steps are
3. Create/re-use and instantiate operators
* Assign operator name that is unique within the application
- * Declare schema upfront for each operator (and thereby its [ports](#h.ihv636)[)](#h.ihv636)
- * (Optional) Set [properties](#h.32hioqz)[ ](#h.32hioqz) and [attributes](#h.41mghml)[ ](#h.41mghml) on the dag as per specification
+ * Declare schema upfront for each operator (and thereby its ports)
+ * (Optional) Set properties and attributes on the dag as per specification
* Connect ports of operators via streams
* Each stream connects one output port of an operator to one or more input ports of other operators.
* (Optional) Set attributes on the streams
@@ -1306,10 +1300,8 @@ the node named “Daily volume” has a physical identifier of 2.
### Operator Interface
-Operator interface in a DAG consists of [ports](#h.ihv636)[,](#h.ihv636) [properties](#h.32hioqz)[,](#h.32hioqz) and
- [attributes](#h.41mghml)
-[.](#h.41mghml) Operators interact with other
-components of the DAG via ports. Functional behavior of the operators
+Operator interface in a DAG consists of ports, properties, and attributes.
+Operators interact with other components of the DAG via ports. Functional behavior of the operators
can be customized via parameters. Run time performance and physical
instantiation is controlled by attributes. Ports and parameters are
fields (variables) of the Operator class/object, while attributes are
@@ -1383,9 +1375,7 @@ connected, to allow a port to go unconnected, you need to add
Attributes can be specified for ports that affect the runtime
behavior. An example of an attribute is parallel partition that specifes
a parallel computation flow per partition. It is described in detail in
-the [Parallel
-Partitions](#h.3vac5uf)[ ](#h.3vac5uf)section.
-Another example is queue capacity that specifies the buffer size for the
+the Parallel Partitions section. Another example is queue capacity that specifies the buffer size for the
port. Details of attributes are covered in Operation and Installation Guide.
#### Properties
@@ -1422,8 +1412,7 @@ Details of attributes are covered in [Configuration Guide](http://docs.datatorr
The state of an operator is defined as the data that it transfers
from one window to a future window. Since the computing model of the
platform is to treat windows like micro-batches, the operator state can
-be [checkpointed](#h.3mzq4wv)[ ](#h.3mzq4wv)every
-Nth window, or every T units of time, where T is significantly greater
+be checkpointed every Nth window, or every T units of time, where T is significantly greater
than the streaming window. When an operator is checkpointed, the entire
object is written to HDFS. The larger the amount of state in an
operator, the longer it takes to recover from a failure. A stateless
@@ -1600,8 +1589,7 @@ will give appropriate warnings or errors if that’s not the case.
-Details are explained in the chapter on Fault Tolerance
-below[.](#h.2nusc19)
+Details are explained in the chapter on Fault Tolerance below.
Streams
--------------------
@@ -1680,9 +1668,7 @@ operator (specially one with multiple input ports) should not depend on
the tuple order from two streams. One way to cope with this
indeterminate order, if necessary, is to wait to get all the tuples of a
window and emit results in endWindow call. All operator templates
-provided as part of [standard operator template
-library](#h.3ep43zb) [ ](#h.3ep43zb)follow
-these principles.
+provided as part of Malhar operator library follow these principles.
@@ -1801,8 +1787,7 @@ handling mechanisms. The checks are exclusively done by the application
business logic, but the platform allows applications to count and audit
these. Some of these features are in the process of development (backend
and UI) and this section will be updated as they are developed. Upon
-completion examples will be added to [demos](#h.upglbi) [t](#h.upglbi)o
-illustrate these.
+completion examples will be added to demos to illustrate these.
http://git-wip-us.apache.org/repos/asf/incubator-apex-core/blob/735ec932/docs/application_packages.md
----------------------------------------------------------------------
diff --git a/docs/application_packages.md b/docs/application_packages.md
index 042ee41..1d1d937 100644
--- a/docs/application_packages.md
+++ b/docs/application_packages.md
@@ -1,4 +1,4 @@
-Apache Apex Application and Configuration Packages
+Apache Apex Packages
==================================================
# Application Packages
http://git-wip-us.apache.org/repos/asf/incubator-apex-core/blob/735ec932/mkdocs.yml
----------------------------------------------------------------------
diff --git a/mkdocs.yml b/mkdocs.yml
index c91718c..646e9f5 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -6,8 +6,7 @@ pages:
- Development:
- Development Setup: apex_development_setup.md
- Applications: application_development.md
- - Application Packages: application_packages.md
- - Configuration Packages: configuration_packages.md
+ - Packages: application_packages.md
- Operators: operator_development.md
- AutoMetric API: autometrics.md
- Operations: