You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@usergrid.apache.org by gr...@apache.org on 2015/05/07 00:35:59 UTC
[08/13] incubator-usergrid git commit: Updated all the diagrams for
newest refactor.
Updated all the diagrams for newest refactor.
Project: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/commit/5312bbc6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/tree/5312bbc6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/diff/5312bbc6
Branch: refs/heads/two-dot-o-dev
Commit: 5312bbc6b6e8c35305a5103c176385643eeb2699
Parents: c3897d3
Author: GERey <gr...@apigee.com>
Authored: Mon May 4 11:21:14 2015 -0700
Committer: GERey <gr...@apigee.com>
Committed: Mon May 4 11:21:14 2015 -0700
----------------------------------------------------------------------
.../usergrid/corepersistence/pipeline/README.md | 36 +++++++++-----------
1 file changed, 16 insertions(+), 20 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/5312bbc6/stack/core/src/main/java/org/apache/usergrid/corepersistence/pipeline/README.md
----------------------------------------------------------------------
diff --git a/stack/core/src/main/java/org/apache/usergrid/corepersistence/pipeline/README.md b/stack/core/src/main/java/org/apache/usergrid/corepersistence/pipeline/README.md
index ad42612..f469e30 100644
--- a/stack/core/src/main/java/org/apache/usergrid/corepersistence/pipeline/README.md
+++ b/stack/core/src/main/java/org/apache/usergrid/corepersistence/pipeline/README.md
@@ -1,25 +1,21 @@
#IO Framework
Based on the Pipes and Filters software pattern
-
-
##How does this all work?
Consider the following example flow:
- ```pipeline = 1. appId -> 2. filter1 -> filter2 -> 3. collector -> 4. PipelineResult```
+ ```pipeline = 1. appId -> 2. filter1 -> filter2 -> 3. collector -> 4. Observable```
1. First we start with an application. You want to do a certain set of operations on this application. The application gets passed into the pipeline. We extract the application id and pass unto the first filter
1. The filter represents some action being done unto the application. This could be a "getCollection" to a "deleteIndex" action. We can have as many as we want. These operations will be done in the order that they are represented in above. The results of each of the filters will be fed into the next one for processing. Thus filter 1 will be done before filter 2 can be applied.
* An important note here is that a cursor may or may not be generated here. this cursor ( if it exists ) will be stored in what is known as a Pipeline Context. The context contains all approriate state cursor information in case we need resume a query after we have run it.
1. After we have applied all the filters, we take the results and feed them into a collector. The collector aggreates the results returned and pushes them into a ResultSet.
- 1. The result set ( along with a cursor is one was generated ) is fed into the PipelineResult class. The PipelineResult is returned as a observable that can be iterated on by the calling method.
-
+ 1. The result set ( along with a cursor is one was generated ) is fed into a observable that can be iterated on.
###Pipeline Module
-
+
1. PipelineModule
-
* This handles all of the guice configuration for the Pipeline Module that we are working in. If you need to wire something, do it here.
1. PipelineBuilderFactory
* This factory is used to instantiate the ReadPipelineBuilder in CpEntityManager and CpRelationshipManager.
@@ -31,20 +27,23 @@ Consider the following example flow:
1. The request and response cursor.
1. The limit on the number of entities we might want to get back from a query
1. Where in the pipeline we are.
- a. The ```execute``` method contains the execution of the pipeline. It retrieves the applications and pipes the operations from one to another until we run out of operations. At that point we aggregate the response into a collection and return an instance of PipelineResult with the results and a cursor.
+ * The ```execute``` method contains the execution of the pipeline. It retrieves the applications and pipes the operations from one to another until we run out of operations. At that point we aggregate the response into a collection and return an observable with the elements.
1. PipelineContext
* This is a stateful representation of the current step in the pipeline. It contains the cusor that we want to run (requestCursor) and the cursor that is returned from the filter (responseCursor).
* Each filter contains a Pipeline Context due to the above reason.
-1. PipelineResult
- * Gets created in the Pipeline after the results are collected from the filters.
- * Depends on the ResponseCursor class.
+1. PipelineOperation
+ * Top class in the Pipeline because it defines what every pipeline operation needs to have and extend. Mandates that every class in the Pipeline extend this class.
+ * Primary interface for Filtering and Collection commands.
1. Cursor Module
* Contains the Request and ResponseCursor classes that are instantiated in Pipeline Module.
* Contains logic that represents the cursor logic.
+1. Read Module
+ * Contains the logic behind reading from graph and elasticsearch.
+
***
###Indepth Cursor Module Explanation
- 
+ 
The Cursor Module is made up of 7 classes.
@@ -61,11 +60,9 @@ The Cursor Module is made up of 7 classes.
***
###Indepth Read Module Explanation
- 
+ 
+
-1. PipelineOperation
- * Top class in the Pipeline because it defines what every pipeline operation needs to have and extend. Mandates that every class in the Pipeline extend this class.
- * Primary interface for Filtering commands.
1. CandidateResultsFilter
* Is an interface
* Extends PipelineOperation
@@ -82,7 +79,6 @@ The Cursor Module is made up of 7 classes.
1. CursorSeek
* Protected internal class that lives in AbstractSeekingFilter
* Whats the deal with only seeking values on the first call? Is this not similar to pagination?
-
1. Collector
* Extends generic PipelineOperation
* Primary used to interact with Entity and Elasticsearch Packages
@@ -106,7 +102,7 @@ The Cursor Module is made up of 7 classes.
1. ReadPipelineBuilderImpl
* Contains the builder implementation of the ReadPipelineBuilder.
* Adds on filters from FilterFactory depending on the type of action we take.
- * Contains execute method when the pipeline is finished building. This pushes results as a PipelineResult back up.
+ * Contains execute method when the pipeline is finished building. This pushes results as an observable back up.
***
###Indepth Entity Module Explanation
@@ -119,7 +115,7 @@ The entity module only contains two classes. So I won't attach the uml diagram a
***
###Indepth Graph Module Explanation
- 
+ 
1. EdgeCursorSerializer
* The serializer we use to decode and make sense of the graph cursor that gets returned.
@@ -135,7 +131,7 @@ The entity module only contains two classes. So I won't attach the uml diagram a
***
###Indepth Elasticsearch Module Explanation
- 
+ 
1. Impl Module
* contains all the implementations and verfiers and loaders for elasticsearch