[jira] [Created] (TINKERPOP-1140) TraversalVertexProgramStep in support of OLAP/OLTP conversions.

["Marko A. Rodriguez (JIRA)" <ji...@apache.org>]

Marko A. Rodriguez created TINKERPOP-1140:
---------------------------------------------

             Summary: TraversalVertexProgramStep in support of OLAP/OLTP conversions.
                 Key: TINKERPOP-1140
                 URL: https://issues.apache.org/jira/browse/TINKERPOP-1140
             Project: TinkerPop
          Issue Type: Improvement
          Components: process
    Affects Versions: 3.1.0-incubating, 3.1.1-incubating, 3.1.2-incubating
            Reporter: Marko A. Rodriguez
             Fix For: 3.2.0-incubating


This was moved from TINKERPOP-971. TINKERPOP-971 was about TraversalSource fluency. This is a feature we can now do because of that work. Isolated this feature in a new ticket.
--------------

Check this idea out:

{code}
g = graph.traversal().withComputer(graph -> graph.compute(SparkGraphComputer.class).workers(10));
g.V().out().values('name')
{code}

This would compile to:

{code}
[TraversalVertexProgramStep([GraphStep,VerticesStep,PropertiesStep]),ComputerResultStep]
{code}

where,

{code}
TraversalVertexProgramStep<Graph,ComputerResult>
ComputerResultStep<ComputerResult,E>
{code}

Next, check this:

{code}
g = graph.traversal().withComputer(graph -> graph.compute(SparkGraphComputer.class).workers(10));
g.V().hasLabel('person').pageRank(out('knows')).order().by('page.rank',decr)
{code}

This will compile to:

{code}
[TraversalVertexProgramStep([GraphStep,HasStep]),PageRankVertexProgramStep([VerticesStep]),TraversalVertexProgramStep([OrderStep]),ComputerResultStep]
{code}

How does this work?!

* TraversalVertexProgramStep will pass a {{ComputerResult}} to PageRankVertexProgram.
** The ComputerResult.graph() will have HALTED_TRAVERSERS properties on all the person vertices (as that is what was computed by the vertex program).
* PageRankVertexProgram will then pass a {{ComputerResult}} to the next TraversalVertexProgram.
** That ComputerResult.graph() will have HALTED_TRAVERSER on all the person vertices and PAGE_RANK on all the vertices.
* TraversalVertexProgram will then execute OrderStep which sorts the person-vertices with HALTED_TRAVERSERS on them by their page.rank properties computed previously.
* ComputerResultStep will then take get the ComputerResult.memory.reducing and iterate it out.

{{ComputerResultStep}} (like now) is smart about either pulling from the graph or from the sideEffect memory. What makes things different from now is that {{TraversalVertexProgramStep}} will come into existence and pull out the respective logic from {{ComputerResultStep}}. In this way, It will be possible to chain {{{XXXVertexProgramStep}}-steps and thus, have a traversal that is multiple OLAP jobs in sequence and thus, this ticket will fall naturally from this https://issues.apache.org/jira/browse/TINKERPOP-570. The whole trick to all of this is that (as we currently do) we save the state of the computation in the graph (and memory) and thus, feeding program into the next just takes over the computation. PageRankVertexProgram doesn't use HALTED_TRAVERSERS in its computation, so it just ignores it. However, TraversalVertexProgram can later access PAGE_RANK and thus, use the results of the previous OLAP computation.

Finally, you want "lambda vertex programs?"

{code}
g = graph.traversal().withComputer(graph -> graph.compute(SparkGraphComputer.class).workers(10));
myProgram = MyVertexProgram.build().create();
g.V().program(myProgram).values('myProgramCounters')
{code}

which compiles to:

{code}
[TraversalVertexProgramStep([GraphStep]),LambdaVertexProgramStep(MyVertexProgram),TraversalVertexProgramStep([PropertiesStep]),ComputerResultStep]
{code}

Thus, just like {{filter()}},{{flatMap()}}, etc....










--
This message was sent by Atlassian JIRA
(v6.3.4#6332)