[jira] [Commented] (TINKERPOP-570) [Proposal] Provide support for OLAP to OLTP to OLAP to OLTP

["Marko A. Rodriguez (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TINKERPOP-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088098#comment-15088098 ] 

Marko A. Rodriguez commented on TINKERPOP-570:
----------------------------------------------

Given https://issues.apache.org/jira/browse/TINKERPOP-971, the solution to this ticket will fall out as a natural consequence. For example:

{code}
g.V().out().order().by('age',decr).limit(10).out('knows').values('name')
{code}

Right now, the query above will throw a {{ComputerVerificationException}} saying something along the lines of "mid-traversal barriers are not allowed in OLAP." That mid-traversal barrier is {{order().by()}}. However, with the concepts in TINKERPOP-971, the above traversal would compile for GraphComputer execution as:

{code}
[TraversalVertexProgramStep(GraphStep,VerticesStep,OrderStep),RangeStep,VerticesStep,PropertiesStep]
{code}

That is, it went OLAP-to-OLTP. How about OLTP-OLAP? Assume the following traversal:

{code}
g.V().has("name","marko").out("knows").has('age',gt(30)).repeat(out()).times(10).limit(10).values('name')
{code}

Given a {{ReasoningStrategy}} (another ticket), this could compile to:

{code}
[GraphStep,HasStep,VerticesStep,HasStep,XXX,TraversalVertexProgramStep([RepeatStep([VerticesStep])]),ComputerResultStep,RangeStep,PropertiesStep]
{code}

The problem was haven't solved is how do we feed traversers into {{TraversalVertexProgram}} (the XXX above)? Well since, TraversalVertexProgram is typed as {{<ComputerResult,ComputerResult>}}, we would need do something like this:

{code}
XXX {
  return new DefaultComputerResult {
    graph() { return this.getTraversal().getGraph() }
    memory() { return Map{{HALTED_TRAVERSERS, this.traversal.toSet()}} }
  }
}
{code}

Now when {{TraversalVertexProgramStep}} gets {{next'd()}} for the first time, it has these options:

* It is the first step in the traversal and thus, simply calls its {{TraversalVertexProgram}} and returns {{ComputerResult}} (currently how things work).
* It pulls a {{ComputerResult}} from the previous step and then calls the TraversalVertexProgram on the computerResult.graph().
* It pulls a {{ComputerResult}} that has a graph and a memory with HALTED_TRAVERSERS. It calls the TraversalVertexProgram (but looks into the memory to see if the current vertex has a HALTED_TRAVERSER).

....something along those lines. I don't like introducing a new step, and perhaps we can have it where TraversalVertexProgramStep is typed as {{TraversalVertexProgramStep<Object,ComputerResult>}} and thus, if the incoming object is a ComputerResult, do one thing, else aggregate all the objects and those are the starts for OLAP.... 

Needs more thinking, but I think we are on the right track.


> [Proposal] Provide support for OLAP to OLTP to OLAP to OLTP
> -----------------------------------------------------------
>
>                 Key: TINKERPOP-570
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-570
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: process
>    Affects Versions: 3.0.2-incubating
>            Reporter: Marko A. Rodriguez
>
> I'm trying to figure out how we can, within a "single traversal", move between OLAP and OLTP at different sections of the traversal. E.g.
> {code}
> [g.V.out.has('age',lt,25)]OLAP[out('parent').out('workPlace')]OLTP[out('coworkers').age.groupCount]OLAP
> {code}
> Going from OLAP to OLTP is easy. We have solved that already as OLAP queries return a {{Traversal<S,E>}} and thus, can be further processed in OLTP. But what about going from OLTP back into OLAP? We need to be able to stream the OLTP results back into traversers on the vertices of the graph -- TinkerGraph (easy), Hadoop (dynamic editing of the disk format!? crazy) .. is there a general pattern that works for all graphs? Finally, what about when the objects are NOT vertices/edges/etc. See the next issue.
> [~mbroecheler]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)