You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by "Marko A. Rodriguez (JIRA)" <ji...@apache.org> on 2015/06/03 00:34:52 UTC
[jira] [Closed] (TINKERPOP3-702) Buffer input to inner traversals
[ https://issues.apache.org/jira/browse/TINKERPOP3-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marko A. Rodriguez closed TINKERPOP3-702.
-----------------------------------------
Resolution: Not A Problem
Fix Version/s: 3.0.0.GA
Assignee: Marko A. Rodriguez
This is not fixable as its part of the language for local traversals to work with one traverser at a time. For {{repeat()}}, note that its sort of a hybrid between local and global. At first, its local, but then as more and more iterations are occurring, {{RepeatStep}} is bulking up Traversers. Thus, no "starts" are pulled until the {{repeat(traversal)}} is drained. Why? To reduce the amount of memory required in this breadth first operation.
In short, you "bulking" with {{ElasticVertexStep}} is good. Some traversals will be able to leverage it to its fullest, others not. One thing I do note that you do weird, you aggregate to a list. Why not just aggregate to {{this.starts.add()}}. This way you can take advantage of {{Traverser.bulk()}}-bulking and you have less data structures to maintain.
> Buffer input to inner traversals
> --------------------------------
>
> Key: TINKERPOP3-702
> URL: https://issues.apache.org/jira/browse/TINKERPOP3-702
> Project: TinkerPop 3
> Issue Type: Improvement
> Components: process
> Reporter: Ran Magen
> Assignee: Marko A. Rodriguez
> Fix For: 3.0.0.GA
>
>
> In elastic-gremlin we implement an optimized VertexStep. Part of its job is to batch/buffer/bulk different traversers and query them together in-order to minizmize the number of queries.
> You can see the implementation here: https://github.com/rmagen/elastic-gremlin/blob/master/src/main/java/org/elasticgremlin/process/optimize/ElasticVertexStep.java#L36
> This works great in regular traversals, the "starts" iterator returns as many traversers as the previous step gave out.
> But when the step is in an innerTraversal (e.g. g.V().repeat(__.out()).times(8)), the "starts" iterator only returns one traverser, and will return the next traverser only in the next call to processNextStart. Thus, there is no way to run a bulk query.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [jira] [Closed] (TINKERPOP3-702) Buffer input to inner traversals
Posted by Ran Magen <rm...@gmail.com>.
OK Marko, that makes sense. Thanks for the thorough explanation.
On יום ד׳, 3 ביוני 2015 at 01:35 Marko A. Rodriguez (JIRA) <ji...@apache.org>
wrote:
>
> [
> https://issues.apache.org/jira/browse/TINKERPOP3-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Marko A. Rodriguez closed TINKERPOP3-702.
> -----------------------------------------
> Resolution: Not A Problem
> Fix Version/s: 3.0.0.GA
> Assignee: Marko A. Rodriguez
>
> This is not fixable as its part of the language for local traversals to
> work with one traverser at a time. For {{repeat()}}, note that its sort of
> a hybrid between local and global. At first, its local, but then as more
> and more iterations are occurring, {{RepeatStep}} is bulking up Traversers.
> Thus, no "starts" are pulled until the {{repeat(traversal)}} is drained.
> Why? To reduce the amount of memory required in this breadth first
> operation.
>
> In short, you "bulking" with {{ElasticVertexStep}} is good. Some
> traversals will be able to leverage it to its fullest, others not. One
> thing I do note that you do weird, you aggregate to a list. Why not just
> aggregate to {{this.starts.add()}}. This way you can take advantage of
> {{Traverser.bulk()}}-bulking and you have less data structures to maintain.
>
> > Buffer input to inner traversals
> > --------------------------------
> >
> > Key: TINKERPOP3-702
> > URL:
> https://issues.apache.org/jira/browse/TINKERPOP3-702
> > Project: TinkerPop 3
> > Issue Type: Improvement
> > Components: process
> > Reporter: Ran Magen
> > Assignee: Marko A. Rodriguez
> > Fix For: 3.0.0.GA
> >
> >
> > In elastic-gremlin we implement an optimized VertexStep. Part of its job
> is to batch/buffer/bulk different traversers and query them together
> in-order to minizmize the number of queries.
> > You can see the implementation here:
> https://github.com/rmagen/elastic-gremlin/blob/master/src/main/java/org/elasticgremlin/process/optimize/ElasticVertexStep.java#L36
> > This works great in regular traversals, the "starts" iterator returns as
> many traversers as the previous step gave out.
> > But when the step is in an innerTraversal (e.g.
> g.V().repeat(__.out()).times(8)), the "starts" iterator only returns one
> traverser, and will return the next traverser only in the next call to
> processNextStart. Thus, there is no way to run a bulk query.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>