[jira] [Commented] (TINKERPOP-2256) processAllStarts of AggregateStep should be called only once

["Wang Xiyu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TINKERPOP-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880738#comment-16880738 ] 

Wang Xiyu commented on TINKERPOP-2256:
--------------------------------------

The Full DSL is like :

 
{code:java}
g.V().hasLabel("tag").has("name", tagName).
sideEffect(
V().has('pid', personId).repeat(both("knows").simplePath()).emit().times(2).aggregate/store("friends")
).
as('src_tag').
in("hasTag").hasLabel("post").filter(__.out("hasCreator").where(within("friends"))).
out("hasTag").where(neq("src_tag")).groupCount().by().
unfold().limit(limit)
{code}
We found that using aggregate here is much more slow than using store here because aggregatestep do a lot of hasNext calls on steps which are already fully aggregated.

 

> processAllStarts of AggregateStep should be called only once
> ------------------------------------------------------------
>
>                 Key: TINKERPOP-2256
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2256
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: process
>            Reporter: Wang Xiyu
>            Priority: Minor
>
> Currently the function processNextStart , hasNextBarrier, nextBarrier of AggregateStep all call AggregateStep.processAllStarts:
> {code:java}
>     protected Traverser.Admin<S> processNextStart() {
>         this.processAllStarts();
>         return this.barrier.remove();
>     }{code}
> Every time we get a traverser from AggregateStep, AggregateStep.processAllStarts will be called. Then processAllStarts calls this.starts.hasNext().
> {code:java}
>     @Override
>     public void processAllStarts() {
>         if (this.starts.hasNext()) {
>             final BulkSet<Object> bulkSet = new BulkSet<>();
>             while (this.starts.hasNext()) {
>                 final Traverser.Admin<S> traverser = this.starts.next();
>                 bulkSet.add(TraversalUtil.applyNullable(traverser, this.aggregateTraversal), traverser.bulk());
>                 traverser.setStepId(this.getNextStep().getId()); // when barrier is reloaded, the traversers should be at the next step
>                 this.barrier.add(traverser);
>             }
>             this.getTraversal().getSideEffects().add(this.sideEffectKey, bulkSet);
>         }
>     }
> {code}
> It results in a lot of hasNext call.
> As document says "The step uses [eager evaluation|http://en.wikipedia.org/wiki/Eager_evaluation] in that no objects continue on until all previous objects have been fully aggregated." maybe we can limit the AggregateStep.processAllStarts only be called once.
>  
>  
>  
> We found this when we run DSL like this :
> {code:java}
> g.V().has('name','wxy').repeat(both("knows").simplePath()).emit().times(2).aggregate("friends"){code}
> and the plan is like this :
> {code:java}
> GraphStep(vertex,[name.eq(wxy)]), RepeatStep([VertexStep(BOTH,[knows],vertex), PathFilterStep(simple), RepeatEndStep],until(loops(2)),emit(true)), AggregateStep(friends)
> {code}
> Then we found thousands of calls to GraphStep(vertex,[name.eq(wxy)]).hasNext.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)