You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by pieter <pi...@gmail.com> on 2017/04/19 16:32:51 UTC
Local children barrier advice needed
Hi,
I have started optimizing Sqlg to do a bulk/barrier for its VertexStep
optimizations.
Sqlg has two optimization strategies.
GraphStepStrategy and VertexStepStrategy. GraphStepStrategy executes
first and then VertexStepStrategy.
GraphStepStrategy starts at the beginning of the traversal optimizing
from left to right till it reaches a step that it can not optimize and
terminates.
After that VertexStepStrategy tries to optimize what remains.
It ultimately replaces optimizable sequential steps with a
SqlgVertexStep.
Thus far the SqlgVertexStep always has one incoming traverser from
where it continues the traversal. Basically it translated to a sql
where clause with the incoming traversal element's id.
The current optimization is to bulk the incoming traversers and execute
the traversal for all incoming traversers in one go. This reduces
latency and has a drastic performance improvement.
I do the same as the existing BarrierSteps and iterate the `starts` to
collect all the left incoming traversers and from there I continue and
all is well.
However for local traversals there is only one start on the traversal
so the barrier idea is not working.
Is there a way barrier all incoming left traversers on local
traversals?
I have looked but can not see a way to accomplish this. Before I start
hacking away do you have ideas as how to accomplish this.
Does Tinkerpop already barrier/bulk incoming traversers on local
traversals?
Thanks
Pieter
Re: Local children barrier advice needed
Posted by pieter <pi...@gmail.com>.
Hi,
Ok thanks for that.
I am trying some trickery.
Will let you know where the trickery leads to.
CheersPieter
On Thu, 2017-04-20 at 13:59 -0600, Marko Rodriguez wrote:
> Pieter\u2026. you have a fascinating problem. Because this touches exactly
> at one of the fundamental concepts of Gremlin \u2014 local vs. global
> traversals.�
> g.V().out().count() == g.V().local(out()).count()��
> However,
>
> g.V().out().count() != g.V().local(out().count())
> ///////////////
>
> gremlin> g.V().out().count()
> ==>6
> gremlin> g.V().local(out()).count()
> ==>6
> gremlin> g.V().out().count()
> ==>6
> gremlin> g.V().local(out().count())
> ==>3
> ==>0
> ==>0
> ==>2
> ==>0
> ==>1
> gremlin>
> So, your ability to grab the "global stream\u201d and not just a single
> object from it within a local traversal will require some trickery.
>
> Look at:
>
> g.V().local(out().count())
> How would we do this so you pull all the V()\u2019s into the Sqlg[out()]
> but still single stream post process? Thats a good question. Hmmmm\u2026
>
>
> g.V().aggregate(\u2018x\u2019).cap(\u2018x\u2019).local(sqlgOut().count())
> Now, Sqlg[out()] would do this:
>
> 1.�Is the input a list? If yes, then execute all in batch.
> 2. Then pop off the first mapping as the output to count()
> only!
> 3. then Sqlg[out()].reset() method doesn\u2019t clear\u2026..
>
> Wait\u2026 no, it can\u2019t do that\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026cause it will not next()
> correctly.
>
> Hm. Wow\u2026. mind blown.
>
> Owwww\u2026\u2026\u2026\u2026.
>
> Marko.
>
>
>
>
>
> > On Apr 20, 2017, at 10:47 AM, pieter <pi...@gmail.com>
> > wrote:
> >
> > Sorry, forwarding was not a good idea either,
> >
> > Here is an example with global children and the batching works
> > well.
> > Sqlg does not currently optimize the 'where' (TraversalFilterStep).
> >
> > ����@Test
> > ����public void testBatchingIncomingTraversersOnVertexStep() {
> > ��������int count = 10_000;
> > ��������for (int i = 0; i < count; i++) {
> > ������������Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
> > ������������Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
> > ������������a1.addEdge("ab", b1);
> > ��������}
> > ��������this.sqlgGraph.tx().commit();
> >
> > ��������GraphTraversal traversal = this.sqlgGraph.traversal()
> > ����������������.V().where(__.hasLabel("A"))
> > ����������������.out();
> > ��������printTraversalForm(traversal);
> > ��������List<Vertex> vertices = traversal.toList();
> > ��������assertEquals(count, vertices.size());
> > ����}
> >
> > This prints out,
> >
> > pre-strategy:[GraphStep(vertex,[]),
> > TraversalFilterStep([HasStep([~label.eq(A)])]),
> > VertexStep(OUT,vertex)]
> > post-
> > strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
> > HasStep([~label.eq(A)]),
> > SqlgVertexStepCompiled@[sqlgPathFakeLabel]]
> >
> > The SqlgVertexStepCompiled is able to iterate all 10 000 incoming
> > traversers and execute one query for the out().
> >
> > This reduced the query time from 12 seconds to 0.4 seconds.
> > Happiness!!
> >
> > An example with a local traversal.
> >
> > ����@Test
> > ����public void testBatchingIncomingTraversersOnLocalVertexStep() {
> > ��������int count = 10_000;
> > ��������for (int i = 0; i < count; i++) {
> > ������������Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
> > ������������Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
> > ������������a1.addEdge("ab", b1);
> > ��������}
> > ��������this.sqlgGraph.tx().commit();
> >
> > ��������GraphTraversal traversal = this.sqlgGraph.traversal()
> > ����������������.V().hasLabel("A")
> > ����������������.local(
> > ������������������������__.out()
> > ����������������);
> > ��������printTraversalForm(traversal);
> > ��������List<Vertex> vertices = traversal.toList();
> > ��������Assert.assertEquals(count, vertices.size());
> > ����}
> >
> > This prints out,
> >
> > pre-strategy:[GraphStep(vertex,[]), HasStep([~label.eq(A)]),
> > LocalStep([VertexStep(OUT,vertex)])]
> > post-
> > strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
> > LocalStep([SqlgVertexStepCompiled@[sqlgPathFakeLabel]])]
> >
> > In this case SqlgVertexStepCompiled is a local traversal of the
> > LocalStep.
> >
> > Iterating the starts only returns one traverser as the LocalStep
> > only
> > puts one on the traversal at a time.
> >
> > I suppose I can replace LocalStep with a custom one but there are
> > many
> > steps with local children which will make things
> > fragile if I were to replace so many steps in a copy paste fashion.
> >
> > Thanks
> > Pieter
> >
> >
> > On Thu, 2017-04-20 at 09:10 -0600, Marko Rodriguez wrote:
> > > Hello,
> > >
> > > > I have started optimizing Sqlg to do a bulk/barrier for its
> > > > VertexStep
> > > > optimizations.
> > >
> > > Cool.
> > >
> > > > Sqlg has two optimization strategies.
> > > >
> > > > GraphStepStrategy and VertexStepStrategy. GraphStepStrategy
> > > > executes
> > > > first and then VertexStepStrategy.
> > > >
> > > > GraphStepStrategy starts at the beginning of the traversal
> > > > optimizing
> > > > from left to right till it reaches a step that it can not
> > > > optimize
> > > > and
> > > > terminates.
> > >
> > > Makes sense.
> > >
> > > > After that VertexStepStrategy tries to optimize what remains.
> > > > It ultimately replaces optimizable sequential steps with a
> > > > SqlgVertexStep.
> > >
> > > Okay...
> > >
> > > > Thus far the SqlgVertexStep always has one incoming traverser
> > > > from
> > > > where it continues the traversal. Basically it translated to a
> > > > sql
> > > > where clause with the incoming traversal element's id.
> > > >
> > > > The current optimization is to bulk the incoming traversers and
> > > > execute
> > > > the traversal for all incoming traversers in one go. This
> > > > reduces
> > > > latency and has a drastic performance improvement.
> > > >
> > > > I do the same as the existing BarrierSteps and iterate the
> > > > `starts`
> > > > to
> > > > collect all the left incoming traversers and from there I
> > > > continue
> > > > and
> > > > all is well.
> > >
> > > Smart. You got chops.
> > >
> > > > However for local traversals there is only one start on the
> > > > traversal
> > > > so the barrier idea is not working.
> > > >
> > > > Is there a way barrier all incoming left traversers on local
> > > > traversals?
> > >
> > > Eeeeeeeeeeeeeeee\u2026\u2026 huuuuuuuhhhhhhhh\u2026\u2026\u2026\u2026\u2026.
> > >
> > > There is a \u201ceasy\u201d and there is a \u201chard.\u201d Give me an example
> > > traversal
> > > and lets discuss from a more specific standpoint before
> > > generalizing...
> > >
> > > Thanks,
> > > Marko.
> > >
> > > http://markorodriguez.com
Re: Local children barrier advice needed
Posted by pieter gmail <pi...@gmail.com>.
Hi,
Ok took a while but got a partial solution working.
However I can not get it right, for now, forg.V().local(out().count())
I am not able to manage the localTraversal.reset() properly.
I should however be able to get it right later when I optimize
ReducingBarrierStep by pushing down the reducing to the db.
So for now I replace LocalStep andChooseStep (that do not have any
ReducingBarrierStep, RangeGlobal orSampleStep) with custom steps to
barrier the starts.
The biggest problem was to not loose the order of the incoming starts.
Basically I keep some state and sort the results afterwards.
There are more steps to replace but for now the strategy seems to be
working.
Cheers
Pieter
On 20/04/2017 21:59, Marko Rodriguez wrote:
> Pieter…. you have a fascinating problem. Because this touches exactly
> at one of the fundamental concepts of Gremlin — local vs. global
> traversals.
>
> g.V().out().count() == g.V().local(out()).count()
>
> However,
>
> g.V().out().count() != g.V().local(out().count())
>
>
> ///////////////
>
> gremlin> g.V().out().count()
> ==>6
> gremlin> g.V().local(out()).count()
> ==>6
> gremlin> g.V().out().count()
> ==>6
> gremlin> g.V().local(out().count())
> ==>3
> ==>0
> ==>0
> ==>2
> ==>0
> ==>1
> gremlin>
>
>
> So, your ability to grab the "global stream” and not just a single
> object from it within a local traversal will require some trickery.
>
> Look at:
>
> g.V().local(out().count())
>
>
> How would we do this so you pull all the V()’s into the Sqlg[out()]
> but still single stream post process? Thats a good question. Hmmmm…
>
>
> g.V().aggregate(‘x’).cap(‘x’).local(sqlgOut().count())
>
>
> Now, Sqlg[out()] would do this:
>
> 1. Is the input a list? If yes, then execute all in batch.
> 2. Then pop off the first mapping as the output to count() only!
> 3. then Sqlg[out()].reset() method doesn’t clear…..
>
> Wait… no, it can’t do that………………………………cause it will not next() correctly.
>
> Hm. Wow…. mind blown.
>
> Owwww………….
>
> Marko.
>
>
>> On Apr 20, 2017, at 10:47 AM, pieter <pieter.martin@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Sorry, forwarding was not a good idea either,
>>
>> Here is an example with global children and the batching works well.
>> Sqlg does not currently optimize the 'where' (TraversalFilterStep).
>>
>> @Test
>> public void testBatchingIncomingTraversersOnVertexStep() {
>> int count = 10_000;
>> for (int i = 0; i < count; i++) {
>> Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
>> Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
>> a1.addEdge("ab", b1);
>> }
>> this.sqlgGraph.tx().commit();
>>
>> GraphTraversal traversal = this.sqlgGraph.traversal()
>> .V().where(__.hasLabel("A"))
>> .out();
>> printTraversalForm(traversal);
>> List<Vertex> vertices = traversal.toList();
>> assertEquals(count, vertices.size());
>> }
>>
>> This prints out,
>>
>> pre-strategy:[GraphStep(vertex,[]),
>> TraversalFilterStep([HasStep([~label.eq(A)])]), VertexStep(OUT,vertex)]
>> post-strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
>> HasStep([~label.eq(A)]), SqlgVertexStepCompiled@[sqlgPathFakeLabel]]
>>
>> The SqlgVertexStepCompiled is able to iterate all 10 000 incoming
>> traversers and execute one query for the out().
>>
>> This reduced the query time from 12 seconds to 0.4 seconds. Happiness!!
>>
>> An example with a local traversal.
>>
>> @Test
>> public void testBatchingIncomingTraversersOnLocalVertexStep() {
>> int count = 10_000;
>> for (int i = 0; i < count; i++) {
>> Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
>> Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
>> a1.addEdge("ab", b1);
>> }
>> this.sqlgGraph.tx().commit();
>>
>> GraphTraversal traversal = this.sqlgGraph.traversal()
>> .V().hasLabel("A")
>> .local(
>> __.out()
>> );
>> printTraversalForm(traversal);
>> List<Vertex> vertices = traversal.toList();
>> Assert.assertEquals(count, vertices.size());
>> }
>>
>> This prints out,
>>
>> pre-strategy:[GraphStep(vertex,[]), HasStep([~label.eq(A)]),
>> LocalStep([VertexStep(OUT,vertex)])]
>> post-strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
>> LocalStep([SqlgVertexStepCompiled@[sqlgPathFakeLabel]])]
>>
>> In this case SqlgVertexStepCompiled is a local traversal of the
>> LocalStep.
>>
>> Iterating the starts only returns one traverser as the LocalStep only
>> puts one on the traversal at a time.
>>
>> I suppose I can replace LocalStep with a custom one but there are many
>> steps with local children which will make things
>> fragile if I were to replace so many steps in a copy paste fashion.
>>
>> Thanks
>> Pieter
>>
>>
>> On Thu, 2017-04-20 at 09:10 -0600, Marko Rodriguez wrote:
>>> Hello,
>>>
>>>> I have started optimizing Sqlg to do a bulk/barrier for its
>>>> VertexStep
>>>> optimizations.
>>>
>>> Cool.
>>>
>>>> Sqlg has two optimization strategies.
>>>>
>>>> GraphStepStrategy and VertexStepStrategy. GraphStepStrategy
>>>> executes
>>>> first and then VertexStepStrategy.
>>>>
>>>> GraphStepStrategy starts at the beginning of the traversal
>>>> optimizing
>>>> from left to right till it reaches a step that it can not optimize
>>>> and
>>>> terminates.
>>>
>>> Makes sense.
>>>
>>>> After that VertexStepStrategy tries to optimize what remains.
>>>> It ultimately replaces optimizable sequential steps with a
>>>> SqlgVertexStep.
>>>
>>> Okay...
>>>
>>>> Thus far the SqlgVertexStep always has one incoming traverser from
>>>> where it continues the traversal. Basically it translated to a sql
>>>> where clause with the incoming traversal element's id.
>>>>
>>>> The current optimization is to bulk the incoming traversers and
>>>> execute
>>>> the traversal for all incoming traversers in one go. This reduces
>>>> latency and has a drastic performance improvement.
>>>>
>>>> I do the same as the existing BarrierSteps and iterate the `starts`
>>>> to
>>>> collect all the left incoming traversers and from there I continue
>>>> and
>>>> all is well.
>>>
>>> Smart. You got chops.
>>>
>>>> However for local traversals there is only one start on the
>>>> traversal
>>>> so the barrier idea is not working.
>>>>
>>>> Is there a way barrier all incoming left traversers on local
>>>> traversals?
>>>
>>>
>>> Eeeeeeeeeeeeeeee…… huuuuuuuhhhhhhhh…………….
>>>
>>> There is a “easy” and there is a “hard.” Give me an example traversal
>>> and lets discuss from a more specific standpoint before
>>> generalizing...
>>>
>>> Thanks,
>>> Marko.
>>>
>>> http://markorodriguez.com
>
Re: Local children barrier advice needed
Posted by Marko Rodriguez <ok...@gmail.com>.
Pieter…. you have a fascinating problem. Because this touches exactly at one of the fundamental concepts of Gremlin — local vs. global traversals.
g.V().out().count() == g.V().local(out()).count()
However,
g.V().out().count() != g.V().local(out().count())
///////////////
gremlin> g.V().out().count()
==>6
gremlin> g.V().local(out()).count()
==>6
gremlin> g.V().out().count()
==>6
gremlin> g.V().local(out().count())
==>3
==>0
==>0
==>2
==>0
==>1
gremlin>
So, your ability to grab the "global stream” and not just a single object from it within a local traversal will require some trickery.
Look at:
g.V().local(out().count())
How would we do this so you pull all the V()’s into the Sqlg[out()] but still single stream post process? Thats a good question. Hmmmm…
g.V().aggregate(‘x’).cap(‘x’).local(sqlgOut().count())
Now, Sqlg[out()] would do this:
1. Is the input a list? If yes, then execute all in batch.
2. Then pop off the first mapping as the output to count() only!
3. then Sqlg[out()].reset() method doesn’t clear…..
Wait… no, it can’t do that………………………………cause it will not next() correctly.
Hm. Wow…. mind blown.
Owwww………….
Marko.
> On Apr 20, 2017, at 10:47 AM, pieter <pi...@gmail.com> wrote:
>
> Sorry, forwarding was not a good idea either,
>
> Here is an example with global children and the batching works well.
> Sqlg does not currently optimize the 'where' (TraversalFilterStep).
>
> @Test
> public void testBatchingIncomingTraversersOnVertexStep() {
> int count = 10_000;
> for (int i = 0; i < count; i++) {
> Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
> Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
> a1.addEdge("ab", b1);
> }
> this.sqlgGraph.tx().commit();
>
> GraphTraversal traversal = this.sqlgGraph.traversal()
> .V().where(__.hasLabel("A"))
> .out();
> printTraversalForm(traversal);
> List<Vertex> vertices = traversal.toList();
> assertEquals(count, vertices.size());
> }
>
> This prints out,
>
> pre-strategy:[GraphStep(vertex,[]),
> TraversalFilterStep([HasStep([~label.eq(A)])]), VertexStep(OUT,vertex)]
> post-strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
> HasStep([~label.eq(A)]), SqlgVertexStepCompiled@[sqlgPathFakeLabel]]
>
> The SqlgVertexStepCompiled is able to iterate all 10 000 incoming
> traversers and execute one query for the out().
>
> This reduced the query time from 12 seconds to 0.4 seconds. Happiness!!
>
> An example with a local traversal.
>
> @Test
> public void testBatchingIncomingTraversersOnLocalVertexStep() {
> int count = 10_000;
> for (int i = 0; i < count; i++) {
> Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
> Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
> a1.addEdge("ab", b1);
> }
> this.sqlgGraph.tx().commit();
>
> GraphTraversal traversal = this.sqlgGraph.traversal()
> .V().hasLabel("A")
> .local(
> __.out()
> );
> printTraversalForm(traversal);
> List<Vertex> vertices = traversal.toList();
> Assert.assertEquals(count, vertices.size());
> }
>
> This prints out,
>
> pre-strategy:[GraphStep(vertex,[]), HasStep([~label.eq(A)]),
> LocalStep([VertexStep(OUT,vertex)])]
> post-strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
> LocalStep([SqlgVertexStepCompiled@[sqlgPathFakeLabel]])]
>
> In this case SqlgVertexStepCompiled is a local traversal of the
> LocalStep.
>
> Iterating the starts only returns one traverser as the LocalStep only
> puts one on the traversal at a time.
>
> I suppose I can replace LocalStep with a custom one but there are many
> steps with local children which will make things
> fragile if I were to replace so many steps in a copy paste fashion.
>
> Thanks
> Pieter
>
>
> On Thu, 2017-04-20 at 09:10 -0600, Marko Rodriguez wrote:
>> Hello,
>>
>>> I have started optimizing Sqlg to do a bulk/barrier for its
>>> VertexStep
>>> optimizations.
>>
>> Cool.
>>
>>> Sqlg has two optimization strategies.
>>>
>>> GraphStepStrategy and VertexStepStrategy. GraphStepStrategy
>>> executes
>>> first and then VertexStepStrategy.
>>>
>>> GraphStepStrategy starts at the beginning of the traversal
>>> optimizing
>>> from left to right till it reaches a step that it can not optimize
>>> and
>>> terminates.
>>
>> Makes sense.
>>
>>> After that VertexStepStrategy tries to optimize what remains.
>>> It ultimately replaces optimizable sequential steps with a
>>> SqlgVertexStep.
>>
>> Okay...
>>
>>> Thus far the SqlgVertexStep always has one incoming traverser from
>>> where it continues the traversal. Basically it translated to a sql
>>> where clause with the incoming traversal element's id.
>>>
>>> The current optimization is to bulk the incoming traversers and
>>> execute
>>> the traversal for all incoming traversers in one go. This reduces
>>> latency and has a drastic performance improvement.
>>>
>>> I do the same as the existing BarrierSteps and iterate the `starts`
>>> to
>>> collect all the left incoming traversers and from there I continue
>>> and
>>> all is well.
>>
>> Smart. You got chops.
>>
>>> However for local traversals there is only one start on the
>>> traversal
>>> so the barrier idea is not working.
>>>
>>> Is there a way barrier all incoming left traversers on local
>>> traversals?
>>
>>
>> Eeeeeeeeeeeeeeee…… huuuuuuuhhhhhhhh…………….
>>
>> There is a “easy” and there is a “hard.” Give me an example traversal
>> and lets discuss from a more specific standpoint before
>> generalizing...
>>
>> Thanks,
>> Marko.
>>
>> http://markorodriguez.com
Re: Local children barrier advice needed
Posted by pieter <pi...@gmail.com>.
Sorry, forwarding was not a good idea either,
Here is an example with global children and the batching works well.
Sqlg does not currently optimize the 'where' (TraversalFilterStep).
����@Test
����public void testBatchingIncomingTraversersOnVertexStep() {
��������int count = 10_000;
��������for (int i = 0; i < count; i++) {
������������Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
������������Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
������������a1.addEdge("ab", b1);
��������}
��������this.sqlgGraph.tx().commit();
��������GraphTraversal traversal = this.sqlgGraph.traversal()
����������������.V().where(__.hasLabel("A"))
����������������.out();
��������printTraversalForm(traversal);
��������List<Vertex> vertices = traversal.toList();
��������assertEquals(count, vertices.size());
����}
This prints out,
pre-strategy:[GraphStep(vertex,[]),
TraversalFilterStep([HasStep([~label.eq(A)])]), VertexStep(OUT,vertex)]
post-strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
HasStep([~label.eq(A)]), SqlgVertexStepCompiled@[sqlgPathFakeLabel]]
The SqlgVertexStepCompiled is able to iterate all 10 000 incoming
traversers and execute one query for the out().
This reduced the query time from 12 seconds to 0.4 seconds. Happiness!!
An example with a local traversal.
����@Test
����public void testBatchingIncomingTraversersOnLocalVertexStep() {
��������int count = 10_000;
��������for (int i = 0; i < count; i++) {
������������Vertex a1 = this.sqlgGraph.addVertex(T.label, "A");
������������Vertex b1 = this.sqlgGraph.addVertex(T.label, "B");
������������a1.addEdge("ab", b1);
��������}
��������this.sqlgGraph.tx().commit();
��������GraphTraversal traversal = this.sqlgGraph.traversal()
����������������.V().hasLabel("A")
����������������.local(
������������������������__.out()
����������������);
��������printTraversalForm(traversal);
��������List<Vertex> vertices = traversal.toList();
��������Assert.assertEquals(count, vertices.size());
����}
This prints out,
pre-strategy:[GraphStep(vertex,[]), HasStep([~label.eq(A)]),
LocalStep([VertexStep(OUT,vertex)])]
post-strategy:[SqlgGraphStepCompiled(vertex,[])@[sqlgPathFakeLabel],
LocalStep([SqlgVertexStepCompiled@[sqlgPathFakeLabel]])]
In this case SqlgVertexStepCompiled is a local traversal of the
LocalStep.
Iterating the starts only returns one traverser as the LocalStep only
puts one on the traversal at a time.
I suppose I can replace LocalStep with a custom one but there are many
steps with local children which will make things
fragile if I were to replace so many steps in a copy paste fashion.
Thanks
Pieter
On Thu, 2017-04-20 at 09:10 -0600, Marko Rodriguez wrote:
> Hello,
>
> > I have started optimizing Sqlg to do a bulk/barrier for its
> > VertexStep
> > optimizations.
>
> Cool.
>
> > Sqlg has two optimization strategies.
> >
> > GraphStepStrategy and VertexStepStrategy. GraphStepStrategy
> > executes
> > first and then VertexStepStrategy.
> >
> > GraphStepStrategy starts at the beginning of the traversal
> > optimizing
> > from left to right till it reaches a step that it can not optimize
> > and
> > terminates.
>
> Makes sense.
>
> > After that VertexStepStrategy tries to optimize what remains.
> > It ultimately replaces optimizable sequential steps with a
> > SqlgVertexStep.
>
> Okay...
>
> > Thus far the SqlgVertexStep always has one incoming traverser from
> > where it continues the traversal. Basically it translated to a sql
> > where clause with the incoming traversal element's id.
> >
> > The current optimization is to bulk the incoming traversers and
> > execute
> > the traversal for all incoming traversers in one go. This reduces
> > latency and has a drastic performance improvement.
> >
> > I do the same as the existing BarrierSteps and iterate the `starts`
> > to
> > collect all the left incoming traversers and from there I continue
> > and
> > all is well.
>
> Smart. You got chops.
>
> > However for local traversals there is only one start on the
> > traversal
> > so the barrier idea is not working.
> >
> > Is there a way barrier all incoming left traversers on local
> > traversals?
>
>
> Eeeeeeeeeeeeeeee\u2026\u2026 huuuuuuuhhhhhhhh\u2026\u2026\u2026\u2026\u2026.
>
> There is a \u201ceasy\u201d and there is a \u201chard.\u201d Give me an example traversal
> and lets discuss from a more specific standpoint before
> generalizing...
>
> Thanks,
> Marko.
>
> http://markorodriguez.com
Re: Local children barrier advice needed
Posted by Marko Rodriguez <ok...@gmail.com>.
Hello,
> I have started optimizing Sqlg to do a bulk/barrier for its VertexStep
> optimizations.
Cool.
> Sqlg has two optimization strategies.
>
> GraphStepStrategy and VertexStepStrategy. GraphStepStrategy executes
> first and then VertexStepStrategy.
>
> GraphStepStrategy starts at the beginning of the traversal optimizing
> from left to right till it reaches a step that it can not optimize and
> terminates.
Makes sense.
> After that VertexStepStrategy tries to optimize what remains.
> It ultimately replaces optimizable sequential steps with a
> SqlgVertexStep.
Okay...
> Thus far the SqlgVertexStep always has one incoming traverser from
> where it continues the traversal. Basically it translated to a sql
> where clause with the incoming traversal element's id.
>
> The current optimization is to bulk the incoming traversers and execute
> the traversal for all incoming traversers in one go. This reduces
> latency and has a drastic performance improvement.
>
> I do the same as the existing BarrierSteps and iterate the `starts` to
> collect all the left incoming traversers and from there I continue and
> all is well.
Smart. You got chops.
> However for local traversals there is only one start on the traversal
> so the barrier idea is not working.
>
> Is there a way barrier all incoming left traversers on local
> traversals?
Eeeeeeeeeeeeeeee…… huuuuuuuhhhhhhhh…………….
There is a “easy” and there is a “hard.” Give me an example traversal and lets discuss from a more specific standpoint before generalizing...
Thanks,
Marko.
http://markorodriguez.com