You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by Andrii Lomakin via dev <de...@tinkerpop.apache.org> on 2025/12/03 06:57:30 UTC
Vendor independent LDBC SNB benchmarks on Gremlin
Good day, colleagues.
Thanks to Vladislav Grinin (author of this project), I am going to present
to you our implementation of vendor-independent LDBC SNB benchmarks. They
use IoC based on Guice, so any vendor can utilize it to measure the
performance of their database.
https://github.com/JetBrains/ldbc-snb-interactive-gremlin
If you have any questions, feedback, or need assistance to incorporate
them, please join our Zulip chat. We will be glad to answer them
https://youtrackdb.zulipchat.com .
We look forward to seeing you there!
P.S. Please refrain from creating issues or discussions on GitHub. We will
close them soon.
--
Andrii Lomakin
YouTrackDB development lead
Re: Vendor independent LDBC SNB benchmarks on Gremlin
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
Hi Stephen,
I also wanted to share an example of how an optimized, vendor-specific
short query can be provided by overriding the default implementation:
https://github.com/JetBrains/ldbc-snb-interactive-gremlin/blob/master/ytdb/src/main/java/com/youtrackdb/ldbc/ytdb/YtdbModule.java
Best regards,
Andrii Lomakin
On Fri, Dec 5, 2025 at 3:02 PM Andrii Lomakin <an...@jetbrains.com>
wrote:
> Here is the default class that you can override to specify each query
> separately
>
> https://github.com/JetBrains/ldbc-snb-interactive-gremlin/blob/master/common/src/main/java/com/youtrackdb/ldbc/common/DefaultQueryModule.java
> .
>
> On Fri, Dec 5, 2025 at 1:58 PM Andrii Lomakin <
> andrey.lomakin@jetbrains.com> wrote:
>
>> Hi Stephen,
>>
>> Thank you for the feedback and for sharing your thoughts on idiomatic
>> Gremlin.
>>
>> Our implementation utilizes Guice, which enables vendors to override the
>> implementation of every query separately, addressing the differences in
>> database-specific optimizations you mentioned.
>> We are currently working on documentation that will provide more details
>> about our design and the framework's capabilities.
>>
>> Best regards,
>> Andrii Lomakin
>>
>> On Fri, Dec 5, 2025 at 1:45 PM Stephen Mallette <sp...@gmail.com>
>> wrote:
>>
>>> Thanks for sharing this. TinkerPop gets a lot of requests for an
>>> implementation of the snb benchmarks. There are some older repositories
>>> out
>>> there that used to support it, but they are no longer maintained to my
>>> knowledge. it will be nice to be able to point folks to one that is
>>> current
>>> the next time we're asked.
>>>
>>> One thing I remember from the earlier implementations of this benchmark
>>> for
>>> TinkerPop, was that Gremlin often optimizes differently per database and
>>> there are multiple ways to write a Gremlin query to get the same result.
>>> One of the earlier implementations was written with a particular graph in
>>> mind and therefore if run against a different graph might cut against it.
>>> It's just my suggestion but I tend to think that a good/fair benchmark
>>> implementation would simply maintain the Gremlin in its most idiomatic
>>> form
>>> (i.e. it would use the form most commonly accepted). I suppose that is
>>> subjective to some degree, but as an example, a lot of folks new to
>>> Gremlin
>>> immediately reach for select():
>>>
>>> gremlin> g.V(2).as('x','y').select('x','y').by('name').by('age')
>>> ==>[x:vadas,y:27]
>>>
>>> which we would almost surely say was idiomatically incorrect for Gremlin.
>>> We'd prefer project() for this case:
>>>
>>> gremlin> g.V(2).project('x','y').by('name').by('age')
>>> ==>[x:vadas,y:27]
>>>
>>> Or, instead of:
>>>
>>> gremlin> g.V().where(out())
>>> ==>v[1]
>>> ==>v[4]
>>> ==>v[6]
>>>
>>> we'd prefer:
>>>
>>> gremlin> g.V().where(outE())
>>> ==>v[1]
>>> ==>v[4]
>>> ==>v[6]
>>>
>>> even though TinkerPop's optimizers would catch that small detail for us.
>>> I
>>> suppose a different approach might be to allow providers to selectively
>>> override a query, if a different writing can achieve better performance
>>> in
>>> their case. Maybe users of the benchmark could then opt to run it with
>>> the
>>> idiomatic Gremlin or with the overrides to compare. I'd be curious if you
>>> had any thoughts on this matter as I think it has some impact on the
>>> direction of the implementation.
>>>
>>> On Wed, Dec 3, 2025 at 3:28 AM Andrii Lomakin via dev <
>>> dev@tinkerpop.apache.org> wrote:
>>>
>>> > The repository is available. Sorry for the confusion.
>>> >
>>> > On Wed, Dec 3, 2025 at 9:18 AM Andrii Lomakin <
>>> > andrey.lomakin@jetbrains.com>
>>> > wrote:
>>> >
>>> > > I do sorry for a bit early publishing repository will be available
>>> soon
>>> > >
>>> > > Andrii Lomakin
>>> > > YouTrackDB development lead
>>> > >
>>> > > On Wed, 3 Dec 2025, 07:57 Andrii Lomakin, <
>>> andrey.lomakin@jetbrains.com>
>>> > > wrote:
>>> > >
>>> > >> Good day, colleagues.
>>> > >>
>>> > >> Thanks to Vladislav Grinin (author of this project), I am going to
>>> > >> present to you our implementation of vendor-independent LDBC SNB
>>> > >> benchmarks. They use IoC based on Guice, so any vendor can utilize
>>> it to
>>> > >> measure the performance of their database.
>>> > >>
>>> > >>
>>> > >> https://github.com/JetBrains/ldbc-snb-interactive-gremlin
>>> > >>
>>> > >> If you have any questions, feedback, or need assistance to
>>> incorporate
>>> > >> them, please join our Zulip chat. We will be glad to answer them
>>> > >> https://youtrackdb.zulipchat.com .
>>> > >>
>>> > >> We look forward to seeing you there!
>>> > >>
>>> > >> P.S. Please refrain from creating issues or discussions on GitHub.
>>> We
>>> > >> will close them soon.
>>> > >>
>>> > >> --
>>> > >> Andrii Lomakin
>>> > >> YouTrackDB development lead
>>> > >>
>>> > >
>>> >
>>> > --
>>> > Andrii Lomakin
>>> > YouTrackDB development lead
>>> >
>>>
>>
>>
>> --
>> Andrii Lomakin
>> YouTrackDB development lead
>>
>
>
> --
> Andrii Lomakin
> YouTrackDB development lead
>
--
Andrii Lomakin
YouTrackDB development lead
Re: Vendor independent LDBC SNB benchmarks on Gremlin
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
Here is the default class that you can override to specify each query
separately
https://github.com/JetBrains/ldbc-snb-interactive-gremlin/blob/master/common/src/main/java/com/youtrackdb/ldbc/common/DefaultQueryModule.java
.
On Fri, Dec 5, 2025 at 1:58 PM Andrii Lomakin <an...@jetbrains.com>
wrote:
> Hi Stephen,
>
> Thank you for the feedback and for sharing your thoughts on idiomatic
> Gremlin.
>
> Our implementation utilizes Guice, which enables vendors to override the
> implementation of every query separately, addressing the differences in
> database-specific optimizations you mentioned.
> We are currently working on documentation that will provide more details
> about our design and the framework's capabilities.
>
> Best regards,
> Andrii Lomakin
>
> On Fri, Dec 5, 2025 at 1:45 PM Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> Thanks for sharing this. TinkerPop gets a lot of requests for an
>> implementation of the snb benchmarks. There are some older repositories
>> out
>> there that used to support it, but they are no longer maintained to my
>> knowledge. it will be nice to be able to point folks to one that is
>> current
>> the next time we're asked.
>>
>> One thing I remember from the earlier implementations of this benchmark
>> for
>> TinkerPop, was that Gremlin often optimizes differently per database and
>> there are multiple ways to write a Gremlin query to get the same result.
>> One of the earlier implementations was written with a particular graph in
>> mind and therefore if run against a different graph might cut against it.
>> It's just my suggestion but I tend to think that a good/fair benchmark
>> implementation would simply maintain the Gremlin in its most idiomatic
>> form
>> (i.e. it would use the form most commonly accepted). I suppose that is
>> subjective to some degree, but as an example, a lot of folks new to
>> Gremlin
>> immediately reach for select():
>>
>> gremlin> g.V(2).as('x','y').select('x','y').by('name').by('age')
>> ==>[x:vadas,y:27]
>>
>> which we would almost surely say was idiomatically incorrect for Gremlin.
>> We'd prefer project() for this case:
>>
>> gremlin> g.V(2).project('x','y').by('name').by('age')
>> ==>[x:vadas,y:27]
>>
>> Or, instead of:
>>
>> gremlin> g.V().where(out())
>> ==>v[1]
>> ==>v[4]
>> ==>v[6]
>>
>> we'd prefer:
>>
>> gremlin> g.V().where(outE())
>> ==>v[1]
>> ==>v[4]
>> ==>v[6]
>>
>> even though TinkerPop's optimizers would catch that small detail for us. I
>> suppose a different approach might be to allow providers to selectively
>> override a query, if a different writing can achieve better performance in
>> their case. Maybe users of the benchmark could then opt to run it with the
>> idiomatic Gremlin or with the overrides to compare. I'd be curious if you
>> had any thoughts on this matter as I think it has some impact on the
>> direction of the implementation.
>>
>> On Wed, Dec 3, 2025 at 3:28 AM Andrii Lomakin via dev <
>> dev@tinkerpop.apache.org> wrote:
>>
>> > The repository is available. Sorry for the confusion.
>> >
>> > On Wed, Dec 3, 2025 at 9:18 AM Andrii Lomakin <
>> > andrey.lomakin@jetbrains.com>
>> > wrote:
>> >
>> > > I do sorry for a bit early publishing repository will be available
>> soon
>> > >
>> > > Andrii Lomakin
>> > > YouTrackDB development lead
>> > >
>> > > On Wed, 3 Dec 2025, 07:57 Andrii Lomakin, <
>> andrey.lomakin@jetbrains.com>
>> > > wrote:
>> > >
>> > >> Good day, colleagues.
>> > >>
>> > >> Thanks to Vladislav Grinin (author of this project), I am going to
>> > >> present to you our implementation of vendor-independent LDBC SNB
>> > >> benchmarks. They use IoC based on Guice, so any vendor can utilize
>> it to
>> > >> measure the performance of their database.
>> > >>
>> > >>
>> > >> https://github.com/JetBrains/ldbc-snb-interactive-gremlin
>> > >>
>> > >> If you have any questions, feedback, or need assistance to
>> incorporate
>> > >> them, please join our Zulip chat. We will be glad to answer them
>> > >> https://youtrackdb.zulipchat.com .
>> > >>
>> > >> We look forward to seeing you there!
>> > >>
>> > >> P.S. Please refrain from creating issues or discussions on GitHub. We
>> > >> will close them soon.
>> > >>
>> > >> --
>> > >> Andrii Lomakin
>> > >> YouTrackDB development lead
>> > >>
>> > >
>> >
>> > --
>> > Andrii Lomakin
>> > YouTrackDB development lead
>> >
>>
>
>
> --
> Andrii Lomakin
> YouTrackDB development lead
>
--
Andrii Lomakin
YouTrackDB development lead
Re: Vendor independent LDBC SNB benchmarks on Gremlin
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
Hi Stephen,
Thank you for the feedback and for sharing your thoughts on idiomatic
Gremlin.
Our implementation utilizes Guice, which enables vendors to override the
implementation of every query separately, addressing the differences in
database-specific optimizations you mentioned.
We are currently working on documentation that will provide more details
about our design and the framework's capabilities.
Best regards,
Andrii Lomakin
On Fri, Dec 5, 2025 at 1:45 PM Stephen Mallette <sp...@gmail.com>
wrote:
> Thanks for sharing this. TinkerPop gets a lot of requests for an
> implementation of the snb benchmarks. There are some older repositories out
> there that used to support it, but they are no longer maintained to my
> knowledge. it will be nice to be able to point folks to one that is current
> the next time we're asked.
>
> One thing I remember from the earlier implementations of this benchmark for
> TinkerPop, was that Gremlin often optimizes differently per database and
> there are multiple ways to write a Gremlin query to get the same result.
> One of the earlier implementations was written with a particular graph in
> mind and therefore if run against a different graph might cut against it.
> It's just my suggestion but I tend to think that a good/fair benchmark
> implementation would simply maintain the Gremlin in its most idiomatic form
> (i.e. it would use the form most commonly accepted). I suppose that is
> subjective to some degree, but as an example, a lot of folks new to Gremlin
> immediately reach for select():
>
> gremlin> g.V(2).as('x','y').select('x','y').by('name').by('age')
> ==>[x:vadas,y:27]
>
> which we would almost surely say was idiomatically incorrect for Gremlin.
> We'd prefer project() for this case:
>
> gremlin> g.V(2).project('x','y').by('name').by('age')
> ==>[x:vadas,y:27]
>
> Or, instead of:
>
> gremlin> g.V().where(out())
> ==>v[1]
> ==>v[4]
> ==>v[6]
>
> we'd prefer:
>
> gremlin> g.V().where(outE())
> ==>v[1]
> ==>v[4]
> ==>v[6]
>
> even though TinkerPop's optimizers would catch that small detail for us. I
> suppose a different approach might be to allow providers to selectively
> override a query, if a different writing can achieve better performance in
> their case. Maybe users of the benchmark could then opt to run it with the
> idiomatic Gremlin or with the overrides to compare. I'd be curious if you
> had any thoughts on this matter as I think it has some impact on the
> direction of the implementation.
>
> On Wed, Dec 3, 2025 at 3:28 AM Andrii Lomakin via dev <
> dev@tinkerpop.apache.org> wrote:
>
> > The repository is available. Sorry for the confusion.
> >
> > On Wed, Dec 3, 2025 at 9:18 AM Andrii Lomakin <
> > andrey.lomakin@jetbrains.com>
> > wrote:
> >
> > > I do sorry for a bit early publishing repository will be available soon
> > >
> > > Andrii Lomakin
> > > YouTrackDB development lead
> > >
> > > On Wed, 3 Dec 2025, 07:57 Andrii Lomakin, <
> andrey.lomakin@jetbrains.com>
> > > wrote:
> > >
> > >> Good day, colleagues.
> > >>
> > >> Thanks to Vladislav Grinin (author of this project), I am going to
> > >> present to you our implementation of vendor-independent LDBC SNB
> > >> benchmarks. They use IoC based on Guice, so any vendor can utilize it
> to
> > >> measure the performance of their database.
> > >>
> > >>
> > >> https://github.com/JetBrains/ldbc-snb-interactive-gremlin
> > >>
> > >> If you have any questions, feedback, or need assistance to incorporate
> > >> them, please join our Zulip chat. We will be glad to answer them
> > >> https://youtrackdb.zulipchat.com .
> > >>
> > >> We look forward to seeing you there!
> > >>
> > >> P.S. Please refrain from creating issues or discussions on GitHub. We
> > >> will close them soon.
> > >>
> > >> --
> > >> Andrii Lomakin
> > >> YouTrackDB development lead
> > >>
> > >
> >
> > --
> > Andrii Lomakin
> > YouTrackDB development lead
> >
>
--
Andrii Lomakin
YouTrackDB development lead
Re: Vendor independent LDBC SNB benchmarks on Gremlin
Posted by Stephen Mallette <sp...@gmail.com>.
Thanks for sharing this. TinkerPop gets a lot of requests for an
implementation of the snb benchmarks. There are some older repositories out
there that used to support it, but they are no longer maintained to my
knowledge. it will be nice to be able to point folks to one that is current
the next time we're asked.
One thing I remember from the earlier implementations of this benchmark for
TinkerPop, was that Gremlin often optimizes differently per database and
there are multiple ways to write a Gremlin query to get the same result.
One of the earlier implementations was written with a particular graph in
mind and therefore if run against a different graph might cut against it.
It's just my suggestion but I tend to think that a good/fair benchmark
implementation would simply maintain the Gremlin in its most idiomatic form
(i.e. it would use the form most commonly accepted). I suppose that is
subjective to some degree, but as an example, a lot of folks new to Gremlin
immediately reach for select():
gremlin> g.V(2).as('x','y').select('x','y').by('name').by('age')
==>[x:vadas,y:27]
which we would almost surely say was idiomatically incorrect for Gremlin.
We'd prefer project() for this case:
gremlin> g.V(2).project('x','y').by('name').by('age')
==>[x:vadas,y:27]
Or, instead of:
gremlin> g.V().where(out())
==>v[1]
==>v[4]
==>v[6]
we'd prefer:
gremlin> g.V().where(outE())
==>v[1]
==>v[4]
==>v[6]
even though TinkerPop's optimizers would catch that small detail for us. I
suppose a different approach might be to allow providers to selectively
override a query, if a different writing can achieve better performance in
their case. Maybe users of the benchmark could then opt to run it with the
idiomatic Gremlin or with the overrides to compare. I'd be curious if you
had any thoughts on this matter as I think it has some impact on the
direction of the implementation.
On Wed, Dec 3, 2025 at 3:28 AM Andrii Lomakin via dev <
dev@tinkerpop.apache.org> wrote:
> The repository is available. Sorry for the confusion.
>
> On Wed, Dec 3, 2025 at 9:18 AM Andrii Lomakin <
> andrey.lomakin@jetbrains.com>
> wrote:
>
> > I do sorry for a bit early publishing repository will be available soon
> >
> > Andrii Lomakin
> > YouTrackDB development lead
> >
> > On Wed, 3 Dec 2025, 07:57 Andrii Lomakin, <an...@jetbrains.com>
> > wrote:
> >
> >> Good day, colleagues.
> >>
> >> Thanks to Vladislav Grinin (author of this project), I am going to
> >> present to you our implementation of vendor-independent LDBC SNB
> >> benchmarks. They use IoC based on Guice, so any vendor can utilize it to
> >> measure the performance of their database.
> >>
> >>
> >> https://github.com/JetBrains/ldbc-snb-interactive-gremlin
> >>
> >> If you have any questions, feedback, or need assistance to incorporate
> >> them, please join our Zulip chat. We will be glad to answer them
> >> https://youtrackdb.zulipchat.com .
> >>
> >> We look forward to seeing you there!
> >>
> >> P.S. Please refrain from creating issues or discussions on GitHub. We
> >> will close them soon.
> >>
> >> --
> >> Andrii Lomakin
> >> YouTrackDB development lead
> >>
> >
>
> --
> Andrii Lomakin
> YouTrackDB development lead
>
Re: Vendor independent LDBC SNB benchmarks on Gremlin
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
The repository is available. Sorry for the confusion.
On Wed, Dec 3, 2025 at 9:18 AM Andrii Lomakin <an...@jetbrains.com>
wrote:
> I do sorry for a bit early publishing repository will be available soon
>
> Andrii Lomakin
> YouTrackDB development lead
>
> On Wed, 3 Dec 2025, 07:57 Andrii Lomakin, <an...@jetbrains.com>
> wrote:
>
>> Good day, colleagues.
>>
>> Thanks to Vladislav Grinin (author of this project), I am going to
>> present to you our implementation of vendor-independent LDBC SNB
>> benchmarks. They use IoC based on Guice, so any vendor can utilize it to
>> measure the performance of their database.
>>
>>
>> https://github.com/JetBrains/ldbc-snb-interactive-gremlin
>>
>> If you have any questions, feedback, or need assistance to incorporate
>> them, please join our Zulip chat. We will be glad to answer them
>> https://youtrackdb.zulipchat.com .
>>
>> We look forward to seeing you there!
>>
>> P.S. Please refrain from creating issues or discussions on GitHub. We
>> will close them soon.
>>
>> --
>> Andrii Lomakin
>> YouTrackDB development lead
>>
>
--
Andrii Lomakin
YouTrackDB development lead
Re: Vendor independent LDBC SNB benchmarks on Gremlin
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
I do sorry for a bit early publishing repository will be available soon
Andrii Lomakin
YouTrackDB development lead
On Wed, 3 Dec 2025, 07:57 Andrii Lomakin, <an...@jetbrains.com>
wrote:
> Good day, colleagues.
>
> Thanks to Vladislav Grinin (author of this project), I am going to present
> to you our implementation of vendor-independent LDBC SNB benchmarks. They
> use IoC based on Guice, so any vendor can utilize it to measure the
> performance of their database.
>
>
> https://github.com/JetBrains/ldbc-snb-interactive-gremlin
>
> If you have any questions, feedback, or need assistance to incorporate
> them, please join our Zulip chat. We will be glad to answer them
> https://youtrackdb.zulipchat.com .
>
> We look forward to seeing you there!
>
> P.S. Please refrain from creating issues or discussions on GitHub. We will
> close them soon.
>
> --
> Andrii Lomakin
> YouTrackDB development lead
>