Re: [TinkerPop] JavaTranslator vs. GroovyTranslator and Gremlin bytecode compilation speeds.

[Stephen Mallette <sp...@gmail.com>]

Your assumptions are correct but i'll add some slight modifications and
there is one other possibility to consider:

REST:  ScriptEngine
Client.submit(String): ScriptEngine
Client.submit(Traversal): Java
Client.submit(Bytecode): Java
traversal().withRemote(DriverRemoteConnection): Java
Client.submit(Traversal [with lambda]): ScriptEngine
Client.submit(Bytecode [with lambda]): ScriptEngine
traversal().withRemote(DriverRemoteConnection [with lambda]): ScriptEngine

Note i say "ScriptEngine" and not "Groovy" because the native language
might be Python, Clojure, whatever. Note that the inclusion of "lambdas" in
the traversal changes the execution method on the server. The language the
lambda is written in determines the ScriptEngine to use to process it.

On Wed, Sep 28, 2016 at 11:19 AM, Robert Dale <ro...@gmail.com> wrote:

> It would be good to understand when each is used. My assumptions:
>
> REST:  Groovy
> Client.submit(String): Groovy
> Client.submit(Traversal): Java
> Client.submit(Bytecode): Java
> traversal().withRemote(DriverRemoteConnection): Java
>
> ??
>
>
> On Wed, Sep 28, 2016 at 10:55 AM, Marko Rodriguez <ok...@gmail.com>
> wrote:
> > Hello,
> >
> > Gremlin bytecode provides a language agnostic way of sending Gremlin
> > traversals between machines — whether physical or virtual. For instance,
> it
> > is possible to send bytecode from one JVM to another or from CPython to
> the
> > JVM across the network. Once bytecode is received, it needs to be
> translated
> > into a representation that the processing VM can then evaluate.
> >
> > GremlinServer is smart in that when bytecode is received it will analyze
> it
> > for lambdas. If there are lambdas, written in language X, then it will
> use
> > XTranslator and XScriptEngine to evaluate the bytecode and create a
> > Traversal for evaluation. However, if there are no lambdas, then it will
> use
> > JavaTranslator to create a Traversal for evaluation.
> >
> > So, the question for me is:
> >
> > Is JavaTranslator (which uses Java reflection to convert bytecode to
> > Traversal) faster than GroovyTranslator/GroovyScriptEngine (which
> creates a
> > String script for and evaluates it in the ScriptEngine)?
> >
> > Lets see. Here is our script in total.
> >
> > import org.apache.tinkerpop.gremlin.jsr223.JavaTranslator
> > import org.apache.tinkerpop.gremlin.groovy.jsr223.GroovyTranslator
> >
> > //// EXECUTED LOCALLY (e.g. CLIENT APPLICATION) ////
> >
> > g = EmptyGraph.instance().traversal()
> >
> > t = g.V().has('name','marko').
> >       repeat(out()).times(2).
> >       groupCount().by('name'); []
> > bytecode = t.bytecode
> > // send the bytecode over the wire
> >
> > //// EXECUTED REMOTELY (e.g. GREMLIN SERVER) ////
> >
> > groovy = new GremlinGroovyScriptEngine()
> > bindings = groovy.createBindings()
> > bindings.put('g',g)
> > compiled = groovy.compile(GroovyTranslator.of('g').translate(bytecode))
> >
> > x = JavaTranslator.of(g).translate(bytecode); []
> > y = compiled.eval(bindings); []
> > z = groovy.eval(GroovyTranslator.of('g').translate(bytecode),
> bindings); []
> > x == y
> > y == z
> > z == x
> > x.toString()
> >
> > clock(1000){ JavaTranslator.of(g).translate(bytecode) }
> > clock(1000){ compiled.eval(bindings) } // caching
> > clock(1000){ groovy.reset();
> > groovy.eval(GroovyTranslator.of('g').translate(bytecode), bindings) }
> // no
> > caching
> >
> >
> > First, lets make sure they all return the same traversal:
> >
> > gremlin> x = JavaTranslator.of(g).translate(bytecode); []
> > gremlin> y = compiled.eval(bindings); []
> > gremlin> z = groovy.eval(GroovyTranslator.of('g').translate(bytecode),
> > bindings); []
> > gremlin> x == y
> > ==>true
> > gremlin> y == z
> > ==>true
> > gremlin> z == x
> > ==>true
> > gremlin> x.toString()
> > ==>[GraphStep(vertex,[]), HasStep([name.eq(marko)]),
> > RepeatStep([VertexStep(OUT,vertex),
> > RepeatEndStep],until(loops(2)),emit(false)),
> GroupCountStep(value(name))]
> > gremlin>
> >
> >
> > Great. They do. Now lets see how fast they are.
> >
> > gremlin> clock(1000){ JavaTranslator.of(g).translate(bytecode) }
> > ==>0.004768085
> > gremlin> clock(1000){ compiled.eval(bindings) } // caching
> > ==>0.015168259
> > gremlin> clock(1000){ groovy.reset();
> > groovy.eval(GroovyTranslator.of('g').translate(bytecode), bindings) }
> // no
> > caching
> > ==>40.790075693
> > gremlin>
> >
> >
> > Cool. JavaTranslator is about 1000x faster than a evaluating a String
> script
> > and about 3x faster than evaluating a compiled script. JavaTranslator
> takes
> > about 40 micro-seconds to translate the bytecode, while an uncached
> String
> > script takes 40 milliseconds.
> >
> > So, what did we learn?
> >
> > 1. Bytecode is slick in that we don’t have to use Gremlin-Groovy to
> evaluate
> > it (if there are no lambdas) and thus, can do everything in Java and
> fast!
> > 2. It very important to always use parameterized queries with
> > GremlinServer/etc. as you can see how costly it is to evaluate a String
> > script repeatedly.
> >
> > What is crazy is that my JavaTranslator code is gheeeeeetto.
> >
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/JavaTranslator.java
> >
> > If anyone wants to submit a PR to make JavaTranslator more efficient,
> please
> > do. However, we are still doing well with what we have regardless.
> >
> > Take care,
> > Marko.
> >
> > http://markorodriguez.com
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Gremlin-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to gremlin-users+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/gremlin-users/05AFFEA0-
> 3982-4F61-856F-6CE0CBD51602%40gmail.com.
> > For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Robert Dale
>