You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tinkerpop.apache.org by Stephen Mallette <sp...@gmail.com> on 2017/09/14 20:17:07 UTC

[DISCUSS] GLV Test Suite

I've brought this issue up in the past and had suggested some options I had
in mind but now I've finally put the basics of those ideas in place so I
figured I'd start a fresh thread. Recall that the issue at hand is that we
don't have a test suite for GLVs as gremlin-test is bound to the JVM. We
have some tricks that let us test gremlin-python with it but those tricks
won't work for every language and we now have the first language in
gremlin-dotnet and upcoming gremlin-javascript which won't support it (yes,
i know that gremlin-javascript can run on the jvm but there are issues with
getting it all to work with the test framework that make it unduly
complicated).

On other threads I offered the idea that we look to use Gherkin to write
general Gremlin test specifications, which then could be read and processed
by the wide variety of test frameworks that can read that format - there
tend to be Gherkin processors in just about every language - for example,
see:

https://cucumber.io/

I just created this issue:

https://issues.apache.org/jira/browse/TINKERPOP-1784

and pushed this branch:

https://github.com/apache/tinkerpop/tree/TINKERPOP-1784

which demonstrates how this works with gremlin-python. The basic anatomy of
this setup involves this new directory in gremlin-test:

https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/gremlin-test/features

It contains the Gherkin .features files. These are the test specifications.
They are written using gremlin-java as the "model" language. GLVs will then
need to write some infrastructure to process these Gherkin files. The key
to making this "easy" to implement will lie in our abiilty to keep the
assertions we want to have relatively simple. The more simplistic the
language in the Gherkin .feature files the easier the job it will be for
GLVs to build their infrastructure. Of course, once that infrastructure is
in place, the GLV developer just has to write the GLV version of the
Gremlin specified in the .feature file. So you can look at all the
"infrastructure" code here in this pair of files:

https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/gremlin-python/src/main/jython/radish

and then you can look at the GLV Gremlin translations specifically here:

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-python/src/main/jython/radish/count_features_step.py#L34-L46

I think this approach works pretty well and solves our general problems for
GLV testing. There is some pain up front in implementing the
"infrastructure" but after that new Gremlin tests added to .feature files
just need to translated in the GLV. I suppose we could "automate" a good
portion of the translation with reflection of some sort. Anything else
could just be handled manually.

Not sure if we need to use this new model to wholly replace the old one.
The process test suite has its place in helping graph database providers
test their stuff. I also imagine that introducing this approach in that
context would create a breaking change which we would then need to push off
to 3.4.0. I suppose that gives us time to think, but for now it might not
be best to conflate the two and just treat them as separate aspects of the
test suite.

Anyway - it's important we settle on an approach to testing as we really
shouldn't do a GA release of the Gremlin .NET GLV without getting the test
suite solid. please yell if you have any ideas or feedback on this approach.

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

Makes sense, as long as we have the basic logic nailed down for the steps /
parameters / results, we can continue adding scenarios afterwards (new
scenarios with the same step structure should work in any language).

On Wed, Nov 8, 2017 at 1:49 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> I've been away for a few days on vacation, but before I left, I made some
> pretty decent progress on the remaining tests and I'm finding less and less
> of my time being focused on the test logic and more and more of it on the
> translation of the tests themselves. I'm starting to feel comfortable that
> most of the remaining pain is in test translation which means I could
> probably issue the PR for what I have sooner than later. I'll probably just
> make sure that I have at least a "good enough" representation of tests for
> each step and then issue the PR. That will allow TINKERPOP-1827 to proceed
> more easily and I can work more at more relaxed pace on migrating all the
> tests. Please let me know if there are any concerns with that approach.
>
> On Tue, Nov 7, 2017 at 5:02 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> >
> wrote:
>
> > I've created TINKERPOP-1827 to track the progress of the test suite
> runner
> > for Gremlin .NET.
> >
> > I have a working gherkin-based runner that executes the steps (parsing
> > gremlin groovy / parameters / results) as defined in the TINKERPOP-1784
> WIP
> > branch.
> >
> > https://github.com/jorgebay/tinkerpop/commits/gherkin-cs-runner
> >
> > I'll wait for TINKERPOP-1784 to be resolved in order the follow the last
> > changes and submit a pull request for it.
> >
> > On Tue, Oct 31, 2017 at 11:09 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com
> > > wrote:
> >
> > > Ok.
> > > Going back to my initial comment regarding enums: don't mind, I've
> > > included the possibility to use enums on the C# parser.
> > >
> > > On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > >> I guess there wouldn't be a problem with that, though having it the
> way
> > it
> > >> works now was a nice check to ensure that the ScriptEngine was
> > configured
> > >> properly. I guess that shouldn't be the focus on this body of tests
> > >> though.
> > >> We should be more concerned that the elements of the Gremlin language
> > >> actually work.
> > >>
> > >> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
> > >> jorgebaygondra@gmail.com>
> > >> wrote:
> > >>
> > >> > I see that for most languages using bytecode to native language
> > >> translator
> > >> > would be the easiest solution. In the case of the JavaScript GLV, it
> > >> would
> > >> > be fairly straightforward to implement (and I plan to do so!).
> > >> > In the case of C#, being more strict regarding typing than java, it
> > >> would
> > >> > be an incomplete solution. For example:
> > >> >
> > >> > Consider the following traversal in Java: g.V().values("age")
> > >> > The type parameter for values() method is inferred at runtime and
> > things
> > >> > "just works" thanks to type erasure.
> > >> >
> > >> > In the case of C#, it should be written as
> g.V().Values<int>("age") ,
> > >> with
> > >> > the type parameter to specify which type of traverser are you
> > expecting
> > >> at
> > >> > a compile time.
> > >> >
> > >> > That's why it wont be possible to make a generic translator from
> > >> bytecode
> > >> > to C# code, without understanding which generic types are expected.
> > >> >
> > >> > In the case of C#, I was going for a translator on the C# side that
> > >> > tokenizes the gremlin traversal. Once the traversal is tokenized, in
> > the
> > >> > moment of invoking the methods (with reflection), it can be made
> aware
> > >> of
> > >> > the modern graph data (with datatypes!), ie: For method Values<>(),
> if
> > >> the
> > >> > property key is "age" -> use int as generic type parameter.
> > >> >
> > >> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <
> > spmallette@gmail.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > I can't help thinking that perhaps non-JVM languages need to
> > leverage
> > >> > > bytecode to make traversal building easier. We already have an
> > >> > established
> > >> > > pattern for converting bytecode into Traversals and while it is
> > >> different
> > >> > > for each language, it's generally governed by the Translator
> > >> interface.
> > >> > > Note that we already do this for Java, Groovy, and Python and the
> > code
> > >> > > isn't too crazy - just 200 lines or so:
> > >> > >
> > >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > >> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> > >> > JavaTranslator.java
> > >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > >> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> > >> > > GroovyTranslator.java
> > >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > >> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> > >> > > PythonTranslator.java
> > >> > >
> > >> > > I think that non-JVM languages could send the Gremlin string from
> > the
> > >> > test
> > >> > > to Gremlin Server as a script first and return the bytecode as
> JSON.
> > >> Then
> > >> > > it could use a "Translator" to then parse it into it's native
> > >> language.
> > >> > > Shouldn't we just build a CSharpTranslator similar to these? does
> > that
> > >> > make
> > >> > > sense?
> > >> > >
> > >> > >
> > >> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> > >> > > jorgebaygondra@gmail.com
> > >> > > > wrote:
> > >> > >
> > >> > > > I've been looking over the current test scenarios on the
> > >> TINKERPOP-1784
> > >> > > > branch, with the C# GLV test suite in mind, and oh man that's an
> > >> > > impressive
> > >> > > > amount of scenarios!
> > >> > > >
> > >> > > > So far, I have a suggestion.
> > >> > > >
> > >> > > > Instead of
> > >> > > >
> > >> > > >   Scenario: g_V_fold_countXlocalX
> > >> > > >     Given the modern graph
> > >> > > >     And the traversal of
> > >> > > >       """
> > >> > > >       g.V().fold().count(Scope.local)
> > >> > > >       """
> > >> > > >
> > >> > > > We could use
> > >> > > >
> > >> > > >   Scenario: g_V_fold_countXlocalX
> > >> > > >     Given the modern graph
> > >> > > >     And parameter 1 being an enum
> > >> > > >       """
> > >> > > >       Scope.local
> > >> > > >       """
> > >> > > >     And the traversal of
> > >> > > >       """
> > >> > > >       g.V().fold().count(param1)
> > >> > > >       """
> > >> > > >
> > >> > > > That way we can exclude enums from possible parameter values,
> > >> narrowing
> > >> > > > down the amount of types of parameters allowed, ie: const
> numeric,
> > >> > const
> > >> > > > strings (wrapped in double quotes), traversals (starting with
> > `__`)
> > >> ,
> > >> > > ...,
> > >> > > > making parsing the traversal a little easier.
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> > >> > spmallette@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Just a quick update on this item - it continues........
> > >> > > > >
> > >> > > > > Went down a bad path earlier in the week and ended up
> shelving a
> > >> lot
> > >> > of
> > >> > > > > work - ended up just coded into a corner. Anyway, I've not
> > really
> > >> > > changed
> > >> > > > > much in the implementation, but I've still not reached a point
> > >> where
> > >> > > the
> > >> > > > > addition of a new .feature files comes without meeting some
> new
> > >> type
> > >> > of
> > >> > > > > assertion that has to be dealt with. This ends up slowing down
> > >> > progress
> > >> > > > on
> > >> > > > > porting over the tests. Please feel free to peruse the latest
> > >> changes
> > >> > > on
> > >> > > > > the branch and let me know if there's any feedback.
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> > >> > > spmallette@gmail.com
> > >> > > > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > I did some pretty heavy refactoring to the python test logic
> > >> > > (altered a
> > >> > > > > > bit by some revision of the gherkin feature file language)
> and
> > >> the
> > >> > > > result
> > >> > > > > > is a much more simplified test logic file:
> > >> > > > > >
> > >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > >> > > > > >
> > >> > > > > > About 120 lines of code (down from about 170). If you
> include
> > >> the
> > >> > > test
> > >> > > > > > logic "setup" file:
> > >> > > > > >
> > >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > > gremlin-python/src/main/jython/radish/terrain.py
> > >> > > > > >
> > >> > > > > > we end up with about 250 lines of test logic total (take out
> > the
> > >> > > > > > comment/license and we're probably well under 200 total -
> not
> > >> too
> > >> > > > bad). I
> > >> > > > > > think I'm closing in on the end to infrastructure building
> > here
> > >> so
> > >> > > the
> > >> > > > > > basic framework is getting close to final at this point I
> > >> believe.
> > >> > > I'll
> > >> > > > > > keep scanning the tests looking for other types of
> assertions
> > >> that
> > >> > > I've
> > >> > > > > not
> > >> > > > > > yet covered, but it's getting pretty solid I think.
> Hopefully,
> > >> > there
> > >> > > > > won't
> > >> > > > > > need to be too many more lines of code needed to express the
> > >> test
> > >> > > logic
> > >> > > > > as
> > >> > > > > > I like how things are looking right now.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > >> > > > > > jorgebaygondra@gmail.com> wrote:
> > >> > > > > >
> > >> > > > > >> Great progress! I like how you avoided using ids, even if
> it
> > >> adds
> > >> > > some
> > >> > > > > >> complexity to the transformation required.
> > >> > > > > >>
> > >> > > > > >> The Python step definitions are still quite simple, I think
> > it
> > >> > would
> > >> > > > be
> > >> > > > > >> mostly the same on the rest of the languages.
> > >> > > > > >>
> > >> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > >> > > > spmallette@gmail.com
> > >> > > > > >
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >> > Just another update on progress with the test suite. The
> > >> > language
> > >> > > of
> > >> > > > > the
> > >> > > > > >> > .feature files is getting slightly more complex as I try
> to
> > >> > > > translate
> > >> > > > > >> more
> > >> > > > > >> > and more of our Java process suite tests into the
> language
> > of
> > >> > the
> > >> > > > > >> gherkin
> > >> > > > > >> > files. You can see here where I've added the ability to
> > >> include
> > >> > > > > >> parameters:
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-test/features/filter/Has.feature#L33
> > >> > > > > >> >
> > >> > > > > >> > I also came up with a method of asserting edges (didn't
> > want
> > >> to
> > >> > > rely
> > >> > > > > on
> > >> > > > > >> ids
> > >> > > > > >> > as it makes the gherkin harder to read, plus i didn't
> want
> > to
> > >> > > assume
> > >> > > > > >> > TinkerGraph identifiers in case these tests were every
> used
> > >> with
> > >> > > > some
> > >> > > > > >> other
> > >> > > > > >> > graph database that didn't use longs):
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > >> > > > > >> >
> > >> > > > > >> > for all those additions (and others) the logic required
> by
> > >> the
> > >> > GLV
> > >> > > > to
> > >> > > > > >> > process these tests has stayed surprisingly simple:
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > >> > > > > >> >
> > >> > > > > >> > There's a fair bit of regex/string manipulation involved
> > >> there,
> > >> > > but
> > >> > > > > it's
> > >> > > > > >> > processing strings from the feature file so that's the
> > >> nature of
> > >> > > it
> > >> > > > I
> > >> > > > > >> > suppose. I think I'm of the mind that I want to port all
> of
> > >> the
> > >> > > > tests
> > >> > > > > to
> > >> > > > > >> > feature files, so I wrote this unit test to help validate
> > >> that
> > >> > > none
> > >> > > > > were
> > >> > > > > >> > missed:
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > >> > > > > >> > structure/FeatureCoverageTest.java
> > >> > > > > >> >
> > >> > > > > >> > I don't know that we have to get to 100% porting right
> > away,
> > >> > but I
> > >> > > > > think
> > >> > > > > >> > that once we have these gherkin files written they not
> only
> > >> > become
> > >> > > > the
> > >> > > > > >> > basis for our current GLV testing work, but we might be
> > able
> > >> to
> > >> > > > simply
> > >> > > > > >> use
> > >> > > > > >> > them for testing the Java stuff as well - that would rid
> us
> > >> of
> > >> > > > having
> > >> > > > > >> the
> > >> > > > > >> > test code duplication. It also sets us up with a portable
> > >> body
> > >> > of
> > >> > > > > tests
> > >> > > > > >> > that can be re-used in TinkerPop 4.x.
> > >> > > > > >> >
> > >> > > > > >> > I'm open to suggestions if anyone has any.
> > >> > > > > >> >
> > >> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > >> > > > > >> > jorgebaygondra@gmail.com
> > >> > > > > >> > > wrote:
> > >> > > > > >> >
> > >> > > > > >> > > I was able to build a proof of concept for a
> > Gherkin-based
> > >> > test
> > >> > > > > >> runner in
> > >> > > > > >> > > C#, that takes the proposed count and select features
> > >> > > > > >> > > <https://github.com/apache/
> > tinkerpop/tree/TINKERPOP-1784/
> > >> > > > > >> > > gremlin-test/features/map>
> > >> > > > > >> > > and runs them using C# step definitions.
> > >> > > > > >> > >
> > >> > > > > >> > > It uses the Gherkin parser <
> > https://github.com/cucumber/g
> > >> > > > > >> herkin-dotnet>
> > >> > > > > >> > > from
> > >> > > > > >> > > cucumber, there isn't a release of the parser with .NET
> > >> Core
> > >> > > > support
> > >> > > > > >> so
> > >> > > > > >> > > I've
> > >> > > > > >> > > asked them to release one
> > >> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
> > >> (there
> > >> > > is
> > >> > > > no
> > >> > > > > >> > > limitation from their source files). If they are not
> able
> > >> to
> > >> > > > release
> > >> > > > > >> it
> > >> > > > > >> > in
> > >> > > > > >> > > the next few days, we can implement our own as it
> should
> > be
> > >> > > pretty
> > >> > > > > >> > straight
> > >> > > > > >> > > forward.
> > >> > > > > >> > >
> > >> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > >> > > > > >> spmallette@gmail.com>
> > >> > > > > >> > > wrote:
> > >> > > > > >> > >
> > >> > > > > >> > > > Thanks for the update. I'm trying to keep the test
> > >> language
> > >> > as
> > >> > > > > >> simple
> > >> > > > > >> > as
> > >> > > > > >> > > > possible so that we don't need an overly complicated
> > test
> > >> > > > > >> > implementation.
> > >> > > > > >> > > > Hopefully that will help make the .NET approach as
> easy
> > >> as
> > >> > > > > possible.
> > >> > > > > >> > > >
> > >> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > >> > > > > >> > > > jorgebaygondra@gmail.com>
> > >> > > > > >> > > > wrote:
> > >> > > > > >> > > >
> > >> > > > > >> > > > > I've been looking into Gherkin support for .NET:
> > >> SpecFlow,
> > >> > > the
> > >> > > > > >> > cucumber
> > >> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > >> > > > > >> > > > cucumber-implementations
> > >> > > > > >> > > > > >,
> > >> > > > > >> > > > > does not support .NET Core platform (we use .NET
> Core
> > >> > build
> > >> > > > > tools
> > >> > > > > >> for
> > >> > > > > >> > > the
> > >> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
> > >> > > > > >> pecFlow/projects/2
> > >> > > > > >> > >,
> > >> > > > > >> > > > > .NET
> > >> > > > > >> > > > > Core support on SpecFlow is coming at a very slow
> > pace
> > >> and
> > >> > > we
> > >> > > > > >> > shouldn't
> > >> > > > > >> > > > > expect to land any time soon (there were some
> design
> > >> > > decisions
> > >> > > > > in
> > >> > > > > >> > > > SpecFlow
> > >> > > > > >> > > > > library that makes supporting other platforms
> > >> non-trivial,
> > >> > > > like
> > >> > > > > >> > > requiring
> > >> > > > > >> > > > > code gen).
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > The alternative would be to implement our own
> harness
> > >> to
> > >> > > > support
> > >> > > > > >> it:
> > >> > > > > >> > > > from a
> > >> > > > > >> > > > > xunit test, look for certain types and parse the
> > >> > > annotations,
> > >> > > > > and
> > >> > > > > >> > > execute
> > >> > > > > >> > > > > them using the Gherkin features.
> > >> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > >> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > >> > > > > >> > > > > I'll continue looking into this option and try to
> > >> > understand
> > >> > > > the
> > >> > > > > >> > effort
> > >> > > > > >> > > > > required...
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > >> > > > > >> > > > > jorgebaygondra@gmail.com>
> > >> > > > > >> > > > > wrote:
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
> > >> growing
> > >> > > > number
> > >> > > > > >> of
> > >> > > > > >> > > GLVs.
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > We should find a way to define the different
> > features
> > >> > > > > supported
> > >> > > > > >> by
> > >> > > > > >> > > each
> > >> > > > > >> > > > > > GLV, as it's reasonable to have different
> maturity
> > >> > levels
> > >> > > > per
> > >> > > > > >> GLV
> > >> > > > > >> > > (ie:
> > >> > > > > >> > > > > > lambdas support, traversal strategy, ...). I
> don't
> > >> know
> > >> > if
> > >> > > > it
> > >> > > > > >> will
> > >> > > > > >> > be
> > >> > > > > >> > > > > > beneficial to do it in the Gherkin files or
> within
> > >> each
> > >> > > GLV
> > >> > > > > >> > > > > implementation.
> > >> > > > > >> > > > > > Also, we should consider the process of rolling
> > out a
> > >> > new
> > >> > > > > >> method /
> > >> > > > > >> > > > class
> > >> > > > > >> > > > > > in the java implementation, how that could affect
> > >> each
> > >> > > GLV.
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen
> Mallette
> > <
> > >> > > > > >> > > > spmallette@gmail.com
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > wrote:
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >> that's what i meant by "reflection" or as you
> > >> suggest
> > >> > > > > eval(). I
> > >> > > > > >> > > guess
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> point is that if the language can support some
> way
> > >> of
> > >> > > > taking
> > >> > > > > >> the
> > >> > > > > >> > > > string
> > >> > > > > >> > > > > >> value and turning it automatically into a
> > traversal
> > >> in
> > >> > > that
> > >> > > > > >> GLVs
> > >> > > > > >> > > style
> > >> > > > > >> > > > > >> then
> > >> > > > > >> > > > > >> we should do that.
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > >> > > > > >> <me...@gremlin.guru>
> > >> > > > > >> > > > > wrote:
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> > For unparameterized queries it can probably be
> > as
> > >> > easy
> > >> > > > as:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > @given("the traversal of")
> > >> > > > > >> > > > > >> > def translate_traversal(step):
> > >> > > > > >> > > > > >> >     g = step.context.g
> > >> > > > > >> > > > > >> >     step.context.traversal = eval(step.text)
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > Cheers,
> > >> > > > > >> > > > > >> > Daniel
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel
> Kuppitz
> > >> > > > > >> <me@gremlin.guru
> > >> > > > > >> > >
> > >> > > > > >> > > > > >> wrote:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber
> /
> > >> > Gherkin
> > >> > > > for
> > >> > > > > >> > years,
> > >> > > > > >> > > > > but I
> > >> > > > > >> > > > > >> > > really like the BDD approach.
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> > >> > translations
> > >> > > > > >> > > specifically
> > >> > > > > >> > > > > >> here:
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > >> > > > > radish/count_features_step.py#
> > >> > > > > >> > > L34-L46
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > This part is the only thing that looks weird
> > to
> > >> me.
> > >> > > > > You're
> > >> > > > > >> > > > basically
> > >> > > > > >> > > > > >> > > writing every query twice; is there really
> no
> > >> > easier
> > >> > > > way
> > >> > > > > >> to do
> > >> > > > > >> > > > that?
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > Cheers,
> > >> > > > > >> > > > > >> > > Daniel
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
> > >> Mallette <
> > >> > > > > >> > > > > >> spmallette@gmail.com>
> > >> > > > > >> > > > > >> > > wrote:
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >> I've brought this issue up in the past and
> > had
> > >> > > > suggested
> > >> > > > > >> some
> > >> > > > > >> > > > > >> options I
> > >> > > > > >> > > > > >> > >> had
> > >> > > > > >> > > > > >> > >> in mind but now I've finally put the basics
> > of
> > >> > those
> > >> > > > > >> ideas in
> > >> > > > > >> > > > place
> > >> > > > > >> > > > > >> so I
> > >> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall
> that
> > >> the
> > >> > > > issue
> > >> > > > > at
> > >> > > > > >> > hand
> > >> > > > > >> > > > is
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > we
> > >> > > > > >> > > > > >> > >> don't have a test suite for GLVs as
> > >> gremlin-test
> > >> > is
> > >> > > > > bound
> > >> > > > > >> to
> > >> > > > > >> > > the
> > >> > > > > >> > > > > >> JVM. We
> > >> > > > > >> > > > > >> > >> have some tricks that let us test
> > >> gremlin-python
> > >> > > with
> > >> > > > it
> > >> > > > > >> but
> > >> > > > > >> > > > those
> > >> > > > > >> > > > > >> > tricks
> > >> > > > > >> > > > > >> > >> won't work for every language and we now
> have
> > >> the
> > >> > > > first
> > >> > > > > >> > > language
> > >> > > > > >> > > > in
> > >> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming
> > gremlin-javascript
> > >> > which
> > >> > > > > won't
> > >> > > > > >> > > > support
> > >> > > > > >> > > > > it
> > >> > > > > >> > > > > >> > >> (yes,
> > >> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on
> the
> > >> jvm
> > >> > > but
> > >> > > > > >> there
> > >> > > > > >> > are
> > >> > > > > >> > > > > >> issues
> > >> > > > > >> > > > > >> > >> with
> > >> > > > > >> > > > > >> > >> getting it all to work with the test
> > framework
> > >> > that
> > >> > > > make
> > >> > > > > >> it
> > >> > > > > >> > > > unduly
> > >> > > > > >> > > > > >> > >> complicated).
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> On other threads I offered the idea that we
> > >> look
> > >> > to
> > >> > > > use
> > >> > > > > >> > Gherkin
> > >> > > > > >> > > > to
> > >> > > > > >> > > > > >> write
> > >> > > > > >> > > > > >> > >> general Gremlin test specifications, which
> > then
> > >> > > could
> > >> > > > be
> > >> > > > > >> read
> > >> > > > > >> > > and
> > >> > > > > >> > > > > >> > >> processed
> > >> > > > > >> > > > > >> > >> by the wide variety of test frameworks that
> > can
> > >> > read
> > >> > > > > that
> > >> > > > > >> > > format
> > >> > > > > >> > > > -
> > >> > > > > >> > > > > >> there
> > >> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
> > >> every
> > >> > > > > >> language -
> > >> > > > > >> > > for
> > >> > > > > >> > > > > >> > example,
> > >> > > > > >> > > > > >> > >> see:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://cucumber.io/
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> I just created this issue:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://issues.apache.org/
> > >> > > jira/browse/TINKERPOP-1784
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> and pushed this branch:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/
> > >> > > > tinkerpop/tree/TINKERPOP-1784
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> which demonstrates how this works with
> > >> > > gremlin-python.
> > >> > > > > The
> > >> > > > > >> > > basic
> > >> > > > > >> > > > > >> anatomy
> > >> > > > > >> > > > > >> > >> of
> > >> > > > > >> > > > > >> > >> this setup involves this new directory in
> > >> > > > gremlin-test:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-test/features
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> It contains the Gherkin .features files.
> > These
> > >> are
> > >> > > the
> > >> > > > > >> test
> > >> > > > > >> > > > > >> > >> specifications.
> > >> > > > > >> > > > > >> > >> They are written using gremlin-java as the
> > >> "model"
> > >> > > > > >> language.
> > >> > > > > >> > > GLVs
> > >> > > > > >> > > > > >> will
> > >> > > > > >> > > > > >> > >> then
> > >> > > > > >> > > > > >> > >> need to write some infrastructure to
> process
> > >> these
> > >> > > > > Gherkin
> > >> > > > > >> > > files.
> > >> > > > > >> > > > > The
> > >> > > > > >> > > > > >> > key
> > >> > > > > >> > > > > >> > >> to making this "easy" to implement will lie
> > in
> > >> our
> > >> > > > > >> abiilty to
> > >> > > > > >> > > > keep
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > >> assertions we want to have relatively
> simple.
> > >> The
> > >> > > more
> > >> > > > > >> > > simplistic
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
> > >> easier
> > >> > > the
> > >> > > > > job
> > >> > > > > >> it
> > >> > > > > >> > > will
> > >> > > > > >> > > > > be
> > >> > > > > >> > > > > >> for
> > >> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of
> > course,
> > >> > once
> > >> > > > that
> > >> > > > > >> > > > > >> infrastructure
> > >> > > > > >> > > > > >> > is
> > >> > > > > >> > > > > >> > >> in place, the GLV developer just has to
> write
> > >> the
> > >> > > GLV
> > >> > > > > >> version
> > >> > > > > >> > > of
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So
> > you
> > >> can
> > >> > > > look
> > >> > > > > at
> > >> > > > > >> > all
> > >> > > > > >> > > > the
> > >> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
> > >> files:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> > >> > > translations
> > >> > > > > >> > > > specifically
> > >> > > > > >> > > > > >> here:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > >> > > > > radish/count_features_step.py#
> > >> > > > > >> > > L34-L46
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> I think this approach works pretty well and
> > >> solves
> > >> > > our
> > >> > > > > >> > general
> > >> > > > > >> > > > > >> problems
> > >> > > > > >> > > > > >> > >> for
> > >> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > >> > > > implementing
> > >> > > > > >> the
> > >> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
> > >> tests
> > >> > > > added
> > >> > > > > to
> > >> > > > > >> > > > .feature
> > >> > > > > >> > > > > >> > files
> > >> > > > > >> > > > > >> > >> just need to translated in the GLV. I
> suppose
> > >> we
> > >> > > could
> > >> > > > > >> > > > "automate" a
> > >> > > > > >> > > > > >> good
> > >> > > > > >> > > > > >> > >> portion of the translation with reflection
> of
> > >> some
> > >> > > > sort.
> > >> > > > > >> > > Anything
> > >> > > > > >> > > > > >> else
> > >> > > > > >> > > > > >> > >> could just be handled manually.
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> Not sure if we need to use this new model
> to
> > >> > wholly
> > >> > > > > >> replace
> > >> > > > > >> > the
> > >> > > > > >> > > > old
> > >> > > > > >> > > > > >> one.
> > >> > > > > >> > > > > >> > >> The process test suite has its place in
> > helping
> > >> > > graph
> > >> > > > > >> > database
> > >> > > > > >> > > > > >> providers
> > >> > > > > >> > > > > >> > >> test their stuff. I also imagine that
> > >> introducing
> > >> > > this
> > >> > > > > >> > approach
> > >> > > > > >> > > > in
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > >> context would create a breaking change
> which
> > we
> > >> > > would
> > >> > > > > then
> > >> > > > > >> > need
> > >> > > > > >> > > > to
> > >> > > > > >> > > > > >> push
> > >> > > > > >> > > > > >> > >> off
> > >> > > > > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to
> > >> think,
> > >> > > but
> > >> > > > > for
> > >> > > > > >> now
> > >> > > > > >> > > it
> > >> > > > > >> > > > > >> might
> > >> > > > > >> > > > > >> > not
> > >> > > > > >> > > > > >> > >> be best to conflate the two and just treat
> > >> them as
> > >> > > > > >> separate
> > >> > > > > >> > > > aspects
> > >> > > > > >> > > > > >> of
> > >> > > > > >> > > > > >> > the
> > >> > > > > >> > > > > >> > >> test suite.
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> Anyway - it's important we settle on an
> > >> approach
> > >> > to
> > >> > > > > >> testing
> > >> > > > > >> > as
> > >> > > > > >> > > we
> > >> > > > > >> > > > > >> really
> > >> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin
> .NET
> > >> GLV
> > >> > > > > without
> > >> > > > > >> > > getting
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > test
> > >> > > > > >> > > > > >> > >> suite solid. please yell if you have any
> > ideas
> > >> or
> > >> > > > > >> feedback on
> > >> > > > > >> > > > this
> > >> > > > > >> > > > > >> > >> approach.
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > >
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

I've been away for a few days on vacation, but before I left, I made some
pretty decent progress on the remaining tests and I'm finding less and less
of my time being focused on the test logic and more and more of it on the
translation of the tests themselves. I'm starting to feel comfortable that
most of the remaining pain is in test translation which means I could
probably issue the PR for what I have sooner than later. I'll probably just
make sure that I have at least a "good enough" representation of tests for
each step and then issue the PR. That will allow TINKERPOP-1827 to proceed
more easily and I can work more at more relaxed pace on migrating all the
tests. Please let me know if there are any concerns with that approach.

On Tue, Nov 7, 2017 at 5:02 AM, Jorge Bay Gondra <jo...@gmail.com>
wrote:

> I've created TINKERPOP-1827 to track the progress of the test suite runner
> for Gremlin .NET.
>
> I have a working gherkin-based runner that executes the steps (parsing
> gremlin groovy / parameters / results) as defined in the TINKERPOP-1784 WIP
> branch.
>
> https://github.com/jorgebay/tinkerpop/commits/gherkin-cs-runner
>
> I'll wait for TINKERPOP-1784 to be resolved in order the follow the last
> changes and submit a pull request for it.
>
> On Tue, Oct 31, 2017 at 11:09 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com
> > wrote:
>
> > Ok.
> > Going back to my initial comment regarding enums: don't mind, I've
> > included the possibility to use enums on the C# parser.
> >
> > On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> >> I guess there wouldn't be a problem with that, though having it the way
> it
> >> works now was a nice check to ensure that the ScriptEngine was
> configured
> >> properly. I guess that shouldn't be the focus on this body of tests
> >> though.
> >> We should be more concerned that the elements of the Gremlin language
> >> actually work.
> >>
> >> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
> >> jorgebaygondra@gmail.com>
> >> wrote:
> >>
> >> > I see that for most languages using bytecode to native language
> >> translator
> >> > would be the easiest solution. In the case of the JavaScript GLV, it
> >> would
> >> > be fairly straightforward to implement (and I plan to do so!).
> >> > In the case of C#, being more strict regarding typing than java, it
> >> would
> >> > be an incomplete solution. For example:
> >> >
> >> > Consider the following traversal in Java: g.V().values("age")
> >> > The type parameter for values() method is inferred at runtime and
> things
> >> > "just works" thanks to type erasure.
> >> >
> >> > In the case of C#, it should be written as  g.V().Values<int>("age") ,
> >> with
> >> > the type parameter to specify which type of traverser are you
> expecting
> >> at
> >> > a compile time.
> >> >
> >> > That's why it wont be possible to make a generic translator from
> >> bytecode
> >> > to C# code, without understanding which generic types are expected.
> >> >
> >> > In the case of C#, I was going for a translator on the C# side that
> >> > tokenizes the gremlin traversal. Once the traversal is tokenized, in
> the
> >> > moment of invoking the methods (with reflection), it can be made aware
> >> of
> >> > the modern graph data (with datatypes!), ie: For method Values<>(), if
> >> the
> >> > property key is "age" -> use int as generic type parameter.
> >> >
> >> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <
> spmallette@gmail.com
> >> >
> >> > wrote:
> >> >
> >> > > I can't help thinking that perhaps non-JVM languages need to
> leverage
> >> > > bytecode to make traversal building easier. We already have an
> >> > established
> >> > > pattern for converting bytecode into Traversals and while it is
> >> different
> >> > > for each language, it's generally governed by the Translator
> >> interface.
> >> > > Note that we already do this for Java, Groovy, and Python and the
> code
> >> > > isn't too crazy - just 200 lines or so:
> >> > >
> >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> >> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> >> > JavaTranslator.java
> >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> >> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> >> > > GroovyTranslator.java
> >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> >> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> >> > > PythonTranslator.java
> >> > >
> >> > > I think that non-JVM languages could send the Gremlin string from
> the
> >> > test
> >> > > to Gremlin Server as a script first and return the bytecode as JSON.
> >> Then
> >> > > it could use a "Translator" to then parse it into it's native
> >> language.
> >> > > Shouldn't we just build a CSharpTranslator similar to these? does
> that
> >> > make
> >> > > sense?
> >> > >
> >> > >
> >> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> >> > > jorgebaygondra@gmail.com
> >> > > > wrote:
> >> > >
> >> > > > I've been looking over the current test scenarios on the
> >> TINKERPOP-1784
> >> > > > branch, with the C# GLV test suite in mind, and oh man that's an
> >> > > impressive
> >> > > > amount of scenarios!
> >> > > >
> >> > > > So far, I have a suggestion.
> >> > > >
> >> > > > Instead of
> >> > > >
> >> > > >   Scenario: g_V_fold_countXlocalX
> >> > > >     Given the modern graph
> >> > > >     And the traversal of
> >> > > >       """
> >> > > >       g.V().fold().count(Scope.local)
> >> > > >       """
> >> > > >
> >> > > > We could use
> >> > > >
> >> > > >   Scenario: g_V_fold_countXlocalX
> >> > > >     Given the modern graph
> >> > > >     And parameter 1 being an enum
> >> > > >       """
> >> > > >       Scope.local
> >> > > >       """
> >> > > >     And the traversal of
> >> > > >       """
> >> > > >       g.V().fold().count(param1)
> >> > > >       """
> >> > > >
> >> > > > That way we can exclude enums from possible parameter values,
> >> narrowing
> >> > > > down the amount of types of parameters allowed, ie: const numeric,
> >> > const
> >> > > > strings (wrapped in double quotes), traversals (starting with
> `__`)
> >> ,
> >> > > ...,
> >> > > > making parsing the traversal a little easier.
> >> > > >
> >> > > >
> >> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> >> > spmallette@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Just a quick update on this item - it continues........
> >> > > > >
> >> > > > > Went down a bad path earlier in the week and ended up shelving a
> >> lot
> >> > of
> >> > > > > work - ended up just coded into a corner. Anyway, I've not
> really
> >> > > changed
> >> > > > > much in the implementation, but I've still not reached a point
> >> where
> >> > > the
> >> > > > > addition of a new .feature files comes without meeting some new
> >> type
> >> > of
> >> > > > > assertion that has to be dealt with. This ends up slowing down
> >> > progress
> >> > > > on
> >> > > > > porting over the tests. Please feel free to peruse the latest
> >> changes
> >> > > on
> >> > > > > the branch and let me know if there's any feedback.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> >> > > spmallette@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > I did some pretty heavy refactoring to the python test logic
> >> > > (altered a
> >> > > > > > bit by some revision of the gherkin feature file language) and
> >> the
> >> > > > result
> >> > > > > > is a much more simplified test logic file:
> >> > > > > >
> >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> >> > > > > >
> >> > > > > > About 120 lines of code (down from about 170). If you include
> >> the
> >> > > test
> >> > > > > > logic "setup" file:
> >> > > > > >
> >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > > gremlin-python/src/main/jython/radish/terrain.py
> >> > > > > >
> >> > > > > > we end up with about 250 lines of test logic total (take out
> the
> >> > > > > > comment/license and we're probably well under 200 total - not
> >> too
> >> > > > bad). I
> >> > > > > > think I'm closing in on the end to infrastructure building
> here
> >> so
> >> > > the
> >> > > > > > basic framework is getting close to final at this point I
> >> believe.
> >> > > I'll
> >> > > > > > keep scanning the tests looking for other types of assertions
> >> that
> >> > > I've
> >> > > > > not
> >> > > > > > yet covered, but it's getting pretty solid I think. Hopefully,
> >> > there
> >> > > > > won't
> >> > > > > > need to be too many more lines of code needed to express the
> >> test
> >> > > logic
> >> > > > > as
> >> > > > > > I like how things are looking right now.
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> >> > > > > > jorgebaygondra@gmail.com> wrote:
> >> > > > > >
> >> > > > > >> Great progress! I like how you avoided using ids, even if it
> >> adds
> >> > > some
> >> > > > > >> complexity to the transformation required.
> >> > > > > >>
> >> > > > > >> The Python step definitions are still quite simple, I think
> it
> >> > would
> >> > > > be
> >> > > > > >> mostly the same on the rest of the languages.
> >> > > > > >>
> >> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> >> > > > spmallette@gmail.com
> >> > > > > >
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >> > Just another update on progress with the test suite. The
> >> > language
> >> > > of
> >> > > > > the
> >> > > > > >> > .feature files is getting slightly more complex as I try to
> >> > > > translate
> >> > > > > >> more
> >> > > > > >> > and more of our Java process suite tests into the language
> of
> >> > the
> >> > > > > >> gherkin
> >> > > > > >> > files. You can see here where I've added the ability to
> >> include
> >> > > > > >> parameters:
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-test/features/filter/Has.feature#L33
> >> > > > > >> >
> >> > > > > >> > I also came up with a method of asserting edges (didn't
> want
> >> to
> >> > > rely
> >> > > > > on
> >> > > > > >> ids
> >> > > > > >> > as it makes the gherkin harder to read, plus i didn't want
> to
> >> > > assume
> >> > > > > >> > TinkerGraph identifiers in case these tests were every used
> >> with
> >> > > > some
> >> > > > > >> other
> >> > > > > >> > graph database that didn't use longs):
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> >> > > > > >> >
> >> > > > > >> > for all those additions (and others) the logic required by
> >> the
> >> > GLV
> >> > > > to
> >> > > > > >> > process these tests has stayed surprisingly simple:
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> >> > > > > >> >
> >> > > > > >> > There's a fair bit of regex/string manipulation involved
> >> there,
> >> > > but
> >> > > > > it's
> >> > > > > >> > processing strings from the feature file so that's the
> >> nature of
> >> > > it
> >> > > > I
> >> > > > > >> > suppose. I think I'm of the mind that I want to port all of
> >> the
> >> > > > tests
> >> > > > > to
> >> > > > > >> > feature files, so I wrote this unit test to help validate
> >> that
> >> > > none
> >> > > > > were
> >> > > > > >> > missed:
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> >> > > > > >> > structure/FeatureCoverageTest.java
> >> > > > > >> >
> >> > > > > >> > I don't know that we have to get to 100% porting right
> away,
> >> > but I
> >> > > > > think
> >> > > > > >> > that once we have these gherkin files written they not only
> >> > become
> >> > > > the
> >> > > > > >> > basis for our current GLV testing work, but we might be
> able
> >> to
> >> > > > simply
> >> > > > > >> use
> >> > > > > >> > them for testing the Java stuff as well - that would rid us
> >> of
> >> > > > having
> >> > > > > >> the
> >> > > > > >> > test code duplication. It also sets us up with a portable
> >> body
> >> > of
> >> > > > > tests
> >> > > > > >> > that can be re-used in TinkerPop 4.x.
> >> > > > > >> >
> >> > > > > >> > I'm open to suggestions if anyone has any.
> >> > > > > >> >
> >> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> >> > > > > >> > jorgebaygondra@gmail.com
> >> > > > > >> > > wrote:
> >> > > > > >> >
> >> > > > > >> > > I was able to build a proof of concept for a
> Gherkin-based
> >> > test
> >> > > > > >> runner in
> >> > > > > >> > > C#, that takes the proposed count and select features
> >> > > > > >> > > <https://github.com/apache/
> tinkerpop/tree/TINKERPOP-1784/
> >> > > > > >> > > gremlin-test/features/map>
> >> > > > > >> > > and runs them using C# step definitions.
> >> > > > > >> > >
> >> > > > > >> > > It uses the Gherkin parser <
> https://github.com/cucumber/g
> >> > > > > >> herkin-dotnet>
> >> > > > > >> > > from
> >> > > > > >> > > cucumber, there isn't a release of the parser with .NET
> >> Core
> >> > > > support
> >> > > > > >> so
> >> > > > > >> > > I've
> >> > > > > >> > > asked them to release one
> >> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
> >> (there
> >> > > is
> >> > > > no
> >> > > > > >> > > limitation from their source files). If they are not able
> >> to
> >> > > > release
> >> > > > > >> it
> >> > > > > >> > in
> >> > > > > >> > > the next few days, we can implement our own as it should
> be
> >> > > pretty
> >> > > > > >> > straight
> >> > > > > >> > > forward.
> >> > > > > >> > >
> >> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> >> > > > > >> spmallette@gmail.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > >
> >> > > > > >> > > > Thanks for the update. I'm trying to keep the test
> >> language
> >> > as
> >> > > > > >> simple
> >> > > > > >> > as
> >> > > > > >> > > > possible so that we don't need an overly complicated
> test
> >> > > > > >> > implementation.
> >> > > > > >> > > > Hopefully that will help make the .NET approach as easy
> >> as
> >> > > > > possible.
> >> > > > > >> > > >
> >> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> >> > > > > >> > > > jorgebaygondra@gmail.com>
> >> > > > > >> > > > wrote:
> >> > > > > >> > > >
> >> > > > > >> > > > > I've been looking into Gherkin support for .NET:
> >> SpecFlow,
> >> > > the
> >> > > > > >> > cucumber
> >> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> >> > > > > >> > > > cucumber-implementations
> >> > > > > >> > > > > >,
> >> > > > > >> > > > > does not support .NET Core platform (we use .NET Core
> >> > build
> >> > > > > tools
> >> > > > > >> for
> >> > > > > >> > > the
> >> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
> >> > > > > >> > > > >
> >> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
> >> > > > > >> pecFlow/projects/2
> >> > > > > >> > >,
> >> > > > > >> > > > > .NET
> >> > > > > >> > > > > Core support on SpecFlow is coming at a very slow
> pace
> >> and
> >> > > we
> >> > > > > >> > shouldn't
> >> > > > > >> > > > > expect to land any time soon (there were some design
> >> > > decisions
> >> > > > > in
> >> > > > > >> > > > SpecFlow
> >> > > > > >> > > > > library that makes supporting other platforms
> >> non-trivial,
> >> > > > like
> >> > > > > >> > > requiring
> >> > > > > >> > > > > code gen).
> >> > > > > >> > > > >
> >> > > > > >> > > > > The alternative would be to implement our own harness
> >> to
> >> > > > support
> >> > > > > >> it:
> >> > > > > >> > > > from a
> >> > > > > >> > > > > xunit test, look for certain types and parse the
> >> > > annotations,
> >> > > > > and
> >> > > > > >> > > execute
> >> > > > > >> > > > > them using the Gherkin features.
> >> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> >> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> >> > > > > >> > > > > I'll continue looking into this option and try to
> >> > understand
> >> > > > the
> >> > > > > >> > effort
> >> > > > > >> > > > > required...
> >> > > > > >> > > > >
> >> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> >> > > > > >> > > > > jorgebaygondra@gmail.com>
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > >
> >> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
> >> growing
> >> > > > number
> >> > > > > >> of
> >> > > > > >> > > GLVs.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > We should find a way to define the different
> features
> >> > > > > supported
> >> > > > > >> by
> >> > > > > >> > > each
> >> > > > > >> > > > > > GLV, as it's reasonable to have different maturity
> >> > levels
> >> > > > per
> >> > > > > >> GLV
> >> > > > > >> > > (ie:
> >> > > > > >> > > > > > lambdas support, traversal strategy, ...). I don't
> >> know
> >> > if
> >> > > > it
> >> > > > > >> will
> >> > > > > >> > be
> >> > > > > >> > > > > > beneficial to do it in the Gherkin files or within
> >> each
> >> > > GLV
> >> > > > > >> > > > > implementation.
> >> > > > > >> > > > > > Also, we should consider the process of rolling
> out a
> >> > new
> >> > > > > >> method /
> >> > > > > >> > > > class
> >> > > > > >> > > > > > in the java implementation, how that could affect
> >> each
> >> > > GLV.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette
> <
> >> > > > > >> > > > spmallette@gmail.com
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > wrote:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >> that's what i meant by "reflection" or as you
> >> suggest
> >> > > > > eval(). I
> >> > > > > >> > > guess
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> point is that if the language can support some way
> >> of
> >> > > > taking
> >> > > > > >> the
> >> > > > > >> > > > string
> >> > > > > >> > > > > >> value and turning it automatically into a
> traversal
> >> in
> >> > > that
> >> > > > > >> GLVs
> >> > > > > >> > > style
> >> > > > > >> > > > > >> then
> >> > > > > >> > > > > >> we should do that.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> >> > > > > >> <me...@gremlin.guru>
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > For unparameterized queries it can probably be
> as
> >> > easy
> >> > > > as:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > @given("the traversal of")
> >> > > > > >> > > > > >> > def translate_traversal(step):
> >> > > > > >> > > > > >> >     g = step.context.g
> >> > > > > >> > > > > >> >     step.context.traversal = eval(step.text)
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > Cheers,
> >> > > > > >> > > > > >> > Daniel
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> >> > > > > >> <me@gremlin.guru
> >> > > > > >> > >
> >> > > > > >> > > > > >> wrote:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
> >> > Gherkin
> >> > > > for
> >> > > > > >> > years,
> >> > > > > >> > > > > but I
> >> > > > > >> > > > > >> > > really like the BDD approach.
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> >> > translations
> >> > > > > >> > > specifically
> >> > > > > >> > > > > >> here:
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> >> > > > > radish/count_features_step.py#
> >> > > > > >> > > L34-L46
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > This part is the only thing that looks weird
> to
> >> me.
> >> > > > > You're
> >> > > > > >> > > > basically
> >> > > > > >> > > > > >> > > writing every query twice; is there really no
> >> > easier
> >> > > > way
> >> > > > > >> to do
> >> > > > > >> > > > that?
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > Cheers,
> >> > > > > >> > > > > >> > > Daniel
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
> >> Mallette <
> >> > > > > >> > > > > >> spmallette@gmail.com>
> >> > > > > >> > > > > >> > > wrote:
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >> I've brought this issue up in the past and
> had
> >> > > > suggested
> >> > > > > >> some
> >> > > > > >> > > > > >> options I
> >> > > > > >> > > > > >> > >> had
> >> > > > > >> > > > > >> > >> in mind but now I've finally put the basics
> of
> >> > those
> >> > > > > >> ideas in
> >> > > > > >> > > > place
> >> > > > > >> > > > > >> so I
> >> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that
> >> the
> >> > > > issue
> >> > > > > at
> >> > > > > >> > hand
> >> > > > > >> > > > is
> >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > we
> >> > > > > >> > > > > >> > >> don't have a test suite for GLVs as
> >> gremlin-test
> >> > is
> >> > > > > bound
> >> > > > > >> to
> >> > > > > >> > > the
> >> > > > > >> > > > > >> JVM. We
> >> > > > > >> > > > > >> > >> have some tricks that let us test
> >> gremlin-python
> >> > > with
> >> > > > it
> >> > > > > >> but
> >> > > > > >> > > > those
> >> > > > > >> > > > > >> > tricks
> >> > > > > >> > > > > >> > >> won't work for every language and we now have
> >> the
> >> > > > first
> >> > > > > >> > > language
> >> > > > > >> > > > in
> >> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming
> gremlin-javascript
> >> > which
> >> > > > > won't
> >> > > > > >> > > > support
> >> > > > > >> > > > > it
> >> > > > > >> > > > > >> > >> (yes,
> >> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on the
> >> jvm
> >> > > but
> >> > > > > >> there
> >> > > > > >> > are
> >> > > > > >> > > > > >> issues
> >> > > > > >> > > > > >> > >> with
> >> > > > > >> > > > > >> > >> getting it all to work with the test
> framework
> >> > that
> >> > > > make
> >> > > > > >> it
> >> > > > > >> > > > unduly
> >> > > > > >> > > > > >> > >> complicated).
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> On other threads I offered the idea that we
> >> look
> >> > to
> >> > > > use
> >> > > > > >> > Gherkin
> >> > > > > >> > > > to
> >> > > > > >> > > > > >> write
> >> > > > > >> > > > > >> > >> general Gremlin test specifications, which
> then
> >> > > could
> >> > > > be
> >> > > > > >> read
> >> > > > > >> > > and
> >> > > > > >> > > > > >> > >> processed
> >> > > > > >> > > > > >> > >> by the wide variety of test frameworks that
> can
> >> > read
> >> > > > > that
> >> > > > > >> > > format
> >> > > > > >> > > > -
> >> > > > > >> > > > > >> there
> >> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
> >> every
> >> > > > > >> language -
> >> > > > > >> > > for
> >> > > > > >> > > > > >> > example,
> >> > > > > >> > > > > >> > >> see:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://cucumber.io/
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> I just created this issue:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://issues.apache.org/
> >> > > jira/browse/TINKERPOP-1784
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> and pushed this branch:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/
> >> > > > tinkerpop/tree/TINKERPOP-1784
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> which demonstrates how this works with
> >> > > gremlin-python.
> >> > > > > The
> >> > > > > >> > > basic
> >> > > > > >> > > > > >> anatomy
> >> > > > > >> > > > > >> > >> of
> >> > > > > >> > > > > >> > >> this setup involves this new directory in
> >> > > > gremlin-test:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-test/features
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> It contains the Gherkin .features files.
> These
> >> are
> >> > > the
> >> > > > > >> test
> >> > > > > >> > > > > >> > >> specifications.
> >> > > > > >> > > > > >> > >> They are written using gremlin-java as the
> >> "model"
> >> > > > > >> language.
> >> > > > > >> > > GLVs
> >> > > > > >> > > > > >> will
> >> > > > > >> > > > > >> > >> then
> >> > > > > >> > > > > >> > >> need to write some infrastructure to process
> >> these
> >> > > > > Gherkin
> >> > > > > >> > > files.
> >> > > > > >> > > > > The
> >> > > > > >> > > > > >> > key
> >> > > > > >> > > > > >> > >> to making this "easy" to implement will lie
> in
> >> our
> >> > > > > >> abiilty to
> >> > > > > >> > > > keep
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > >> assertions we want to have relatively simple.
> >> The
> >> > > more
> >> > > > > >> > > simplistic
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
> >> easier
> >> > > the
> >> > > > > job
> >> > > > > >> it
> >> > > > > >> > > will
> >> > > > > >> > > > > be
> >> > > > > >> > > > > >> for
> >> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of
> course,
> >> > once
> >> > > > that
> >> > > > > >> > > > > >> infrastructure
> >> > > > > >> > > > > >> > is
> >> > > > > >> > > > > >> > >> in place, the GLV developer just has to write
> >> the
> >> > > GLV
> >> > > > > >> version
> >> > > > > >> > > of
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So
> you
> >> can
> >> > > > look
> >> > > > > at
> >> > > > > >> > all
> >> > > > > >> > > > the
> >> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
> >> files:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> >> > > translations
> >> > > > > >> > > > specifically
> >> > > > > >> > > > > >> here:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> >> > > > > radish/count_features_step.py#
> >> > > > > >> > > L34-L46
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> I think this approach works pretty well and
> >> solves
> >> > > our
> >> > > > > >> > general
> >> > > > > >> > > > > >> problems
> >> > > > > >> > > > > >> > >> for
> >> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> >> > > > implementing
> >> > > > > >> the
> >> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
> >> tests
> >> > > > added
> >> > > > > to
> >> > > > > >> > > > .feature
> >> > > > > >> > > > > >> > files
> >> > > > > >> > > > > >> > >> just need to translated in the GLV. I suppose
> >> we
> >> > > could
> >> > > > > >> > > > "automate" a
> >> > > > > >> > > > > >> good
> >> > > > > >> > > > > >> > >> portion of the translation with reflection of
> >> some
> >> > > > sort.
> >> > > > > >> > > Anything
> >> > > > > >> > > > > >> else
> >> > > > > >> > > > > >> > >> could just be handled manually.
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> Not sure if we need to use this new model to
> >> > wholly
> >> > > > > >> replace
> >> > > > > >> > the
> >> > > > > >> > > > old
> >> > > > > >> > > > > >> one.
> >> > > > > >> > > > > >> > >> The process test suite has its place in
> helping
> >> > > graph
> >> > > > > >> > database
> >> > > > > >> > > > > >> providers
> >> > > > > >> > > > > >> > >> test their stuff. I also imagine that
> >> introducing
> >> > > this
> >> > > > > >> > approach
> >> > > > > >> > > > in
> >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > >> context would create a breaking change which
> we
> >> > > would
> >> > > > > then
> >> > > > > >> > need
> >> > > > > >> > > > to
> >> > > > > >> > > > > >> push
> >> > > > > >> > > > > >> > >> off
> >> > > > > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to
> >> think,
> >> > > but
> >> > > > > for
> >> > > > > >> now
> >> > > > > >> > > it
> >> > > > > >> > > > > >> might
> >> > > > > >> > > > > >> > not
> >> > > > > >> > > > > >> > >> be best to conflate the two and just treat
> >> them as
> >> > > > > >> separate
> >> > > > > >> > > > aspects
> >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > >> test suite.
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> Anyway - it's important we settle on an
> >> approach
> >> > to
> >> > > > > >> testing
> >> > > > > >> > as
> >> > > > > >> > > we
> >> > > > > >> > > > > >> really
> >> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET
> >> GLV
> >> > > > > without
> >> > > > > >> > > getting
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > test
> >> > > > > >> > > > > >> > >> suite solid. please yell if you have any
> ideas
> >> or
> >> > > > > >> feedback on
> >> > > > > >> > > > this
> >> > > > > >> > > > > >> > >> approach.
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

I've created TINKERPOP-1827 to track the progress of the test suite runner
for Gremlin .NET.

I have a working gherkin-based runner that executes the steps (parsing
gremlin groovy / parameters / results) as defined in the TINKERPOP-1784 WIP
branch.

https://github.com/jorgebay/tinkerpop/commits/gherkin-cs-runner

I'll wait for TINKERPOP-1784 to be resolved in order the follow the last
changes and submit a pull request for it.

On Tue, Oct 31, 2017 at 11:09 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> wrote:

> Ok.
> Going back to my initial comment regarding enums: don't mind, I've
> included the possibility to use enums on the C# parser.
>
> On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> I guess there wouldn't be a problem with that, though having it the way it
>> works now was a nice check to ensure that the ScriptEngine was configured
>> properly. I guess that shouldn't be the focus on this body of tests
>> though.
>> We should be more concerned that the elements of the Gremlin language
>> actually work.
>>
>> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
>> jorgebaygondra@gmail.com>
>> wrote:
>>
>> > I see that for most languages using bytecode to native language
>> translator
>> > would be the easiest solution. In the case of the JavaScript GLV, it
>> would
>> > be fairly straightforward to implement (and I plan to do so!).
>> > In the case of C#, being more strict regarding typing than java, it
>> would
>> > be an incomplete solution. For example:
>> >
>> > Consider the following traversal in Java: g.V().values("age")
>> > The type parameter for values() method is inferred at runtime and things
>> > "just works" thanks to type erasure.
>> >
>> > In the case of C#, it should be written as  g.V().Values<int>("age") ,
>> with
>> > the type parameter to specify which type of traverser are you expecting
>> at
>> > a compile time.
>> >
>> > That's why it wont be possible to make a generic translator from
>> bytecode
>> > to C# code, without understanding which generic types are expected.
>> >
>> > In the case of C#, I was going for a translator on the C# side that
>> > tokenizes the gremlin traversal. Once the traversal is tokenized, in the
>> > moment of invoking the methods (with reflection), it can be made aware
>> of
>> > the modern graph data (with datatypes!), ie: For method Values<>(), if
>> the
>> > property key is "age" -> use int as generic type parameter.
>> >
>> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <spmallette@gmail.com
>> >
>> > wrote:
>> >
>> > > I can't help thinking that perhaps non-JVM languages need to leverage
>> > > bytecode to make traversal building easier. We already have an
>> > established
>> > > pattern for converting bytecode into Traversals and while it is
>> different
>> > > for each language, it's generally governed by the Translator
>> interface.
>> > > Note that we already do this for Java, Groovy, and Python and the code
>> > > isn't too crazy - just 200 lines or so:
>> > >
>> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
>> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
>> > JavaTranslator.java
>> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
>> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
>> > > GroovyTranslator.java
>> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
>> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
>> > > PythonTranslator.java
>> > >
>> > > I think that non-JVM languages could send the Gremlin string from the
>> > test
>> > > to Gremlin Server as a script first and return the bytecode as JSON.
>> Then
>> > > it could use a "Translator" to then parse it into it's native
>> language.
>> > > Shouldn't we just build a CSharpTranslator similar to these? does that
>> > make
>> > > sense?
>> > >
>> > >
>> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
>> > > jorgebaygondra@gmail.com
>> > > > wrote:
>> > >
>> > > > I've been looking over the current test scenarios on the
>> TINKERPOP-1784
>> > > > branch, with the C# GLV test suite in mind, and oh man that's an
>> > > impressive
>> > > > amount of scenarios!
>> > > >
>> > > > So far, I have a suggestion.
>> > > >
>> > > > Instead of
>> > > >
>> > > >   Scenario: g_V_fold_countXlocalX
>> > > >     Given the modern graph
>> > > >     And the traversal of
>> > > >       """
>> > > >       g.V().fold().count(Scope.local)
>> > > >       """
>> > > >
>> > > > We could use
>> > > >
>> > > >   Scenario: g_V_fold_countXlocalX
>> > > >     Given the modern graph
>> > > >     And parameter 1 being an enum
>> > > >       """
>> > > >       Scope.local
>> > > >       """
>> > > >     And the traversal of
>> > > >       """
>> > > >       g.V().fold().count(param1)
>> > > >       """
>> > > >
>> > > > That way we can exclude enums from possible parameter values,
>> narrowing
>> > > > down the amount of types of parameters allowed, ie: const numeric,
>> > const
>> > > > strings (wrapped in double quotes), traversals (starting with `__`)
>> ,
>> > > ...,
>> > > > making parsing the traversal a little easier.
>> > > >
>> > > >
>> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
>> > spmallette@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Just a quick update on this item - it continues........
>> > > > >
>> > > > > Went down a bad path earlier in the week and ended up shelving a
>> lot
>> > of
>> > > > > work - ended up just coded into a corner. Anyway, I've not really
>> > > changed
>> > > > > much in the implementation, but I've still not reached a point
>> where
>> > > the
>> > > > > addition of a new .feature files comes without meeting some new
>> type
>> > of
>> > > > > assertion that has to be dealt with. This ends up slowing down
>> > progress
>> > > > on
>> > > > > porting over the tests. Please feel free to peruse the latest
>> changes
>> > > on
>> > > > > the branch and let me know if there's any feedback.
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
>> > > spmallette@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I did some pretty heavy refactoring to the python test logic
>> > > (altered a
>> > > > > > bit by some revision of the gherkin feature file language) and
>> the
>> > > > result
>> > > > > > is a much more simplified test logic file:
>> > > > > >
>> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
>> > > > > >
>> > > > > > About 120 lines of code (down from about 170). If you include
>> the
>> > > test
>> > > > > > logic "setup" file:
>> > > > > >
>> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > > gremlin-python/src/main/jython/radish/terrain.py
>> > > > > >
>> > > > > > we end up with about 250 lines of test logic total (take out the
>> > > > > > comment/license and we're probably well under 200 total - not
>> too
>> > > > bad). I
>> > > > > > think I'm closing in on the end to infrastructure building here
>> so
>> > > the
>> > > > > > basic framework is getting close to final at this point I
>> believe.
>> > > I'll
>> > > > > > keep scanning the tests looking for other types of assertions
>> that
>> > > I've
>> > > > > not
>> > > > > > yet covered, but it's getting pretty solid I think. Hopefully,
>> > there
>> > > > > won't
>> > > > > > need to be too many more lines of code needed to express the
>> test
>> > > logic
>> > > > > as
>> > > > > > I like how things are looking right now.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
>> > > > > > jorgebaygondra@gmail.com> wrote:
>> > > > > >
>> > > > > >> Great progress! I like how you avoided using ids, even if it
>> adds
>> > > some
>> > > > > >> complexity to the transformation required.
>> > > > > >>
>> > > > > >> The Python step definitions are still quite simple, I think it
>> > would
>> > > > be
>> > > > > >> mostly the same on the rest of the languages.
>> > > > > >>
>> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
>> > > > spmallette@gmail.com
>> > > > > >
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >> > Just another update on progress with the test suite. The
>> > language
>> > > of
>> > > > > the
>> > > > > >> > .feature files is getting slightly more complex as I try to
>> > > > translate
>> > > > > >> more
>> > > > > >> > and more of our Java process suite tests into the language of
>> > the
>> > > > > >> gherkin
>> > > > > >> > files. You can see here where I've added the ability to
>> include
>> > > > > >> parameters:
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-test/features/filter/Has.feature#L33
>> > > > > >> >
>> > > > > >> > I also came up with a method of asserting edges (didn't want
>> to
>> > > rely
>> > > > > on
>> > > > > >> ids
>> > > > > >> > as it makes the gherkin harder to read, plus i didn't want to
>> > > assume
>> > > > > >> > TinkerGraph identifiers in case these tests were every used
>> with
>> > > > some
>> > > > > >> other
>> > > > > >> > graph database that didn't use longs):
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
>> > > > > >> >
>> > > > > >> > for all those additions (and others) the logic required by
>> the
>> > GLV
>> > > > to
>> > > > > >> > process these tests has stayed surprisingly simple:
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
>> > > > > >> >
>> > > > > >> > There's a fair bit of regex/string manipulation involved
>> there,
>> > > but
>> > > > > it's
>> > > > > >> > processing strings from the feature file so that's the
>> nature of
>> > > it
>> > > > I
>> > > > > >> > suppose. I think I'm of the mind that I want to port all of
>> the
>> > > > tests
>> > > > > to
>> > > > > >> > feature files, so I wrote this unit test to help validate
>> that
>> > > none
>> > > > > were
>> > > > > >> > missed:
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
>> > > > > >> > structure/FeatureCoverageTest.java
>> > > > > >> >
>> > > > > >> > I don't know that we have to get to 100% porting right away,
>> > but I
>> > > > > think
>> > > > > >> > that once we have these gherkin files written they not only
>> > become
>> > > > the
>> > > > > >> > basis for our current GLV testing work, but we might be able
>> to
>> > > > simply
>> > > > > >> use
>> > > > > >> > them for testing the Java stuff as well - that would rid us
>> of
>> > > > having
>> > > > > >> the
>> > > > > >> > test code duplication. It also sets us up with a portable
>> body
>> > of
>> > > > > tests
>> > > > > >> > that can be re-used in TinkerPop 4.x.
>> > > > > >> >
>> > > > > >> > I'm open to suggestions if anyone has any.
>> > > > > >> >
>> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
>> > > > > >> > jorgebaygondra@gmail.com
>> > > > > >> > > wrote:
>> > > > > >> >
>> > > > > >> > > I was able to build a proof of concept for a Gherkin-based
>> > test
>> > > > > >> runner in
>> > > > > >> > > C#, that takes the proposed count and select features
>> > > > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
>> > > > > >> > > gremlin-test/features/map>
>> > > > > >> > > and runs them using C# step definitions.
>> > > > > >> > >
>> > > > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
>> > > > > >> herkin-dotnet>
>> > > > > >> > > from
>> > > > > >> > > cucumber, there isn't a release of the parser with .NET
>> Core
>> > > > support
>> > > > > >> so
>> > > > > >> > > I've
>> > > > > >> > > asked them to release one
>> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
>> (there
>> > > is
>> > > > no
>> > > > > >> > > limitation from their source files). If they are not able
>> to
>> > > > release
>> > > > > >> it
>> > > > > >> > in
>> > > > > >> > > the next few days, we can implement our own as it should be
>> > > pretty
>> > > > > >> > straight
>> > > > > >> > > forward.
>> > > > > >> > >
>> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
>> > > > > >> spmallette@gmail.com>
>> > > > > >> > > wrote:
>> > > > > >> > >
>> > > > > >> > > > Thanks for the update. I'm trying to keep the test
>> language
>> > as
>> > > > > >> simple
>> > > > > >> > as
>> > > > > >> > > > possible so that we don't need an overly complicated test
>> > > > > >> > implementation.
>> > > > > >> > > > Hopefully that will help make the .NET approach as easy
>> as
>> > > > > possible.
>> > > > > >> > > >
>> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
>> > > > > >> > > > jorgebaygondra@gmail.com>
>> > > > > >> > > > wrote:
>> > > > > >> > > >
>> > > > > >> > > > > I've been looking into Gherkin support for .NET:
>> SpecFlow,
>> > > the
>> > > > > >> > cucumber
>> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
>> > > > > >> > > > cucumber-implementations
>> > > > > >> > > > > >,
>> > > > > >> > > > > does not support .NET Core platform (we use .NET Core
>> > build
>> > > > > tools
>> > > > > >> for
>> > > > > >> > > the
>> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
>> > > > > >> > > > >
>> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
>> > > > > >> pecFlow/projects/2
>> > > > > >> > >,
>> > > > > >> > > > > .NET
>> > > > > >> > > > > Core support on SpecFlow is coming at a very slow pace
>> and
>> > > we
>> > > > > >> > shouldn't
>> > > > > >> > > > > expect to land any time soon (there were some design
>> > > decisions
>> > > > > in
>> > > > > >> > > > SpecFlow
>> > > > > >> > > > > library that makes supporting other platforms
>> non-trivial,
>> > > > like
>> > > > > >> > > requiring
>> > > > > >> > > > > code gen).
>> > > > > >> > > > >
>> > > > > >> > > > > The alternative would be to implement our own harness
>> to
>> > > > support
>> > > > > >> it:
>> > > > > >> > > > from a
>> > > > > >> > > > > xunit test, look for certain types and parse the
>> > > annotations,
>> > > > > and
>> > > > > >> > > execute
>> > > > > >> > > > > them using the Gherkin features.
>> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
>> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
>> > > > > >> > > > > I'll continue looking into this option and try to
>> > understand
>> > > > the
>> > > > > >> > effort
>> > > > > >> > > > > required...
>> > > > > >> > > > >
>> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
>> > > > > >> > > > > jorgebaygondra@gmail.com>
>> > > > > >> > > > > wrote:
>> > > > > >> > > > >
>> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
>> growing
>> > > > number
>> > > > > >> of
>> > > > > >> > > GLVs.
>> > > > > >> > > > > >
>> > > > > >> > > > > > We should find a way to define the different features
>> > > > > supported
>> > > > > >> by
>> > > > > >> > > each
>> > > > > >> > > > > > GLV, as it's reasonable to have different maturity
>> > levels
>> > > > per
>> > > > > >> GLV
>> > > > > >> > > (ie:
>> > > > > >> > > > > > lambdas support, traversal strategy, ...). I don't
>> know
>> > if
>> > > > it
>> > > > > >> will
>> > > > > >> > be
>> > > > > >> > > > > > beneficial to do it in the Gherkin files or within
>> each
>> > > GLV
>> > > > > >> > > > > implementation.
>> > > > > >> > > > > > Also, we should consider the process of rolling out a
>> > new
>> > > > > >> method /
>> > > > > >> > > > class
>> > > > > >> > > > > > in the java implementation, how that could affect
>> each
>> > > GLV.
>> > > > > >> > > > > >
>> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
>> > > > > >> > > > spmallette@gmail.com
>> > > > > >> > > > > >
>> > > > > >> > > > > > wrote:
>> > > > > >> > > > > >
>> > > > > >> > > > > >> that's what i meant by "reflection" or as you
>> suggest
>> > > > > eval(). I
>> > > > > >> > > guess
>> > > > > >> > > > > the
>> > > > > >> > > > > >> point is that if the language can support some way
>> of
>> > > > taking
>> > > > > >> the
>> > > > > >> > > > string
>> > > > > >> > > > > >> value and turning it automatically into a traversal
>> in
>> > > that
>> > > > > >> GLVs
>> > > > > >> > > style
>> > > > > >> > > > > >> then
>> > > > > >> > > > > >> we should do that.
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
>> > > > > >> <me...@gremlin.guru>
>> > > > > >> > > > > wrote:
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> > For unparameterized queries it can probably be as
>> > easy
>> > > > as:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > @given("the traversal of")
>> > > > > >> > > > > >> > def translate_traversal(step):
>> > > > > >> > > > > >> >     g = step.context.g
>> > > > > >> > > > > >> >     step.context.traversal = eval(step.text)
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > Cheers,
>> > > > > >> > > > > >> > Daniel
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
>> > > > > >> <me@gremlin.guru
>> > > > > >> > >
>> > > > > >> > > > > >> wrote:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
>> > Gherkin
>> > > > for
>> > > > > >> > years,
>> > > > > >> > > > > but I
>> > > > > >> > > > > >> > > really like the BDD approach.
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
>> > translations
>> > > > > >> > > specifically
>> > > > > >> > > > > >> here:
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-python/src/main/jython/
>> > > > > radish/count_features_step.py#
>> > > > > >> > > L34-L46
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > This part is the only thing that looks weird to
>> me.
>> > > > > You're
>> > > > > >> > > > basically
>> > > > > >> > > > > >> > > writing every query twice; is there really no
>> > easier
>> > > > way
>> > > > > >> to do
>> > > > > >> > > > that?
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > Cheers,
>> > > > > >> > > > > >> > > Daniel
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
>> Mallette <
>> > > > > >> > > > > >> spmallette@gmail.com>
>> > > > > >> > > > > >> > > wrote:
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >> I've brought this issue up in the past and had
>> > > > suggested
>> > > > > >> some
>> > > > > >> > > > > >> options I
>> > > > > >> > > > > >> > >> had
>> > > > > >> > > > > >> > >> in mind but now I've finally put the basics of
>> > those
>> > > > > >> ideas in
>> > > > > >> > > > place
>> > > > > >> > > > > >> so I
>> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that
>> the
>> > > > issue
>> > > > > at
>> > > > > >> > hand
>> > > > > >> > > > is
>> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > we
>> > > > > >> > > > > >> > >> don't have a test suite for GLVs as
>> gremlin-test
>> > is
>> > > > > bound
>> > > > > >> to
>> > > > > >> > > the
>> > > > > >> > > > > >> JVM. We
>> > > > > >> > > > > >> > >> have some tricks that let us test
>> gremlin-python
>> > > with
>> > > > it
>> > > > > >> but
>> > > > > >> > > > those
>> > > > > >> > > > > >> > tricks
>> > > > > >> > > > > >> > >> won't work for every language and we now have
>> the
>> > > > first
>> > > > > >> > > language
>> > > > > >> > > > in
>> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript
>> > which
>> > > > > won't
>> > > > > >> > > > support
>> > > > > >> > > > > it
>> > > > > >> > > > > >> > >> (yes,
>> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on the
>> jvm
>> > > but
>> > > > > >> there
>> > > > > >> > are
>> > > > > >> > > > > >> issues
>> > > > > >> > > > > >> > >> with
>> > > > > >> > > > > >> > >> getting it all to work with the test framework
>> > that
>> > > > make
>> > > > > >> it
>> > > > > >> > > > unduly
>> > > > > >> > > > > >> > >> complicated).
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> On other threads I offered the idea that we
>> look
>> > to
>> > > > use
>> > > > > >> > Gherkin
>> > > > > >> > > > to
>> > > > > >> > > > > >> write
>> > > > > >> > > > > >> > >> general Gremlin test specifications, which then
>> > > could
>> > > > be
>> > > > > >> read
>> > > > > >> > > and
>> > > > > >> > > > > >> > >> processed
>> > > > > >> > > > > >> > >> by the wide variety of test frameworks that can
>> > read
>> > > > > that
>> > > > > >> > > format
>> > > > > >> > > > -
>> > > > > >> > > > > >> there
>> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
>> every
>> > > > > >> language -
>> > > > > >> > > for
>> > > > > >> > > > > >> > example,
>> > > > > >> > > > > >> > >> see:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://cucumber.io/
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> I just created this issue:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://issues.apache.org/
>> > > jira/browse/TINKERPOP-1784
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> and pushed this branch:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/
>> > > > tinkerpop/tree/TINKERPOP-1784
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> which demonstrates how this works with
>> > > gremlin-python.
>> > > > > The
>> > > > > >> > > basic
>> > > > > >> > > > > >> anatomy
>> > > > > >> > > > > >> > >> of
>> > > > > >> > > > > >> > >> this setup involves this new directory in
>> > > > gremlin-test:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-test/features
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> It contains the Gherkin .features files. These
>> are
>> > > the
>> > > > > >> test
>> > > > > >> > > > > >> > >> specifications.
>> > > > > >> > > > > >> > >> They are written using gremlin-java as the
>> "model"
>> > > > > >> language.
>> > > > > >> > > GLVs
>> > > > > >> > > > > >> will
>> > > > > >> > > > > >> > >> then
>> > > > > >> > > > > >> > >> need to write some infrastructure to process
>> these
>> > > > > Gherkin
>> > > > > >> > > files.
>> > > > > >> > > > > The
>> > > > > >> > > > > >> > key
>> > > > > >> > > > > >> > >> to making this "easy" to implement will lie in
>> our
>> > > > > >> abiilty to
>> > > > > >> > > > keep
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > >> assertions we want to have relatively simple.
>> The
>> > > more
>> > > > > >> > > simplistic
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
>> easier
>> > > the
>> > > > > job
>> > > > > >> it
>> > > > > >> > > will
>> > > > > >> > > > > be
>> > > > > >> > > > > >> for
>> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of course,
>> > once
>> > > > that
>> > > > > >> > > > > >> infrastructure
>> > > > > >> > > > > >> > is
>> > > > > >> > > > > >> > >> in place, the GLV developer just has to write
>> the
>> > > GLV
>> > > > > >> version
>> > > > > >> > > of
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So you
>> can
>> > > > look
>> > > > > at
>> > > > > >> > all
>> > > > > >> > > > the
>> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
>> files:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
>> > > translations
>> > > > > >> > > > specifically
>> > > > > >> > > > > >> here:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-python/src/main/jython/
>> > > > > radish/count_features_step.py#
>> > > > > >> > > L34-L46
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> I think this approach works pretty well and
>> solves
>> > > our
>> > > > > >> > general
>> > > > > >> > > > > >> problems
>> > > > > >> > > > > >> > >> for
>> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
>> > > > implementing
>> > > > > >> the
>> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
>> tests
>> > > > added
>> > > > > to
>> > > > > >> > > > .feature
>> > > > > >> > > > > >> > files
>> > > > > >> > > > > >> > >> just need to translated in the GLV. I suppose
>> we
>> > > could
>> > > > > >> > > > "automate" a
>> > > > > >> > > > > >> good
>> > > > > >> > > > > >> > >> portion of the translation with reflection of
>> some
>> > > > sort.
>> > > > > >> > > Anything
>> > > > > >> > > > > >> else
>> > > > > >> > > > > >> > >> could just be handled manually.
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> Not sure if we need to use this new model to
>> > wholly
>> > > > > >> replace
>> > > > > >> > the
>> > > > > >> > > > old
>> > > > > >> > > > > >> one.
>> > > > > >> > > > > >> > >> The process test suite has its place in helping
>> > > graph
>> > > > > >> > database
>> > > > > >> > > > > >> providers
>> > > > > >> > > > > >> > >> test their stuff. I also imagine that
>> introducing
>> > > this
>> > > > > >> > approach
>> > > > > >> > > > in
>> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > >> context would create a breaking change which we
>> > > would
>> > > > > then
>> > > > > >> > need
>> > > > > >> > > > to
>> > > > > >> > > > > >> push
>> > > > > >> > > > > >> > >> off
>> > > > > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to
>> think,
>> > > but
>> > > > > for
>> > > > > >> now
>> > > > > >> > > it
>> > > > > >> > > > > >> might
>> > > > > >> > > > > >> > not
>> > > > > >> > > > > >> > >> be best to conflate the two and just treat
>> them as
>> > > > > >> separate
>> > > > > >> > > > aspects
>> > > > > >> > > > > >> of
>> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > >> test suite.
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> Anyway - it's important we settle on an
>> approach
>> > to
>> > > > > >> testing
>> > > > > >> > as
>> > > > > >> > > we
>> > > > > >> > > > > >> really
>> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET
>> GLV
>> > > > > without
>> > > > > >> > > getting
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > test
>> > > > > >> > > > > >> > >> suite solid. please yell if you have any ideas
>> or
>> > > > > >> feedback on
>> > > > > >> > > > this
>> > > > > >> > > > > >> > >> approach.
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >>
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > >
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

Ok.
Going back to my initial comment regarding enums: don't mind, I've included
the possibility to use enums on the C# parser.

On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> I guess there wouldn't be a problem with that, though having it the way it
> works now was a nice check to ensure that the ScriptEngine was configured
> properly. I guess that shouldn't be the focus on this body of tests though.
> We should be more concerned that the elements of the Gremlin language
> actually work.
>
> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com>
> wrote:
>
> > I see that for most languages using bytecode to native language
> translator
> > would be the easiest solution. In the case of the JavaScript GLV, it
> would
> > be fairly straightforward to implement (and I plan to do so!).
> > In the case of C#, being more strict regarding typing than java, it would
> > be an incomplete solution. For example:
> >
> > Consider the following traversal in Java: g.V().values("age")
> > The type parameter for values() method is inferred at runtime and things
> > "just works" thanks to type erasure.
> >
> > In the case of C#, it should be written as  g.V().Values<int>("age") ,
> with
> > the type parameter to specify which type of traverser are you expecting
> at
> > a compile time.
> >
> > That's why it wont be possible to make a generic translator from bytecode
> > to C# code, without understanding which generic types are expected.
> >
> > In the case of C#, I was going for a translator on the C# side that
> > tokenizes the gremlin traversal. Once the traversal is tokenized, in the
> > moment of invoking the methods (with reflection), it can be made aware of
> > the modern graph data (with datatypes!), ie: For method Values<>(), if
> the
> > property key is "age" -> use int as generic type parameter.
> >
> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > I can't help thinking that perhaps non-JVM languages need to leverage
> > > bytecode to make traversal building easier. We already have an
> > established
> > > pattern for converting bytecode into Traversals and while it is
> different
> > > for each language, it's generally governed by the Translator interface.
> > > Note that we already do this for Java, Groovy, and Python and the code
> > > isn't too crazy - just 200 lines or so:
> > >
> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> > JavaTranslator.java
> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> > > GroovyTranslator.java
> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> > > PythonTranslator.java
> > >
> > > I think that non-JVM languages could send the Gremlin string from the
> > test
> > > to Gremlin Server as a script first and return the bytecode as JSON.
> Then
> > > it could use a "Translator" to then parse it into it's native language.
> > > Shouldn't we just build a CSharpTranslator similar to these? does that
> > make
> > > sense?
> > >
> > >
> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> > > jorgebaygondra@gmail.com
> > > > wrote:
> > >
> > > > I've been looking over the current test scenarios on the
> TINKERPOP-1784
> > > > branch, with the C# GLV test suite in mind, and oh man that's an
> > > impressive
> > > > amount of scenarios!
> > > >
> > > > So far, I have a suggestion.
> > > >
> > > > Instead of
> > > >
> > > >   Scenario: g_V_fold_countXlocalX
> > > >     Given the modern graph
> > > >     And the traversal of
> > > >       """
> > > >       g.V().fold().count(Scope.local)
> > > >       """
> > > >
> > > > We could use
> > > >
> > > >   Scenario: g_V_fold_countXlocalX
> > > >     Given the modern graph
> > > >     And parameter 1 being an enum
> > > >       """
> > > >       Scope.local
> > > >       """
> > > >     And the traversal of
> > > >       """
> > > >       g.V().fold().count(param1)
> > > >       """
> > > >
> > > > That way we can exclude enums from possible parameter values,
> narrowing
> > > > down the amount of types of parameters allowed, ie: const numeric,
> > const
> > > > strings (wrapped in double quotes), traversals (starting with `__`) ,
> > > ...,
> > > > making parsing the traversal a little easier.
> > > >
> > > >
> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> > spmallette@gmail.com>
> > > > wrote:
> > > >
> > > > > Just a quick update on this item - it continues........
> > > > >
> > > > > Went down a bad path earlier in the week and ended up shelving a
> lot
> > of
> > > > > work - ended up just coded into a corner. Anyway, I've not really
> > > changed
> > > > > much in the implementation, but I've still not reached a point
> where
> > > the
> > > > > addition of a new .feature files comes without meeting some new
> type
> > of
> > > > > assertion that has to be dealt with. This ends up slowing down
> > progress
> > > > on
> > > > > porting over the tests. Please feel free to peruse the latest
> changes
> > > on
> > > > > the branch and let me know if there's any feedback.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> > > spmallette@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > I did some pretty heavy refactoring to the python test logic
> > > (altered a
> > > > > > bit by some revision of the gherkin feature file language) and
> the
> > > > result
> > > > > > is a much more simplified test logic file:
> > > > > >
> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > > >
> > > > > > About 120 lines of code (down from about 170). If you include the
> > > test
> > > > > > logic "setup" file:
> > > > > >
> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > > gremlin-python/src/main/jython/radish/terrain.py
> > > > > >
> > > > > > we end up with about 250 lines of test logic total (take out the
> > > > > > comment/license and we're probably well under 200 total - not too
> > > > bad). I
> > > > > > think I'm closing in on the end to infrastructure building here
> so
> > > the
> > > > > > basic framework is getting close to final at this point I
> believe.
> > > I'll
> > > > > > keep scanning the tests looking for other types of assertions
> that
> > > I've
> > > > > not
> > > > > > yet covered, but it's getting pretty solid I think. Hopefully,
> > there
> > > > > won't
> > > > > > need to be too many more lines of code needed to express the test
> > > logic
> > > > > as
> > > > > > I like how things are looking right now.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > > > > jorgebaygondra@gmail.com> wrote:
> > > > > >
> > > > > >> Great progress! I like how you avoided using ids, even if it
> adds
> > > some
> > > > > >> complexity to the transformation required.
> > > > > >>
> > > > > >> The Python step definitions are still quite simple, I think it
> > would
> > > > be
> > > > > >> mostly the same on the rest of the languages.
> > > > > >>
> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > > > spmallette@gmail.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Just another update on progress with the test suite. The
> > language
> > > of
> > > > > the
> > > > > >> > .feature files is getting slightly more complex as I try to
> > > > translate
> > > > > >> more
> > > > > >> > and more of our Java process suite tests into the language of
> > the
> > > > > >> gherkin
> > > > > >> > files. You can see here where I've added the ability to
> include
> > > > > >> parameters:
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-test/features/filter/Has.feature#L33
> > > > > >> >
> > > > > >> > I also came up with a method of asserting edges (didn't want
> to
> > > rely
> > > > > on
> > > > > >> ids
> > > > > >> > as it makes the gherkin harder to read, plus i didn't want to
> > > assume
> > > > > >> > TinkerGraph identifiers in case these tests were every used
> with
> > > > some
> > > > > >> other
> > > > > >> > graph database that didn't use longs):
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > > > > >> >
> > > > > >> > for all those additions (and others) the logic required by the
> > GLV
> > > > to
> > > > > >> > process these tests has stayed surprisingly simple:
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > > >> >
> > > > > >> > There's a fair bit of regex/string manipulation involved
> there,
> > > but
> > > > > it's
> > > > > >> > processing strings from the feature file so that's the nature
> of
> > > it
> > > > I
> > > > > >> > suppose. I think I'm of the mind that I want to port all of
> the
> > > > tests
> > > > > to
> > > > > >> > feature files, so I wrote this unit test to help validate that
> > > none
> > > > > were
> > > > > >> > missed:
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > > > > >> > structure/FeatureCoverageTest.java
> > > > > >> >
> > > > > >> > I don't know that we have to get to 100% porting right away,
> > but I
> > > > > think
> > > > > >> > that once we have these gherkin files written they not only
> > become
> > > > the
> > > > > >> > basis for our current GLV testing work, but we might be able
> to
> > > > simply
> > > > > >> use
> > > > > >> > them for testing the Java stuff as well - that would rid us of
> > > > having
> > > > > >> the
> > > > > >> > test code duplication. It also sets us up with a portable body
> > of
> > > > > tests
> > > > > >> > that can be re-used in TinkerPop 4.x.
> > > > > >> >
> > > > > >> > I'm open to suggestions if anyone has any.
> > > > > >> >
> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > > > > >> > jorgebaygondra@gmail.com
> > > > > >> > > wrote:
> > > > > >> >
> > > > > >> > > I was able to build a proof of concept for a Gherkin-based
> > test
> > > > > >> runner in
> > > > > >> > > C#, that takes the proposed count and select features
> > > > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > > > >> > > gremlin-test/features/map>
> > > > > >> > > and runs them using C# step definitions.
> > > > > >> > >
> > > > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > > > > >> herkin-dotnet>
> > > > > >> > > from
> > > > > >> > > cucumber, there isn't a release of the parser with .NET Core
> > > > support
> > > > > >> so
> > > > > >> > > I've
> > > > > >> > > asked them to release one
> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
> (there
> > > is
> > > > no
> > > > > >> > > limitation from their source files). If they are not able to
> > > > release
> > > > > >> it
> > > > > >> > in
> > > > > >> > > the next few days, we can implement our own as it should be
> > > pretty
> > > > > >> > straight
> > > > > >> > > forward.
> > > > > >> > >
> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > > > > >> spmallette@gmail.com>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Thanks for the update. I'm trying to keep the test
> language
> > as
> > > > > >> simple
> > > > > >> > as
> > > > > >> > > > possible so that we don't need an overly complicated test
> > > > > >> > implementation.
> > > > > >> > > > Hopefully that will help make the .NET approach as easy as
> > > > > possible.
> > > > > >> > > >
> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > > > >> > > > jorgebaygondra@gmail.com>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > I've been looking into Gherkin support for .NET:
> SpecFlow,
> > > the
> > > > > >> > cucumber
> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > > > > >> > > > cucumber-implementations
> > > > > >> > > > > >,
> > > > > >> > > > > does not support .NET Core platform (we use .NET Core
> > build
> > > > > tools
> > > > > >> for
> > > > > >> > > the
> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
> > > > > >> > > > >
> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
> > > > > >> pecFlow/projects/2
> > > > > >> > >,
> > > > > >> > > > > .NET
> > > > > >> > > > > Core support on SpecFlow is coming at a very slow pace
> and
> > > we
> > > > > >> > shouldn't
> > > > > >> > > > > expect to land any time soon (there were some design
> > > decisions
> > > > > in
> > > > > >> > > > SpecFlow
> > > > > >> > > > > library that makes supporting other platforms
> non-trivial,
> > > > like
> > > > > >> > > requiring
> > > > > >> > > > > code gen).
> > > > > >> > > > >
> > > > > >> > > > > The alternative would be to implement our own harness to
> > > > support
> > > > > >> it:
> > > > > >> > > > from a
> > > > > >> > > > > xunit test, look for certain types and parse the
> > > annotations,
> > > > > and
> > > > > >> > > execute
> > > > > >> > > > > them using the Gherkin features.
> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > > > > >> > > > > I'll continue looking into this option and try to
> > understand
> > > > the
> > > > > >> > effort
> > > > > >> > > > > required...
> > > > > >> > > > >
> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > > > >> > > > > jorgebaygondra@gmail.com>
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
> growing
> > > > number
> > > > > >> of
> > > > > >> > > GLVs.
> > > > > >> > > > > >
> > > > > >> > > > > > We should find a way to define the different features
> > > > > supported
> > > > > >> by
> > > > > >> > > each
> > > > > >> > > > > > GLV, as it's reasonable to have different maturity
> > levels
> > > > per
> > > > > >> GLV
> > > > > >> > > (ie:
> > > > > >> > > > > > lambdas support, traversal strategy, ...). I don't
> know
> > if
> > > > it
> > > > > >> will
> > > > > >> > be
> > > > > >> > > > > > beneficial to do it in the Gherkin files or within
> each
> > > GLV
> > > > > >> > > > > implementation.
> > > > > >> > > > > > Also, we should consider the process of rolling out a
> > new
> > > > > >> method /
> > > > > >> > > > class
> > > > > >> > > > > > in the java implementation, how that could affect each
> > > GLV.
> > > > > >> > > > > >
> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > > > >> > > > spmallette@gmail.com
> > > > > >> > > > > >
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > > > > eval(). I
> > > > > >> > > guess
> > > > > >> > > > > the
> > > > > >> > > > > >> point is that if the language can support some way of
> > > > taking
> > > > > >> the
> > > > > >> > > > string
> > > > > >> > > > > >> value and turning it automatically into a traversal
> in
> > > that
> > > > > >> GLVs
> > > > > >> > > style
> > > > > >> > > > > >> then
> > > > > >> > > > > >> we should do that.
> > > > > >> > > > > >>
> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > > > > >> <me...@gremlin.guru>
> > > > > >> > > > > wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> > For unparameterized queries it can probably be as
> > easy
> > > > as:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > @given("the traversal of")
> > > > > >> > > > > >> > def translate_traversal(step):
> > > > > >> > > > > >> >     g = step.context.g
> > > > > >> > > > > >> >     step.context.traversal = eval(step.text)
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > Cheers,
> > > > > >> > > > > >> > Daniel
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > > > > >> <me@gremlin.guru
> > > > > >> > >
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
> > Gherkin
> > > > for
> > > > > >> > years,
> > > > > >> > > > > but I
> > > > > >> > > > > >> > > really like the BDD approach.
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> > translations
> > > > > >> > > specifically
> > > > > >> > > > > >> here:
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > > radish/count_features_step.py#
> > > > > >> > > L34-L46
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > This part is the only thing that looks weird to
> me.
> > > > > You're
> > > > > >> > > > basically
> > > > > >> > > > > >> > > writing every query twice; is there really no
> > easier
> > > > way
> > > > > >> to do
> > > > > >> > > > that?
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > Cheers,
> > > > > >> > > > > >> > > Daniel
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
> Mallette <
> > > > > >> > > > > >> spmallette@gmail.com>
> > > > > >> > > > > >> > > wrote:
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >> I've brought this issue up in the past and had
> > > > suggested
> > > > > >> some
> > > > > >> > > > > >> options I
> > > > > >> > > > > >> > >> had
> > > > > >> > > > > >> > >> in mind but now I've finally put the basics of
> > those
> > > > > >> ideas in
> > > > > >> > > > place
> > > > > >> > > > > >> so I
> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that
> the
> > > > issue
> > > > > at
> > > > > >> > hand
> > > > > >> > > > is
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > we
> > > > > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test
> > is
> > > > > bound
> > > > > >> to
> > > > > >> > > the
> > > > > >> > > > > >> JVM. We
> > > > > >> > > > > >> > >> have some tricks that let us test gremlin-python
> > > with
> > > > it
> > > > > >> but
> > > > > >> > > > those
> > > > > >> > > > > >> > tricks
> > > > > >> > > > > >> > >> won't work for every language and we now have
> the
> > > > first
> > > > > >> > > language
> > > > > >> > > > in
> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript
> > which
> > > > > won't
> > > > > >> > > > support
> > > > > >> > > > > it
> > > > > >> > > > > >> > >> (yes,
> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on the
> jvm
> > > but
> > > > > >> there
> > > > > >> > are
> > > > > >> > > > > >> issues
> > > > > >> > > > > >> > >> with
> > > > > >> > > > > >> > >> getting it all to work with the test framework
> > that
> > > > make
> > > > > >> it
> > > > > >> > > > unduly
> > > > > >> > > > > >> > >> complicated).
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> On other threads I offered the idea that we look
> > to
> > > > use
> > > > > >> > Gherkin
> > > > > >> > > > to
> > > > > >> > > > > >> write
> > > > > >> > > > > >> > >> general Gremlin test specifications, which then
> > > could
> > > > be
> > > > > >> read
> > > > > >> > > and
> > > > > >> > > > > >> > >> processed
> > > > > >> > > > > >> > >> by the wide variety of test frameworks that can
> > read
> > > > > that
> > > > > >> > > format
> > > > > >> > > > -
> > > > > >> > > > > >> there
> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
> every
> > > > > >> language -
> > > > > >> > > for
> > > > > >> > > > > >> > example,
> > > > > >> > > > > >> > >> see:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://cucumber.io/
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> I just created this issue:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://issues.apache.org/
> > > jira/browse/TINKERPOP-1784
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> and pushed this branch:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/
> > > > tinkerpop/tree/TINKERPOP-1784
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> which demonstrates how this works with
> > > gremlin-python.
> > > > > The
> > > > > >> > > basic
> > > > > >> > > > > >> anatomy
> > > > > >> > > > > >> > >> of
> > > > > >> > > > > >> > >> this setup involves this new directory in
> > > > gremlin-test:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-test/features
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> It contains the Gherkin .features files. These
> are
> > > the
> > > > > >> test
> > > > > >> > > > > >> > >> specifications.
> > > > > >> > > > > >> > >> They are written using gremlin-java as the
> "model"
> > > > > >> language.
> > > > > >> > > GLVs
> > > > > >> > > > > >> will
> > > > > >> > > > > >> > >> then
> > > > > >> > > > > >> > >> need to write some infrastructure to process
> these
> > > > > Gherkin
> > > > > >> > > files.
> > > > > >> > > > > The
> > > > > >> > > > > >> > key
> > > > > >> > > > > >> > >> to making this "easy" to implement will lie in
> our
> > > > > >> abiilty to
> > > > > >> > > > keep
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > >> assertions we want to have relatively simple.
> The
> > > more
> > > > > >> > > simplistic
> > > > > >> > > > > the
> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
> easier
> > > the
> > > > > job
> > > > > >> it
> > > > > >> > > will
> > > > > >> > > > > be
> > > > > >> > > > > >> for
> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of course,
> > once
> > > > that
> > > > > >> > > > > >> infrastructure
> > > > > >> > > > > >> > is
> > > > > >> > > > > >> > >> in place, the GLV developer just has to write
> the
> > > GLV
> > > > > >> version
> > > > > >> > > of
> > > > > >> > > > > the
> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So you
> can
> > > > look
> > > > > at
> > > > > >> > all
> > > > > >> > > > the
> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
> files:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> > > translations
> > > > > >> > > > specifically
> > > > > >> > > > > >> here:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > > radish/count_features_step.py#
> > > > > >> > > L34-L46
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> I think this approach works pretty well and
> solves
> > > our
> > > > > >> > general
> > > > > >> > > > > >> problems
> > > > > >> > > > > >> > >> for
> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > > > implementing
> > > > > >> the
> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
> tests
> > > > added
> > > > > to
> > > > > >> > > > .feature
> > > > > >> > > > > >> > files
> > > > > >> > > > > >> > >> just need to translated in the GLV. I suppose we
> > > could
> > > > > >> > > > "automate" a
> > > > > >> > > > > >> good
> > > > > >> > > > > >> > >> portion of the translation with reflection of
> some
> > > > sort.
> > > > > >> > > Anything
> > > > > >> > > > > >> else
> > > > > >> > > > > >> > >> could just be handled manually.
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> Not sure if we need to use this new model to
> > wholly
> > > > > >> replace
> > > > > >> > the
> > > > > >> > > > old
> > > > > >> > > > > >> one.
> > > > > >> > > > > >> > >> The process test suite has its place in helping
> > > graph
> > > > > >> > database
> > > > > >> > > > > >> providers
> > > > > >> > > > > >> > >> test their stuff. I also imagine that
> introducing
> > > this
> > > > > >> > approach
> > > > > >> > > > in
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > >> context would create a breaking change which we
> > > would
> > > > > then
> > > > > >> > need
> > > > > >> > > > to
> > > > > >> > > > > >> push
> > > > > >> > > > > >> > >> off
> > > > > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to
> think,
> > > but
> > > > > for
> > > > > >> now
> > > > > >> > > it
> > > > > >> > > > > >> might
> > > > > >> > > > > >> > not
> > > > > >> > > > > >> > >> be best to conflate the two and just treat them
> as
> > > > > >> separate
> > > > > >> > > > aspects
> > > > > >> > > > > >> of
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > >> test suite.
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> Anyway - it's important we settle on an approach
> > to
> > > > > >> testing
> > > > > >> > as
> > > > > >> > > we
> > > > > >> > > > > >> really
> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET
> GLV
> > > > > without
> > > > > >> > > getting
> > > > > >> > > > > the
> > > > > >> > > > > >> > test
> > > > > >> > > > > >> > >> suite solid. please yell if you have any ideas
> or
> > > > > >> feedback on
> > > > > >> > > > this
> > > > > >> > > > > >> > >> approach.
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> >
> > > > > >> > > > > >>
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

I guess there wouldn't be a problem with that, though having it the way it
works now was a nice check to ensure that the ScriptEngine was configured
properly. I guess that shouldn't be the focus on this body of tests though.
We should be more concerned that the elements of the Gremlin language
actually work.

On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <jo...@gmail.com>
wrote:

> I see that for most languages using bytecode to native language translator
> would be the easiest solution. In the case of the JavaScript GLV, it would
> be fairly straightforward to implement (and I plan to do so!).
> In the case of C#, being more strict regarding typing than java, it would
> be an incomplete solution. For example:
>
> Consider the following traversal in Java: g.V().values("age")
> The type parameter for values() method is inferred at runtime and things
> "just works" thanks to type erasure.
>
> In the case of C#, it should be written as  g.V().Values<int>("age") , with
> the type parameter to specify which type of traverser are you expecting at
> a compile time.
>
> That's why it wont be possible to make a generic translator from bytecode
> to C# code, without understanding which generic types are expected.
>
> In the case of C#, I was going for a translator on the C# side that
> tokenizes the gremlin traversal. Once the traversal is tokenized, in the
> moment of invoking the methods (with reflection), it can be made aware of
> the modern graph data (with datatypes!), ie: For method Values<>(), if the
> property key is "age" -> use int as generic type parameter.
>
> On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > I can't help thinking that perhaps non-JVM languages need to leverage
> > bytecode to make traversal building easier. We already have an
> established
> > pattern for converting bytecode into Traversals and while it is different
> > for each language, it's generally governed by the Translator interface.
> > Note that we already do this for Java, Groovy, and Python and the code
> > isn't too crazy - just 200 lines or so:
> >
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> JavaTranslator.java
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> > GroovyTranslator.java
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> > PythonTranslator.java
> >
> > I think that non-JVM languages could send the Gremlin string from the
> test
> > to Gremlin Server as a script first and return the bytecode as JSON. Then
> > it could use a "Translator" to then parse it into it's native language.
> > Shouldn't we just build a CSharpTranslator similar to these? does that
> make
> > sense?
> >
> >
> > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com
> > > wrote:
> >
> > > I've been looking over the current test scenarios on the TINKERPOP-1784
> > > branch, with the C# GLV test suite in mind, and oh man that's an
> > impressive
> > > amount of scenarios!
> > >
> > > So far, I have a suggestion.
> > >
> > > Instead of
> > >
> > >   Scenario: g_V_fold_countXlocalX
> > >     Given the modern graph
> > >     And the traversal of
> > >       """
> > >       g.V().fold().count(Scope.local)
> > >       """
> > >
> > > We could use
> > >
> > >   Scenario: g_V_fold_countXlocalX
> > >     Given the modern graph
> > >     And parameter 1 being an enum
> > >       """
> > >       Scope.local
> > >       """
> > >     And the traversal of
> > >       """
> > >       g.V().fold().count(param1)
> > >       """
> > >
> > > That way we can exclude enums from possible parameter values, narrowing
> > > down the amount of types of parameters allowed, ie: const numeric,
> const
> > > strings (wrapped in double quotes), traversals (starting with `__`) ,
> > ...,
> > > making parsing the traversal a little easier.
> > >
> > >
> > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > > > Just a quick update on this item - it continues........
> > > >
> > > > Went down a bad path earlier in the week and ended up shelving a lot
> of
> > > > work - ended up just coded into a corner. Anyway, I've not really
> > changed
> > > > much in the implementation, but I've still not reached a point where
> > the
> > > > addition of a new .feature files comes without meeting some new type
> of
> > > > assertion that has to be dealt with. This ends up slowing down
> progress
> > > on
> > > > porting over the tests. Please feel free to peruse the latest changes
> > on
> > > > the branch and let me know if there's any feedback.
> > > >
> > > >
> > > >
> > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> > spmallette@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I did some pretty heavy refactoring to the python test logic
> > (altered a
> > > > > bit by some revision of the gherkin feature file language) and the
> > > result
> > > > > is a much more simplified test logic file:
> > > > >
> > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > >
> > > > > About 120 lines of code (down from about 170). If you include the
> > test
> > > > > logic "setup" file:
> > > > >
> > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > gremlin-python/src/main/jython/radish/terrain.py
> > > > >
> > > > > we end up with about 250 lines of test logic total (take out the
> > > > > comment/license and we're probably well under 200 total - not too
> > > bad). I
> > > > > think I'm closing in on the end to infrastructure building here so
> > the
> > > > > basic framework is getting close to final at this point I believe.
> > I'll
> > > > > keep scanning the tests looking for other types of assertions that
> > I've
> > > > not
> > > > > yet covered, but it's getting pretty solid I think. Hopefully,
> there
> > > > won't
> > > > > need to be too many more lines of code needed to express the test
> > logic
> > > > as
> > > > > I like how things are looking right now.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > > > jorgebaygondra@gmail.com> wrote:
> > > > >
> > > > >> Great progress! I like how you avoided using ids, even if it adds
> > some
> > > > >> complexity to the transformation required.
> > > > >>
> > > > >> The Python step definitions are still quite simple, I think it
> would
> > > be
> > > > >> mostly the same on the rest of the languages.
> > > > >>
> > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > > spmallette@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Just another update on progress with the test suite. The
> language
> > of
> > > > the
> > > > >> > .feature files is getting slightly more complex as I try to
> > > translate
> > > > >> more
> > > > >> > and more of our Java process suite tests into the language of
> the
> > > > >> gherkin
> > > > >> > files. You can see here where I've added the ability to include
> > > > >> parameters:
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-test/features/filter/Has.feature#L33
> > > > >> >
> > > > >> > I also came up with a method of asserting edges (didn't want to
> > rely
> > > > on
> > > > >> ids
> > > > >> > as it makes the gherkin harder to read, plus i didn't want to
> > assume
> > > > >> > TinkerGraph identifiers in case these tests were every used with
> > > some
> > > > >> other
> > > > >> > graph database that didn't use longs):
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > > > >> >
> > > > >> > for all those additions (and others) the logic required by the
> GLV
> > > to
> > > > >> > process these tests has stayed surprisingly simple:
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > >> >
> > > > >> > There's a fair bit of regex/string manipulation involved there,
> > but
> > > > it's
> > > > >> > processing strings from the feature file so that's the nature of
> > it
> > > I
> > > > >> > suppose. I think I'm of the mind that I want to port all of the
> > > tests
> > > > to
> > > > >> > feature files, so I wrote this unit test to help validate that
> > none
> > > > were
> > > > >> > missed:
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > > > >> > structure/FeatureCoverageTest.java
> > > > >> >
> > > > >> > I don't know that we have to get to 100% porting right away,
> but I
> > > > think
> > > > >> > that once we have these gherkin files written they not only
> become
> > > the
> > > > >> > basis for our current GLV testing work, but we might be able to
> > > simply
> > > > >> use
> > > > >> > them for testing the Java stuff as well - that would rid us of
> > > having
> > > > >> the
> > > > >> > test code duplication. It also sets us up with a portable body
> of
> > > > tests
> > > > >> > that can be re-used in TinkerPop 4.x.
> > > > >> >
> > > > >> > I'm open to suggestions if anyone has any.
> > > > >> >
> > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > > > >> > jorgebaygondra@gmail.com
> > > > >> > > wrote:
> > > > >> >
> > > > >> > > I was able to build a proof of concept for a Gherkin-based
> test
> > > > >> runner in
> > > > >> > > C#, that takes the proposed count and select features
> > > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > > >> > > gremlin-test/features/map>
> > > > >> > > and runs them using C# step definitions.
> > > > >> > >
> > > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > > > >> herkin-dotnet>
> > > > >> > > from
> > > > >> > > cucumber, there isn't a release of the parser with .NET Core
> > > support
> > > > >> so
> > > > >> > > I've
> > > > >> > > asked them to release one
> > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there
> > is
> > > no
> > > > >> > > limitation from their source files). If they are not able to
> > > release
> > > > >> it
> > > > >> > in
> > > > >> > > the next few days, we can implement our own as it should be
> > pretty
> > > > >> > straight
> > > > >> > > forward.
> > > > >> > >
> > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > > > >> spmallette@gmail.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Thanks for the update. I'm trying to keep the test language
> as
> > > > >> simple
> > > > >> > as
> > > > >> > > > possible so that we don't need an overly complicated test
> > > > >> > implementation.
> > > > >> > > > Hopefully that will help make the .NET approach as easy as
> > > > possible.
> > > > >> > > >
> > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > > >> > > > jorgebaygondra@gmail.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > I've been looking into Gherkin support for .NET: SpecFlow,
> > the
> > > > >> > cucumber
> > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > > > >> > > > cucumber-implementations
> > > > >> > > > > >,
> > > > >> > > > > does not support .NET Core platform (we use .NET Core
> build
> > > > tools
> > > > >> for
> > > > >> > > the
> > > > >> > > > > .NET GLV) and only supports .NET Framework.
> > > > >> > > > >
> > > > >> > > > > From what I can see <https://github.com/techtalk/S
> > > > >> pecFlow/projects/2
> > > > >> > >,
> > > > >> > > > > .NET
> > > > >> > > > > Core support on SpecFlow is coming at a very slow pace and
> > we
> > > > >> > shouldn't
> > > > >> > > > > expect to land any time soon (there were some design
> > decisions
> > > > in
> > > > >> > > > SpecFlow
> > > > >> > > > > library that makes supporting other platforms non-trivial,
> > > like
> > > > >> > > requiring
> > > > >> > > > > code gen).
> > > > >> > > > >
> > > > >> > > > > The alternative would be to implement our own harness to
> > > support
> > > > >> it:
> > > > >> > > > from a
> > > > >> > > > > xunit test, look for certain types and parse the
> > annotations,
> > > > and
> > > > >> > > execute
> > > > >> > > > > them using the Gherkin features.
> > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > > > >> > > > > I'll continue looking into this option and try to
> understand
> > > the
> > > > >> > effort
> > > > >> > > > > required...
> > > > >> > > > >
> > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > > >> > > > > jorgebaygondra@gmail.com>
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Nice! Gherkin will make our lives easier with a growing
> > > number
> > > > >> of
> > > > >> > > GLVs.
> > > > >> > > > > >
> > > > >> > > > > > We should find a way to define the different features
> > > > supported
> > > > >> by
> > > > >> > > each
> > > > >> > > > > > GLV, as it's reasonable to have different maturity
> levels
> > > per
> > > > >> GLV
> > > > >> > > (ie:
> > > > >> > > > > > lambdas support, traversal strategy, ...). I don't know
> if
> > > it
> > > > >> will
> > > > >> > be
> > > > >> > > > > > beneficial to do it in the Gherkin files or within each
> > GLV
> > > > >> > > > > implementation.
> > > > >> > > > > > Also, we should consider the process of rolling out a
> new
> > > > >> method /
> > > > >> > > > class
> > > > >> > > > > > in the java implementation, how that could affect each
> > GLV.
> > > > >> > > > > >
> > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > > >> > > > spmallette@gmail.com
> > > > >> > > > > >
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > > > eval(). I
> > > > >> > > guess
> > > > >> > > > > the
> > > > >> > > > > >> point is that if the language can support some way of
> > > taking
> > > > >> the
> > > > >> > > > string
> > > > >> > > > > >> value and turning it automatically into a traversal in
> > that
> > > > >> GLVs
> > > > >> > > style
> > > > >> > > > > >> then
> > > > >> > > > > >> we should do that.
> > > > >> > > > > >>
> > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > > > >> <me...@gremlin.guru>
> > > > >> > > > > wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > For unparameterized queries it can probably be as
> easy
> > > as:
> > > > >> > > > > >> >
> > > > >> > > > > >> > @given("the traversal of")
> > > > >> > > > > >> > def translate_traversal(step):
> > > > >> > > > > >> >     g = step.context.g
> > > > >> > > > > >> >     step.context.traversal = eval(step.text)
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > Cheers,
> > > > >> > > > > >> > Daniel
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > > > >> <me@gremlin.guru
> > > > >> > >
> > > > >> > > > > >> wrote:
> > > > >> > > > > >> >
> > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
> Gherkin
> > > for
> > > > >> > years,
> > > > >> > > > > but I
> > > > >> > > > > >> > > really like the BDD approach.
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> translations
> > > > >> > > specifically
> > > > >> > > > > >> here:
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > radish/count_features_step.py#
> > > > >> > > L34-L46
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > This part is the only thing that looks weird to me.
> > > > You're
> > > > >> > > > basically
> > > > >> > > > > >> > > writing every query twice; is there really no
> easier
> > > way
> > > > >> to do
> > > > >> > > > that?
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > Cheers,
> > > > >> > > > > >> > > Daniel
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > > >> > > > > >> spmallette@gmail.com>
> > > > >> > > > > >> > > wrote:
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >> I've brought this issue up in the past and had
> > > suggested
> > > > >> some
> > > > >> > > > > >> options I
> > > > >> > > > > >> > >> had
> > > > >> > > > > >> > >> in mind but now I've finally put the basics of
> those
> > > > >> ideas in
> > > > >> > > > place
> > > > >> > > > > >> so I
> > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that the
> > > issue
> > > > at
> > > > >> > hand
> > > > >> > > > is
> > > > >> > > > > >> that
> > > > >> > > > > >> > we
> > > > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test
> is
> > > > bound
> > > > >> to
> > > > >> > > the
> > > > >> > > > > >> JVM. We
> > > > >> > > > > >> > >> have some tricks that let us test gremlin-python
> > with
> > > it
> > > > >> but
> > > > >> > > > those
> > > > >> > > > > >> > tricks
> > > > >> > > > > >> > >> won't work for every language and we now have the
> > > first
> > > > >> > > language
> > > > >> > > > in
> > > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript
> which
> > > > won't
> > > > >> > > > support
> > > > >> > > > > it
> > > > >> > > > > >> > >> (yes,
> > > > >> > > > > >> > >> i know that gremlin-javascript can run on the jvm
> > but
> > > > >> there
> > > > >> > are
> > > > >> > > > > >> issues
> > > > >> > > > > >> > >> with
> > > > >> > > > > >> > >> getting it all to work with the test framework
> that
> > > make
> > > > >> it
> > > > >> > > > unduly
> > > > >> > > > > >> > >> complicated).
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> On other threads I offered the idea that we look
> to
> > > use
> > > > >> > Gherkin
> > > > >> > > > to
> > > > >> > > > > >> write
> > > > >> > > > > >> > >> general Gremlin test specifications, which then
> > could
> > > be
> > > > >> read
> > > > >> > > and
> > > > >> > > > > >> > >> processed
> > > > >> > > > > >> > >> by the wide variety of test frameworks that can
> read
> > > > that
> > > > >> > > format
> > > > >> > > > -
> > > > >> > > > > >> there
> > > > >> > > > > >> > >> tend to be Gherkin processors in just about every
> > > > >> language -
> > > > >> > > for
> > > > >> > > > > >> > example,
> > > > >> > > > > >> > >> see:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://cucumber.io/
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> I just created this issue:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://issues.apache.org/
> > jira/browse/TINKERPOP-1784
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> and pushed this branch:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/
> > > tinkerpop/tree/TINKERPOP-1784
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> which demonstrates how this works with
> > gremlin-python.
> > > > The
> > > > >> > > basic
> > > > >> > > > > >> anatomy
> > > > >> > > > > >> > >> of
> > > > >> > > > > >> > >> this setup involves this new directory in
> > > gremlin-test:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-test/features
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> It contains the Gherkin .features files. These are
> > the
> > > > >> test
> > > > >> > > > > >> > >> specifications.
> > > > >> > > > > >> > >> They are written using gremlin-java as the "model"
> > > > >> language.
> > > > >> > > GLVs
> > > > >> > > > > >> will
> > > > >> > > > > >> > >> then
> > > > >> > > > > >> > >> need to write some infrastructure to process these
> > > > Gherkin
> > > > >> > > files.
> > > > >> > > > > The
> > > > >> > > > > >> > key
> > > > >> > > > > >> > >> to making this "easy" to implement will lie in our
> > > > >> abiilty to
> > > > >> > > > keep
> > > > >> > > > > >> the
> > > > >> > > > > >> > >> assertions we want to have relatively simple. The
> > more
> > > > >> > > simplistic
> > > > >> > > > > the
> > > > >> > > > > >> > >> language in the Gherkin .feature files the easier
> > the
> > > > job
> > > > >> it
> > > > >> > > will
> > > > >> > > > > be
> > > > >> > > > > >> for
> > > > >> > > > > >> > >> GLVs to build their infrastructure. Of course,
> once
> > > that
> > > > >> > > > > >> infrastructure
> > > > >> > > > > >> > is
> > > > >> > > > > >> > >> in place, the GLV developer just has to write the
> > GLV
> > > > >> version
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > >> > >> Gremlin specified in the .feature file. So you can
> > > look
> > > > at
> > > > >> > all
> > > > >> > > > the
> > > > >> > > > > >> > >> "infrastructure" code here in this pair of files:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> > translations
> > > > >> > > > specifically
> > > > >> > > > > >> here:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > radish/count_features_step.py#
> > > > >> > > L34-L46
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> I think this approach works pretty well and solves
> > our
> > > > >> > general
> > > > >> > > > > >> problems
> > > > >> > > > > >> > >> for
> > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > > implementing
> > > > >> the
> > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin tests
> > > added
> > > > to
> > > > >> > > > .feature
> > > > >> > > > > >> > files
> > > > >> > > > > >> > >> just need to translated in the GLV. I suppose we
> > could
> > > > >> > > > "automate" a
> > > > >> > > > > >> good
> > > > >> > > > > >> > >> portion of the translation with reflection of some
> > > sort.
> > > > >> > > Anything
> > > > >> > > > > >> else
> > > > >> > > > > >> > >> could just be handled manually.
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> Not sure if we need to use this new model to
> wholly
> > > > >> replace
> > > > >> > the
> > > > >> > > > old
> > > > >> > > > > >> one.
> > > > >> > > > > >> > >> The process test suite has its place in helping
> > graph
> > > > >> > database
> > > > >> > > > > >> providers
> > > > >> > > > > >> > >> test their stuff. I also imagine that introducing
> > this
> > > > >> > approach
> > > > >> > > > in
> > > > >> > > > > >> that
> > > > >> > > > > >> > >> context would create a breaking change which we
> > would
> > > > then
> > > > >> > need
> > > > >> > > > to
> > > > >> > > > > >> push
> > > > >> > > > > >> > >> off
> > > > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to think,
> > but
> > > > for
> > > > >> now
> > > > >> > > it
> > > > >> > > > > >> might
> > > > >> > > > > >> > not
> > > > >> > > > > >> > >> be best to conflate the two and just treat them as
> > > > >> separate
> > > > >> > > > aspects
> > > > >> > > > > >> of
> > > > >> > > > > >> > the
> > > > >> > > > > >> > >> test suite.
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> Anyway - it's important we settle on an approach
> to
> > > > >> testing
> > > > >> > as
> > > > >> > > we
> > > > >> > > > > >> really
> > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> > > > without
> > > > >> > > getting
> > > > >> > > > > the
> > > > >> > > > > >> > test
> > > > >> > > > > >> > >> suite solid. please yell if you have any ideas or
> > > > >> feedback on
> > > > >> > > > this
> > > > >> > > > > >> > >> approach.
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

I see that for most languages using bytecode to native language translator
would be the easiest solution. In the case of the JavaScript GLV, it would
be fairly straightforward to implement (and I plan to do so!).
In the case of C#, being more strict regarding typing than java, it would
be an incomplete solution. For example:

Consider the following traversal in Java: g.V().values("age")
The type parameter for values() method is inferred at runtime and things
"just works" thanks to type erasure.

In the case of C#, it should be written as  g.V().Values<int>("age") , with
the type parameter to specify which type of traverser are you expecting at
a compile time.

That's why it wont be possible to make a generic translator from bytecode
to C# code, without understanding which generic types are expected.

In the case of C#, I was going for a translator on the C# side that
tokenizes the gremlin traversal. Once the traversal is tokenized, in the
moment of invoking the methods (with reflection), it can be made aware of
the modern graph data (with datatypes!), ie: For method Values<>(), if the
property key is "age" -> use int as generic type parameter.

On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> I can't help thinking that perhaps non-JVM languages need to leverage
> bytecode to make traversal building easier. We already have an established
> pattern for converting bytecode into Traversals and while it is different
> for each language, it's generally governed by the Translator interface.
> Note that we already do this for Java, Groovy, and Python and the code
> isn't too crazy - just 200 lines or so:
>
> https://github.com/apache/tinkerpop/blob/master/gremlin-
> core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/JavaTranslator.java
> https://github.com/apache/tinkerpop/blob/master/gremlin-
> groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> GroovyTranslator.java
> https://github.com/apache/tinkerpop/blob/master/gremlin-
> python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> PythonTranslator.java
>
> I think that non-JVM languages could send the Gremlin string from the test
> to Gremlin Server as a script first and return the bytecode as JSON. Then
> it could use a "Translator" to then parse it into it's native language.
> Shouldn't we just build a CSharpTranslator similar to these? does that make
> sense?
>
>
> On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com
> > wrote:
>
> > I've been looking over the current test scenarios on the TINKERPOP-1784
> > branch, with the C# GLV test suite in mind, and oh man that's an
> impressive
> > amount of scenarios!
> >
> > So far, I have a suggestion.
> >
> > Instead of
> >
> >   Scenario: g_V_fold_countXlocalX
> >     Given the modern graph
> >     And the traversal of
> >       """
> >       g.V().fold().count(Scope.local)
> >       """
> >
> > We could use
> >
> >   Scenario: g_V_fold_countXlocalX
> >     Given the modern graph
> >     And parameter 1 being an enum
> >       """
> >       Scope.local
> >       """
> >     And the traversal of
> >       """
> >       g.V().fold().count(param1)
> >       """
> >
> > That way we can exclude enums from possible parameter values, narrowing
> > down the amount of types of parameters allowed, ie: const numeric, const
> > strings (wrapped in double quotes), traversals (starting with `__`) ,
> ...,
> > making parsing the traversal a little easier.
> >
> >
> > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > Just a quick update on this item - it continues........
> > >
> > > Went down a bad path earlier in the week and ended up shelving a lot of
> > > work - ended up just coded into a corner. Anyway, I've not really
> changed
> > > much in the implementation, but I've still not reached a point where
> the
> > > addition of a new .feature files comes without meeting some new type of
> > > assertion that has to be dealt with. This ends up slowing down progress
> > on
> > > porting over the tests. Please feel free to peruse the latest changes
> on
> > > the branch and let me know if there's any feedback.
> > >
> > >
> > >
> > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> spmallette@gmail.com
> > >
> > > wrote:
> > >
> > > > I did some pretty heavy refactoring to the python test logic
> (altered a
> > > > bit by some revision of the gherkin feature file language) and the
> > result
> > > > is a much more simplified test logic file:
> > > >
> > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > > >
> > > > About 120 lines of code (down from about 170). If you include the
> test
> > > > logic "setup" file:
> > > >
> > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > gremlin-python/src/main/jython/radish/terrain.py
> > > >
> > > > we end up with about 250 lines of test logic total (take out the
> > > > comment/license and we're probably well under 200 total - not too
> > bad). I
> > > > think I'm closing in on the end to infrastructure building here so
> the
> > > > basic framework is getting close to final at this point I believe.
> I'll
> > > > keep scanning the tests looking for other types of assertions that
> I've
> > > not
> > > > yet covered, but it's getting pretty solid I think. Hopefully, there
> > > won't
> > > > need to be too many more lines of code needed to express the test
> logic
> > > as
> > > > I like how things are looking right now.
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > > jorgebaygondra@gmail.com> wrote:
> > > >
> > > >> Great progress! I like how you avoided using ids, even if it adds
> some
> > > >> complexity to the transformation required.
> > > >>
> > > >> The Python step definitions are still quite simple, I think it would
> > be
> > > >> mostly the same on the rest of the languages.
> > > >>
> > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > spmallette@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Just another update on progress with the test suite. The language
> of
> > > the
> > > >> > .feature files is getting slightly more complex as I try to
> > translate
> > > >> more
> > > >> > and more of our Java process suite tests into the language of the
> > > >> gherkin
> > > >> > files. You can see here where I've added the ability to include
> > > >> parameters:
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-test/features/filter/Has.feature#L33
> > > >> >
> > > >> > I also came up with a method of asserting edges (didn't want to
> rely
> > > on
> > > >> ids
> > > >> > as it makes the gherkin harder to read, plus i didn't want to
> assume
> > > >> > TinkerGraph identifiers in case these tests were every used with
> > some
> > > >> other
> > > >> > graph database that didn't use longs):
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > > >> >
> > > >> > for all those additions (and others) the logic required by the GLV
> > to
> > > >> > process these tests has stayed surprisingly simple:
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > > >> >
> > > >> > There's a fair bit of regex/string manipulation involved there,
> but
> > > it's
> > > >> > processing strings from the feature file so that's the nature of
> it
> > I
> > > >> > suppose. I think I'm of the mind that I want to port all of the
> > tests
> > > to
> > > >> > feature files, so I wrote this unit test to help validate that
> none
> > > were
> > > >> > missed:
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > > >> > structure/FeatureCoverageTest.java
> > > >> >
> > > >> > I don't know that we have to get to 100% porting right away, but I
> > > think
> > > >> > that once we have these gherkin files written they not only become
> > the
> > > >> > basis for our current GLV testing work, but we might be able to
> > simply
> > > >> use
> > > >> > them for testing the Java stuff as well - that would rid us of
> > having
> > > >> the
> > > >> > test code duplication. It also sets us up with a portable body of
> > > tests
> > > >> > that can be re-used in TinkerPop 4.x.
> > > >> >
> > > >> > I'm open to suggestions if anyone has any.
> > > >> >
> > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > > >> > jorgebaygondra@gmail.com
> > > >> > > wrote:
> > > >> >
> > > >> > > I was able to build a proof of concept for a Gherkin-based test
> > > >> runner in
> > > >> > > C#, that takes the proposed count and select features
> > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > >> > > gremlin-test/features/map>
> > > >> > > and runs them using C# step definitions.
> > > >> > >
> > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > > >> herkin-dotnet>
> > > >> > > from
> > > >> > > cucumber, there isn't a release of the parser with .NET Core
> > support
> > > >> so
> > > >> > > I've
> > > >> > > asked them to release one
> > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there
> is
> > no
> > > >> > > limitation from their source files). If they are not able to
> > release
> > > >> it
> > > >> > in
> > > >> > > the next few days, we can implement our own as it should be
> pretty
> > > >> > straight
> > > >> > > forward.
> > > >> > >
> > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > > >> spmallette@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Thanks for the update. I'm trying to keep the test language as
> > > >> simple
> > > >> > as
> > > >> > > > possible so that we don't need an overly complicated test
> > > >> > implementation.
> > > >> > > > Hopefully that will help make the .NET approach as easy as
> > > possible.
> > > >> > > >
> > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > >> > > > jorgebaygondra@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > I've been looking into Gherkin support for .NET: SpecFlow,
> the
> > > >> > cucumber
> > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > > >> > > > cucumber-implementations
> > > >> > > > > >,
> > > >> > > > > does not support .NET Core platform (we use .NET Core build
> > > tools
> > > >> for
> > > >> > > the
> > > >> > > > > .NET GLV) and only supports .NET Framework.
> > > >> > > > >
> > > >> > > > > From what I can see <https://github.com/techtalk/S
> > > >> pecFlow/projects/2
> > > >> > >,
> > > >> > > > > .NET
> > > >> > > > > Core support on SpecFlow is coming at a very slow pace and
> we
> > > >> > shouldn't
> > > >> > > > > expect to land any time soon (there were some design
> decisions
> > > in
> > > >> > > > SpecFlow
> > > >> > > > > library that makes supporting other platforms non-trivial,
> > like
> > > >> > > requiring
> > > >> > > > > code gen).
> > > >> > > > >
> > > >> > > > > The alternative would be to implement our own harness to
> > support
> > > >> it:
> > > >> > > > from a
> > > >> > > > > xunit test, look for certain types and parse the
> annotations,
> > > and
> > > >> > > execute
> > > >> > > > > them using the Gherkin features.
> > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > > >> > > > > I'll continue looking into this option and try to understand
> > the
> > > >> > effort
> > > >> > > > > required...
> > > >> > > > >
> > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > >> > > > > jorgebaygondra@gmail.com>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Nice! Gherkin will make our lives easier with a growing
> > number
> > > >> of
> > > >> > > GLVs.
> > > >> > > > > >
> > > >> > > > > > We should find a way to define the different features
> > > supported
> > > >> by
> > > >> > > each
> > > >> > > > > > GLV, as it's reasonable to have different maturity levels
> > per
> > > >> GLV
> > > >> > > (ie:
> > > >> > > > > > lambdas support, traversal strategy, ...). I don't know if
> > it
> > > >> will
> > > >> > be
> > > >> > > > > > beneficial to do it in the Gherkin files or within each
> GLV
> > > >> > > > > implementation.
> > > >> > > > > > Also, we should consider the process of rolling out a new
> > > >> method /
> > > >> > > > class
> > > >> > > > > > in the java implementation, how that could affect each
> GLV.
> > > >> > > > > >
> > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > >> > > > spmallette@gmail.com
> > > >> > > > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > > eval(). I
> > > >> > > guess
> > > >> > > > > the
> > > >> > > > > >> point is that if the language can support some way of
> > taking
> > > >> the
> > > >> > > > string
> > > >> > > > > >> value and turning it automatically into a traversal in
> that
> > > >> GLVs
> > > >> > > style
> > > >> > > > > >> then
> > > >> > > > > >> we should do that.
> > > >> > > > > >>
> > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > > >> <me...@gremlin.guru>
> > > >> > > > > wrote:
> > > >> > > > > >>
> > > >> > > > > >> > For unparameterized queries it can probably be as easy
> > as:
> > > >> > > > > >> >
> > > >> > > > > >> > @given("the traversal of")
> > > >> > > > > >> > def translate_traversal(step):
> > > >> > > > > >> >     g = step.context.g
> > > >> > > > > >> >     step.context.traversal = eval(step.text)
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > Cheers,
> > > >> > > > > >> > Daniel
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > > >> <me@gremlin.guru
> > > >> > >
> > > >> > > > > >> wrote:
> > > >> > > > > >> >
> > > >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin
> > for
> > > >> > years,
> > > >> > > > > but I
> > > >> > > > > >> > > really like the BDD approach.
> > > >> > > > > >> > >
> > > >> > > > > >> > > and then you can look at the GLV Gremlin translations
> > > >> > > specifically
> > > >> > > > > >> here:
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/blob/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-python/src/main/jython/
> > > radish/count_features_step.py#
> > > >> > > L34-L46
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> > > This part is the only thing that looks weird to me.
> > > You're
> > > >> > > > basically
> > > >> > > > > >> > > writing every query twice; is there really no easier
> > way
> > > >> to do
> > > >> > > > that?
> > > >> > > > > >> > >
> > > >> > > > > >> > > Cheers,
> > > >> > > > > >> > > Daniel
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > >> > > > > >> spmallette@gmail.com>
> > > >> > > > > >> > > wrote:
> > > >> > > > > >> > >
> > > >> > > > > >> > >> I've brought this issue up in the past and had
> > suggested
> > > >> some
> > > >> > > > > >> options I
> > > >> > > > > >> > >> had
> > > >> > > > > >> > >> in mind but now I've finally put the basics of those
> > > >> ideas in
> > > >> > > > place
> > > >> > > > > >> so I
> > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that the
> > issue
> > > at
> > > >> > hand
> > > >> > > > is
> > > >> > > > > >> that
> > > >> > > > > >> > we
> > > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> > > bound
> > > >> to
> > > >> > > the
> > > >> > > > > >> JVM. We
> > > >> > > > > >> > >> have some tricks that let us test gremlin-python
> with
> > it
> > > >> but
> > > >> > > > those
> > > >> > > > > >> > tricks
> > > >> > > > > >> > >> won't work for every language and we now have the
> > first
> > > >> > > language
> > > >> > > > in
> > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> > > won't
> > > >> > > > support
> > > >> > > > > it
> > > >> > > > > >> > >> (yes,
> > > >> > > > > >> > >> i know that gremlin-javascript can run on the jvm
> but
> > > >> there
> > > >> > are
> > > >> > > > > >> issues
> > > >> > > > > >> > >> with
> > > >> > > > > >> > >> getting it all to work with the test framework that
> > make
> > > >> it
> > > >> > > > unduly
> > > >> > > > > >> > >> complicated).
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> On other threads I offered the idea that we look to
> > use
> > > >> > Gherkin
> > > >> > > > to
> > > >> > > > > >> write
> > > >> > > > > >> > >> general Gremlin test specifications, which then
> could
> > be
> > > >> read
> > > >> > > and
> > > >> > > > > >> > >> processed
> > > >> > > > > >> > >> by the wide variety of test frameworks that can read
> > > that
> > > >> > > format
> > > >> > > > -
> > > >> > > > > >> there
> > > >> > > > > >> > >> tend to be Gherkin processors in just about every
> > > >> language -
> > > >> > > for
> > > >> > > > > >> > example,
> > > >> > > > > >> > >> see:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://cucumber.io/
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> I just created this issue:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://issues.apache.org/
> jira/browse/TINKERPOP-1784
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> and pushed this branch:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/
> > tinkerpop/tree/TINKERPOP-1784
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> which demonstrates how this works with
> gremlin-python.
> > > The
> > > >> > > basic
> > > >> > > > > >> anatomy
> > > >> > > > > >> > >> of
> > > >> > > > > >> > >> this setup involves this new directory in
> > gremlin-test:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/tree/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-test/features
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> It contains the Gherkin .features files. These are
> the
> > > >> test
> > > >> > > > > >> > >> specifications.
> > > >> > > > > >> > >> They are written using gremlin-java as the "model"
> > > >> language.
> > > >> > > GLVs
> > > >> > > > > >> will
> > > >> > > > > >> > >> then
> > > >> > > > > >> > >> need to write some infrastructure to process these
> > > Gherkin
> > > >> > > files.
> > > >> > > > > The
> > > >> > > > > >> > key
> > > >> > > > > >> > >> to making this "easy" to implement will lie in our
> > > >> abiilty to
> > > >> > > > keep
> > > >> > > > > >> the
> > > >> > > > > >> > >> assertions we want to have relatively simple. The
> more
> > > >> > > simplistic
> > > >> > > > > the
> > > >> > > > > >> > >> language in the Gherkin .feature files the easier
> the
> > > job
> > > >> it
> > > >> > > will
> > > >> > > > > be
> > > >> > > > > >> for
> > > >> > > > > >> > >> GLVs to build their infrastructure. Of course, once
> > that
> > > >> > > > > >> infrastructure
> > > >> > > > > >> > is
> > > >> > > > > >> > >> in place, the GLV developer just has to write the
> GLV
> > > >> version
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > >> > >> Gremlin specified in the .feature file. So you can
> > look
> > > at
> > > >> > all
> > > >> > > > the
> > > >> > > > > >> > >> "infrastructure" code here in this pair of files:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/tree/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> translations
> > > >> > > > specifically
> > > >> > > > > >> here:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/blob/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-python/src/main/jython/
> > > radish/count_features_step.py#
> > > >> > > L34-L46
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> I think this approach works pretty well and solves
> our
> > > >> > general
> > > >> > > > > >> problems
> > > >> > > > > >> > >> for
> > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > implementing
> > > >> the
> > > >> > > > > >> > >> "infrastructure" but after that new Gremlin tests
> > added
> > > to
> > > >> > > > .feature
> > > >> > > > > >> > files
> > > >> > > > > >> > >> just need to translated in the GLV. I suppose we
> could
> > > >> > > > "automate" a
> > > >> > > > > >> good
> > > >> > > > > >> > >> portion of the translation with reflection of some
> > sort.
> > > >> > > Anything
> > > >> > > > > >> else
> > > >> > > > > >> > >> could just be handled manually.
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> Not sure if we need to use this new model to wholly
> > > >> replace
> > > >> > the
> > > >> > > > old
> > > >> > > > > >> one.
> > > >> > > > > >> > >> The process test suite has its place in helping
> graph
> > > >> > database
> > > >> > > > > >> providers
> > > >> > > > > >> > >> test their stuff. I also imagine that introducing
> this
> > > >> > approach
> > > >> > > > in
> > > >> > > > > >> that
> > > >> > > > > >> > >> context would create a breaking change which we
> would
> > > then
> > > >> > need
> > > >> > > > to
> > > >> > > > > >> push
> > > >> > > > > >> > >> off
> > > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to think,
> but
> > > for
> > > >> now
> > > >> > > it
> > > >> > > > > >> might
> > > >> > > > > >> > not
> > > >> > > > > >> > >> be best to conflate the two and just treat them as
> > > >> separate
> > > >> > > > aspects
> > > >> > > > > >> of
> > > >> > > > > >> > the
> > > >> > > > > >> > >> test suite.
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> Anyway - it's important we settle on an approach to
> > > >> testing
> > > >> > as
> > > >> > > we
> > > >> > > > > >> really
> > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> > > without
> > > >> > > getting
> > > >> > > > > the
> > > >> > > > > >> > test
> > > >> > > > > >> > >> suite solid. please yell if you have any ideas or
> > > >> feedback on
> > > >> > > > this
> > > >> > > > > >> > >> approach.
> > > >> > > > > >> > >>
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

I can't help thinking that perhaps non-JVM languages need to leverage
bytecode to make traversal building easier. We already have an established
pattern for converting bytecode into Traversals and while it is different
for each language, it's generally governed by the Translator interface.
Note that we already do this for Java, Groovy, and Python and the code
isn't too crazy - just 200 lines or so:

https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/JavaTranslator.java
https://github.com/apache/tinkerpop/blob/master/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/GroovyTranslator.java
https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/PythonTranslator.java

I think that non-JVM languages could send the Gremlin string from the test
to Gremlin Server as a script first and return the bytecode as JSON. Then
it could use a "Translator" to then parse it into it's native language.
Shouldn't we just build a CSharpTranslator similar to these? does that make
sense?


On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> wrote:

> I've been looking over the current test scenarios on the TINKERPOP-1784
> branch, with the C# GLV test suite in mind, and oh man that's an impressive
> amount of scenarios!
>
> So far, I have a suggestion.
>
> Instead of
>
>   Scenario: g_V_fold_countXlocalX
>     Given the modern graph
>     And the traversal of
>       """
>       g.V().fold().count(Scope.local)
>       """
>
> We could use
>
>   Scenario: g_V_fold_countXlocalX
>     Given the modern graph
>     And parameter 1 being an enum
>       """
>       Scope.local
>       """
>     And the traversal of
>       """
>       g.V().fold().count(param1)
>       """
>
> That way we can exclude enums from possible parameter values, narrowing
> down the amount of types of parameters allowed, ie: const numeric, const
> strings (wrapped in double quotes), traversals (starting with `__`) , ...,
> making parsing the traversal a little easier.
>
>
> On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > Just a quick update on this item - it continues........
> >
> > Went down a bad path earlier in the week and ended up shelving a lot of
> > work - ended up just coded into a corner. Anyway, I've not really changed
> > much in the implementation, but I've still not reached a point where the
> > addition of a new .feature files comes without meeting some new type of
> > assertion that has to be dealt with. This ends up slowing down progress
> on
> > porting over the tests. Please feel free to peruse the latest changes on
> > the branch and let me know if there's any feedback.
> >
> >
> >
> > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <spmallette@gmail.com
> >
> > wrote:
> >
> > > I did some pretty heavy refactoring to the python test logic (altered a
> > > bit by some revision of the gherkin feature file language) and the
> result
> > > is a much more simplified test logic file:
> > >
> > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > gremlin-python/src/main/jython/radish/feature_steps.py
> > >
> > > About 120 lines of code (down from about 170). If you include the test
> > > logic "setup" file:
> > >
> > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > gremlin-python/src/main/jython/radish/terrain.py
> > >
> > > we end up with about 250 lines of test logic total (take out the
> > > comment/license and we're probably well under 200 total - not too
> bad). I
> > > think I'm closing in on the end to infrastructure building here so the
> > > basic framework is getting close to final at this point I believe. I'll
> > > keep scanning the tests looking for other types of assertions that I've
> > not
> > > yet covered, but it's getting pretty solid I think. Hopefully, there
> > won't
> > > need to be too many more lines of code needed to express the test logic
> > as
> > > I like how things are looking right now.
> > >
> > >
> > >
> > >
> > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > jorgebaygondra@gmail.com> wrote:
> > >
> > >> Great progress! I like how you avoided using ids, even if it adds some
> > >> complexity to the transformation required.
> > >>
> > >> The Python step definitions are still quite simple, I think it would
> be
> > >> mostly the same on the rest of the languages.
> > >>
> > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> spmallette@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Just another update on progress with the test suite. The language of
> > the
> > >> > .feature files is getting slightly more complex as I try to
> translate
> > >> more
> > >> > and more of our Java process suite tests into the language of the
> > >> gherkin
> > >> > files. You can see here where I've added the ability to include
> > >> parameters:
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-test/features/filter/Has.feature#L33
> > >> >
> > >> > I also came up with a method of asserting edges (didn't want to rely
> > on
> > >> ids
> > >> > as it makes the gherkin harder to read, plus i didn't want to assume
> > >> > TinkerGraph identifiers in case these tests were every used with
> some
> > >> other
> > >> > graph database that didn't use longs):
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > >> >
> > >> > for all those additions (and others) the logic required by the GLV
> to
> > >> > process these tests has stayed surprisingly simple:
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > >> >
> > >> > There's a fair bit of regex/string manipulation involved there, but
> > it's
> > >> > processing strings from the feature file so that's the nature of it
> I
> > >> > suppose. I think I'm of the mind that I want to port all of the
> tests
> > to
> > >> > feature files, so I wrote this unit test to help validate that none
> > were
> > >> > missed:
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > >> > structure/FeatureCoverageTest.java
> > >> >
> > >> > I don't know that we have to get to 100% porting right away, but I
> > think
> > >> > that once we have these gherkin files written they not only become
> the
> > >> > basis for our current GLV testing work, but we might be able to
> simply
> > >> use
> > >> > them for testing the Java stuff as well - that would rid us of
> having
> > >> the
> > >> > test code duplication. It also sets us up with a portable body of
> > tests
> > >> > that can be re-used in TinkerPop 4.x.
> > >> >
> > >> > I'm open to suggestions if anyone has any.
> > >> >
> > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > >> > jorgebaygondra@gmail.com
> > >> > > wrote:
> > >> >
> > >> > > I was able to build a proof of concept for a Gherkin-based test
> > >> runner in
> > >> > > C#, that takes the proposed count and select features
> > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > >> > > gremlin-test/features/map>
> > >> > > and runs them using C# step definitions.
> > >> > >
> > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > >> herkin-dotnet>
> > >> > > from
> > >> > > cucumber, there isn't a release of the parser with .NET Core
> support
> > >> so
> > >> > > I've
> > >> > > asked them to release one
> > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is
> no
> > >> > > limitation from their source files). If they are not able to
> release
> > >> it
> > >> > in
> > >> > > the next few days, we can implement our own as it should be pretty
> > >> > straight
> > >> > > forward.
> > >> > >
> > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > >> spmallette@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Thanks for the update. I'm trying to keep the test language as
> > >> simple
> > >> > as
> > >> > > > possible so that we don't need an overly complicated test
> > >> > implementation.
> > >> > > > Hopefully that will help make the .NET approach as easy as
> > possible.
> > >> > > >
> > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > >> > > > jorgebaygondra@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> > >> > cucumber
> > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > >> > > > cucumber-implementations
> > >> > > > > >,
> > >> > > > > does not support .NET Core platform (we use .NET Core build
> > tools
> > >> for
> > >> > > the
> > >> > > > > .NET GLV) and only supports .NET Framework.
> > >> > > > >
> > >> > > > > From what I can see <https://github.com/techtalk/S
> > >> pecFlow/projects/2
> > >> > >,
> > >> > > > > .NET
> > >> > > > > Core support on SpecFlow is coming at a very slow pace and we
> > >> > shouldn't
> > >> > > > > expect to land any time soon (there were some design decisions
> > in
> > >> > > > SpecFlow
> > >> > > > > library that makes supporting other platforms non-trivial,
> like
> > >> > > requiring
> > >> > > > > code gen).
> > >> > > > >
> > >> > > > > The alternative would be to implement our own harness to
> support
> > >> it:
> > >> > > > from a
> > >> > > > > xunit test, look for certain types and parse the annotations,
> > and
> > >> > > execute
> > >> > > > > them using the Gherkin features.
> > >> > > > > There is a .NET cross-platform Gherkin parser:
> > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > >> > > > > I'll continue looking into this option and try to understand
> the
> > >> > effort
> > >> > > > > required...
> > >> > > > >
> > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > >> > > > > jorgebaygondra@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Nice! Gherkin will make our lives easier with a growing
> number
> > >> of
> > >> > > GLVs.
> > >> > > > > >
> > >> > > > > > We should find a way to define the different features
> > supported
> > >> by
> > >> > > each
> > >> > > > > > GLV, as it's reasonable to have different maturity levels
> per
> > >> GLV
> > >> > > (ie:
> > >> > > > > > lambdas support, traversal strategy, ...). I don't know if
> it
> > >> will
> > >> > be
> > >> > > > > > beneficial to do it in the Gherkin files or within each GLV
> > >> > > > > implementation.
> > >> > > > > > Also, we should consider the process of rolling out a new
> > >> method /
> > >> > > > class
> > >> > > > > > in the java implementation, how that could affect each GLV.
> > >> > > > > >
> > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > >> > > > spmallette@gmail.com
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > eval(). I
> > >> > > guess
> > >> > > > > the
> > >> > > > > >> point is that if the language can support some way of
> taking
> > >> the
> > >> > > > string
> > >> > > > > >> value and turning it automatically into a traversal in that
> > >> GLVs
> > >> > > style
> > >> > > > > >> then
> > >> > > > > >> we should do that.
> > >> > > > > >>
> > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > >> <me...@gremlin.guru>
> > >> > > > > wrote:
> > >> > > > > >>
> > >> > > > > >> > For unparameterized queries it can probably be as easy
> as:
> > >> > > > > >> >
> > >> > > > > >> > @given("the traversal of")
> > >> > > > > >> > def translate_traversal(step):
> > >> > > > > >> >     g = step.context.g
> > >> > > > > >> >     step.context.traversal = eval(step.text)
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > Cheers,
> > >> > > > > >> > Daniel
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > >> <me@gremlin.guru
> > >> > >
> > >> > > > > >> wrote:
> > >> > > > > >> >
> > >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin
> for
> > >> > years,
> > >> > > > > but I
> > >> > > > > >> > > really like the BDD approach.
> > >> > > > > >> > >
> > >> > > > > >> > > and then you can look at the GLV Gremlin translations
> > >> > > specifically
> > >> > > > > >> here:
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-python/src/main/jython/
> > radish/count_features_step.py#
> > >> > > L34-L46
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > This part is the only thing that looks weird to me.
> > You're
> > >> > > > basically
> > >> > > > > >> > > writing every query twice; is there really no easier
> way
> > >> to do
> > >> > > > that?
> > >> > > > > >> > >
> > >> > > > > >> > > Cheers,
> > >> > > > > >> > > Daniel
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > >> > > > > >> spmallette@gmail.com>
> > >> > > > > >> > > wrote:
> > >> > > > > >> > >
> > >> > > > > >> > >> I've brought this issue up in the past and had
> suggested
> > >> some
> > >> > > > > >> options I
> > >> > > > > >> > >> had
> > >> > > > > >> > >> in mind but now I've finally put the basics of those
> > >> ideas in
> > >> > > > place
> > >> > > > > >> so I
> > >> > > > > >> > >> figured I'd start a fresh thread. Recall that the
> issue
> > at
> > >> > hand
> > >> > > > is
> > >> > > > > >> that
> > >> > > > > >> > we
> > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> > bound
> > >> to
> > >> > > the
> > >> > > > > >> JVM. We
> > >> > > > > >> > >> have some tricks that let us test gremlin-python with
> it
> > >> but
> > >> > > > those
> > >> > > > > >> > tricks
> > >> > > > > >> > >> won't work for every language and we now have the
> first
> > >> > > language
> > >> > > > in
> > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> > won't
> > >> > > > support
> > >> > > > > it
> > >> > > > > >> > >> (yes,
> > >> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
> > >> there
> > >> > are
> > >> > > > > >> issues
> > >> > > > > >> > >> with
> > >> > > > > >> > >> getting it all to work with the test framework that
> make
> > >> it
> > >> > > > unduly
> > >> > > > > >> > >> complicated).
> > >> > > > > >> > >>
> > >> > > > > >> > >> On other threads I offered the idea that we look to
> use
> > >> > Gherkin
> > >> > > > to
> > >> > > > > >> write
> > >> > > > > >> > >> general Gremlin test specifications, which then could
> be
> > >> read
> > >> > > and
> > >> > > > > >> > >> processed
> > >> > > > > >> > >> by the wide variety of test frameworks that can read
> > that
> > >> > > format
> > >> > > > -
> > >> > > > > >> there
> > >> > > > > >> > >> tend to be Gherkin processors in just about every
> > >> language -
> > >> > > for
> > >> > > > > >> > example,
> > >> > > > > >> > >> see:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://cucumber.io/
> > >> > > > > >> > >>
> > >> > > > > >> > >> I just created this issue:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > >> > > > > >> > >>
> > >> > > > > >> > >> and pushed this branch:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/
> tinkerpop/tree/TINKERPOP-1784
> > >> > > > > >> > >>
> > >> > > > > >> > >> which demonstrates how this works with gremlin-python.
> > The
> > >> > > basic
> > >> > > > > >> anatomy
> > >> > > > > >> > >> of
> > >> > > > > >> > >> this setup involves this new directory in
> gremlin-test:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-test/features
> > >> > > > > >> > >>
> > >> > > > > >> > >> It contains the Gherkin .features files. These are the
> > >> test
> > >> > > > > >> > >> specifications.
> > >> > > > > >> > >> They are written using gremlin-java as the "model"
> > >> language.
> > >> > > GLVs
> > >> > > > > >> will
> > >> > > > > >> > >> then
> > >> > > > > >> > >> need to write some infrastructure to process these
> > Gherkin
> > >> > > files.
> > >> > > > > The
> > >> > > > > >> > key
> > >> > > > > >> > >> to making this "easy" to implement will lie in our
> > >> abiilty to
> > >> > > > keep
> > >> > > > > >> the
> > >> > > > > >> > >> assertions we want to have relatively simple. The more
> > >> > > simplistic
> > >> > > > > the
> > >> > > > > >> > >> language in the Gherkin .feature files the easier the
> > job
> > >> it
> > >> > > will
> > >> > > > > be
> > >> > > > > >> for
> > >> > > > > >> > >> GLVs to build their infrastructure. Of course, once
> that
> > >> > > > > >> infrastructure
> > >> > > > > >> > is
> > >> > > > > >> > >> in place, the GLV developer just has to write the GLV
> > >> version
> > >> > > of
> > >> > > > > the
> > >> > > > > >> > >> Gremlin specified in the .feature file. So you can
> look
> > at
> > >> > all
> > >> > > > the
> > >> > > > > >> > >> "infrastructure" code here in this pair of files:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-python/src/main/jython/radish
> > >> > > > > >> > >>
> > >> > > > > >> > >> and then you can look at the GLV Gremlin translations
> > >> > > > specifically
> > >> > > > > >> here:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-python/src/main/jython/
> > radish/count_features_step.py#
> > >> > > L34-L46
> > >> > > > > >> > >>
> > >> > > > > >> > >> I think this approach works pretty well and solves our
> > >> > general
> > >> > > > > >> problems
> > >> > > > > >> > >> for
> > >> > > > > >> > >> GLV testing. There is some pain up front in
> implementing
> > >> the
> > >> > > > > >> > >> "infrastructure" but after that new Gremlin tests
> added
> > to
> > >> > > > .feature
> > >> > > > > >> > files
> > >> > > > > >> > >> just need to translated in the GLV. I suppose we could
> > >> > > > "automate" a
> > >> > > > > >> good
> > >> > > > > >> > >> portion of the translation with reflection of some
> sort.
> > >> > > Anything
> > >> > > > > >> else
> > >> > > > > >> > >> could just be handled manually.
> > >> > > > > >> > >>
> > >> > > > > >> > >> Not sure if we need to use this new model to wholly
> > >> replace
> > >> > the
> > >> > > > old
> > >> > > > > >> one.
> > >> > > > > >> > >> The process test suite has its place in helping graph
> > >> > database
> > >> > > > > >> providers
> > >> > > > > >> > >> test their stuff. I also imagine that introducing this
> > >> > approach
> > >> > > > in
> > >> > > > > >> that
> > >> > > > > >> > >> context would create a breaking change which we would
> > then
> > >> > need
> > >> > > > to
> > >> > > > > >> push
> > >> > > > > >> > >> off
> > >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to think, but
> > for
> > >> now
> > >> > > it
> > >> > > > > >> might
> > >> > > > > >> > not
> > >> > > > > >> > >> be best to conflate the two and just treat them as
> > >> separate
> > >> > > > aspects
> > >> > > > > >> of
> > >> > > > > >> > the
> > >> > > > > >> > >> test suite.
> > >> > > > > >> > >>
> > >> > > > > >> > >> Anyway - it's important we settle on an approach to
> > >> testing
> > >> > as
> > >> > > we
> > >> > > > > >> really
> > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> > without
> > >> > > getting
> > >> > > > > the
> > >> > > > > >> > test
> > >> > > > > >> > >> suite solid. please yell if you have any ideas or
> > >> feedback on
> > >> > > > this
> > >> > > > > >> > >> approach.
> > >> > > > > >> > >>
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

I've been looking over the current test scenarios on the TINKERPOP-1784
branch, with the C# GLV test suite in mind, and oh man that's an impressive
amount of scenarios!

So far, I have a suggestion.

Instead of

  Scenario: g_V_fold_countXlocalX
    Given the modern graph
    And the traversal of
      """
      g.V().fold().count(Scope.local)
      """

We could use

  Scenario: g_V_fold_countXlocalX
    Given the modern graph
    And parameter 1 being an enum
      """
      Scope.local
      """
    And the traversal of
      """
      g.V().fold().count(param1)
      """

That way we can exclude enums from possible parameter values, narrowing
down the amount of types of parameters allowed, ie: const numeric, const
strings (wrapped in double quotes), traversals (starting with `__`) , ...,
making parsing the traversal a little easier.


On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> Just a quick update on this item - it continues........
>
> Went down a bad path earlier in the week and ended up shelving a lot of
> work - ended up just coded into a corner. Anyway, I've not really changed
> much in the implementation, but I've still not reached a point where the
> addition of a new .feature files comes without meeting some new type of
> assertion that has to be dealt with. This ends up slowing down progress on
> porting over the tests. Please feel free to peruse the latest changes on
> the branch and let me know if there's any feedback.
>
>
>
> On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > I did some pretty heavy refactoring to the python test logic (altered a
> > bit by some revision of the gherkin feature file language) and the result
> > is a much more simplified test logic file:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/feature_steps.py
> >
> > About 120 lines of code (down from about 170). If you include the test
> > logic "setup" file:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/terrain.py
> >
> > we end up with about 250 lines of test logic total (take out the
> > comment/license and we're probably well under 200 total - not too bad). I
> > think I'm closing in on the end to infrastructure building here so the
> > basic framework is getting close to final at this point I believe. I'll
> > keep scanning the tests looking for other types of assertions that I've
> not
> > yet covered, but it's getting pretty solid I think. Hopefully, there
> won't
> > need to be too many more lines of code needed to express the test logic
> as
> > I like how things are looking right now.
> >
> >
> >
> >
> > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com> wrote:
> >
> >> Great progress! I like how you avoided using ids, even if it adds some
> >> complexity to the transformation required.
> >>
> >> The Python step definitions are still quite simple, I think it would be
> >> mostly the same on the rest of the languages.
> >>
> >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <spmallette@gmail.com
> >
> >> wrote:
> >>
> >> > Just another update on progress with the test suite. The language of
> the
> >> > .feature files is getting slightly more complex as I try to translate
> >> more
> >> > and more of our Java process suite tests into the language of the
> >> gherkin
> >> > files. You can see here where I've added the ability to include
> >> parameters:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/features/filter/Has.feature#L33
> >> >
> >> > I also came up with a method of asserting edges (didn't want to rely
> on
> >> ids
> >> > as it makes the gherkin harder to read, plus i didn't want to assume
> >> > TinkerGraph identifiers in case these tests were every used with some
> >> other
> >> > graph database that didn't use longs):
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/features/map/Vertex.feature#L105-L111
> >> >
> >> > for all those additions (and others) the logic required by the GLV to
> >> > process these tests has stayed surprisingly simple:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-python/src/main/jython/radish/feature_steps.py
> >> >
> >> > There's a fair bit of regex/string manipulation involved there, but
> it's
> >> > processing strings from the feature file so that's the nature of it I
> >> > suppose. I think I'm of the mind that I want to port all of the tests
> to
> >> > feature files, so I wrote this unit test to help validate that none
> were
> >> > missed:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> >> > structure/FeatureCoverageTest.java
> >> >
> >> > I don't know that we have to get to 100% porting right away, but I
> think
> >> > that once we have these gherkin files written they not only become the
> >> > basis for our current GLV testing work, but we might be able to simply
> >> use
> >> > them for testing the Java stuff as well - that would rid us of having
> >> the
> >> > test code duplication. It also sets us up with a portable body of
> tests
> >> > that can be re-used in TinkerPop 4.x.
> >> >
> >> > I'm open to suggestions if anyone has any.
> >> >
> >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> >> > jorgebaygondra@gmail.com
> >> > > wrote:
> >> >
> >> > > I was able to build a proof of concept for a Gherkin-based test
> >> runner in
> >> > > C#, that takes the proposed count and select features
> >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> >> > > gremlin-test/features/map>
> >> > > and runs them using C# step definitions.
> >> > >
> >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> >> herkin-dotnet>
> >> > > from
> >> > > cucumber, there isn't a release of the parser with .NET Core support
> >> so
> >> > > I've
> >> > > asked them to release one
> >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
> >> > > limitation from their source files). If they are not able to release
> >> it
> >> > in
> >> > > the next few days, we can implement our own as it should be pretty
> >> > straight
> >> > > forward.
> >> > >
> >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> >> spmallette@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Thanks for the update. I'm trying to keep the test language as
> >> simple
> >> > as
> >> > > > possible so that we don't need an overly complicated test
> >> > implementation.
> >> > > > Hopefully that will help make the .NET approach as easy as
> possible.
> >> > > >
> >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> >> > > > jorgebaygondra@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> >> > cucumber
> >> > > > > implementation for .NET <https://cucumber.io/docs#
> >> > > > cucumber-implementations
> >> > > > > >,
> >> > > > > does not support .NET Core platform (we use .NET Core build
> tools
> >> for
> >> > > the
> >> > > > > .NET GLV) and only supports .NET Framework.
> >> > > > >
> >> > > > > From what I can see <https://github.com/techtalk/S
> >> pecFlow/projects/2
> >> > >,
> >> > > > > .NET
> >> > > > > Core support on SpecFlow is coming at a very slow pace and we
> >> > shouldn't
> >> > > > > expect to land any time soon (there were some design decisions
> in
> >> > > > SpecFlow
> >> > > > > library that makes supporting other platforms non-trivial, like
> >> > > requiring
> >> > > > > code gen).
> >> > > > >
> >> > > > > The alternative would be to implement our own harness to support
> >> it:
> >> > > > from a
> >> > > > > xunit test, look for certain types and parse the annotations,
> and
> >> > > execute
> >> > > > > them using the Gherkin features.
> >> > > > > There is a .NET cross-platform Gherkin parser:
> >> > > > > https://github.com/cucumber/gherkin-dotnet
> >> > > > > I'll continue looking into this option and try to understand the
> >> > effort
> >> > > > > required...
> >> > > > >
> >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> >> > > > > jorgebaygondra@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Nice! Gherkin will make our lives easier with a growing number
> >> of
> >> > > GLVs.
> >> > > > > >
> >> > > > > > We should find a way to define the different features
> supported
> >> by
> >> > > each
> >> > > > > > GLV, as it's reasonable to have different maturity levels per
> >> GLV
> >> > > (ie:
> >> > > > > > lambdas support, traversal strategy, ...). I don't know if it
> >> will
> >> > be
> >> > > > > > beneficial to do it in the Gherkin files or within each GLV
> >> > > > > implementation.
> >> > > > > > Also, we should consider the process of rolling out a new
> >> method /
> >> > > > class
> >> > > > > > in the java implementation, how that could affect each GLV.
> >> > > > > >
> >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> >> > > > spmallette@gmail.com
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> that's what i meant by "reflection" or as you suggest
> eval(). I
> >> > > guess
> >> > > > > the
> >> > > > > >> point is that if the language can support some way of taking
> >> the
> >> > > > string
> >> > > > > >> value and turning it automatically into a traversal in that
> >> GLVs
> >> > > style
> >> > > > > >> then
> >> > > > > >> we should do that.
> >> > > > > >>
> >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> >> <me...@gremlin.guru>
> >> > > > > wrote:
> >> > > > > >>
> >> > > > > >> > For unparameterized queries it can probably be as easy as:
> >> > > > > >> >
> >> > > > > >> > @given("the traversal of")
> >> > > > > >> > def translate_traversal(step):
> >> > > > > >> >     g = step.context.g
> >> > > > > >> >     step.context.traversal = eval(step.text)
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > Cheers,
> >> > > > > >> > Daniel
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> >> <me@gremlin.guru
> >> > >
> >> > > > > >> wrote:
> >> > > > > >> >
> >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
> >> > years,
> >> > > > > but I
> >> > > > > >> > > really like the BDD approach.
> >> > > > > >> > >
> >> > > > > >> > > and then you can look at the GLV Gremlin translations
> >> > > specifically
> >> > > > > >> here:
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/
> radish/count_features_step.py#
> >> > > L34-L46
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > This part is the only thing that looks weird to me.
> You're
> >> > > > basically
> >> > > > > >> > > writing every query twice; is there really no easier way
> >> to do
> >> > > > that?
> >> > > > > >> > >
> >> > > > > >> > > Cheers,
> >> > > > > >> > > Daniel
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> >> > > > > >> spmallette@gmail.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > >
> >> > > > > >> > >> I've brought this issue up in the past and had suggested
> >> some
> >> > > > > >> options I
> >> > > > > >> > >> had
> >> > > > > >> > >> in mind but now I've finally put the basics of those
> >> ideas in
> >> > > > place
> >> > > > > >> so I
> >> > > > > >> > >> figured I'd start a fresh thread. Recall that the issue
> at
> >> > hand
> >> > > > is
> >> > > > > >> that
> >> > > > > >> > we
> >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> bound
> >> to
> >> > > the
> >> > > > > >> JVM. We
> >> > > > > >> > >> have some tricks that let us test gremlin-python with it
> >> but
> >> > > > those
> >> > > > > >> > tricks
> >> > > > > >> > >> won't work for every language and we now have the first
> >> > > language
> >> > > > in
> >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> won't
> >> > > > support
> >> > > > > it
> >> > > > > >> > >> (yes,
> >> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
> >> there
> >> > are
> >> > > > > >> issues
> >> > > > > >> > >> with
> >> > > > > >> > >> getting it all to work with the test framework that make
> >> it
> >> > > > unduly
> >> > > > > >> > >> complicated).
> >> > > > > >> > >>
> >> > > > > >> > >> On other threads I offered the idea that we look to use
> >> > Gherkin
> >> > > > to
> >> > > > > >> write
> >> > > > > >> > >> general Gremlin test specifications, which then could be
> >> read
> >> > > and
> >> > > > > >> > >> processed
> >> > > > > >> > >> by the wide variety of test frameworks that can read
> that
> >> > > format
> >> > > > -
> >> > > > > >> there
> >> > > > > >> > >> tend to be Gherkin processors in just about every
> >> language -
> >> > > for
> >> > > > > >> > example,
> >> > > > > >> > >> see:
> >> > > > > >> > >>
> >> > > > > >> > >> https://cucumber.io/
> >> > > > > >> > >>
> >> > > > > >> > >> I just created this issue:
> >> > > > > >> > >>
> >> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> >> > > > > >> > >>
> >> > > > > >> > >> and pushed this branch:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> >> > > > > >> > >>
> >> > > > > >> > >> which demonstrates how this works with gremlin-python.
> The
> >> > > basic
> >> > > > > >> anatomy
> >> > > > > >> > >> of
> >> > > > > >> > >> this setup involves this new directory in gremlin-test:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-test/features
> >> > > > > >> > >>
> >> > > > > >> > >> It contains the Gherkin .features files. These are the
> >> test
> >> > > > > >> > >> specifications.
> >> > > > > >> > >> They are written using gremlin-java as the "model"
> >> language.
> >> > > GLVs
> >> > > > > >> will
> >> > > > > >> > >> then
> >> > > > > >> > >> need to write some infrastructure to process these
> Gherkin
> >> > > files.
> >> > > > > The
> >> > > > > >> > key
> >> > > > > >> > >> to making this "easy" to implement will lie in our
> >> abiilty to
> >> > > > keep
> >> > > > > >> the
> >> > > > > >> > >> assertions we want to have relatively simple. The more
> >> > > simplistic
> >> > > > > the
> >> > > > > >> > >> language in the Gherkin .feature files the easier the
> job
> >> it
> >> > > will
> >> > > > > be
> >> > > > > >> for
> >> > > > > >> > >> GLVs to build their infrastructure. Of course, once that
> >> > > > > >> infrastructure
> >> > > > > >> > is
> >> > > > > >> > >> in place, the GLV developer just has to write the GLV
> >> version
> >> > > of
> >> > > > > the
> >> > > > > >> > >> Gremlin specified in the .feature file. So you can look
> at
> >> > all
> >> > > > the
> >> > > > > >> > >> "infrastructure" code here in this pair of files:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/radish
> >> > > > > >> > >>
> >> > > > > >> > >> and then you can look at the GLV Gremlin translations
> >> > > > specifically
> >> > > > > >> here:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/
> radish/count_features_step.py#
> >> > > L34-L46
> >> > > > > >> > >>
> >> > > > > >> > >> I think this approach works pretty well and solves our
> >> > general
> >> > > > > >> problems
> >> > > > > >> > >> for
> >> > > > > >> > >> GLV testing. There is some pain up front in implementing
> >> the
> >> > > > > >> > >> "infrastructure" but after that new Gremlin tests added
> to
> >> > > > .feature
> >> > > > > >> > files
> >> > > > > >> > >> just need to translated in the GLV. I suppose we could
> >> > > > "automate" a
> >> > > > > >> good
> >> > > > > >> > >> portion of the translation with reflection of some sort.
> >> > > Anything
> >> > > > > >> else
> >> > > > > >> > >> could just be handled manually.
> >> > > > > >> > >>
> >> > > > > >> > >> Not sure if we need to use this new model to wholly
> >> replace
> >> > the
> >> > > > old
> >> > > > > >> one.
> >> > > > > >> > >> The process test suite has its place in helping graph
> >> > database
> >> > > > > >> providers
> >> > > > > >> > >> test their stuff. I also imagine that introducing this
> >> > approach
> >> > > > in
> >> > > > > >> that
> >> > > > > >> > >> context would create a breaking change which we would
> then
> >> > need
> >> > > > to
> >> > > > > >> push
> >> > > > > >> > >> off
> >> > > > > >> > >> to 3.4.0.  I suppose that gives us time to think, but
> for
> >> now
> >> > > it
> >> > > > > >> might
> >> > > > > >> > not
> >> > > > > >> > >> be best to conflate the two and just treat them as
> >> separate
> >> > > > aspects
> >> > > > > >> of
> >> > > > > >> > the
> >> > > > > >> > >> test suite.
> >> > > > > >> > >>
> >> > > > > >> > >> Anyway - it's important we settle on an approach to
> >> testing
> >> > as
> >> > > we
> >> > > > > >> really
> >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> without
> >> > > getting
> >> > > > > the
> >> > > > > >> > test
> >> > > > > >> > >> suite solid. please yell if you have any ideas or
> >> feedback on
> >> > > > this
> >> > > > > >> > >> approach.
> >> > > > > >> > >>
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

Just a quick update on this item - it continues........

Went down a bad path earlier in the week and ended up shelving a lot of
work - ended up just coded into a corner. Anyway, I've not really changed
much in the implementation, but I've still not reached a point where the
addition of a new .feature files comes without meeting some new type of
assertion that has to be dealt with. This ends up slowing down progress on
porting over the tests. Please feel free to peruse the latest changes on
the branch and let me know if there's any feedback.



On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> I did some pretty heavy refactoring to the python test logic (altered a
> bit by some revision of the gherkin feature file language) and the result
> is a much more simplified test logic file:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/feature_steps.py
>
> About 120 lines of code (down from about 170). If you include the test
> logic "setup" file:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/terrain.py
>
> we end up with about 250 lines of test logic total (take out the
> comment/license and we're probably well under 200 total - not too bad). I
> think I'm closing in on the end to infrastructure building here so the
> basic framework is getting close to final at this point I believe. I'll
> keep scanning the tests looking for other types of assertions that I've not
> yet covered, but it's getting pretty solid I think. Hopefully, there won't
> need to be too many more lines of code needed to express the test logic as
> I like how things are looking right now.
>
>
>
>
> On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com> wrote:
>
>> Great progress! I like how you avoided using ids, even if it adds some
>> complexity to the transformation required.
>>
>> The Python step definitions are still quite simple, I think it would be
>> mostly the same on the rest of the languages.
>>
>> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <sp...@gmail.com>
>> wrote:
>>
>> > Just another update on progress with the test suite. The language of the
>> > .feature files is getting slightly more complex as I try to translate
>> more
>> > and more of our Java process suite tests into the language of the
>> gherkin
>> > files. You can see here where I've added the ability to include
>> parameters:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/features/filter/Has.feature#L33
>> >
>> > I also came up with a method of asserting edges (didn't want to rely on
>> ids
>> > as it makes the gherkin harder to read, plus i didn't want to assume
>> > TinkerGraph identifiers in case these tests were every used with some
>> other
>> > graph database that didn't use longs):
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/features/map/Vertex.feature#L105-L111
>> >
>> > for all those additions (and others) the logic required by the GLV to
>> > process these tests has stayed surprisingly simple:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-python/src/main/jython/radish/feature_steps.py
>> >
>> > There's a fair bit of regex/string manipulation involved there, but it's
>> > processing strings from the feature file so that's the nature of it I
>> > suppose. I think I'm of the mind that I want to port all of the tests to
>> > feature files, so I wrote this unit test to help validate that none were
>> > missed:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
>> > structure/FeatureCoverageTest.java
>> >
>> > I don't know that we have to get to 100% porting right away, but I think
>> > that once we have these gherkin files written they not only become the
>> > basis for our current GLV testing work, but we might be able to simply
>> use
>> > them for testing the Java stuff as well - that would rid us of having
>> the
>> > test code duplication. It also sets us up with a portable body of tests
>> > that can be re-used in TinkerPop 4.x.
>> >
>> > I'm open to suggestions if anyone has any.
>> >
>> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
>> > jorgebaygondra@gmail.com
>> > > wrote:
>> >
>> > > I was able to build a proof of concept for a Gherkin-based test
>> runner in
>> > > C#, that takes the proposed count and select features
>> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
>> > > gremlin-test/features/map>
>> > > and runs them using C# step definitions.
>> > >
>> > > It uses the Gherkin parser <https://github.com/cucumber/g
>> herkin-dotnet>
>> > > from
>> > > cucumber, there isn't a release of the parser with .NET Core support
>> so
>> > > I've
>> > > asked them to release one
>> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
>> > > limitation from their source files). If they are not able to release
>> it
>> > in
>> > > the next few days, we can implement our own as it should be pretty
>> > straight
>> > > forward.
>> > >
>> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
>> spmallette@gmail.com>
>> > > wrote:
>> > >
>> > > > Thanks for the update. I'm trying to keep the test language as
>> simple
>> > as
>> > > > possible so that we don't need an overly complicated test
>> > implementation.
>> > > > Hopefully that will help make the .NET approach as easy as possible.
>> > > >
>> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
>> > > > jorgebaygondra@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
>> > cucumber
>> > > > > implementation for .NET <https://cucumber.io/docs#
>> > > > cucumber-implementations
>> > > > > >,
>> > > > > does not support .NET Core platform (we use .NET Core build tools
>> for
>> > > the
>> > > > > .NET GLV) and only supports .NET Framework.
>> > > > >
>> > > > > From what I can see <https://github.com/techtalk/S
>> pecFlow/projects/2
>> > >,
>> > > > > .NET
>> > > > > Core support on SpecFlow is coming at a very slow pace and we
>> > shouldn't
>> > > > > expect to land any time soon (there were some design decisions in
>> > > > SpecFlow
>> > > > > library that makes supporting other platforms non-trivial, like
>> > > requiring
>> > > > > code gen).
>> > > > >
>> > > > > The alternative would be to implement our own harness to support
>> it:
>> > > > from a
>> > > > > xunit test, look for certain types and parse the annotations, and
>> > > execute
>> > > > > them using the Gherkin features.
>> > > > > There is a .NET cross-platform Gherkin parser:
>> > > > > https://github.com/cucumber/gherkin-dotnet
>> > > > > I'll continue looking into this option and try to understand the
>> > effort
>> > > > > required...
>> > > > >
>> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
>> > > > > jorgebaygondra@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Nice! Gherkin will make our lives easier with a growing number
>> of
>> > > GLVs.
>> > > > > >
>> > > > > > We should find a way to define the different features supported
>> by
>> > > each
>> > > > > > GLV, as it's reasonable to have different maturity levels per
>> GLV
>> > > (ie:
>> > > > > > lambdas support, traversal strategy, ...). I don't know if it
>> will
>> > be
>> > > > > > beneficial to do it in the Gherkin files or within each GLV
>> > > > > implementation.
>> > > > > > Also, we should consider the process of rolling out a new
>> method /
>> > > > class
>> > > > > > in the java implementation, how that could affect each GLV.
>> > > > > >
>> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
>> > > > spmallette@gmail.com
>> > > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > >> that's what i meant by "reflection" or as you suggest eval(). I
>> > > guess
>> > > > > the
>> > > > > >> point is that if the language can support some way of taking
>> the
>> > > > string
>> > > > > >> value and turning it automatically into a traversal in that
>> GLVs
>> > > style
>> > > > > >> then
>> > > > > >> we should do that.
>> > > > > >>
>> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
>> <me...@gremlin.guru>
>> > > > > wrote:
>> > > > > >>
>> > > > > >> > For unparameterized queries it can probably be as easy as:
>> > > > > >> >
>> > > > > >> > @given("the traversal of")
>> > > > > >> > def translate_traversal(step):
>> > > > > >> >     g = step.context.g
>> > > > > >> >     step.context.traversal = eval(step.text)
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > Cheers,
>> > > > > >> > Daniel
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
>> <me@gremlin.guru
>> > >
>> > > > > >> wrote:
>> > > > > >> >
>> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
>> > years,
>> > > > > but I
>> > > > > >> > > really like the BDD approach.
>> > > > > >> > >
>> > > > > >> > > and then you can look at the GLV Gremlin translations
>> > > specifically
>> > > > > >> here:
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
>> > > L34-L46
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > This part is the only thing that looks weird to me. You're
>> > > > basically
>> > > > > >> > > writing every query twice; is there really no easier way
>> to do
>> > > > that?
>> > > > > >> > >
>> > > > > >> > > Cheers,
>> > > > > >> > > Daniel
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
>> > > > > >> spmallette@gmail.com>
>> > > > > >> > > wrote:
>> > > > > >> > >
>> > > > > >> > >> I've brought this issue up in the past and had suggested
>> some
>> > > > > >> options I
>> > > > > >> > >> had
>> > > > > >> > >> in mind but now I've finally put the basics of those
>> ideas in
>> > > > place
>> > > > > >> so I
>> > > > > >> > >> figured I'd start a fresh thread. Recall that the issue at
>> > hand
>> > > > is
>> > > > > >> that
>> > > > > >> > we
>> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is bound
>> to
>> > > the
>> > > > > >> JVM. We
>> > > > > >> > >> have some tricks that let us test gremlin-python with it
>> but
>> > > > those
>> > > > > >> > tricks
>> > > > > >> > >> won't work for every language and we now have the first
>> > > language
>> > > > in
>> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't
>> > > > support
>> > > > > it
>> > > > > >> > >> (yes,
>> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
>> there
>> > are
>> > > > > >> issues
>> > > > > >> > >> with
>> > > > > >> > >> getting it all to work with the test framework that make
>> it
>> > > > unduly
>> > > > > >> > >> complicated).
>> > > > > >> > >>
>> > > > > >> > >> On other threads I offered the idea that we look to use
>> > Gherkin
>> > > > to
>> > > > > >> write
>> > > > > >> > >> general Gremlin test specifications, which then could be
>> read
>> > > and
>> > > > > >> > >> processed
>> > > > > >> > >> by the wide variety of test frameworks that can read that
>> > > format
>> > > > -
>> > > > > >> there
>> > > > > >> > >> tend to be Gherkin processors in just about every
>> language -
>> > > for
>> > > > > >> > example,
>> > > > > >> > >> see:
>> > > > > >> > >>
>> > > > > >> > >> https://cucumber.io/
>> > > > > >> > >>
>> > > > > >> > >> I just created this issue:
>> > > > > >> > >>
>> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
>> > > > > >> > >>
>> > > > > >> > >> and pushed this branch:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
>> > > > > >> > >>
>> > > > > >> > >> which demonstrates how this works with gremlin-python. The
>> > > basic
>> > > > > >> anatomy
>> > > > > >> > >> of
>> > > > > >> > >> this setup involves this new directory in gremlin-test:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > >> lin-test/features
>> > > > > >> > >>
>> > > > > >> > >> It contains the Gherkin .features files. These are the
>> test
>> > > > > >> > >> specifications.
>> > > > > >> > >> They are written using gremlin-java as the "model"
>> language.
>> > > GLVs
>> > > > > >> will
>> > > > > >> > >> then
>> > > > > >> > >> need to write some infrastructure to process these Gherkin
>> > > files.
>> > > > > The
>> > > > > >> > key
>> > > > > >> > >> to making this "easy" to implement will lie in our
>> abiilty to
>> > > > keep
>> > > > > >> the
>> > > > > >> > >> assertions we want to have relatively simple. The more
>> > > simplistic
>> > > > > the
>> > > > > >> > >> language in the Gherkin .feature files the easier the job
>> it
>> > > will
>> > > > > be
>> > > > > >> for
>> > > > > >> > >> GLVs to build their infrastructure. Of course, once that
>> > > > > >> infrastructure
>> > > > > >> > is
>> > > > > >> > >> in place, the GLV developer just has to write the GLV
>> version
>> > > of
>> > > > > the
>> > > > > >> > >> Gremlin specified in the .feature file. So you can look at
>> > all
>> > > > the
>> > > > > >> > >> "infrastructure" code here in this pair of files:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > >> lin-python/src/main/jython/radish
>> > > > > >> > >>
>> > > > > >> > >> and then you can look at the GLV Gremlin translations
>> > > > specifically
>> > > > > >> here:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
>> > > L34-L46
>> > > > > >> > >>
>> > > > > >> > >> I think this approach works pretty well and solves our
>> > general
>> > > > > >> problems
>> > > > > >> > >> for
>> > > > > >> > >> GLV testing. There is some pain up front in implementing
>> the
>> > > > > >> > >> "infrastructure" but after that new Gremlin tests added to
>> > > > .feature
>> > > > > >> > files
>> > > > > >> > >> just need to translated in the GLV. I suppose we could
>> > > > "automate" a
>> > > > > >> good
>> > > > > >> > >> portion of the translation with reflection of some sort.
>> > > Anything
>> > > > > >> else
>> > > > > >> > >> could just be handled manually.
>> > > > > >> > >>
>> > > > > >> > >> Not sure if we need to use this new model to wholly
>> replace
>> > the
>> > > > old
>> > > > > >> one.
>> > > > > >> > >> The process test suite has its place in helping graph
>> > database
>> > > > > >> providers
>> > > > > >> > >> test their stuff. I also imagine that introducing this
>> > approach
>> > > > in
>> > > > > >> that
>> > > > > >> > >> context would create a breaking change which we would then
>> > need
>> > > > to
>> > > > > >> push
>> > > > > >> > >> off
>> > > > > >> > >> to 3.4.0.  I suppose that gives us time to think, but for
>> now
>> > > it
>> > > > > >> might
>> > > > > >> > not
>> > > > > >> > >> be best to conflate the two and just treat them as
>> separate
>> > > > aspects
>> > > > > >> of
>> > > > > >> > the
>> > > > > >> > >> test suite.
>> > > > > >> > >>
>> > > > > >> > >> Anyway - it's important we settle on an approach to
>> testing
>> > as
>> > > we
>> > > > > >> really
>> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without
>> > > getting
>> > > > > the
>> > > > > >> > test
>> > > > > >> > >> suite solid. please yell if you have any ideas or
>> feedback on
>> > > > this
>> > > > > >> > >> approach.
>> > > > > >> > >>
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

I did some pretty heavy refactoring to the python test logic (altered a bit
by some revision of the gherkin feature file language) and the result is a
much more simplified test logic file:

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-python/src/main/jython/radish/feature_steps.py

About 120 lines of code (down from about 170). If you include the test
logic "setup" file:

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-python/src/main/jython/radish/terrain.py

we end up with about 250 lines of test logic total (take out the
comment/license and we're probably well under 200 total - not too bad). I
think I'm closing in on the end to infrastructure building here so the
basic framework is getting close to final at this point I believe. I'll
keep scanning the tests looking for other types of assertions that I've not
yet covered, but it's getting pretty solid I think. Hopefully, there won't
need to be too many more lines of code needed to express the test logic as
I like how things are looking right now.




On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <jo...@gmail.com>
wrote:

> Great progress! I like how you avoided using ids, even if it adds some
> complexity to the transformation required.
>
> The Python step definitions are still quite simple, I think it would be
> mostly the same on the rest of the languages.
>
> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > Just another update on progress with the test suite. The language of the
> > .feature files is getting slightly more complex as I try to translate
> more
> > and more of our Java process suite tests into the language of the gherkin
> > files. You can see here where I've added the ability to include
> parameters:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-test/features/filter/Has.feature#L33
> >
> > I also came up with a method of asserting edges (didn't want to rely on
> ids
> > as it makes the gherkin harder to read, plus i didn't want to assume
> > TinkerGraph identifiers in case these tests were every used with some
> other
> > graph database that didn't use longs):
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-test/features/map/Vertex.feature#L105-L111
> >
> > for all those additions (and others) the logic required by the GLV to
> > process these tests has stayed surprisingly simple:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/feature_steps.py
> >
> > There's a fair bit of regex/string manipulation involved there, but it's
> > processing strings from the feature file so that's the nature of it I
> > suppose. I think I'm of the mind that I want to port all of the tests to
> > feature files, so I wrote this unit test to help validate that none were
> > missed:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > structure/FeatureCoverageTest.java
> >
> > I don't know that we have to get to 100% porting right away, but I think
> > that once we have these gherkin files written they not only become the
> > basis for our current GLV testing work, but we might be able to simply
> use
> > them for testing the Java stuff as well - that would rid us of having the
> > test code duplication. It also sets us up with a portable body of tests
> > that can be re-used in TinkerPop 4.x.
> >
> > I'm open to suggestions if anyone has any.
> >
> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com
> > > wrote:
> >
> > > I was able to build a proof of concept for a Gherkin-based test runner
> in
> > > C#, that takes the proposed count and select features
> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > gremlin-test/features/map>
> > > and runs them using C# step definitions.
> > >
> > > It uses the Gherkin parser <https://github.com/cucumber/gherkin-dotnet
> >
> > > from
> > > cucumber, there isn't a release of the parser with .NET Core support so
> > > I've
> > > asked them to release one
> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
> > > limitation from their source files). If they are not able to release it
> > in
> > > the next few days, we can implement our own as it should be pretty
> > straight
> > > forward.
> > >
> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > > > Thanks for the update. I'm trying to keep the test language as simple
> > as
> > > > possible so that we don't need an overly complicated test
> > implementation.
> > > > Hopefully that will help make the .NET approach as easy as possible.
> > > >
> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > > jorgebaygondra@gmail.com>
> > > > wrote:
> > > >
> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> > cucumber
> > > > > implementation for .NET <https://cucumber.io/docs#
> > > > cucumber-implementations
> > > > > >,
> > > > > does not support .NET Core platform (we use .NET Core build tools
> for
> > > the
> > > > > .NET GLV) and only supports .NET Framework.
> > > > >
> > > > > From what I can see <https://github.com/techtalk/
> SpecFlow/projects/2
> > >,
> > > > > .NET
> > > > > Core support on SpecFlow is coming at a very slow pace and we
> > shouldn't
> > > > > expect to land any time soon (there were some design decisions in
> > > > SpecFlow
> > > > > library that makes supporting other platforms non-trivial, like
> > > requiring
> > > > > code gen).
> > > > >
> > > > > The alternative would be to implement our own harness to support
> it:
> > > > from a
> > > > > xunit test, look for certain types and parse the annotations, and
> > > execute
> > > > > them using the Gherkin features.
> > > > > There is a .NET cross-platform Gherkin parser:
> > > > > https://github.com/cucumber/gherkin-dotnet
> > > > > I'll continue looking into this option and try to understand the
> > effort
> > > > > required...
> > > > >
> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > > > jorgebaygondra@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Nice! Gherkin will make our lives easier with a growing number of
> > > GLVs.
> > > > > >
> > > > > > We should find a way to define the different features supported
> by
> > > each
> > > > > > GLV, as it's reasonable to have different maturity levels per GLV
> > > (ie:
> > > > > > lambdas support, traversal strategy, ...). I don't know if it
> will
> > be
> > > > > > beneficial to do it in the Gherkin files or within each GLV
> > > > > implementation.
> > > > > > Also, we should consider the process of rolling out a new method
> /
> > > > class
> > > > > > in the java implementation, how that could affect each GLV.
> > > > > >
> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > > spmallette@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> that's what i meant by "reflection" or as you suggest eval(). I
> > > guess
> > > > > the
> > > > > >> point is that if the language can support some way of taking the
> > > > string
> > > > > >> value and turning it automatically into a traversal in that GLVs
> > > style
> > > > > >> then
> > > > > >> we should do that.
> > > > > >>
> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me@gremlin.guru
> >
> > > > > wrote:
> > > > > >>
> > > > > >> > For unparameterized queries it can probably be as easy as:
> > > > > >> >
> > > > > >> > @given("the traversal of")
> > > > > >> > def translate_traversal(step):
> > > > > >> >     g = step.context.g
> > > > > >> >     step.context.traversal = eval(step.text)
> > > > > >> >
> > > > > >> >
> > > > > >> > Cheers,
> > > > > >> > Daniel
> > > > > >> >
> > > > > >> >
> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> <me@gremlin.guru
> > >
> > > > > >> wrote:
> > > > > >> >
> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
> > years,
> > > > > but I
> > > > > >> > > really like the BDD approach.
> > > > > >> > >
> > > > > >> > > and then you can look at the GLV Gremlin translations
> > > specifically
> > > > > >> here:
> > > > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> grem
> > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
> > > L34-L46
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > This part is the only thing that looks weird to me. You're
> > > > basically
> > > > > >> > > writing every query twice; is there really no easier way to
> do
> > > > that?
> > > > > >> > >
> > > > > >> > > Cheers,
> > > > > >> > > Daniel
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > > > >> spmallette@gmail.com>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > >> I've brought this issue up in the past and had suggested
> some
> > > > > >> options I
> > > > > >> > >> had
> > > > > >> > >> in mind but now I've finally put the basics of those ideas
> in
> > > > place
> > > > > >> so I
> > > > > >> > >> figured I'd start a fresh thread. Recall that the issue at
> > hand
> > > > is
> > > > > >> that
> > > > > >> > we
> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is bound
> to
> > > the
> > > > > >> JVM. We
> > > > > >> > >> have some tricks that let us test gremlin-python with it
> but
> > > > those
> > > > > >> > tricks
> > > > > >> > >> won't work for every language and we now have the first
> > > language
> > > > in
> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't
> > > > support
> > > > > it
> > > > > >> > >> (yes,
> > > > > >> > >> i know that gremlin-javascript can run on the jvm but there
> > are
> > > > > >> issues
> > > > > >> > >> with
> > > > > >> > >> getting it all to work with the test framework that make it
> > > > unduly
> > > > > >> > >> complicated).
> > > > > >> > >>
> > > > > >> > >> On other threads I offered the idea that we look to use
> > Gherkin
> > > > to
> > > > > >> write
> > > > > >> > >> general Gremlin test specifications, which then could be
> read
> > > and
> > > > > >> > >> processed
> > > > > >> > >> by the wide variety of test frameworks that can read that
> > > format
> > > > -
> > > > > >> there
> > > > > >> > >> tend to be Gherkin processors in just about every language
> -
> > > for
> > > > > >> > example,
> > > > > >> > >> see:
> > > > > >> > >>
> > > > > >> > >> https://cucumber.io/
> > > > > >> > >>
> > > > > >> > >> I just created this issue:
> > > > > >> > >>
> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > > > > >> > >>
> > > > > >> > >> and pushed this branch:
> > > > > >> > >>
> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> > > > > >> > >>
> > > > > >> > >> which demonstrates how this works with gremlin-python. The
> > > basic
> > > > > >> anatomy
> > > > > >> > >> of
> > > > > >> > >> this setup involves this new directory in gremlin-test:
> > > > > >> > >>
> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> grem
> > > > > >> > >> lin-test/features
> > > > > >> > >>
> > > > > >> > >> It contains the Gherkin .features files. These are the test
> > > > > >> > >> specifications.
> > > > > >> > >> They are written using gremlin-java as the "model"
> language.
> > > GLVs
> > > > > >> will
> > > > > >> > >> then
> > > > > >> > >> need to write some infrastructure to process these Gherkin
> > > files.
> > > > > The
> > > > > >> > key
> > > > > >> > >> to making this "easy" to implement will lie in our abiilty
> to
> > > > keep
> > > > > >> the
> > > > > >> > >> assertions we want to have relatively simple. The more
> > > simplistic
> > > > > the
> > > > > >> > >> language in the Gherkin .feature files the easier the job
> it
> > > will
> > > > > be
> > > > > >> for
> > > > > >> > >> GLVs to build their infrastructure. Of course, once that
> > > > > >> infrastructure
> > > > > >> > is
> > > > > >> > >> in place, the GLV developer just has to write the GLV
> version
> > > of
> > > > > the
> > > > > >> > >> Gremlin specified in the .feature file. So you can look at
> > all
> > > > the
> > > > > >> > >> "infrastructure" code here in this pair of files:
> > > > > >> > >>
> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> grem
> > > > > >> > >> lin-python/src/main/jython/radish
> > > > > >> > >>
> > > > > >> > >> and then you can look at the GLV Gremlin translations
> > > > specifically
> > > > > >> here:
> > > > > >> > >>
> > > > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> grem
> > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
> > > L34-L46
> > > > > >> > >>
> > > > > >> > >> I think this approach works pretty well and solves our
> > general
> > > > > >> problems
> > > > > >> > >> for
> > > > > >> > >> GLV testing. There is some pain up front in implementing
> the
> > > > > >> > >> "infrastructure" but after that new Gremlin tests added to
> > > > .feature
> > > > > >> > files
> > > > > >> > >> just need to translated in the GLV. I suppose we could
> > > > "automate" a
> > > > > >> good
> > > > > >> > >> portion of the translation with reflection of some sort.
> > > Anything
> > > > > >> else
> > > > > >> > >> could just be handled manually.
> > > > > >> > >>
> > > > > >> > >> Not sure if we need to use this new model to wholly replace
> > the
> > > > old
> > > > > >> one.
> > > > > >> > >> The process test suite has its place in helping graph
> > database
> > > > > >> providers
> > > > > >> > >> test their stuff. I also imagine that introducing this
> > approach
> > > > in
> > > > > >> that
> > > > > >> > >> context would create a breaking change which we would then
> > need
> > > > to
> > > > > >> push
> > > > > >> > >> off
> > > > > >> > >> to 3.4.0.  I suppose that gives us time to think, but for
> now
> > > it
> > > > > >> might
> > > > > >> > not
> > > > > >> > >> be best to conflate the two and just treat them as separate
> > > > aspects
> > > > > >> of
> > > > > >> > the
> > > > > >> > >> test suite.
> > > > > >> > >>
> > > > > >> > >> Anyway - it's important we settle on an approach to testing
> > as
> > > we
> > > > > >> really
> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without
> > > getting
> > > > > the
> > > > > >> > test
> > > > > >> > >> suite solid. please yell if you have any ideas or feedback
> on
> > > > this
> > > > > >> > >> approach.
> > > > > >> > >>
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

Great progress! I like how you avoided using ids, even if it adds some
complexity to the transformation required.

The Python step definitions are still quite simple, I think it would be
mostly the same on the rest of the languages.

On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> Just another update on progress with the test suite. The language of the
> .feature files is getting slightly more complex as I try to translate more
> and more of our Java process suite tests into the language of the gherkin
> files. You can see here where I've added the ability to include parameters:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-test/features/filter/Has.feature#L33
>
> I also came up with a method of asserting edges (didn't want to rely on ids
> as it makes the gherkin harder to read, plus i didn't want to assume
> TinkerGraph identifiers in case these tests were every used with some other
> graph database that didn't use longs):
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-test/features/map/Vertex.feature#L105-L111
>
> for all those additions (and others) the logic required by the GLV to
> process these tests has stayed surprisingly simple:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/feature_steps.py
>
> There's a fair bit of regex/string manipulation involved there, but it's
> processing strings from the feature file so that's the nature of it I
> suppose. I think I'm of the mind that I want to port all of the tests to
> feature files, so I wrote this unit test to help validate that none were
> missed:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> structure/FeatureCoverageTest.java
>
> I don't know that we have to get to 100% porting right away, but I think
> that once we have these gherkin files written they not only become the
> basis for our current GLV testing work, but we might be able to simply use
> them for testing the Java stuff as well - that would rid us of having the
> test code duplication. It also sets us up with a portable body of tests
> that can be re-used in TinkerPop 4.x.
>
> I'm open to suggestions if anyone has any.
>
> On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com
> > wrote:
>
> > I was able to build a proof of concept for a Gherkin-based test runner in
> > C#, that takes the proposed count and select features
> > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > gremlin-test/features/map>
> > and runs them using C# step definitions.
> >
> > It uses the Gherkin parser <https://github.com/cucumber/gherkin-dotnet>
> > from
> > cucumber, there isn't a release of the parser with .NET Core support so
> > I've
> > asked them to release one
> > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
> > limitation from their source files). If they are not able to release it
> in
> > the next few days, we can implement our own as it should be pretty
> straight
> > forward.
> >
> > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > Thanks for the update. I'm trying to keep the test language as simple
> as
> > > possible so that we don't need an overly complicated test
> implementation.
> > > Hopefully that will help make the .NET approach as easy as possible.
> > >
> > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > jorgebaygondra@gmail.com>
> > > wrote:
> > >
> > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> cucumber
> > > > implementation for .NET <https://cucumber.io/docs#
> > > cucumber-implementations
> > > > >,
> > > > does not support .NET Core platform (we use .NET Core build tools for
> > the
> > > > .NET GLV) and only supports .NET Framework.
> > > >
> > > > From what I can see <https://github.com/techtalk/SpecFlow/projects/2
> >,
> > > > .NET
> > > > Core support on SpecFlow is coming at a very slow pace and we
> shouldn't
> > > > expect to land any time soon (there were some design decisions in
> > > SpecFlow
> > > > library that makes supporting other platforms non-trivial, like
> > requiring
> > > > code gen).
> > > >
> > > > The alternative would be to implement our own harness to support it:
> > > from a
> > > > xunit test, look for certain types and parse the annotations, and
> > execute
> > > > them using the Gherkin features.
> > > > There is a .NET cross-platform Gherkin parser:
> > > > https://github.com/cucumber/gherkin-dotnet
> > > > I'll continue looking into this option and try to understand the
> effort
> > > > required...
> > > >
> > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > > jorgebaygondra@gmail.com>
> > > > wrote:
> > > >
> > > > > Nice! Gherkin will make our lives easier with a growing number of
> > GLVs.
> > > > >
> > > > > We should find a way to define the different features supported by
> > each
> > > > > GLV, as it's reasonable to have different maturity levels per GLV
> > (ie:
> > > > > lambdas support, traversal strategy, ...). I don't know if it will
> be
> > > > > beneficial to do it in the Gherkin files or within each GLV
> > > > implementation.
> > > > > Also, we should consider the process of rolling out a new method /
> > > class
> > > > > in the java implementation, how that could affect each GLV.
> > > > >
> > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > spmallette@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > >> that's what i meant by "reflection" or as you suggest eval(). I
> > guess
> > > > the
> > > > >> point is that if the language can support some way of taking the
> > > string
> > > > >> value and turning it automatically into a traversal in that GLVs
> > style
> > > > >> then
> > > > >> we should do that.
> > > > >>
> > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru>
> > > > wrote:
> > > > >>
> > > > >> > For unparameterized queries it can probably be as easy as:
> > > > >> >
> > > > >> > @given("the traversal of")
> > > > >> > def translate_traversal(step):
> > > > >> >     g = step.context.g
> > > > >> >     step.context.traversal = eval(step.text)
> > > > >> >
> > > > >> >
> > > > >> > Cheers,
> > > > >> > Daniel
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me@gremlin.guru
> >
> > > > >> wrote:
> > > > >> >
> > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
> years,
> > > > but I
> > > > >> > > really like the BDD approach.
> > > > >> > >
> > > > >> > > and then you can look at the GLV Gremlin translations
> > specifically
> > > > >> here:
> > > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
> > L34-L46
> > > > >> > >
> > > > >> > >
> > > > >> > > This part is the only thing that looks weird to me. You're
> > > basically
> > > > >> > > writing every query twice; is there really no easier way to do
> > > that?
> > > > >> > >
> > > > >> > > Cheers,
> > > > >> > > Daniel
> > > > >> > >
> > > > >> > >
> > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > > >> spmallette@gmail.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > >> I've brought this issue up in the past and had suggested some
> > > > >> options I
> > > > >> > >> had
> > > > >> > >> in mind but now I've finally put the basics of those ideas in
> > > place
> > > > >> so I
> > > > >> > >> figured I'd start a fresh thread. Recall that the issue at
> hand
> > > is
> > > > >> that
> > > > >> > we
> > > > >> > >> don't have a test suite for GLVs as gremlin-test is bound to
> > the
> > > > >> JVM. We
> > > > >> > >> have some tricks that let us test gremlin-python with it but
> > > those
> > > > >> > tricks
> > > > >> > >> won't work for every language and we now have the first
> > language
> > > in
> > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't
> > > support
> > > > it
> > > > >> > >> (yes,
> > > > >> > >> i know that gremlin-javascript can run on the jvm but there
> are
> > > > >> issues
> > > > >> > >> with
> > > > >> > >> getting it all to work with the test framework that make it
> > > unduly
> > > > >> > >> complicated).
> > > > >> > >>
> > > > >> > >> On other threads I offered the idea that we look to use
> Gherkin
> > > to
> > > > >> write
> > > > >> > >> general Gremlin test specifications, which then could be read
> > and
> > > > >> > >> processed
> > > > >> > >> by the wide variety of test frameworks that can read that
> > format
> > > -
> > > > >> there
> > > > >> > >> tend to be Gherkin processors in just about every language -
> > for
> > > > >> > example,
> > > > >> > >> see:
> > > > >> > >>
> > > > >> > >> https://cucumber.io/
> > > > >> > >>
> > > > >> > >> I just created this issue:
> > > > >> > >>
> > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > > > >> > >>
> > > > >> > >> and pushed this branch:
> > > > >> > >>
> > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> > > > >> > >>
> > > > >> > >> which demonstrates how this works with gremlin-python. The
> > basic
> > > > >> anatomy
> > > > >> > >> of
> > > > >> > >> this setup involves this new directory in gremlin-test:
> > > > >> > >>
> > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > > > >> > >> lin-test/features
> > > > >> > >>
> > > > >> > >> It contains the Gherkin .features files. These are the test
> > > > >> > >> specifications.
> > > > >> > >> They are written using gremlin-java as the "model" language.
> > GLVs
> > > > >> will
> > > > >> > >> then
> > > > >> > >> need to write some infrastructure to process these Gherkin
> > files.
> > > > The
> > > > >> > key
> > > > >> > >> to making this "easy" to implement will lie in our abiilty to
> > > keep
> > > > >> the
> > > > >> > >> assertions we want to have relatively simple. The more
> > simplistic
> > > > the
> > > > >> > >> language in the Gherkin .feature files the easier the job it
> > will
> > > > be
> > > > >> for
> > > > >> > >> GLVs to build their infrastructure. Of course, once that
> > > > >> infrastructure
> > > > >> > is
> > > > >> > >> in place, the GLV developer just has to write the GLV version
> > of
> > > > the
> > > > >> > >> Gremlin specified in the .feature file. So you can look at
> all
> > > the
> > > > >> > >> "infrastructure" code here in this pair of files:
> > > > >> > >>
> > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > > > >> > >> lin-python/src/main/jython/radish
> > > > >> > >>
> > > > >> > >> and then you can look at the GLV Gremlin translations
> > > specifically
> > > > >> here:
> > > > >> > >>
> > > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
> > L34-L46
> > > > >> > >>
> > > > >> > >> I think this approach works pretty well and solves our
> general
> > > > >> problems
> > > > >> > >> for
> > > > >> > >> GLV testing. There is some pain up front in implementing the
> > > > >> > >> "infrastructure" but after that new Gremlin tests added to
> > > .feature
> > > > >> > files
> > > > >> > >> just need to translated in the GLV. I suppose we could
> > > "automate" a
> > > > >> good
> > > > >> > >> portion of the translation with reflection of some sort.
> > Anything
> > > > >> else
> > > > >> > >> could just be handled manually.
> > > > >> > >>
> > > > >> > >> Not sure if we need to use this new model to wholly replace
> the
> > > old
> > > > >> one.
> > > > >> > >> The process test suite has its place in helping graph
> database
> > > > >> providers
> > > > >> > >> test their stuff. I also imagine that introducing this
> approach
> > > in
> > > > >> that
> > > > >> > >> context would create a breaking change which we would then
> need
> > > to
> > > > >> push
> > > > >> > >> off
> > > > >> > >> to 3.4.0.  I suppose that gives us time to think, but for now
> > it
> > > > >> might
> > > > >> > not
> > > > >> > >> be best to conflate the two and just treat them as separate
> > > aspects
> > > > >> of
> > > > >> > the
> > > > >> > >> test suite.
> > > > >> > >>
> > > > >> > >> Anyway - it's important we settle on an approach to testing
> as
> > we
> > > > >> really
> > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without
> > getting
> > > > the
> > > > >> > test
> > > > >> > >> suite solid. please yell if you have any ideas or feedback on
> > > this
> > > > >> > >> approach.
> > > > >> > >>
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

Just another update on progress with the test suite. The language of the
.feature files is getting slightly more complex as I try to translate more
and more of our Java process suite tests into the language of the gherkin
files. You can see here where I've added the ability to include parameters:

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-test/features/filter/Has.feature#L33

I also came up with a method of asserting edges (didn't want to rely on ids
as it makes the gherkin harder to read, plus i didn't want to assume
TinkerGraph identifiers in case these tests were every used with some other
graph database that didn't use longs):

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-test/features/map/Vertex.feature#L105-L111

for all those additions (and others) the logic required by the GLV to
process these tests has stayed surprisingly simple:

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-python/src/main/jython/radish/feature_steps.py

There's a fair bit of regex/string manipulation involved there, but it's
processing strings from the feature file so that's the nature of it I
suppose. I think I'm of the mind that I want to port all of the tests to
feature files, so I wrote this unit test to help validate that none were
missed:

https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/structure/FeatureCoverageTest.java

I don't know that we have to get to 100% porting right away, but I think
that once we have these gherkin files written they not only become the
basis for our current GLV testing work, but we might be able to simply use
them for testing the Java stuff as well - that would rid us of having the
test code duplication. It also sets us up with a portable body of tests
that can be re-used in TinkerPop 4.x.

I'm open to suggestions if anyone has any.

On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> wrote:

> I was able to build a proof of concept for a Gherkin-based test runner in
> C#, that takes the proposed count and select features
> <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> gremlin-test/features/map>
> and runs them using C# step definitions.
>
> It uses the Gherkin parser <https://github.com/cucumber/gherkin-dotnet>
> from
> cucumber, there isn't a release of the parser with .NET Core support so
> I've
> asked them to release one
> <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
> limitation from their source files). If they are not able to release it in
> the next few days, we can implement our own as it should be pretty straight
> forward.
>
> On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > Thanks for the update. I'm trying to keep the test language as simple as
> > possible so that we don't need an overly complicated test implementation.
> > Hopefully that will help make the .NET approach as easy as possible.
> >
> > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com>
> > wrote:
> >
> > > I've been looking into Gherkin support for .NET: SpecFlow, the cucumber
> > > implementation for .NET <https://cucumber.io/docs#
> > cucumber-implementations
> > > >,
> > > does not support .NET Core platform (we use .NET Core build tools for
> the
> > > .NET GLV) and only supports .NET Framework.
> > >
> > > From what I can see <https://github.com/techtalk/SpecFlow/projects/2>,
> > > .NET
> > > Core support on SpecFlow is coming at a very slow pace and we shouldn't
> > > expect to land any time soon (there were some design decisions in
> > SpecFlow
> > > library that makes supporting other platforms non-trivial, like
> requiring
> > > code gen).
> > >
> > > The alternative would be to implement our own harness to support it:
> > from a
> > > xunit test, look for certain types and parse the annotations, and
> execute
> > > them using the Gherkin features.
> > > There is a .NET cross-platform Gherkin parser:
> > > https://github.com/cucumber/gherkin-dotnet
> > > I'll continue looking into this option and try to understand the effort
> > > required...
> > >
> > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > jorgebaygondra@gmail.com>
> > > wrote:
> > >
> > > > Nice! Gherkin will make our lives easier with a growing number of
> GLVs.
> > > >
> > > > We should find a way to define the different features supported by
> each
> > > > GLV, as it's reasonable to have different maturity levels per GLV
> (ie:
> > > > lambdas support, traversal strategy, ...). I don't know if it will be
> > > > beneficial to do it in the Gherkin files or within each GLV
> > > implementation.
> > > > Also, we should consider the process of rolling out a new method /
> > class
> > > > in the java implementation, how that could affect each GLV.
> > > >
> > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > spmallette@gmail.com
> > > >
> > > > wrote:
> > > >
> > > >> that's what i meant by "reflection" or as you suggest eval(). I
> guess
> > > the
> > > >> point is that if the language can support some way of taking the
> > string
> > > >> value and turning it automatically into a traversal in that GLVs
> style
> > > >> then
> > > >> we should do that.
> > > >>
> > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru>
> > > wrote:
> > > >>
> > > >> > For unparameterized queries it can probably be as easy as:
> > > >> >
> > > >> > @given("the traversal of")
> > > >> > def translate_traversal(step):
> > > >> >     g = step.context.g
> > > >> >     step.context.traversal = eval(step.text)
> > > >> >
> > > >> >
> > > >> > Cheers,
> > > >> > Daniel
> > > >> >
> > > >> >
> > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru>
> > > >> wrote:
> > > >> >
> > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for years,
> > > but I
> > > >> > > really like the BDD approach.
> > > >> > >
> > > >> > > and then you can look at the GLV Gremlin translations
> specifically
> > > >> here:
> > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
> L34-L46
> > > >> > >
> > > >> > >
> > > >> > > This part is the only thing that looks weird to me. You're
> > basically
> > > >> > > writing every query twice; is there really no easier way to do
> > that?
> > > >> > >
> > > >> > > Cheers,
> > > >> > > Daniel
> > > >> > >
> > > >> > >
> > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > >> spmallette@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > >> I've brought this issue up in the past and had suggested some
> > > >> options I
> > > >> > >> had
> > > >> > >> in mind but now I've finally put the basics of those ideas in
> > place
> > > >> so I
> > > >> > >> figured I'd start a fresh thread. Recall that the issue at hand
> > is
> > > >> that
> > > >> > we
> > > >> > >> don't have a test suite for GLVs as gremlin-test is bound to
> the
> > > >> JVM. We
> > > >> > >> have some tricks that let us test gremlin-python with it but
> > those
> > > >> > tricks
> > > >> > >> won't work for every language and we now have the first
> language
> > in
> > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't
> > support
> > > it
> > > >> > >> (yes,
> > > >> > >> i know that gremlin-javascript can run on the jvm but there are
> > > >> issues
> > > >> > >> with
> > > >> > >> getting it all to work with the test framework that make it
> > unduly
> > > >> > >> complicated).
> > > >> > >>
> > > >> > >> On other threads I offered the idea that we look to use Gherkin
> > to
> > > >> write
> > > >> > >> general Gremlin test specifications, which then could be read
> and
> > > >> > >> processed
> > > >> > >> by the wide variety of test frameworks that can read that
> format
> > -
> > > >> there
> > > >> > >> tend to be Gherkin processors in just about every language -
> for
> > > >> > example,
> > > >> > >> see:
> > > >> > >>
> > > >> > >> https://cucumber.io/
> > > >> > >>
> > > >> > >> I just created this issue:
> > > >> > >>
> > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > > >> > >>
> > > >> > >> and pushed this branch:
> > > >> > >>
> > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> > > >> > >>
> > > >> > >> which demonstrates how this works with gremlin-python. The
> basic
> > > >> anatomy
> > > >> > >> of
> > > >> > >> this setup involves this new directory in gremlin-test:
> > > >> > >>
> > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > > >> > >> lin-test/features
> > > >> > >>
> > > >> > >> It contains the Gherkin .features files. These are the test
> > > >> > >> specifications.
> > > >> > >> They are written using gremlin-java as the "model" language.
> GLVs
> > > >> will
> > > >> > >> then
> > > >> > >> need to write some infrastructure to process these Gherkin
> files.
> > > The
> > > >> > key
> > > >> > >> to making this "easy" to implement will lie in our abiilty to
> > keep
> > > >> the
> > > >> > >> assertions we want to have relatively simple. The more
> simplistic
> > > the
> > > >> > >> language in the Gherkin .feature files the easier the job it
> will
> > > be
> > > >> for
> > > >> > >> GLVs to build their infrastructure. Of course, once that
> > > >> infrastructure
> > > >> > is
> > > >> > >> in place, the GLV developer just has to write the GLV version
> of
> > > the
> > > >> > >> Gremlin specified in the .feature file. So you can look at all
> > the
> > > >> > >> "infrastructure" code here in this pair of files:
> > > >> > >>
> > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > > >> > >> lin-python/src/main/jython/radish
> > > >> > >>
> > > >> > >> and then you can look at the GLV Gremlin translations
> > specifically
> > > >> here:
> > > >> > >>
> > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
> L34-L46
> > > >> > >>
> > > >> > >> I think this approach works pretty well and solves our general
> > > >> problems
> > > >> > >> for
> > > >> > >> GLV testing. There is some pain up front in implementing the
> > > >> > >> "infrastructure" but after that new Gremlin tests added to
> > .feature
> > > >> > files
> > > >> > >> just need to translated in the GLV. I suppose we could
> > "automate" a
> > > >> good
> > > >> > >> portion of the translation with reflection of some sort.
> Anything
> > > >> else
> > > >> > >> could just be handled manually.
> > > >> > >>
> > > >> > >> Not sure if we need to use this new model to wholly replace the
> > old
> > > >> one.
> > > >> > >> The process test suite has its place in helping graph database
> > > >> providers
> > > >> > >> test their stuff. I also imagine that introducing this approach
> > in
> > > >> that
> > > >> > >> context would create a breaking change which we would then need
> > to
> > > >> push
> > > >> > >> off
> > > >> > >> to 3.4.0.  I suppose that gives us time to think, but for now
> it
> > > >> might
> > > >> > not
> > > >> > >> be best to conflate the two and just treat them as separate
> > aspects
> > > >> of
> > > >> > the
> > > >> > >> test suite.
> > > >> > >>
> > > >> > >> Anyway - it's important we settle on an approach to testing as
> we
> > > >> really
> > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without
> getting
> > > the
> > > >> > test
> > > >> > >> suite solid. please yell if you have any ideas or feedback on
> > this
> > > >> > >> approach.
> > > >> > >>
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

I was able to build a proof of concept for a Gherkin-based test runner in
C#, that takes the proposed count and select features
<https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/gremlin-test/features/map>
and runs them using C# step definitions.

It uses the Gherkin parser <https://github.com/cucumber/gherkin-dotnet> from
cucumber, there isn't a release of the parser with .NET Core support so I've
asked them to release one
<https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
limitation from their source files). If they are not able to release it in
the next few days, we can implement our own as it should be pretty straight
forward.

On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> Thanks for the update. I'm trying to keep the test language as simple as
> possible so that we don't need an overly complicated test implementation.
> Hopefully that will help make the .NET approach as easy as possible.
>
> On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com>
> wrote:
>
> > I've been looking into Gherkin support for .NET: SpecFlow, the cucumber
> > implementation for .NET <https://cucumber.io/docs#
> cucumber-implementations
> > >,
> > does not support .NET Core platform (we use .NET Core build tools for the
> > .NET GLV) and only supports .NET Framework.
> >
> > From what I can see <https://github.com/techtalk/SpecFlow/projects/2>,
> > .NET
> > Core support on SpecFlow is coming at a very slow pace and we shouldn't
> > expect to land any time soon (there were some design decisions in
> SpecFlow
> > library that makes supporting other platforms non-trivial, like requiring
> > code gen).
> >
> > The alternative would be to implement our own harness to support it:
> from a
> > xunit test, look for certain types and parse the annotations, and execute
> > them using the Gherkin features.
> > There is a .NET cross-platform Gherkin parser:
> > https://github.com/cucumber/gherkin-dotnet
> > I'll continue looking into this option and try to understand the effort
> > required...
> >
> > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com>
> > wrote:
> >
> > > Nice! Gherkin will make our lives easier with a growing number of GLVs.
> > >
> > > We should find a way to define the different features supported by each
> > > GLV, as it's reasonable to have different maturity levels per GLV (ie:
> > > lambdas support, traversal strategy, ...). I don't know if it will be
> > > beneficial to do it in the Gherkin files or within each GLV
> > implementation.
> > > Also, we should consider the process of rolling out a new method /
> class
> > > in the java implementation, how that could affect each GLV.
> > >
> > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> spmallette@gmail.com
> > >
> > > wrote:
> > >
> > >> that's what i meant by "reflection" or as you suggest eval(). I guess
> > the
> > >> point is that if the language can support some way of taking the
> string
> > >> value and turning it automatically into a traversal in that GLVs style
> > >> then
> > >> we should do that.
> > >>
> > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru>
> > wrote:
> > >>
> > >> > For unparameterized queries it can probably be as easy as:
> > >> >
> > >> > @given("the traversal of")
> > >> > def translate_traversal(step):
> > >> >     g = step.context.g
> > >> >     step.context.traversal = eval(step.text)
> > >> >
> > >> >
> > >> > Cheers,
> > >> > Daniel
> > >> >
> > >> >
> > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru>
> > >> wrote:
> > >> >
> > >> > > That's great stuff. I haven't used Cucumber / Gherkin for years,
> > but I
> > >> > > really like the BDD approach.
> > >> > >
> > >> > > and then you can look at the GLV Gremlin translations specifically
> > >> here:
> > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > >> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> > >> > >
> > >> > >
> > >> > > This part is the only thing that looks weird to me. You're
> basically
> > >> > > writing every query twice; is there really no easier way to do
> that?
> > >> > >
> > >> > > Cheers,
> > >> > > Daniel
> > >> > >
> > >> > >
> > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > >> spmallette@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > >> I've brought this issue up in the past and had suggested some
> > >> options I
> > >> > >> had
> > >> > >> in mind but now I've finally put the basics of those ideas in
> place
> > >> so I
> > >> > >> figured I'd start a fresh thread. Recall that the issue at hand
> is
> > >> that
> > >> > we
> > >> > >> don't have a test suite for GLVs as gremlin-test is bound to the
> > >> JVM. We
> > >> > >> have some tricks that let us test gremlin-python with it but
> those
> > >> > tricks
> > >> > >> won't work for every language and we now have the first language
> in
> > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't
> support
> > it
> > >> > >> (yes,
> > >> > >> i know that gremlin-javascript can run on the jvm but there are
> > >> issues
> > >> > >> with
> > >> > >> getting it all to work with the test framework that make it
> unduly
> > >> > >> complicated).
> > >> > >>
> > >> > >> On other threads I offered the idea that we look to use Gherkin
> to
> > >> write
> > >> > >> general Gremlin test specifications, which then could be read and
> > >> > >> processed
> > >> > >> by the wide variety of test frameworks that can read that format
> -
> > >> there
> > >> > >> tend to be Gherkin processors in just about every language - for
> > >> > example,
> > >> > >> see:
> > >> > >>
> > >> > >> https://cucumber.io/
> > >> > >>
> > >> > >> I just created this issue:
> > >> > >>
> > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > >> > >>
> > >> > >> and pushed this branch:
> > >> > >>
> > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> > >> > >>
> > >> > >> which demonstrates how this works with gremlin-python. The basic
> > >> anatomy
> > >> > >> of
> > >> > >> this setup involves this new directory in gremlin-test:
> > >> > >>
> > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > >> > >> lin-test/features
> > >> > >>
> > >> > >> It contains the Gherkin .features files. These are the test
> > >> > >> specifications.
> > >> > >> They are written using gremlin-java as the "model" language. GLVs
> > >> will
> > >> > >> then
> > >> > >> need to write some infrastructure to process these Gherkin files.
> > The
> > >> > key
> > >> > >> to making this "easy" to implement will lie in our abiilty to
> keep
> > >> the
> > >> > >> assertions we want to have relatively simple. The more simplistic
> > the
> > >> > >> language in the Gherkin .feature files the easier the job it will
> > be
> > >> for
> > >> > >> GLVs to build their infrastructure. Of course, once that
> > >> infrastructure
> > >> > is
> > >> > >> in place, the GLV developer just has to write the GLV version of
> > the
> > >> > >> Gremlin specified in the .feature file. So you can look at all
> the
> > >> > >> "infrastructure" code here in this pair of files:
> > >> > >>
> > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > >> > >> lin-python/src/main/jython/radish
> > >> > >>
> > >> > >> and then you can look at the GLV Gremlin translations
> specifically
> > >> here:
> > >> > >>
> > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > >> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> > >> > >>
> > >> > >> I think this approach works pretty well and solves our general
> > >> problems
> > >> > >> for
> > >> > >> GLV testing. There is some pain up front in implementing the
> > >> > >> "infrastructure" but after that new Gremlin tests added to
> .feature
> > >> > files
> > >> > >> just need to translated in the GLV. I suppose we could
> "automate" a
> > >> good
> > >> > >> portion of the translation with reflection of some sort. Anything
> > >> else
> > >> > >> could just be handled manually.
> > >> > >>
> > >> > >> Not sure if we need to use this new model to wholly replace the
> old
> > >> one.
> > >> > >> The process test suite has its place in helping graph database
> > >> providers
> > >> > >> test their stuff. I also imagine that introducing this approach
> in
> > >> that
> > >> > >> context would create a breaking change which we would then need
> to
> > >> push
> > >> > >> off
> > >> > >> to 3.4.0.  I suppose that gives us time to think, but for now it
> > >> might
> > >> > not
> > >> > >> be best to conflate the two and just treat them as separate
> aspects
> > >> of
> > >> > the
> > >> > >> test suite.
> > >> > >>
> > >> > >> Anyway - it's important we settle on an approach to testing as we
> > >> really
> > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without getting
> > the
> > >> > test
> > >> > >> suite solid. please yell if you have any ideas or feedback on
> this
> > >> > >> approach.
> > >> > >>
> > >> > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

Thanks for the update. I'm trying to keep the test language as simple as
possible so that we don't need an overly complicated test implementation.
Hopefully that will help make the .NET approach as easy as possible.

On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <jo...@gmail.com>
wrote:

> I've been looking into Gherkin support for .NET: SpecFlow, the cucumber
> implementation for .NET <https://cucumber.io/docs#cucumber-implementations
> >,
> does not support .NET Core platform (we use .NET Core build tools for the
> .NET GLV) and only supports .NET Framework.
>
> From what I can see <https://github.com/techtalk/SpecFlow/projects/2>,
> .NET
> Core support on SpecFlow is coming at a very slow pace and we shouldn't
> expect to land any time soon (there were some design decisions in SpecFlow
> library that makes supporting other platforms non-trivial, like requiring
> code gen).
>
> The alternative would be to implement our own harness to support it: from a
> xunit test, look for certain types and parse the annotations, and execute
> them using the Gherkin features.
> There is a .NET cross-platform Gherkin parser:
> https://github.com/cucumber/gherkin-dotnet
> I'll continue looking into this option and try to understand the effort
> required...
>
> On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com>
> wrote:
>
> > Nice! Gherkin will make our lives easier with a growing number of GLVs.
> >
> > We should find a way to define the different features supported by each
> > GLV, as it's reasonable to have different maturity levels per GLV (ie:
> > lambdas support, traversal strategy, ...). I don't know if it will be
> > beneficial to do it in the Gherkin files or within each GLV
> implementation.
> > Also, we should consider the process of rolling out a new method / class
> > in the java implementation, how that could affect each GLV.
> >
> > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <spmallette@gmail.com
> >
> > wrote:
> >
> >> that's what i meant by "reflection" or as you suggest eval(). I guess
> the
> >> point is that if the language can support some way of taking the string
> >> value and turning it automatically into a traversal in that GLVs style
> >> then
> >> we should do that.
> >>
> >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru>
> wrote:
> >>
> >> > For unparameterized queries it can probably be as easy as:
> >> >
> >> > @given("the traversal of")
> >> > def translate_traversal(step):
> >> >     g = step.context.g
> >> >     step.context.traversal = eval(step.text)
> >> >
> >> >
> >> > Cheers,
> >> > Daniel
> >> >
> >> >
> >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru>
> >> wrote:
> >> >
> >> > > That's great stuff. I haven't used Cucumber / Gherkin for years,
> but I
> >> > > really like the BDD approach.
> >> > >
> >> > > and then you can look at the GLV Gremlin translations specifically
> >> here:
> >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> >> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> >> > >
> >> > >
> >> > > This part is the only thing that looks weird to me. You're basically
> >> > > writing every query twice; is there really no easier way to do that?
> >> > >
> >> > > Cheers,
> >> > > Daniel
> >> > >
> >> > >
> >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> >> spmallette@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> I've brought this issue up in the past and had suggested some
> >> options I
> >> > >> had
> >> > >> in mind but now I've finally put the basics of those ideas in place
> >> so I
> >> > >> figured I'd start a fresh thread. Recall that the issue at hand is
> >> that
> >> > we
> >> > >> don't have a test suite for GLVs as gremlin-test is bound to the
> >> JVM. We
> >> > >> have some tricks that let us test gremlin-python with it but those
> >> > tricks
> >> > >> won't work for every language and we now have the first language in
> >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't support
> it
> >> > >> (yes,
> >> > >> i know that gremlin-javascript can run on the jvm but there are
> >> issues
> >> > >> with
> >> > >> getting it all to work with the test framework that make it unduly
> >> > >> complicated).
> >> > >>
> >> > >> On other threads I offered the idea that we look to use Gherkin to
> >> write
> >> > >> general Gremlin test specifications, which then could be read and
> >> > >> processed
> >> > >> by the wide variety of test frameworks that can read that format -
> >> there
> >> > >> tend to be Gherkin processors in just about every language - for
> >> > example,
> >> > >> see:
> >> > >>
> >> > >> https://cucumber.io/
> >> > >>
> >> > >> I just created this issue:
> >> > >>
> >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> >> > >>
> >> > >> and pushed this branch:
> >> > >>
> >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> >> > >>
> >> > >> which demonstrates how this works with gremlin-python. The basic
> >> anatomy
> >> > >> of
> >> > >> this setup involves this new directory in gremlin-test:
> >> > >>
> >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> >> > >> lin-test/features
> >> > >>
> >> > >> It contains the Gherkin .features files. These are the test
> >> > >> specifications.
> >> > >> They are written using gremlin-java as the "model" language. GLVs
> >> will
> >> > >> then
> >> > >> need to write some infrastructure to process these Gherkin files.
> The
> >> > key
> >> > >> to making this "easy" to implement will lie in our abiilty to keep
> >> the
> >> > >> assertions we want to have relatively simple. The more simplistic
> the
> >> > >> language in the Gherkin .feature files the easier the job it will
> be
> >> for
> >> > >> GLVs to build their infrastructure. Of course, once that
> >> infrastructure
> >> > is
> >> > >> in place, the GLV developer just has to write the GLV version of
> the
> >> > >> Gremlin specified in the .feature file. So you can look at all the
> >> > >> "infrastructure" code here in this pair of files:
> >> > >>
> >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> >> > >> lin-python/src/main/jython/radish
> >> > >>
> >> > >> and then you can look at the GLV Gremlin translations specifically
> >> here:
> >> > >>
> >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> >> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> >> > >>
> >> > >> I think this approach works pretty well and solves our general
> >> problems
> >> > >> for
> >> > >> GLV testing. There is some pain up front in implementing the
> >> > >> "infrastructure" but after that new Gremlin tests added to .feature
> >> > files
> >> > >> just need to translated in the GLV. I suppose we could "automate" a
> >> good
> >> > >> portion of the translation with reflection of some sort. Anything
> >> else
> >> > >> could just be handled manually.
> >> > >>
> >> > >> Not sure if we need to use this new model to wholly replace the old
> >> one.
> >> > >> The process test suite has its place in helping graph database
> >> providers
> >> > >> test their stuff. I also imagine that introducing this approach in
> >> that
> >> > >> context would create a breaking change which we would then need to
> >> push
> >> > >> off
> >> > >> to 3.4.0.  I suppose that gives us time to think, but for now it
> >> might
> >> > not
> >> > >> be best to conflate the two and just treat them as separate aspects
> >> of
> >> > the
> >> > >> test suite.
> >> > >>
> >> > >> Anyway - it's important we settle on an approach to testing as we
> >> really
> >> > >> shouldn't do a GA release of the Gremlin .NET GLV without getting
> the
> >> > test
> >> > >> suite solid. please yell if you have any ideas or feedback on this
> >> > >> approach.
> >> > >>
> >> > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

I've been looking into Gherkin support for .NET: SpecFlow, the cucumber
implementation for .NET <https://cucumber.io/docs#cucumber-implementations>,
does not support .NET Core platform (we use .NET Core build tools for the
.NET GLV) and only supports .NET Framework.

From what I can see <https://github.com/techtalk/SpecFlow/projects/2>, .NET
Core support on SpecFlow is coming at a very slow pace and we shouldn't
expect to land any time soon (there were some design decisions in SpecFlow
library that makes supporting other platforms non-trivial, like requiring
code gen).

The alternative would be to implement our own harness to support it: from a
xunit test, look for certain types and parse the annotations, and execute
them using the Gherkin features.
There is a .NET cross-platform Gherkin parser:
https://github.com/cucumber/gherkin-dotnet
I'll continue looking into this option and try to understand the effort
required...

On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <jo...@gmail.com>
wrote:

> Nice! Gherkin will make our lives easier with a growing number of GLVs.
>
> We should find a way to define the different features supported by each
> GLV, as it's reasonable to have different maturity levels per GLV (ie:
> lambdas support, traversal strategy, ...). I don't know if it will be
> beneficial to do it in the Gherkin files or within each GLV implementation.
> Also, we should consider the process of rolling out a new method / class
> in the java implementation, how that could affect each GLV.
>
> On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> that's what i meant by "reflection" or as you suggest eval(). I guess the
>> point is that if the language can support some way of taking the string
>> value and turning it automatically into a traversal in that GLVs style
>> then
>> we should do that.
>>
>> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:
>>
>> > For unparameterized queries it can probably be as easy as:
>> >
>> > @given("the traversal of")
>> > def translate_traversal(step):
>> >     g = step.context.g
>> >     step.context.traversal = eval(step.text)
>> >
>> >
>> > Cheers,
>> > Daniel
>> >
>> >
>> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru>
>> wrote:
>> >
>> > > That's great stuff. I haven't used Cucumber / Gherkin for years, but I
>> > > really like the BDD approach.
>> > >
>> > > and then you can look at the GLV Gremlin translations specifically
>> here:
>> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
>> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
>> > >
>> > >
>> > > This part is the only thing that looks weird to me. You're basically
>> > > writing every query twice; is there really no easier way to do that?
>> > >
>> > > Cheers,
>> > > Daniel
>> > >
>> > >
>> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
>> spmallette@gmail.com>
>> > > wrote:
>> > >
>> > >> I've brought this issue up in the past and had suggested some
>> options I
>> > >> had
>> > >> in mind but now I've finally put the basics of those ideas in place
>> so I
>> > >> figured I'd start a fresh thread. Recall that the issue at hand is
>> that
>> > we
>> > >> don't have a test suite for GLVs as gremlin-test is bound to the
>> JVM. We
>> > >> have some tricks that let us test gremlin-python with it but those
>> > tricks
>> > >> won't work for every language and we now have the first language in
>> > >> gremlin-dotnet and upcoming gremlin-javascript which won't support it
>> > >> (yes,
>> > >> i know that gremlin-javascript can run on the jvm but there are
>> issues
>> > >> with
>> > >> getting it all to work with the test framework that make it unduly
>> > >> complicated).
>> > >>
>> > >> On other threads I offered the idea that we look to use Gherkin to
>> write
>> > >> general Gremlin test specifications, which then could be read and
>> > >> processed
>> > >> by the wide variety of test frameworks that can read that format -
>> there
>> > >> tend to be Gherkin processors in just about every language - for
>> > example,
>> > >> see:
>> > >>
>> > >> https://cucumber.io/
>> > >>
>> > >> I just created this issue:
>> > >>
>> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
>> > >>
>> > >> and pushed this branch:
>> > >>
>> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
>> > >>
>> > >> which demonstrates how this works with gremlin-python. The basic
>> anatomy
>> > >> of
>> > >> this setup involves this new directory in gremlin-test:
>> > >>
>> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
>> > >> lin-test/features
>> > >>
>> > >> It contains the Gherkin .features files. These are the test
>> > >> specifications.
>> > >> They are written using gremlin-java as the "model" language. GLVs
>> will
>> > >> then
>> > >> need to write some infrastructure to process these Gherkin files. The
>> > key
>> > >> to making this "easy" to implement will lie in our abiilty to keep
>> the
>> > >> assertions we want to have relatively simple. The more simplistic the
>> > >> language in the Gherkin .feature files the easier the job it will be
>> for
>> > >> GLVs to build their infrastructure. Of course, once that
>> infrastructure
>> > is
>> > >> in place, the GLV developer just has to write the GLV version of the
>> > >> Gremlin specified in the .feature file. So you can look at all the
>> > >> "infrastructure" code here in this pair of files:
>> > >>
>> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
>> > >> lin-python/src/main/jython/radish
>> > >>
>> > >> and then you can look at the GLV Gremlin translations specifically
>> here:
>> > >>
>> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
>> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
>> > >>
>> > >> I think this approach works pretty well and solves our general
>> problems
>> > >> for
>> > >> GLV testing. There is some pain up front in implementing the
>> > >> "infrastructure" but after that new Gremlin tests added to .feature
>> > files
>> > >> just need to translated in the GLV. I suppose we could "automate" a
>> good
>> > >> portion of the translation with reflection of some sort. Anything
>> else
>> > >> could just be handled manually.
>> > >>
>> > >> Not sure if we need to use this new model to wholly replace the old
>> one.
>> > >> The process test suite has its place in helping graph database
>> providers
>> > >> test their stuff. I also imagine that introducing this approach in
>> that
>> > >> context would create a breaking change which we would then need to
>> push
>> > >> off
>> > >> to 3.4.0.  I suppose that gives us time to think, but for now it
>> might
>> > not
>> > >> be best to conflate the two and just treat them as separate aspects
>> of
>> > the
>> > >> test suite.
>> > >>
>> > >> Anyway - it's important we settle on an approach to testing as we
>> really
>> > >> shouldn't do a GA release of the Gremlin .NET GLV without getting the
>> > test
>> > >> suite solid. please yell if you have any ideas or feedback on this
>> > >> approach.
>> > >>
>> > >
>> > >
>> >
>>
>
>

Re: [DISCUSS] GLV Test Suite

Posted by Jorge Bay Gondra <jo...@gmail.com>.

Nice! Gherkin will make our lives easier with a growing number of GLVs.

We should find a way to define the different features supported by each
GLV, as it's reasonable to have different maturity levels per GLV (ie:
lambdas support, traversal strategy, ...). I don't know if it will be
beneficial to do it in the Gherkin files or within each GLV implementation.
Also, we should consider the process of rolling out a new method / class in
the java implementation, how that could affect each GLV.

On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> that's what i meant by "reflection" or as you suggest eval(). I guess the
> point is that if the language can support some way of taking the string
> value and turning it automatically into a traversal in that GLVs style then
> we should do that.
>
> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:
>
> > For unparameterized queries it can probably be as easy as:
> >
> > @given("the traversal of")
> > def translate_traversal(step):
> >     g = step.context.g
> >     step.context.traversal = eval(step.text)
> >
> >
> > Cheers,
> > Daniel
> >
> >
> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:
> >
> > > That's great stuff. I haven't used Cucumber / Gherkin for years, but I
> > > really like the BDD approach.
> > >
> > > and then you can look at the GLV Gremlin translations specifically
> here:
> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> > >
> > >
> > > This part is the only thing that looks weird to me. You're basically
> > > writing every query twice; is there really no easier way to do that?
> > >
> > > Cheers,
> > > Daniel
> > >
> > >
> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > >> I've brought this issue up in the past and had suggested some options
> I
> > >> had
> > >> in mind but now I've finally put the basics of those ideas in place
> so I
> > >> figured I'd start a fresh thread. Recall that the issue at hand is
> that
> > we
> > >> don't have a test suite for GLVs as gremlin-test is bound to the JVM.
> We
> > >> have some tricks that let us test gremlin-python with it but those
> > tricks
> > >> won't work for every language and we now have the first language in
> > >> gremlin-dotnet and upcoming gremlin-javascript which won't support it
> > >> (yes,
> > >> i know that gremlin-javascript can run on the jvm but there are issues
> > >> with
> > >> getting it all to work with the test framework that make it unduly
> > >> complicated).
> > >>
> > >> On other threads I offered the idea that we look to use Gherkin to
> write
> > >> general Gremlin test specifications, which then could be read and
> > >> processed
> > >> by the wide variety of test frameworks that can read that format -
> there
> > >> tend to be Gherkin processors in just about every language - for
> > example,
> > >> see:
> > >>
> > >> https://cucumber.io/
> > >>
> > >> I just created this issue:
> > >>
> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > >>
> > >> and pushed this branch:
> > >>
> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> > >>
> > >> which demonstrates how this works with gremlin-python. The basic
> anatomy
> > >> of
> > >> this setup involves this new directory in gremlin-test:
> > >>
> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > >> lin-test/features
> > >>
> > >> It contains the Gherkin .features files. These are the test
> > >> specifications.
> > >> They are written using gremlin-java as the "model" language. GLVs will
> > >> then
> > >> need to write some infrastructure to process these Gherkin files. The
> > key
> > >> to making this "easy" to implement will lie in our abiilty to keep the
> > >> assertions we want to have relatively simple. The more simplistic the
> > >> language in the Gherkin .feature files the easier the job it will be
> for
> > >> GLVs to build their infrastructure. Of course, once that
> infrastructure
> > is
> > >> in place, the GLV developer just has to write the GLV version of the
> > >> Gremlin specified in the .feature file. So you can look at all the
> > >> "infrastructure" code here in this pair of files:
> > >>
> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> > >> lin-python/src/main/jython/radish
> > >>
> > >> and then you can look at the GLV Gremlin translations specifically
> here:
> > >>
> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> > >>
> > >> I think this approach works pretty well and solves our general
> problems
> > >> for
> > >> GLV testing. There is some pain up front in implementing the
> > >> "infrastructure" but after that new Gremlin tests added to .feature
> > files
> > >> just need to translated in the GLV. I suppose we could "automate" a
> good
> > >> portion of the translation with reflection of some sort. Anything else
> > >> could just be handled manually.
> > >>
> > >> Not sure if we need to use this new model to wholly replace the old
> one.
> > >> The process test suite has its place in helping graph database
> providers
> > >> test their stuff. I also imagine that introducing this approach in
> that
> > >> context would create a breaking change which we would then need to
> push
> > >> off
> > >> to 3.4.0.  I suppose that gives us time to think, but for now it might
> > not
> > >> be best to conflate the two and just treat them as separate aspects of
> > the
> > >> test suite.
> > >>
> > >> Anyway - it's important we settle on an approach to testing as we
> really
> > >> shouldn't do a GA release of the Gremlin .NET GLV without getting the
> > test
> > >> suite solid. please yell if you have any ideas or feedback on this
> > >> approach.
> > >>
> > >
> > >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Stephen Mallette <sp...@gmail.com>.

that's what i meant by "reflection" or as you suggest eval(). I guess the
point is that if the language can support some way of taking the string
value and turning it automatically into a traversal in that GLVs style then
we should do that.

On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:

> For unparameterized queries it can probably be as easy as:
>
> @given("the traversal of")
> def translate_traversal(step):
>     g = step.context.g
>     step.context.traversal = eval(step.text)
>
>
> Cheers,
> Daniel
>
>
> On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:
>
> > That's great stuff. I haven't used Cucumber / Gherkin for years, but I
> > really like the BDD approach.
> >
> > and then you can look at the GLV Gremlin translations specifically here:
> >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> >
> >
> > This part is the only thing that looks weird to me. You're basically
> > writing every query twice; is there really no easier way to do that?
> >
> > Cheers,
> > Daniel
> >
> >
> > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> >> I've brought this issue up in the past and had suggested some options I
> >> had
> >> in mind but now I've finally put the basics of those ideas in place so I
> >> figured I'd start a fresh thread. Recall that the issue at hand is that
> we
> >> don't have a test suite for GLVs as gremlin-test is bound to the JVM. We
> >> have some tricks that let us test gremlin-python with it but those
> tricks
> >> won't work for every language and we now have the first language in
> >> gremlin-dotnet and upcoming gremlin-javascript which won't support it
> >> (yes,
> >> i know that gremlin-javascript can run on the jvm but there are issues
> >> with
> >> getting it all to work with the test framework that make it unduly
> >> complicated).
> >>
> >> On other threads I offered the idea that we look to use Gherkin to write
> >> general Gremlin test specifications, which then could be read and
> >> processed
> >> by the wide variety of test frameworks that can read that format - there
> >> tend to be Gherkin processors in just about every language - for
> example,
> >> see:
> >>
> >> https://cucumber.io/
> >>
> >> I just created this issue:
> >>
> >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> >>
> >> and pushed this branch:
> >>
> >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> >>
> >> which demonstrates how this works with gremlin-python. The basic anatomy
> >> of
> >> this setup involves this new directory in gremlin-test:
> >>
> >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> >> lin-test/features
> >>
> >> It contains the Gherkin .features files. These are the test
> >> specifications.
> >> They are written using gremlin-java as the "model" language. GLVs will
> >> then
> >> need to write some infrastructure to process these Gherkin files. The
> key
> >> to making this "easy" to implement will lie in our abiilty to keep the
> >> assertions we want to have relatively simple. The more simplistic the
> >> language in the Gherkin .feature files the easier the job it will be for
> >> GLVs to build their infrastructure. Of course, once that infrastructure
> is
> >> in place, the GLV developer just has to write the GLV version of the
> >> Gremlin specified in the .feature file. So you can look at all the
> >> "infrastructure" code here in this pair of files:
> >>
> >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
> >> lin-python/src/main/jython/radish
> >>
> >> and then you can look at the GLV Gremlin translations specifically here:
> >>
> >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
> >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
> >>
> >> I think this approach works pretty well and solves our general problems
> >> for
> >> GLV testing. There is some pain up front in implementing the
> >> "infrastructure" but after that new Gremlin tests added to .feature
> files
> >> just need to translated in the GLV. I suppose we could "automate" a good
> >> portion of the translation with reflection of some sort. Anything else
> >> could just be handled manually.
> >>
> >> Not sure if we need to use this new model to wholly replace the old one.
> >> The process test suite has its place in helping graph database providers
> >> test their stuff. I also imagine that introducing this approach in that
> >> context would create a breaking change which we would then need to push
> >> off
> >> to 3.4.0.  I suppose that gives us time to think, but for now it might
> not
> >> be best to conflate the two and just treat them as separate aspects of
> the
> >> test suite.
> >>
> >> Anyway - it's important we settle on an approach to testing as we really
> >> shouldn't do a GA release of the Gremlin .NET GLV without getting the
> test
> >> suite solid. please yell if you have any ideas or feedback on this
> >> approach.
> >>
> >
> >
>

Re: [DISCUSS] GLV Test Suite

Posted by Daniel Kuppitz <me...@gremlin.guru>.

For unparameterized queries it can probably be as easy as:

@given("the traversal of")
def translate_traversal(step):
    g = step.context.g
    step.context.traversal = eval(step.text)


Cheers,
Daniel


On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:

> That's great stuff. I haven't used Cucumber / Gherkin for years, but I
> really like the BDD approach.
>
> and then you can look at the GLV Gremlin translations specifically here:
>> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
>> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
>
>
> This part is the only thing that looks weird to me. You're basically
> writing every query twice; is there really no easier way to do that?
>
> Cheers,
> Daniel
>
>
> On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> I've brought this issue up in the past and had suggested some options I
>> had
>> in mind but now I've finally put the basics of those ideas in place so I
>> figured I'd start a fresh thread. Recall that the issue at hand is that we
>> don't have a test suite for GLVs as gremlin-test is bound to the JVM. We
>> have some tricks that let us test gremlin-python with it but those tricks
>> won't work for every language and we now have the first language in
>> gremlin-dotnet and upcoming gremlin-javascript which won't support it
>> (yes,
>> i know that gremlin-javascript can run on the jvm but there are issues
>> with
>> getting it all to work with the test framework that make it unduly
>> complicated).
>>
>> On other threads I offered the idea that we look to use Gherkin to write
>> general Gremlin test specifications, which then could be read and
>> processed
>> by the wide variety of test frameworks that can read that format - there
>> tend to be Gherkin processors in just about every language - for example,
>> see:
>>
>> https://cucumber.io/
>>
>> I just created this issue:
>>
>> https://issues.apache.org/jira/browse/TINKERPOP-1784
>>
>> and pushed this branch:
>>
>> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
>>
>> which demonstrates how this works with gremlin-python. The basic anatomy
>> of
>> this setup involves this new directory in gremlin-test:
>>
>> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
>> lin-test/features
>>
>> It contains the Gherkin .features files. These are the test
>> specifications.
>> They are written using gremlin-java as the "model" language. GLVs will
>> then
>> need to write some infrastructure to process these Gherkin files. The key
>> to making this "easy" to implement will lie in our abiilty to keep the
>> assertions we want to have relatively simple. The more simplistic the
>> language in the Gherkin .feature files the easier the job it will be for
>> GLVs to build their infrastructure. Of course, once that infrastructure is
>> in place, the GLV developer just has to write the GLV version of the
>> Gremlin specified in the .feature file. So you can look at all the
>> "infrastructure" code here in this pair of files:
>>
>> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem
>> lin-python/src/main/jython/radish
>>
>> and then you can look at the GLV Gremlin translations specifically here:
>>
>> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem
>> lin-python/src/main/jython/radish/count_features_step.py#L34-L46
>>
>> I think this approach works pretty well and solves our general problems
>> for
>> GLV testing. There is some pain up front in implementing the
>> "infrastructure" but after that new Gremlin tests added to .feature files
>> just need to translated in the GLV. I suppose we could "automate" a good
>> portion of the translation with reflection of some sort. Anything else
>> could just be handled manually.
>>
>> Not sure if we need to use this new model to wholly replace the old one.
>> The process test suite has its place in helping graph database providers
>> test their stuff. I also imagine that introducing this approach in that
>> context would create a breaking change which we would then need to push
>> off
>> to 3.4.0.  I suppose that gives us time to think, but for now it might not
>> be best to conflate the two and just treat them as separate aspects of the
>> test suite.
>>
>> Anyway - it's important we settle on an approach to testing as we really
>> shouldn't do a GA release of the Gremlin .NET GLV without getting the test
>> suite solid. please yell if you have any ideas or feedback on this
>> approach.
>>
>
>

Re: [DISCUSS] GLV Test Suite

Posted by Daniel Kuppitz <me...@gremlin.guru>.

That's great stuff. I haven't used Cucumber / Gherkin for years, but I
really like the BDD approach.

and then you can look at the GLV Gremlin translations specifically here:
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/count_features_step.py#L34-L46


This part is the only thing that looks weird to me. You're basically
writing every query twice; is there really no easier way to do that?

Cheers,
Daniel


On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> I've brought this issue up in the past and had suggested some options I had
> in mind but now I've finally put the basics of those ideas in place so I
> figured I'd start a fresh thread. Recall that the issue at hand is that we
> don't have a test suite for GLVs as gremlin-test is bound to the JVM. We
> have some tricks that let us test gremlin-python with it but those tricks
> won't work for every language and we now have the first language in
> gremlin-dotnet and upcoming gremlin-javascript which won't support it (yes,
> i know that gremlin-javascript can run on the jvm but there are issues with
> getting it all to work with the test framework that make it unduly
> complicated).
>
> On other threads I offered the idea that we look to use Gherkin to write
> general Gremlin test specifications, which then could be read and processed
> by the wide variety of test frameworks that can read that format - there
> tend to be Gherkin processors in just about every language - for example,
> see:
>
> https://cucumber.io/
>
> I just created this issue:
>
> https://issues.apache.org/jira/browse/TINKERPOP-1784
>
> and pushed this branch:
>
> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
>
> which demonstrates how this works with gremlin-python. The basic anatomy of
> this setup involves this new directory in gremlin-test:
>
> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> gremlin-test/features
>
> It contains the Gherkin .features files. These are the test specifications.
> They are written using gremlin-java as the "model" language. GLVs will then
> need to write some infrastructure to process these Gherkin files. The key
> to making this "easy" to implement will lie in our abiilty to keep the
> assertions we want to have relatively simple. The more simplistic the
> language in the Gherkin .feature files the easier the job it will be for
> GLVs to build their infrastructure. Of course, once that infrastructure is
> in place, the GLV developer just has to write the GLV version of the
> Gremlin specified in the .feature file. So you can look at all the
> "infrastructure" code here in this pair of files:
>
> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish
>
> and then you can look at the GLV Gremlin translations specifically here:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/count_features_step.py#L34-L46
>
> I think this approach works pretty well and solves our general problems for
> GLV testing. There is some pain up front in implementing the
> "infrastructure" but after that new Gremlin tests added to .feature files
> just need to translated in the GLV. I suppose we could "automate" a good
> portion of the translation with reflection of some sort. Anything else
> could just be handled manually.
>
> Not sure if we need to use this new model to wholly replace the old one.
> The process test suite has its place in helping graph database providers
> test their stuff. I also imagine that introducing this approach in that
> context would create a breaking change which we would then need to push off
> to 3.4.0.  I suppose that gives us time to think, but for now it might not
> be best to conflate the two and just treat them as separate aspects of the
> test suite.
>
> Anyway - it's important we settle on an approach to testing as we really
> shouldn't do a GA release of the Gremlin .NET GLV without getting the test
> suite solid. please yell if you have any ideas or feedback on this
> approach.
>