You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by Stephen Mallette <sp...@gmail.com> on 2017/10/12 18:54:36 UTC
Re: [DISCUSS] GLV Test Suite
Just a quick update on this item - it continues........
Went down a bad path earlier in the week and ended up shelving a lot of
work - ended up just coded into a corner. Anyway, I've not really changed
much in the implementation, but I've still not reached a point where the
addition of a new .feature files comes without meeting some new type of
assertion that has to be dealt with. This ends up slowing down progress on
porting over the tests. Please feel free to peruse the latest changes on
the branch and let me know if there's any feedback.
On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <sp...@gmail.com>
wrote:
> I did some pretty heavy refactoring to the python test logic (altered a
> bit by some revision of the gherkin feature file language) and the result
> is a much more simplified test logic file:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/feature_steps.py
>
> About 120 lines of code (down from about 170). If you include the test
> logic "setup" file:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> gremlin-python/src/main/jython/radish/terrain.py
>
> we end up with about 250 lines of test logic total (take out the
> comment/license and we're probably well under 200 total - not too bad). I
> think I'm closing in on the end to infrastructure building here so the
> basic framework is getting close to final at this point I believe. I'll
> keep scanning the tests looking for other types of assertions that I've not
> yet covered, but it's getting pretty solid I think. Hopefully, there won't
> need to be too many more lines of code needed to express the test logic as
> I like how things are looking right now.
>
>
>
>
> On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com> wrote:
>
>> Great progress! I like how you avoided using ids, even if it adds some
>> complexity to the transformation required.
>>
>> The Python step definitions are still quite simple, I think it would be
>> mostly the same on the rest of the languages.
>>
>> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <sp...@gmail.com>
>> wrote:
>>
>> > Just another update on progress with the test suite. The language of the
>> > .feature files is getting slightly more complex as I try to translate
>> more
>> > and more of our Java process suite tests into the language of the
>> gherkin
>> > files. You can see here where I've added the ability to include
>> parameters:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/features/filter/Has.feature#L33
>> >
>> > I also came up with a method of asserting edges (didn't want to rely on
>> ids
>> > as it makes the gherkin harder to read, plus i didn't want to assume
>> > TinkerGraph identifiers in case these tests were every used with some
>> other
>> > graph database that didn't use longs):
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/features/map/Vertex.feature#L105-L111
>> >
>> > for all those additions (and others) the logic required by the GLV to
>> > process these tests has stayed surprisingly simple:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-python/src/main/jython/radish/feature_steps.py
>> >
>> > There's a fair bit of regex/string manipulation involved there, but it's
>> > processing strings from the feature file so that's the nature of it I
>> > suppose. I think I'm of the mind that I want to port all of the tests to
>> > feature files, so I wrote this unit test to help validate that none were
>> > missed:
>> >
>> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
>> > structure/FeatureCoverageTest.java
>> >
>> > I don't know that we have to get to 100% porting right away, but I think
>> > that once we have these gherkin files written they not only become the
>> > basis for our current GLV testing work, but we might be able to simply
>> use
>> > them for testing the Java stuff as well - that would rid us of having
>> the
>> > test code duplication. It also sets us up with a portable body of tests
>> > that can be re-used in TinkerPop 4.x.
>> >
>> > I'm open to suggestions if anyone has any.
>> >
>> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
>> > jorgebaygondra@gmail.com
>> > > wrote:
>> >
>> > > I was able to build a proof of concept for a Gherkin-based test
>> runner in
>> > > C#, that takes the proposed count and select features
>> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
>> > > gremlin-test/features/map>
>> > > and runs them using C# step definitions.
>> > >
>> > > It uses the Gherkin parser <https://github.com/cucumber/g
>> herkin-dotnet>
>> > > from
>> > > cucumber, there isn't a release of the parser with .NET Core support
>> so
>> > > I've
>> > > asked them to release one
>> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
>> > > limitation from their source files). If they are not able to release
>> it
>> > in
>> > > the next few days, we can implement our own as it should be pretty
>> > straight
>> > > forward.
>> > >
>> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
>> spmallette@gmail.com>
>> > > wrote:
>> > >
>> > > > Thanks for the update. I'm trying to keep the test language as
>> simple
>> > as
>> > > > possible so that we don't need an overly complicated test
>> > implementation.
>> > > > Hopefully that will help make the .NET approach as easy as possible.
>> > > >
>> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
>> > > > jorgebaygondra@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
>> > cucumber
>> > > > > implementation for .NET <https://cucumber.io/docs#
>> > > > cucumber-implementations
>> > > > > >,
>> > > > > does not support .NET Core platform (we use .NET Core build tools
>> for
>> > > the
>> > > > > .NET GLV) and only supports .NET Framework.
>> > > > >
>> > > > > From what I can see <https://github.com/techtalk/S
>> pecFlow/projects/2
>> > >,
>> > > > > .NET
>> > > > > Core support on SpecFlow is coming at a very slow pace and we
>> > shouldn't
>> > > > > expect to land any time soon (there were some design decisions in
>> > > > SpecFlow
>> > > > > library that makes supporting other platforms non-trivial, like
>> > > requiring
>> > > > > code gen).
>> > > > >
>> > > > > The alternative would be to implement our own harness to support
>> it:
>> > > > from a
>> > > > > xunit test, look for certain types and parse the annotations, and
>> > > execute
>> > > > > them using the Gherkin features.
>> > > > > There is a .NET cross-platform Gherkin parser:
>> > > > > https://github.com/cucumber/gherkin-dotnet
>> > > > > I'll continue looking into this option and try to understand the
>> > effort
>> > > > > required...
>> > > > >
>> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
>> > > > > jorgebaygondra@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Nice! Gherkin will make our lives easier with a growing number
>> of
>> > > GLVs.
>> > > > > >
>> > > > > > We should find a way to define the different features supported
>> by
>> > > each
>> > > > > > GLV, as it's reasonable to have different maturity levels per
>> GLV
>> > > (ie:
>> > > > > > lambdas support, traversal strategy, ...). I don't know if it
>> will
>> > be
>> > > > > > beneficial to do it in the Gherkin files or within each GLV
>> > > > > implementation.
>> > > > > > Also, we should consider the process of rolling out a new
>> method /
>> > > > class
>> > > > > > in the java implementation, how that could affect each GLV.
>> > > > > >
>> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
>> > > > spmallette@gmail.com
>> > > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > >> that's what i meant by "reflection" or as you suggest eval(). I
>> > > guess
>> > > > > the
>> > > > > >> point is that if the language can support some way of taking
>> the
>> > > > string
>> > > > > >> value and turning it automatically into a traversal in that
>> GLVs
>> > > style
>> > > > > >> then
>> > > > > >> we should do that.
>> > > > > >>
>> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
>> <me...@gremlin.guru>
>> > > > > wrote:
>> > > > > >>
>> > > > > >> > For unparameterized queries it can probably be as easy as:
>> > > > > >> >
>> > > > > >> > @given("the traversal of")
>> > > > > >> > def translate_traversal(step):
>> > > > > >> > g = step.context.g
>> > > > > >> > step.context.traversal = eval(step.text)
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > Cheers,
>> > > > > >> > Daniel
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
>> <me@gremlin.guru
>> > >
>> > > > > >> wrote:
>> > > > > >> >
>> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
>> > years,
>> > > > > but I
>> > > > > >> > > really like the BDD approach.
>> > > > > >> > >
>> > > > > >> > > and then you can look at the GLV Gremlin translations
>> > > specifically
>> > > > > >> here:
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
>> > > L34-L46
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > This part is the only thing that looks weird to me. You're
>> > > > basically
>> > > > > >> > > writing every query twice; is there really no easier way
>> to do
>> > > > that?
>> > > > > >> > >
>> > > > > >> > > Cheers,
>> > > > > >> > > Daniel
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
>> > > > > >> spmallette@gmail.com>
>> > > > > >> > > wrote:
>> > > > > >> > >
>> > > > > >> > >> I've brought this issue up in the past and had suggested
>> some
>> > > > > >> options I
>> > > > > >> > >> had
>> > > > > >> > >> in mind but now I've finally put the basics of those
>> ideas in
>> > > > place
>> > > > > >> so I
>> > > > > >> > >> figured I'd start a fresh thread. Recall that the issue at
>> > hand
>> > > > is
>> > > > > >> that
>> > > > > >> > we
>> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is bound
>> to
>> > > the
>> > > > > >> JVM. We
>> > > > > >> > >> have some tricks that let us test gremlin-python with it
>> but
>> > > > those
>> > > > > >> > tricks
>> > > > > >> > >> won't work for every language and we now have the first
>> > > language
>> > > > in
>> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't
>> > > > support
>> > > > > it
>> > > > > >> > >> (yes,
>> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
>> there
>> > are
>> > > > > >> issues
>> > > > > >> > >> with
>> > > > > >> > >> getting it all to work with the test framework that make
>> it
>> > > > unduly
>> > > > > >> > >> complicated).
>> > > > > >> > >>
>> > > > > >> > >> On other threads I offered the idea that we look to use
>> > Gherkin
>> > > > to
>> > > > > >> write
>> > > > > >> > >> general Gremlin test specifications, which then could be
>> read
>> > > and
>> > > > > >> > >> processed
>> > > > > >> > >> by the wide variety of test frameworks that can read that
>> > > format
>> > > > -
>> > > > > >> there
>> > > > > >> > >> tend to be Gherkin processors in just about every
>> language -
>> > > for
>> > > > > >> > example,
>> > > > > >> > >> see:
>> > > > > >> > >>
>> > > > > >> > >> https://cucumber.io/
>> > > > > >> > >>
>> > > > > >> > >> I just created this issue:
>> > > > > >> > >>
>> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
>> > > > > >> > >>
>> > > > > >> > >> and pushed this branch:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
>> > > > > >> > >>
>> > > > > >> > >> which demonstrates how this works with gremlin-python. The
>> > > basic
>> > > > > >> anatomy
>> > > > > >> > >> of
>> > > > > >> > >> this setup involves this new directory in gremlin-test:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > >> lin-test/features
>> > > > > >> > >>
>> > > > > >> > >> It contains the Gherkin .features files. These are the
>> test
>> > > > > >> > >> specifications.
>> > > > > >> > >> They are written using gremlin-java as the "model"
>> language.
>> > > GLVs
>> > > > > >> will
>> > > > > >> > >> then
>> > > > > >> > >> need to write some infrastructure to process these Gherkin
>> > > files.
>> > > > > The
>> > > > > >> > key
>> > > > > >> > >> to making this "easy" to implement will lie in our
>> abiilty to
>> > > > keep
>> > > > > >> the
>> > > > > >> > >> assertions we want to have relatively simple. The more
>> > > simplistic
>> > > > > the
>> > > > > >> > >> language in the Gherkin .feature files the easier the job
>> it
>> > > will
>> > > > > be
>> > > > > >> for
>> > > > > >> > >> GLVs to build their infrastructure. Of course, once that
>> > > > > >> infrastructure
>> > > > > >> > is
>> > > > > >> > >> in place, the GLV developer just has to write the GLV
>> version
>> > > of
>> > > > > the
>> > > > > >> > >> Gremlin specified in the .feature file. So you can look at
>> > all
>> > > > the
>> > > > > >> > >> "infrastructure" code here in this pair of files:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > >> lin-python/src/main/jython/radish
>> > > > > >> > >>
>> > > > > >> > >> and then you can look at the GLV Gremlin translations
>> > > > specifically
>> > > > > >> here:
>> > > > > >> > >>
>> > > > > >> > >> https://github.com/apache/tink
>> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#
>> > > L34-L46
>> > > > > >> > >>
>> > > > > >> > >> I think this approach works pretty well and solves our
>> > general
>> > > > > >> problems
>> > > > > >> > >> for
>> > > > > >> > >> GLV testing. There is some pain up front in implementing
>> the
>> > > > > >> > >> "infrastructure" but after that new Gremlin tests added to
>> > > > .feature
>> > > > > >> > files
>> > > > > >> > >> just need to translated in the GLV. I suppose we could
>> > > > "automate" a
>> > > > > >> good
>> > > > > >> > >> portion of the translation with reflection of some sort.
>> > > Anything
>> > > > > >> else
>> > > > > >> > >> could just be handled manually.
>> > > > > >> > >>
>> > > > > >> > >> Not sure if we need to use this new model to wholly
>> replace
>> > the
>> > > > old
>> > > > > >> one.
>> > > > > >> > >> The process test suite has its place in helping graph
>> > database
>> > > > > >> providers
>> > > > > >> > >> test their stuff. I also imagine that introducing this
>> > approach
>> > > > in
>> > > > > >> that
>> > > > > >> > >> context would create a breaking change which we would then
>> > need
>> > > > to
>> > > > > >> push
>> > > > > >> > >> off
>> > > > > >> > >> to 3.4.0. I suppose that gives us time to think, but for
>> now
>> > > it
>> > > > > >> might
>> > > > > >> > not
>> > > > > >> > >> be best to conflate the two and just treat them as
>> separate
>> > > > aspects
>> > > > > >> of
>> > > > > >> > the
>> > > > > >> > >> test suite.
>> > > > > >> > >>
>> > > > > >> > >> Anyway - it's important we settle on an approach to
>> testing
>> > as
>> > > we
>> > > > > >> really
>> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without
>> > > getting
>> > > > > the
>> > > > > >> > test
>> > > > > >> > >> suite solid. please yell if you have any ideas or
>> feedback on
>> > > > this
>> > > > > >> > >> approach.
>> > > > > >> > >>
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [DISCUSS] GLV Test Suite
Posted by Jorge Bay Gondra <jo...@gmail.com>.
Makes sense, as long as we have the basic logic nailed down for the steps /
parameters / results, we can continue adding scenarios afterwards (new
scenarios with the same step structure should work in any language).
On Wed, Nov 8, 2017 at 1:49 PM, Stephen Mallette <sp...@gmail.com>
wrote:
> I've been away for a few days on vacation, but before I left, I made some
> pretty decent progress on the remaining tests and I'm finding less and less
> of my time being focused on the test logic and more and more of it on the
> translation of the tests themselves. I'm starting to feel comfortable that
> most of the remaining pain is in test translation which means I could
> probably issue the PR for what I have sooner than later. I'll probably just
> make sure that I have at least a "good enough" representation of tests for
> each step and then issue the PR. That will allow TINKERPOP-1827 to proceed
> more easily and I can work more at more relaxed pace on migrating all the
> tests. Please let me know if there are any concerns with that approach.
>
> On Tue, Nov 7, 2017 at 5:02 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> >
> wrote:
>
> > I've created TINKERPOP-1827 to track the progress of the test suite
> runner
> > for Gremlin .NET.
> >
> > I have a working gherkin-based runner that executes the steps (parsing
> > gremlin groovy / parameters / results) as defined in the TINKERPOP-1784
> WIP
> > branch.
> >
> > https://github.com/jorgebay/tinkerpop/commits/gherkin-cs-runner
> >
> > I'll wait for TINKERPOP-1784 to be resolved in order the follow the last
> > changes and submit a pull request for it.
> >
> > On Tue, Oct 31, 2017 at 11:09 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com
> > > wrote:
> >
> > > Ok.
> > > Going back to my initial comment regarding enums: don't mind, I've
> > > included the possibility to use enums on the C# parser.
> > >
> > > On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > >> I guess there wouldn't be a problem with that, though having it the
> way
> > it
> > >> works now was a nice check to ensure that the ScriptEngine was
> > configured
> > >> properly. I guess that shouldn't be the focus on this body of tests
> > >> though.
> > >> We should be more concerned that the elements of the Gremlin language
> > >> actually work.
> > >>
> > >> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
> > >> jorgebaygondra@gmail.com>
> > >> wrote:
> > >>
> > >> > I see that for most languages using bytecode to native language
> > >> translator
> > >> > would be the easiest solution. In the case of the JavaScript GLV, it
> > >> would
> > >> > be fairly straightforward to implement (and I plan to do so!).
> > >> > In the case of C#, being more strict regarding typing than java, it
> > >> would
> > >> > be an incomplete solution. For example:
> > >> >
> > >> > Consider the following traversal in Java: g.V().values("age")
> > >> > The type parameter for values() method is inferred at runtime and
> > things
> > >> > "just works" thanks to type erasure.
> > >> >
> > >> > In the case of C#, it should be written as
> g.V().Values<int>("age") ,
> > >> with
> > >> > the type parameter to specify which type of traverser are you
> > expecting
> > >> at
> > >> > a compile time.
> > >> >
> > >> > That's why it wont be possible to make a generic translator from
> > >> bytecode
> > >> > to C# code, without understanding which generic types are expected.
> > >> >
> > >> > In the case of C#, I was going for a translator on the C# side that
> > >> > tokenizes the gremlin traversal. Once the traversal is tokenized, in
> > the
> > >> > moment of invoking the methods (with reflection), it can be made
> aware
> > >> of
> > >> > the modern graph data (with datatypes!), ie: For method Values<>(),
> if
> > >> the
> > >> > property key is "age" -> use int as generic type parameter.
> > >> >
> > >> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <
> > spmallette@gmail.com
> > >> >
> > >> > wrote:
> > >> >
> > >> > > I can't help thinking that perhaps non-JVM languages need to
> > leverage
> > >> > > bytecode to make traversal building easier. We already have an
> > >> > established
> > >> > > pattern for converting bytecode into Traversals and while it is
> > >> different
> > >> > > for each language, it's generally governed by the Translator
> > >> interface.
> > >> > > Note that we already do this for Java, Groovy, and Python and the
> > code
> > >> > > isn't too crazy - just 200 lines or so:
> > >> > >
> > >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > >> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> > >> > JavaTranslator.java
> > >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > >> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> > >> > > GroovyTranslator.java
> > >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > >> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> > >> > > PythonTranslator.java
> > >> > >
> > >> > > I think that non-JVM languages could send the Gremlin string from
> > the
> > >> > test
> > >> > > to Gremlin Server as a script first and return the bytecode as
> JSON.
> > >> Then
> > >> > > it could use a "Translator" to then parse it into it's native
> > >> language.
> > >> > > Shouldn't we just build a CSharpTranslator similar to these? does
> > that
> > >> > make
> > >> > > sense?
> > >> > >
> > >> > >
> > >> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> > >> > > jorgebaygondra@gmail.com
> > >> > > > wrote:
> > >> > >
> > >> > > > I've been looking over the current test scenarios on the
> > >> TINKERPOP-1784
> > >> > > > branch, with the C# GLV test suite in mind, and oh man that's an
> > >> > > impressive
> > >> > > > amount of scenarios!
> > >> > > >
> > >> > > > So far, I have a suggestion.
> > >> > > >
> > >> > > > Instead of
> > >> > > >
> > >> > > > Scenario: g_V_fold_countXlocalX
> > >> > > > Given the modern graph
> > >> > > > And the traversal of
> > >> > > > """
> > >> > > > g.V().fold().count(Scope.local)
> > >> > > > """
> > >> > > >
> > >> > > > We could use
> > >> > > >
> > >> > > > Scenario: g_V_fold_countXlocalX
> > >> > > > Given the modern graph
> > >> > > > And parameter 1 being an enum
> > >> > > > """
> > >> > > > Scope.local
> > >> > > > """
> > >> > > > And the traversal of
> > >> > > > """
> > >> > > > g.V().fold().count(param1)
> > >> > > > """
> > >> > > >
> > >> > > > That way we can exclude enums from possible parameter values,
> > >> narrowing
> > >> > > > down the amount of types of parameters allowed, ie: const
> numeric,
> > >> > const
> > >> > > > strings (wrapped in double quotes), traversals (starting with
> > `__`)
> > >> ,
> > >> > > ...,
> > >> > > > making parsing the traversal a little easier.
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> > >> > spmallette@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Just a quick update on this item - it continues........
> > >> > > > >
> > >> > > > > Went down a bad path earlier in the week and ended up
> shelving a
> > >> lot
> > >> > of
> > >> > > > > work - ended up just coded into a corner. Anyway, I've not
> > really
> > >> > > changed
> > >> > > > > much in the implementation, but I've still not reached a point
> > >> where
> > >> > > the
> > >> > > > > addition of a new .feature files comes without meeting some
> new
> > >> type
> > >> > of
> > >> > > > > assertion that has to be dealt with. This ends up slowing down
> > >> > progress
> > >> > > > on
> > >> > > > > porting over the tests. Please feel free to peruse the latest
> > >> changes
> > >> > > on
> > >> > > > > the branch and let me know if there's any feedback.
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> > >> > > spmallette@gmail.com
> > >> > > > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > I did some pretty heavy refactoring to the python test logic
> > >> > > (altered a
> > >> > > > > > bit by some revision of the gherkin feature file language)
> and
> > >> the
> > >> > > > result
> > >> > > > > > is a much more simplified test logic file:
> > >> > > > > >
> > >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > >> > > > > >
> > >> > > > > > About 120 lines of code (down from about 170). If you
> include
> > >> the
> > >> > > test
> > >> > > > > > logic "setup" file:
> > >> > > > > >
> > >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > > gremlin-python/src/main/jython/radish/terrain.py
> > >> > > > > >
> > >> > > > > > we end up with about 250 lines of test logic total (take out
> > the
> > >> > > > > > comment/license and we're probably well under 200 total -
> not
> > >> too
> > >> > > > bad). I
> > >> > > > > > think I'm closing in on the end to infrastructure building
> > here
> > >> so
> > >> > > the
> > >> > > > > > basic framework is getting close to final at this point I
> > >> believe.
> > >> > > I'll
> > >> > > > > > keep scanning the tests looking for other types of
> assertions
> > >> that
> > >> > > I've
> > >> > > > > not
> > >> > > > > > yet covered, but it's getting pretty solid I think.
> Hopefully,
> > >> > there
> > >> > > > > won't
> > >> > > > > > need to be too many more lines of code needed to express the
> > >> test
> > >> > > logic
> > >> > > > > as
> > >> > > > > > I like how things are looking right now.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > >> > > > > > jorgebaygondra@gmail.com> wrote:
> > >> > > > > >
> > >> > > > > >> Great progress! I like how you avoided using ids, even if
> it
> > >> adds
> > >> > > some
> > >> > > > > >> complexity to the transformation required.
> > >> > > > > >>
> > >> > > > > >> The Python step definitions are still quite simple, I think
> > it
> > >> > would
> > >> > > > be
> > >> > > > > >> mostly the same on the rest of the languages.
> > >> > > > > >>
> > >> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > >> > > > spmallette@gmail.com
> > >> > > > > >
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >> > Just another update on progress with the test suite. The
> > >> > language
> > >> > > of
> > >> > > > > the
> > >> > > > > >> > .feature files is getting slightly more complex as I try
> to
> > >> > > > translate
> > >> > > > > >> more
> > >> > > > > >> > and more of our Java process suite tests into the
> language
> > of
> > >> > the
> > >> > > > > >> gherkin
> > >> > > > > >> > files. You can see here where I've added the ability to
> > >> include
> > >> > > > > >> parameters:
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-test/features/filter/Has.feature#L33
> > >> > > > > >> >
> > >> > > > > >> > I also came up with a method of asserting edges (didn't
> > want
> > >> to
> > >> > > rely
> > >> > > > > on
> > >> > > > > >> ids
> > >> > > > > >> > as it makes the gherkin harder to read, plus i didn't
> want
> > to
> > >> > > assume
> > >> > > > > >> > TinkerGraph identifiers in case these tests were every
> used
> > >> with
> > >> > > > some
> > >> > > > > >> other
> > >> > > > > >> > graph database that didn't use longs):
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > >> > > > > >> >
> > >> > > > > >> > for all those additions (and others) the logic required
> by
> > >> the
> > >> > GLV
> > >> > > > to
> > >> > > > > >> > process these tests has stayed surprisingly simple:
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > >> > > > > >> >
> > >> > > > > >> > There's a fair bit of regex/string manipulation involved
> > >> there,
> > >> > > but
> > >> > > > > it's
> > >> > > > > >> > processing strings from the feature file so that's the
> > >> nature of
> > >> > > it
> > >> > > > I
> > >> > > > > >> > suppose. I think I'm of the mind that I want to port all
> of
> > >> the
> > >> > > > tests
> > >> > > > > to
> > >> > > > > >> > feature files, so I wrote this unit test to help validate
> > >> that
> > >> > > none
> > >> > > > > were
> > >> > > > > >> > missed:
> > >> > > > > >> >
> > >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > >> > > > > >> > structure/FeatureCoverageTest.java
> > >> > > > > >> >
> > >> > > > > >> > I don't know that we have to get to 100% porting right
> > away,
> > >> > but I
> > >> > > > > think
> > >> > > > > >> > that once we have these gherkin files written they not
> only
> > >> > become
> > >> > > > the
> > >> > > > > >> > basis for our current GLV testing work, but we might be
> > able
> > >> to
> > >> > > > simply
> > >> > > > > >> use
> > >> > > > > >> > them for testing the Java stuff as well - that would rid
> us
> > >> of
> > >> > > > having
> > >> > > > > >> the
> > >> > > > > >> > test code duplication. It also sets us up with a portable
> > >> body
> > >> > of
> > >> > > > > tests
> > >> > > > > >> > that can be re-used in TinkerPop 4.x.
> > >> > > > > >> >
> > >> > > > > >> > I'm open to suggestions if anyone has any.
> > >> > > > > >> >
> > >> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > >> > > > > >> > jorgebaygondra@gmail.com
> > >> > > > > >> > > wrote:
> > >> > > > > >> >
> > >> > > > > >> > > I was able to build a proof of concept for a
> > Gherkin-based
> > >> > test
> > >> > > > > >> runner in
> > >> > > > > >> > > C#, that takes the proposed count and select features
> > >> > > > > >> > > <https://github.com/apache/
> > tinkerpop/tree/TINKERPOP-1784/
> > >> > > > > >> > > gremlin-test/features/map>
> > >> > > > > >> > > and runs them using C# step definitions.
> > >> > > > > >> > >
> > >> > > > > >> > > It uses the Gherkin parser <
> > https://github.com/cucumber/g
> > >> > > > > >> herkin-dotnet>
> > >> > > > > >> > > from
> > >> > > > > >> > > cucumber, there isn't a release of the parser with .NET
> > >> Core
> > >> > > > support
> > >> > > > > >> so
> > >> > > > > >> > > I've
> > >> > > > > >> > > asked them to release one
> > >> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
> > >> (there
> > >> > > is
> > >> > > > no
> > >> > > > > >> > > limitation from their source files). If they are not
> able
> > >> to
> > >> > > > release
> > >> > > > > >> it
> > >> > > > > >> > in
> > >> > > > > >> > > the next few days, we can implement our own as it
> should
> > be
> > >> > > pretty
> > >> > > > > >> > straight
> > >> > > > > >> > > forward.
> > >> > > > > >> > >
> > >> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > >> > > > > >> spmallette@gmail.com>
> > >> > > > > >> > > wrote:
> > >> > > > > >> > >
> > >> > > > > >> > > > Thanks for the update. I'm trying to keep the test
> > >> language
> > >> > as
> > >> > > > > >> simple
> > >> > > > > >> > as
> > >> > > > > >> > > > possible so that we don't need an overly complicated
> > test
> > >> > > > > >> > implementation.
> > >> > > > > >> > > > Hopefully that will help make the .NET approach as
> easy
> > >> as
> > >> > > > > possible.
> > >> > > > > >> > > >
> > >> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > >> > > > > >> > > > jorgebaygondra@gmail.com>
> > >> > > > > >> > > > wrote:
> > >> > > > > >> > > >
> > >> > > > > >> > > > > I've been looking into Gherkin support for .NET:
> > >> SpecFlow,
> > >> > > the
> > >> > > > > >> > cucumber
> > >> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > >> > > > > >> > > > cucumber-implementations
> > >> > > > > >> > > > > >,
> > >> > > > > >> > > > > does not support .NET Core platform (we use .NET
> Core
> > >> > build
> > >> > > > > tools
> > >> > > > > >> for
> > >> > > > > >> > > the
> > >> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
> > >> > > > > >> pecFlow/projects/2
> > >> > > > > >> > >,
> > >> > > > > >> > > > > .NET
> > >> > > > > >> > > > > Core support on SpecFlow is coming at a very slow
> > pace
> > >> and
> > >> > > we
> > >> > > > > >> > shouldn't
> > >> > > > > >> > > > > expect to land any time soon (there were some
> design
> > >> > > decisions
> > >> > > > > in
> > >> > > > > >> > > > SpecFlow
> > >> > > > > >> > > > > library that makes supporting other platforms
> > >> non-trivial,
> > >> > > > like
> > >> > > > > >> > > requiring
> > >> > > > > >> > > > > code gen).
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > The alternative would be to implement our own
> harness
> > >> to
> > >> > > > support
> > >> > > > > >> it:
> > >> > > > > >> > > > from a
> > >> > > > > >> > > > > xunit test, look for certain types and parse the
> > >> > > annotations,
> > >> > > > > and
> > >> > > > > >> > > execute
> > >> > > > > >> > > > > them using the Gherkin features.
> > >> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > >> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > >> > > > > >> > > > > I'll continue looking into this option and try to
> > >> > understand
> > >> > > > the
> > >> > > > > >> > effort
> > >> > > > > >> > > > > required...
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > >> > > > > >> > > > > jorgebaygondra@gmail.com>
> > >> > > > > >> > > > > wrote:
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
> > >> growing
> > >> > > > number
> > >> > > > > >> of
> > >> > > > > >> > > GLVs.
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > We should find a way to define the different
> > features
> > >> > > > > supported
> > >> > > > > >> by
> > >> > > > > >> > > each
> > >> > > > > >> > > > > > GLV, as it's reasonable to have different
> maturity
> > >> > levels
> > >> > > > per
> > >> > > > > >> GLV
> > >> > > > > >> > > (ie:
> > >> > > > > >> > > > > > lambdas support, traversal strategy, ...). I
> don't
> > >> know
> > >> > if
> > >> > > > it
> > >> > > > > >> will
> > >> > > > > >> > be
> > >> > > > > >> > > > > > beneficial to do it in the Gherkin files or
> within
> > >> each
> > >> > > GLV
> > >> > > > > >> > > > > implementation.
> > >> > > > > >> > > > > > Also, we should consider the process of rolling
> > out a
> > >> > new
> > >> > > > > >> method /
> > >> > > > > >> > > > class
> > >> > > > > >> > > > > > in the java implementation, how that could affect
> > >> each
> > >> > > GLV.
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen
> Mallette
> > <
> > >> > > > > >> > > > spmallette@gmail.com
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > wrote:
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >> that's what i meant by "reflection" or as you
> > >> suggest
> > >> > > > > eval(). I
> > >> > > > > >> > > guess
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> point is that if the language can support some
> way
> > >> of
> > >> > > > taking
> > >> > > > > >> the
> > >> > > > > >> > > > string
> > >> > > > > >> > > > > >> value and turning it automatically into a
> > traversal
> > >> in
> > >> > > that
> > >> > > > > >> GLVs
> > >> > > > > >> > > style
> > >> > > > > >> > > > > >> then
> > >> > > > > >> > > > > >> we should do that.
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > >> > > > > >> <me...@gremlin.guru>
> > >> > > > > >> > > > > wrote:
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> > For unparameterized queries it can probably be
> > as
> > >> > easy
> > >> > > > as:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > @given("the traversal of")
> > >> > > > > >> > > > > >> > def translate_traversal(step):
> > >> > > > > >> > > > > >> > g = step.context.g
> > >> > > > > >> > > > > >> > step.context.traversal = eval(step.text)
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > Cheers,
> > >> > > > > >> > > > > >> > Daniel
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel
> Kuppitz
> > >> > > > > >> <me@gremlin.guru
> > >> > > > > >> > >
> > >> > > > > >> > > > > >> wrote:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber
> /
> > >> > Gherkin
> > >> > > > for
> > >> > > > > >> > years,
> > >> > > > > >> > > > > but I
> > >> > > > > >> > > > > >> > > really like the BDD approach.
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> > >> > translations
> > >> > > > > >> > > specifically
> > >> > > > > >> > > > > >> here:
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > >> > > > > radish/count_features_step.py#
> > >> > > > > >> > > L34-L46
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > This part is the only thing that looks weird
> > to
> > >> me.
> > >> > > > > You're
> > >> > > > > >> > > > basically
> > >> > > > > >> > > > > >> > > writing every query twice; is there really
> no
> > >> > easier
> > >> > > > way
> > >> > > > > >> to do
> > >> > > > > >> > > > that?
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > Cheers,
> > >> > > > > >> > > > > >> > > Daniel
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
> > >> Mallette <
> > >> > > > > >> > > > > >> spmallette@gmail.com>
> > >> > > > > >> > > > > >> > > wrote:
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >> I've brought this issue up in the past and
> > had
> > >> > > > suggested
> > >> > > > > >> some
> > >> > > > > >> > > > > >> options I
> > >> > > > > >> > > > > >> > >> had
> > >> > > > > >> > > > > >> > >> in mind but now I've finally put the basics
> > of
> > >> > those
> > >> > > > > >> ideas in
> > >> > > > > >> > > > place
> > >> > > > > >> > > > > >> so I
> > >> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall
> that
> > >> the
> > >> > > > issue
> > >> > > > > at
> > >> > > > > >> > hand
> > >> > > > > >> > > > is
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > we
> > >> > > > > >> > > > > >> > >> don't have a test suite for GLVs as
> > >> gremlin-test
> > >> > is
> > >> > > > > bound
> > >> > > > > >> to
> > >> > > > > >> > > the
> > >> > > > > >> > > > > >> JVM. We
> > >> > > > > >> > > > > >> > >> have some tricks that let us test
> > >> gremlin-python
> > >> > > with
> > >> > > > it
> > >> > > > > >> but
> > >> > > > > >> > > > those
> > >> > > > > >> > > > > >> > tricks
> > >> > > > > >> > > > > >> > >> won't work for every language and we now
> have
> > >> the
> > >> > > > first
> > >> > > > > >> > > language
> > >> > > > > >> > > > in
> > >> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming
> > gremlin-javascript
> > >> > which
> > >> > > > > won't
> > >> > > > > >> > > > support
> > >> > > > > >> > > > > it
> > >> > > > > >> > > > > >> > >> (yes,
> > >> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on
> the
> > >> jvm
> > >> > > but
> > >> > > > > >> there
> > >> > > > > >> > are
> > >> > > > > >> > > > > >> issues
> > >> > > > > >> > > > > >> > >> with
> > >> > > > > >> > > > > >> > >> getting it all to work with the test
> > framework
> > >> > that
> > >> > > > make
> > >> > > > > >> it
> > >> > > > > >> > > > unduly
> > >> > > > > >> > > > > >> > >> complicated).
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> On other threads I offered the idea that we
> > >> look
> > >> > to
> > >> > > > use
> > >> > > > > >> > Gherkin
> > >> > > > > >> > > > to
> > >> > > > > >> > > > > >> write
> > >> > > > > >> > > > > >> > >> general Gremlin test specifications, which
> > then
> > >> > > could
> > >> > > > be
> > >> > > > > >> read
> > >> > > > > >> > > and
> > >> > > > > >> > > > > >> > >> processed
> > >> > > > > >> > > > > >> > >> by the wide variety of test frameworks that
> > can
> > >> > read
> > >> > > > > that
> > >> > > > > >> > > format
> > >> > > > > >> > > > -
> > >> > > > > >> > > > > >> there
> > >> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
> > >> every
> > >> > > > > >> language -
> > >> > > > > >> > > for
> > >> > > > > >> > > > > >> > example,
> > >> > > > > >> > > > > >> > >> see:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://cucumber.io/
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> I just created this issue:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://issues.apache.org/
> > >> > > jira/browse/TINKERPOP-1784
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> and pushed this branch:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/
> > >> > > > tinkerpop/tree/TINKERPOP-1784
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> which demonstrates how this works with
> > >> > > gremlin-python.
> > >> > > > > The
> > >> > > > > >> > > basic
> > >> > > > > >> > > > > >> anatomy
> > >> > > > > >> > > > > >> > >> of
> > >> > > > > >> > > > > >> > >> this setup involves this new directory in
> > >> > > > gremlin-test:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-test/features
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> It contains the Gherkin .features files.
> > These
> > >> are
> > >> > > the
> > >> > > > > >> test
> > >> > > > > >> > > > > >> > >> specifications.
> > >> > > > > >> > > > > >> > >> They are written using gremlin-java as the
> > >> "model"
> > >> > > > > >> language.
> > >> > > > > >> > > GLVs
> > >> > > > > >> > > > > >> will
> > >> > > > > >> > > > > >> > >> then
> > >> > > > > >> > > > > >> > >> need to write some infrastructure to
> process
> > >> these
> > >> > > > > Gherkin
> > >> > > > > >> > > files.
> > >> > > > > >> > > > > The
> > >> > > > > >> > > > > >> > key
> > >> > > > > >> > > > > >> > >> to making this "easy" to implement will lie
> > in
> > >> our
> > >> > > > > >> abiilty to
> > >> > > > > >> > > > keep
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > >> assertions we want to have relatively
> simple.
> > >> The
> > >> > > more
> > >> > > > > >> > > simplistic
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
> > >> easier
> > >> > > the
> > >> > > > > job
> > >> > > > > >> it
> > >> > > > > >> > > will
> > >> > > > > >> > > > > be
> > >> > > > > >> > > > > >> for
> > >> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of
> > course,
> > >> > once
> > >> > > > that
> > >> > > > > >> > > > > >> infrastructure
> > >> > > > > >> > > > > >> > is
> > >> > > > > >> > > > > >> > >> in place, the GLV developer just has to
> write
> > >> the
> > >> > > GLV
> > >> > > > > >> version
> > >> > > > > >> > > of
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So
> > you
> > >> can
> > >> > > > look
> > >> > > > > at
> > >> > > > > >> > all
> > >> > > > > >> > > > the
> > >> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
> > >> files:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> > >> > > translations
> > >> > > > > >> > > > specifically
> > >> > > > > >> > > > > >> here:
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > >> > > > > radish/count_features_step.py#
> > >> > > > > >> > > L34-L46
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> I think this approach works pretty well and
> > >> solves
> > >> > > our
> > >> > > > > >> > general
> > >> > > > > >> > > > > >> problems
> > >> > > > > >> > > > > >> > >> for
> > >> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > >> > > > implementing
> > >> > > > > >> the
> > >> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
> > >> tests
> > >> > > > added
> > >> > > > > to
> > >> > > > > >> > > > .feature
> > >> > > > > >> > > > > >> > files
> > >> > > > > >> > > > > >> > >> just need to translated in the GLV. I
> suppose
> > >> we
> > >> > > could
> > >> > > > > >> > > > "automate" a
> > >> > > > > >> > > > > >> good
> > >> > > > > >> > > > > >> > >> portion of the translation with reflection
> of
> > >> some
> > >> > > > sort.
> > >> > > > > >> > > Anything
> > >> > > > > >> > > > > >> else
> > >> > > > > >> > > > > >> > >> could just be handled manually.
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> Not sure if we need to use this new model
> to
> > >> > wholly
> > >> > > > > >> replace
> > >> > > > > >> > the
> > >> > > > > >> > > > old
> > >> > > > > >> > > > > >> one.
> > >> > > > > >> > > > > >> > >> The process test suite has its place in
> > helping
> > >> > > graph
> > >> > > > > >> > database
> > >> > > > > >> > > > > >> providers
> > >> > > > > >> > > > > >> > >> test their stuff. I also imagine that
> > >> introducing
> > >> > > this
> > >> > > > > >> > approach
> > >> > > > > >> > > > in
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > >> context would create a breaking change
> which
> > we
> > >> > > would
> > >> > > > > then
> > >> > > > > >> > need
> > >> > > > > >> > > > to
> > >> > > > > >> > > > > >> push
> > >> > > > > >> > > > > >> > >> off
> > >> > > > > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to
> > >> think,
> > >> > > but
> > >> > > > > for
> > >> > > > > >> now
> > >> > > > > >> > > it
> > >> > > > > >> > > > > >> might
> > >> > > > > >> > > > > >> > not
> > >> > > > > >> > > > > >> > >> be best to conflate the two and just treat
> > >> them as
> > >> > > > > >> separate
> > >> > > > > >> > > > aspects
> > >> > > > > >> > > > > >> of
> > >> > > > > >> > > > > >> > the
> > >> > > > > >> > > > > >> > >> test suite.
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >> Anyway - it's important we settle on an
> > >> approach
> > >> > to
> > >> > > > > >> testing
> > >> > > > > >> > as
> > >> > > > > >> > > we
> > >> > > > > >> > > > > >> really
> > >> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin
> .NET
> > >> GLV
> > >> > > > > without
> > >> > > > > >> > > getting
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > test
> > >> > > > > >> > > > > >> > >> suite solid. please yell if you have any
> > ideas
> > >> or
> > >> > > > > >> feedback on
> > >> > > > > >> > > > this
> > >> > > > > >> > > > > >> > >> approach.
> > >> > > > > >> > > > > >> > >>
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > >
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: [DISCUSS] GLV Test Suite
Posted by Stephen Mallette <sp...@gmail.com>.
I've been away for a few days on vacation, but before I left, I made some
pretty decent progress on the remaining tests and I'm finding less and less
of my time being focused on the test logic and more and more of it on the
translation of the tests themselves. I'm starting to feel comfortable that
most of the remaining pain is in test translation which means I could
probably issue the PR for what I have sooner than later. I'll probably just
make sure that I have at least a "good enough" representation of tests for
each step and then issue the PR. That will allow TINKERPOP-1827 to proceed
more easily and I can work more at more relaxed pace on migrating all the
tests. Please let me know if there are any concerns with that approach.
On Tue, Nov 7, 2017 at 5:02 AM, Jorge Bay Gondra <jo...@gmail.com>
wrote:
> I've created TINKERPOP-1827 to track the progress of the test suite runner
> for Gremlin .NET.
>
> I have a working gherkin-based runner that executes the steps (parsing
> gremlin groovy / parameters / results) as defined in the TINKERPOP-1784 WIP
> branch.
>
> https://github.com/jorgebay/tinkerpop/commits/gherkin-cs-runner
>
> I'll wait for TINKERPOP-1784 to be resolved in order the follow the last
> changes and submit a pull request for it.
>
> On Tue, Oct 31, 2017 at 11:09 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com
> > wrote:
>
> > Ok.
> > Going back to my initial comment regarding enums: don't mind, I've
> > included the possibility to use enums on the C# parser.
> >
> > On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> >> I guess there wouldn't be a problem with that, though having it the way
> it
> >> works now was a nice check to ensure that the ScriptEngine was
> configured
> >> properly. I guess that shouldn't be the focus on this body of tests
> >> though.
> >> We should be more concerned that the elements of the Gremlin language
> >> actually work.
> >>
> >> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
> >> jorgebaygondra@gmail.com>
> >> wrote:
> >>
> >> > I see that for most languages using bytecode to native language
> >> translator
> >> > would be the easiest solution. In the case of the JavaScript GLV, it
> >> would
> >> > be fairly straightforward to implement (and I plan to do so!).
> >> > In the case of C#, being more strict regarding typing than java, it
> >> would
> >> > be an incomplete solution. For example:
> >> >
> >> > Consider the following traversal in Java: g.V().values("age")
> >> > The type parameter for values() method is inferred at runtime and
> things
> >> > "just works" thanks to type erasure.
> >> >
> >> > In the case of C#, it should be written as g.V().Values<int>("age") ,
> >> with
> >> > the type parameter to specify which type of traverser are you
> expecting
> >> at
> >> > a compile time.
> >> >
> >> > That's why it wont be possible to make a generic translator from
> >> bytecode
> >> > to C# code, without understanding which generic types are expected.
> >> >
> >> > In the case of C#, I was going for a translator on the C# side that
> >> > tokenizes the gremlin traversal. Once the traversal is tokenized, in
> the
> >> > moment of invoking the methods (with reflection), it can be made aware
> >> of
> >> > the modern graph data (with datatypes!), ie: For method Values<>(), if
> >> the
> >> > property key is "age" -> use int as generic type parameter.
> >> >
> >> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <
> spmallette@gmail.com
> >> >
> >> > wrote:
> >> >
> >> > > I can't help thinking that perhaps non-JVM languages need to
> leverage
> >> > > bytecode to make traversal building easier. We already have an
> >> > established
> >> > > pattern for converting bytecode into Traversals and while it is
> >> different
> >> > > for each language, it's generally governed by the Translator
> >> interface.
> >> > > Note that we already do this for Java, Groovy, and Python and the
> code
> >> > > isn't too crazy - just 200 lines or so:
> >> > >
> >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> >> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> >> > JavaTranslator.java
> >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> >> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> >> > > GroovyTranslator.java
> >> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> >> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> >> > > PythonTranslator.java
> >> > >
> >> > > I think that non-JVM languages could send the Gremlin string from
> the
> >> > test
> >> > > to Gremlin Server as a script first and return the bytecode as JSON.
> >> Then
> >> > > it could use a "Translator" to then parse it into it's native
> >> language.
> >> > > Shouldn't we just build a CSharpTranslator similar to these? does
> that
> >> > make
> >> > > sense?
> >> > >
> >> > >
> >> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> >> > > jorgebaygondra@gmail.com
> >> > > > wrote:
> >> > >
> >> > > > I've been looking over the current test scenarios on the
> >> TINKERPOP-1784
> >> > > > branch, with the C# GLV test suite in mind, and oh man that's an
> >> > > impressive
> >> > > > amount of scenarios!
> >> > > >
> >> > > > So far, I have a suggestion.
> >> > > >
> >> > > > Instead of
> >> > > >
> >> > > > Scenario: g_V_fold_countXlocalX
> >> > > > Given the modern graph
> >> > > > And the traversal of
> >> > > > """
> >> > > > g.V().fold().count(Scope.local)
> >> > > > """
> >> > > >
> >> > > > We could use
> >> > > >
> >> > > > Scenario: g_V_fold_countXlocalX
> >> > > > Given the modern graph
> >> > > > And parameter 1 being an enum
> >> > > > """
> >> > > > Scope.local
> >> > > > """
> >> > > > And the traversal of
> >> > > > """
> >> > > > g.V().fold().count(param1)
> >> > > > """
> >> > > >
> >> > > > That way we can exclude enums from possible parameter values,
> >> narrowing
> >> > > > down the amount of types of parameters allowed, ie: const numeric,
> >> > const
> >> > > > strings (wrapped in double quotes), traversals (starting with
> `__`)
> >> ,
> >> > > ...,
> >> > > > making parsing the traversal a little easier.
> >> > > >
> >> > > >
> >> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> >> > spmallette@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Just a quick update on this item - it continues........
> >> > > > >
> >> > > > > Went down a bad path earlier in the week and ended up shelving a
> >> lot
> >> > of
> >> > > > > work - ended up just coded into a corner. Anyway, I've not
> really
> >> > > changed
> >> > > > > much in the implementation, but I've still not reached a point
> >> where
> >> > > the
> >> > > > > addition of a new .feature files comes without meeting some new
> >> type
> >> > of
> >> > > > > assertion that has to be dealt with. This ends up slowing down
> >> > progress
> >> > > > on
> >> > > > > porting over the tests. Please feel free to peruse the latest
> >> changes
> >> > > on
> >> > > > > the branch and let me know if there's any feedback.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> >> > > spmallette@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > I did some pretty heavy refactoring to the python test logic
> >> > > (altered a
> >> > > > > > bit by some revision of the gherkin feature file language) and
> >> the
> >> > > > result
> >> > > > > > is a much more simplified test logic file:
> >> > > > > >
> >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> >> > > > > >
> >> > > > > > About 120 lines of code (down from about 170). If you include
> >> the
> >> > > test
> >> > > > > > logic "setup" file:
> >> > > > > >
> >> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > > gremlin-python/src/main/jython/radish/terrain.py
> >> > > > > >
> >> > > > > > we end up with about 250 lines of test logic total (take out
> the
> >> > > > > > comment/license and we're probably well under 200 total - not
> >> too
> >> > > > bad). I
> >> > > > > > think I'm closing in on the end to infrastructure building
> here
> >> so
> >> > > the
> >> > > > > > basic framework is getting close to final at this point I
> >> believe.
> >> > > I'll
> >> > > > > > keep scanning the tests looking for other types of assertions
> >> that
> >> > > I've
> >> > > > > not
> >> > > > > > yet covered, but it's getting pretty solid I think. Hopefully,
> >> > there
> >> > > > > won't
> >> > > > > > need to be too many more lines of code needed to express the
> >> test
> >> > > logic
> >> > > > > as
> >> > > > > > I like how things are looking right now.
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> >> > > > > > jorgebaygondra@gmail.com> wrote:
> >> > > > > >
> >> > > > > >> Great progress! I like how you avoided using ids, even if it
> >> adds
> >> > > some
> >> > > > > >> complexity to the transformation required.
> >> > > > > >>
> >> > > > > >> The Python step definitions are still quite simple, I think
> it
> >> > would
> >> > > > be
> >> > > > > >> mostly the same on the rest of the languages.
> >> > > > > >>
> >> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> >> > > > spmallette@gmail.com
> >> > > > > >
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >> > Just another update on progress with the test suite. The
> >> > language
> >> > > of
> >> > > > > the
> >> > > > > >> > .feature files is getting slightly more complex as I try to
> >> > > > translate
> >> > > > > >> more
> >> > > > > >> > and more of our Java process suite tests into the language
> of
> >> > the
> >> > > > > >> gherkin
> >> > > > > >> > files. You can see here where I've added the ability to
> >> include
> >> > > > > >> parameters:
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-test/features/filter/Has.feature#L33
> >> > > > > >> >
> >> > > > > >> > I also came up with a method of asserting edges (didn't
> want
> >> to
> >> > > rely
> >> > > > > on
> >> > > > > >> ids
> >> > > > > >> > as it makes the gherkin harder to read, plus i didn't want
> to
> >> > > assume
> >> > > > > >> > TinkerGraph identifiers in case these tests were every used
> >> with
> >> > > > some
> >> > > > > >> other
> >> > > > > >> > graph database that didn't use longs):
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> >> > > > > >> >
> >> > > > > >> > for all those additions (and others) the logic required by
> >> the
> >> > GLV
> >> > > > to
> >> > > > > >> > process these tests has stayed surprisingly simple:
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> >> > > > > >> >
> >> > > > > >> > There's a fair bit of regex/string manipulation involved
> >> there,
> >> > > but
> >> > > > > it's
> >> > > > > >> > processing strings from the feature file so that's the
> >> nature of
> >> > > it
> >> > > > I
> >> > > > > >> > suppose. I think I'm of the mind that I want to port all of
> >> the
> >> > > > tests
> >> > > > > to
> >> > > > > >> > feature files, so I wrote this unit test to help validate
> >> that
> >> > > none
> >> > > > > were
> >> > > > > >> > missed:
> >> > > > > >> >
> >> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> >> > > > > >> > structure/FeatureCoverageTest.java
> >> > > > > >> >
> >> > > > > >> > I don't know that we have to get to 100% porting right
> away,
> >> > but I
> >> > > > > think
> >> > > > > >> > that once we have these gherkin files written they not only
> >> > become
> >> > > > the
> >> > > > > >> > basis for our current GLV testing work, but we might be
> able
> >> to
> >> > > > simply
> >> > > > > >> use
> >> > > > > >> > them for testing the Java stuff as well - that would rid us
> >> of
> >> > > > having
> >> > > > > >> the
> >> > > > > >> > test code duplication. It also sets us up with a portable
> >> body
> >> > of
> >> > > > > tests
> >> > > > > >> > that can be re-used in TinkerPop 4.x.
> >> > > > > >> >
> >> > > > > >> > I'm open to suggestions if anyone has any.
> >> > > > > >> >
> >> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> >> > > > > >> > jorgebaygondra@gmail.com
> >> > > > > >> > > wrote:
> >> > > > > >> >
> >> > > > > >> > > I was able to build a proof of concept for a
> Gherkin-based
> >> > test
> >> > > > > >> runner in
> >> > > > > >> > > C#, that takes the proposed count and select features
> >> > > > > >> > > <https://github.com/apache/
> tinkerpop/tree/TINKERPOP-1784/
> >> > > > > >> > > gremlin-test/features/map>
> >> > > > > >> > > and runs them using C# step definitions.
> >> > > > > >> > >
> >> > > > > >> > > It uses the Gherkin parser <
> https://github.com/cucumber/g
> >> > > > > >> herkin-dotnet>
> >> > > > > >> > > from
> >> > > > > >> > > cucumber, there isn't a release of the parser with .NET
> >> Core
> >> > > > support
> >> > > > > >> so
> >> > > > > >> > > I've
> >> > > > > >> > > asked them to release one
> >> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
> >> (there
> >> > > is
> >> > > > no
> >> > > > > >> > > limitation from their source files). If they are not able
> >> to
> >> > > > release
> >> > > > > >> it
> >> > > > > >> > in
> >> > > > > >> > > the next few days, we can implement our own as it should
> be
> >> > > pretty
> >> > > > > >> > straight
> >> > > > > >> > > forward.
> >> > > > > >> > >
> >> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> >> > > > > >> spmallette@gmail.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > >
> >> > > > > >> > > > Thanks for the update. I'm trying to keep the test
> >> language
> >> > as
> >> > > > > >> simple
> >> > > > > >> > as
> >> > > > > >> > > > possible so that we don't need an overly complicated
> test
> >> > > > > >> > implementation.
> >> > > > > >> > > > Hopefully that will help make the .NET approach as easy
> >> as
> >> > > > > possible.
> >> > > > > >> > > >
> >> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> >> > > > > >> > > > jorgebaygondra@gmail.com>
> >> > > > > >> > > > wrote:
> >> > > > > >> > > >
> >> > > > > >> > > > > I've been looking into Gherkin support for .NET:
> >> SpecFlow,
> >> > > the
> >> > > > > >> > cucumber
> >> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> >> > > > > >> > > > cucumber-implementations
> >> > > > > >> > > > > >,
> >> > > > > >> > > > > does not support .NET Core platform (we use .NET Core
> >> > build
> >> > > > > tools
> >> > > > > >> for
> >> > > > > >> > > the
> >> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
> >> > > > > >> > > > >
> >> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
> >> > > > > >> pecFlow/projects/2
> >> > > > > >> > >,
> >> > > > > >> > > > > .NET
> >> > > > > >> > > > > Core support on SpecFlow is coming at a very slow
> pace
> >> and
> >> > > we
> >> > > > > >> > shouldn't
> >> > > > > >> > > > > expect to land any time soon (there were some design
> >> > > decisions
> >> > > > > in
> >> > > > > >> > > > SpecFlow
> >> > > > > >> > > > > library that makes supporting other platforms
> >> non-trivial,
> >> > > > like
> >> > > > > >> > > requiring
> >> > > > > >> > > > > code gen).
> >> > > > > >> > > > >
> >> > > > > >> > > > > The alternative would be to implement our own harness
> >> to
> >> > > > support
> >> > > > > >> it:
> >> > > > > >> > > > from a
> >> > > > > >> > > > > xunit test, look for certain types and parse the
> >> > > annotations,
> >> > > > > and
> >> > > > > >> > > execute
> >> > > > > >> > > > > them using the Gherkin features.
> >> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> >> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> >> > > > > >> > > > > I'll continue looking into this option and try to
> >> > understand
> >> > > > the
> >> > > > > >> > effort
> >> > > > > >> > > > > required...
> >> > > > > >> > > > >
> >> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> >> > > > > >> > > > > jorgebaygondra@gmail.com>
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > >
> >> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
> >> growing
> >> > > > number
> >> > > > > >> of
> >> > > > > >> > > GLVs.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > We should find a way to define the different
> features
> >> > > > > supported
> >> > > > > >> by
> >> > > > > >> > > each
> >> > > > > >> > > > > > GLV, as it's reasonable to have different maturity
> >> > levels
> >> > > > per
> >> > > > > >> GLV
> >> > > > > >> > > (ie:
> >> > > > > >> > > > > > lambdas support, traversal strategy, ...). I don't
> >> know
> >> > if
> >> > > > it
> >> > > > > >> will
> >> > > > > >> > be
> >> > > > > >> > > > > > beneficial to do it in the Gherkin files or within
> >> each
> >> > > GLV
> >> > > > > >> > > > > implementation.
> >> > > > > >> > > > > > Also, we should consider the process of rolling
> out a
> >> > new
> >> > > > > >> method /
> >> > > > > >> > > > class
> >> > > > > >> > > > > > in the java implementation, how that could affect
> >> each
> >> > > GLV.
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette
> <
> >> > > > > >> > > > spmallette@gmail.com
> >> > > > > >> > > > > >
> >> > > > > >> > > > > > wrote:
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >> that's what i meant by "reflection" or as you
> >> suggest
> >> > > > > eval(). I
> >> > > > > >> > > guess
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> point is that if the language can support some way
> >> of
> >> > > > taking
> >> > > > > >> the
> >> > > > > >> > > > string
> >> > > > > >> > > > > >> value and turning it automatically into a
> traversal
> >> in
> >> > > that
> >> > > > > >> GLVs
> >> > > > > >> > > style
> >> > > > > >> > > > > >> then
> >> > > > > >> > > > > >> we should do that.
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> >> > > > > >> <me...@gremlin.guru>
> >> > > > > >> > > > > wrote:
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >> > For unparameterized queries it can probably be
> as
> >> > easy
> >> > > > as:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > @given("the traversal of")
> >> > > > > >> > > > > >> > def translate_traversal(step):
> >> > > > > >> > > > > >> > g = step.context.g
> >> > > > > >> > > > > >> > step.context.traversal = eval(step.text)
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > Cheers,
> >> > > > > >> > > > > >> > Daniel
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> >> > > > > >> <me@gremlin.guru
> >> > > > > >> > >
> >> > > > > >> > > > > >> wrote:
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
> >> > Gherkin
> >> > > > for
> >> > > > > >> > years,
> >> > > > > >> > > > > but I
> >> > > > > >> > > > > >> > > really like the BDD approach.
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> >> > translations
> >> > > > > >> > > specifically
> >> > > > > >> > > > > >> here:
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> >> > > > > radish/count_features_step.py#
> >> > > > > >> > > L34-L46
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > This part is the only thing that looks weird
> to
> >> me.
> >> > > > > You're
> >> > > > > >> > > > basically
> >> > > > > >> > > > > >> > > writing every query twice; is there really no
> >> > easier
> >> > > > way
> >> > > > > >> to do
> >> > > > > >> > > > that?
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > Cheers,
> >> > > > > >> > > > > >> > > Daniel
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
> >> Mallette <
> >> > > > > >> > > > > >> spmallette@gmail.com>
> >> > > > > >> > > > > >> > > wrote:
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >> I've brought this issue up in the past and
> had
> >> > > > suggested
> >> > > > > >> some
> >> > > > > >> > > > > >> options I
> >> > > > > >> > > > > >> > >> had
> >> > > > > >> > > > > >> > >> in mind but now I've finally put the basics
> of
> >> > those
> >> > > > > >> ideas in
> >> > > > > >> > > > place
> >> > > > > >> > > > > >> so I
> >> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that
> >> the
> >> > > > issue
> >> > > > > at
> >> > > > > >> > hand
> >> > > > > >> > > > is
> >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > we
> >> > > > > >> > > > > >> > >> don't have a test suite for GLVs as
> >> gremlin-test
> >> > is
> >> > > > > bound
> >> > > > > >> to
> >> > > > > >> > > the
> >> > > > > >> > > > > >> JVM. We
> >> > > > > >> > > > > >> > >> have some tricks that let us test
> >> gremlin-python
> >> > > with
> >> > > > it
> >> > > > > >> but
> >> > > > > >> > > > those
> >> > > > > >> > > > > >> > tricks
> >> > > > > >> > > > > >> > >> won't work for every language and we now have
> >> the
> >> > > > first
> >> > > > > >> > > language
> >> > > > > >> > > > in
> >> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming
> gremlin-javascript
> >> > which
> >> > > > > won't
> >> > > > > >> > > > support
> >> > > > > >> > > > > it
> >> > > > > >> > > > > >> > >> (yes,
> >> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on the
> >> jvm
> >> > > but
> >> > > > > >> there
> >> > > > > >> > are
> >> > > > > >> > > > > >> issues
> >> > > > > >> > > > > >> > >> with
> >> > > > > >> > > > > >> > >> getting it all to work with the test
> framework
> >> > that
> >> > > > make
> >> > > > > >> it
> >> > > > > >> > > > unduly
> >> > > > > >> > > > > >> > >> complicated).
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> On other threads I offered the idea that we
> >> look
> >> > to
> >> > > > use
> >> > > > > >> > Gherkin
> >> > > > > >> > > > to
> >> > > > > >> > > > > >> write
> >> > > > > >> > > > > >> > >> general Gremlin test specifications, which
> then
> >> > > could
> >> > > > be
> >> > > > > >> read
> >> > > > > >> > > and
> >> > > > > >> > > > > >> > >> processed
> >> > > > > >> > > > > >> > >> by the wide variety of test frameworks that
> can
> >> > read
> >> > > > > that
> >> > > > > >> > > format
> >> > > > > >> > > > -
> >> > > > > >> > > > > >> there
> >> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
> >> every
> >> > > > > >> language -
> >> > > > > >> > > for
> >> > > > > >> > > > > >> > example,
> >> > > > > >> > > > > >> > >> see:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://cucumber.io/
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> I just created this issue:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://issues.apache.org/
> >> > > jira/browse/TINKERPOP-1784
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> and pushed this branch:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/
> >> > > > tinkerpop/tree/TINKERPOP-1784
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> which demonstrates how this works with
> >> > > gremlin-python.
> >> > > > > The
> >> > > > > >> > > basic
> >> > > > > >> > > > > >> anatomy
> >> > > > > >> > > > > >> > >> of
> >> > > > > >> > > > > >> > >> this setup involves this new directory in
> >> > > > gremlin-test:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-test/features
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> It contains the Gherkin .features files.
> These
> >> are
> >> > > the
> >> > > > > >> test
> >> > > > > >> > > > > >> > >> specifications.
> >> > > > > >> > > > > >> > >> They are written using gremlin-java as the
> >> "model"
> >> > > > > >> language.
> >> > > > > >> > > GLVs
> >> > > > > >> > > > > >> will
> >> > > > > >> > > > > >> > >> then
> >> > > > > >> > > > > >> > >> need to write some infrastructure to process
> >> these
> >> > > > > Gherkin
> >> > > > > >> > > files.
> >> > > > > >> > > > > The
> >> > > > > >> > > > > >> > key
> >> > > > > >> > > > > >> > >> to making this "easy" to implement will lie
> in
> >> our
> >> > > > > >> abiilty to
> >> > > > > >> > > > keep
> >> > > > > >> > > > > >> the
> >> > > > > >> > > > > >> > >> assertions we want to have relatively simple.
> >> The
> >> > > more
> >> > > > > >> > > simplistic
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
> >> easier
> >> > > the
> >> > > > > job
> >> > > > > >> it
> >> > > > > >> > > will
> >> > > > > >> > > > > be
> >> > > > > >> > > > > >> for
> >> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of
> course,
> >> > once
> >> > > > that
> >> > > > > >> > > > > >> infrastructure
> >> > > > > >> > > > > >> > is
> >> > > > > >> > > > > >> > >> in place, the GLV developer just has to write
> >> the
> >> > > GLV
> >> > > > > >> version
> >> > > > > >> > > of
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So
> you
> >> can
> >> > > > look
> >> > > > > at
> >> > > > > >> > all
> >> > > > > >> > > > the
> >> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
> >> files:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> >> > > translations
> >> > > > > >> > > > specifically
> >> > > > > >> > > > > >> here:
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> https://github.com/apache/tink
> >> > > > > >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> >> > > > > radish/count_features_step.py#
> >> > > > > >> > > L34-L46
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> I think this approach works pretty well and
> >> solves
> >> > > our
> >> > > > > >> > general
> >> > > > > >> > > > > >> problems
> >> > > > > >> > > > > >> > >> for
> >> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> >> > > > implementing
> >> > > > > >> the
> >> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
> >> tests
> >> > > > added
> >> > > > > to
> >> > > > > >> > > > .feature
> >> > > > > >> > > > > >> > files
> >> > > > > >> > > > > >> > >> just need to translated in the GLV. I suppose
> >> we
> >> > > could
> >> > > > > >> > > > "automate" a
> >> > > > > >> > > > > >> good
> >> > > > > >> > > > > >> > >> portion of the translation with reflection of
> >> some
> >> > > > sort.
> >> > > > > >> > > Anything
> >> > > > > >> > > > > >> else
> >> > > > > >> > > > > >> > >> could just be handled manually.
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> Not sure if we need to use this new model to
> >> > wholly
> >> > > > > >> replace
> >> > > > > >> > the
> >> > > > > >> > > > old
> >> > > > > >> > > > > >> one.
> >> > > > > >> > > > > >> > >> The process test suite has its place in
> helping
> >> > > graph
> >> > > > > >> > database
> >> > > > > >> > > > > >> providers
> >> > > > > >> > > > > >> > >> test their stuff. I also imagine that
> >> introducing
> >> > > this
> >> > > > > >> > approach
> >> > > > > >> > > > in
> >> > > > > >> > > > > >> that
> >> > > > > >> > > > > >> > >> context would create a breaking change which
> we
> >> > > would
> >> > > > > then
> >> > > > > >> > need
> >> > > > > >> > > > to
> >> > > > > >> > > > > >> push
> >> > > > > >> > > > > >> > >> off
> >> > > > > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to
> >> think,
> >> > > but
> >> > > > > for
> >> > > > > >> now
> >> > > > > >> > > it
> >> > > > > >> > > > > >> might
> >> > > > > >> > > > > >> > not
> >> > > > > >> > > > > >> > >> be best to conflate the two and just treat
> >> them as
> >> > > > > >> separate
> >> > > > > >> > > > aspects
> >> > > > > >> > > > > >> of
> >> > > > > >> > > > > >> > the
> >> > > > > >> > > > > >> > >> test suite.
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >> Anyway - it's important we settle on an
> >> approach
> >> > to
> >> > > > > >> testing
> >> > > > > >> > as
> >> > > > > >> > > we
> >> > > > > >> > > > > >> really
> >> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET
> >> GLV
> >> > > > > without
> >> > > > > >> > > getting
> >> > > > > >> > > > > the
> >> > > > > >> > > > > >> > test
> >> > > > > >> > > > > >> > >> suite solid. please yell if you have any
> ideas
> >> or
> >> > > > > >> feedback on
> >> > > > > >> > > > this
> >> > > > > >> > > > > >> > >> approach.
> >> > > > > >> > > > > >> > >>
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> > >
> >> > > > > >> > > > > >> >
> >> > > > > >> > > > > >>
> >> > > > > >> > > > > >
> >> > > > > >> > > > > >
> >> > > > > >> > > > >
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] GLV Test Suite
Posted by Jorge Bay Gondra <jo...@gmail.com>.
I've created TINKERPOP-1827 to track the progress of the test suite runner
for Gremlin .NET.
I have a working gherkin-based runner that executes the steps (parsing
gremlin groovy / parameters / results) as defined in the TINKERPOP-1784 WIP
branch.
https://github.com/jorgebay/tinkerpop/commits/gherkin-cs-runner
I'll wait for TINKERPOP-1784 to be resolved in order the follow the last
changes and submit a pull request for it.
On Tue, Oct 31, 2017 at 11:09 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> wrote:
> Ok.
> Going back to my initial comment regarding enums: don't mind, I've
> included the possibility to use enums on the C# parser.
>
> On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> I guess there wouldn't be a problem with that, though having it the way it
>> works now was a nice check to ensure that the ScriptEngine was configured
>> properly. I guess that shouldn't be the focus on this body of tests
>> though.
>> We should be more concerned that the elements of the Gremlin language
>> actually work.
>>
>> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
>> jorgebaygondra@gmail.com>
>> wrote:
>>
>> > I see that for most languages using bytecode to native language
>> translator
>> > would be the easiest solution. In the case of the JavaScript GLV, it
>> would
>> > be fairly straightforward to implement (and I plan to do so!).
>> > In the case of C#, being more strict regarding typing than java, it
>> would
>> > be an incomplete solution. For example:
>> >
>> > Consider the following traversal in Java: g.V().values("age")
>> > The type parameter for values() method is inferred at runtime and things
>> > "just works" thanks to type erasure.
>> >
>> > In the case of C#, it should be written as g.V().Values<int>("age") ,
>> with
>> > the type parameter to specify which type of traverser are you expecting
>> at
>> > a compile time.
>> >
>> > That's why it wont be possible to make a generic translator from
>> bytecode
>> > to C# code, without understanding which generic types are expected.
>> >
>> > In the case of C#, I was going for a translator on the C# side that
>> > tokenizes the gremlin traversal. Once the traversal is tokenized, in the
>> > moment of invoking the methods (with reflection), it can be made aware
>> of
>> > the modern graph data (with datatypes!), ie: For method Values<>(), if
>> the
>> > property key is "age" -> use int as generic type parameter.
>> >
>> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <spmallette@gmail.com
>> >
>> > wrote:
>> >
>> > > I can't help thinking that perhaps non-JVM languages need to leverage
>> > > bytecode to make traversal building easier. We already have an
>> > established
>> > > pattern for converting bytecode into Traversals and while it is
>> different
>> > > for each language, it's generally governed by the Translator
>> interface.
>> > > Note that we already do this for Java, Groovy, and Python and the code
>> > > isn't too crazy - just 200 lines or so:
>> > >
>> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
>> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
>> > JavaTranslator.java
>> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
>> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
>> > > GroovyTranslator.java
>> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
>> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
>> > > PythonTranslator.java
>> > >
>> > > I think that non-JVM languages could send the Gremlin string from the
>> > test
>> > > to Gremlin Server as a script first and return the bytecode as JSON.
>> Then
>> > > it could use a "Translator" to then parse it into it's native
>> language.
>> > > Shouldn't we just build a CSharpTranslator similar to these? does that
>> > make
>> > > sense?
>> > >
>> > >
>> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
>> > > jorgebaygondra@gmail.com
>> > > > wrote:
>> > >
>> > > > I've been looking over the current test scenarios on the
>> TINKERPOP-1784
>> > > > branch, with the C# GLV test suite in mind, and oh man that's an
>> > > impressive
>> > > > amount of scenarios!
>> > > >
>> > > > So far, I have a suggestion.
>> > > >
>> > > > Instead of
>> > > >
>> > > > Scenario: g_V_fold_countXlocalX
>> > > > Given the modern graph
>> > > > And the traversal of
>> > > > """
>> > > > g.V().fold().count(Scope.local)
>> > > > """
>> > > >
>> > > > We could use
>> > > >
>> > > > Scenario: g_V_fold_countXlocalX
>> > > > Given the modern graph
>> > > > And parameter 1 being an enum
>> > > > """
>> > > > Scope.local
>> > > > """
>> > > > And the traversal of
>> > > > """
>> > > > g.V().fold().count(param1)
>> > > > """
>> > > >
>> > > > That way we can exclude enums from possible parameter values,
>> narrowing
>> > > > down the amount of types of parameters allowed, ie: const numeric,
>> > const
>> > > > strings (wrapped in double quotes), traversals (starting with `__`)
>> ,
>> > > ...,
>> > > > making parsing the traversal a little easier.
>> > > >
>> > > >
>> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
>> > spmallette@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Just a quick update on this item - it continues........
>> > > > >
>> > > > > Went down a bad path earlier in the week and ended up shelving a
>> lot
>> > of
>> > > > > work - ended up just coded into a corner. Anyway, I've not really
>> > > changed
>> > > > > much in the implementation, but I've still not reached a point
>> where
>> > > the
>> > > > > addition of a new .feature files comes without meeting some new
>> type
>> > of
>> > > > > assertion that has to be dealt with. This ends up slowing down
>> > progress
>> > > > on
>> > > > > porting over the tests. Please feel free to peruse the latest
>> changes
>> > > on
>> > > > > the branch and let me know if there's any feedback.
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
>> > > spmallette@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I did some pretty heavy refactoring to the python test logic
>> > > (altered a
>> > > > > > bit by some revision of the gherkin feature file language) and
>> the
>> > > > result
>> > > > > > is a much more simplified test logic file:
>> > > > > >
>> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
>> > > > > >
>> > > > > > About 120 lines of code (down from about 170). If you include
>> the
>> > > test
>> > > > > > logic "setup" file:
>> > > > > >
>> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > > gremlin-python/src/main/jython/radish/terrain.py
>> > > > > >
>> > > > > > we end up with about 250 lines of test logic total (take out the
>> > > > > > comment/license and we're probably well under 200 total - not
>> too
>> > > > bad). I
>> > > > > > think I'm closing in on the end to infrastructure building here
>> so
>> > > the
>> > > > > > basic framework is getting close to final at this point I
>> believe.
>> > > I'll
>> > > > > > keep scanning the tests looking for other types of assertions
>> that
>> > > I've
>> > > > > not
>> > > > > > yet covered, but it's getting pretty solid I think. Hopefully,
>> > there
>> > > > > won't
>> > > > > > need to be too many more lines of code needed to express the
>> test
>> > > logic
>> > > > > as
>> > > > > > I like how things are looking right now.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
>> > > > > > jorgebaygondra@gmail.com> wrote:
>> > > > > >
>> > > > > >> Great progress! I like how you avoided using ids, even if it
>> adds
>> > > some
>> > > > > >> complexity to the transformation required.
>> > > > > >>
>> > > > > >> The Python step definitions are still quite simple, I think it
>> > would
>> > > > be
>> > > > > >> mostly the same on the rest of the languages.
>> > > > > >>
>> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
>> > > > spmallette@gmail.com
>> > > > > >
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >> > Just another update on progress with the test suite. The
>> > language
>> > > of
>> > > > > the
>> > > > > >> > .feature files is getting slightly more complex as I try to
>> > > > translate
>> > > > > >> more
>> > > > > >> > and more of our Java process suite tests into the language of
>> > the
>> > > > > >> gherkin
>> > > > > >> > files. You can see here where I've added the ability to
>> include
>> > > > > >> parameters:
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-test/features/filter/Has.feature#L33
>> > > > > >> >
>> > > > > >> > I also came up with a method of asserting edges (didn't want
>> to
>> > > rely
>> > > > > on
>> > > > > >> ids
>> > > > > >> > as it makes the gherkin harder to read, plus i didn't want to
>> > > assume
>> > > > > >> > TinkerGraph identifiers in case these tests were every used
>> with
>> > > > some
>> > > > > >> other
>> > > > > >> > graph database that didn't use longs):
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
>> > > > > >> >
>> > > > > >> > for all those additions (and others) the logic required by
>> the
>> > GLV
>> > > > to
>> > > > > >> > process these tests has stayed surprisingly simple:
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
>> > > > > >> >
>> > > > > >> > There's a fair bit of regex/string manipulation involved
>> there,
>> > > but
>> > > > > it's
>> > > > > >> > processing strings from the feature file so that's the
>> nature of
>> > > it
>> > > > I
>> > > > > >> > suppose. I think I'm of the mind that I want to port all of
>> the
>> > > > tests
>> > > > > to
>> > > > > >> > feature files, so I wrote this unit test to help validate
>> that
>> > > none
>> > > > > were
>> > > > > >> > missed:
>> > > > > >> >
>> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
>> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
>> > > > > >> > structure/FeatureCoverageTest.java
>> > > > > >> >
>> > > > > >> > I don't know that we have to get to 100% porting right away,
>> > but I
>> > > > > think
>> > > > > >> > that once we have these gherkin files written they not only
>> > become
>> > > > the
>> > > > > >> > basis for our current GLV testing work, but we might be able
>> to
>> > > > simply
>> > > > > >> use
>> > > > > >> > them for testing the Java stuff as well - that would rid us
>> of
>> > > > having
>> > > > > >> the
>> > > > > >> > test code duplication. It also sets us up with a portable
>> body
>> > of
>> > > > > tests
>> > > > > >> > that can be re-used in TinkerPop 4.x.
>> > > > > >> >
>> > > > > >> > I'm open to suggestions if anyone has any.
>> > > > > >> >
>> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
>> > > > > >> > jorgebaygondra@gmail.com
>> > > > > >> > > wrote:
>> > > > > >> >
>> > > > > >> > > I was able to build a proof of concept for a Gherkin-based
>> > test
>> > > > > >> runner in
>> > > > > >> > > C#, that takes the proposed count and select features
>> > > > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
>> > > > > >> > > gremlin-test/features/map>
>> > > > > >> > > and runs them using C# step definitions.
>> > > > > >> > >
>> > > > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
>> > > > > >> herkin-dotnet>
>> > > > > >> > > from
>> > > > > >> > > cucumber, there isn't a release of the parser with .NET
>> Core
>> > > > support
>> > > > > >> so
>> > > > > >> > > I've
>> > > > > >> > > asked them to release one
>> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
>> (there
>> > > is
>> > > > no
>> > > > > >> > > limitation from their source files). If they are not able
>> to
>> > > > release
>> > > > > >> it
>> > > > > >> > in
>> > > > > >> > > the next few days, we can implement our own as it should be
>> > > pretty
>> > > > > >> > straight
>> > > > > >> > > forward.
>> > > > > >> > >
>> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
>> > > > > >> spmallette@gmail.com>
>> > > > > >> > > wrote:
>> > > > > >> > >
>> > > > > >> > > > Thanks for the update. I'm trying to keep the test
>> language
>> > as
>> > > > > >> simple
>> > > > > >> > as
>> > > > > >> > > > possible so that we don't need an overly complicated test
>> > > > > >> > implementation.
>> > > > > >> > > > Hopefully that will help make the .NET approach as easy
>> as
>> > > > > possible.
>> > > > > >> > > >
>> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
>> > > > > >> > > > jorgebaygondra@gmail.com>
>> > > > > >> > > > wrote:
>> > > > > >> > > >
>> > > > > >> > > > > I've been looking into Gherkin support for .NET:
>> SpecFlow,
>> > > the
>> > > > > >> > cucumber
>> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
>> > > > > >> > > > cucumber-implementations
>> > > > > >> > > > > >,
>> > > > > >> > > > > does not support .NET Core platform (we use .NET Core
>> > build
>> > > > > tools
>> > > > > >> for
>> > > > > >> > > the
>> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
>> > > > > >> > > > >
>> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
>> > > > > >> pecFlow/projects/2
>> > > > > >> > >,
>> > > > > >> > > > > .NET
>> > > > > >> > > > > Core support on SpecFlow is coming at a very slow pace
>> and
>> > > we
>> > > > > >> > shouldn't
>> > > > > >> > > > > expect to land any time soon (there were some design
>> > > decisions
>> > > > > in
>> > > > > >> > > > SpecFlow
>> > > > > >> > > > > library that makes supporting other platforms
>> non-trivial,
>> > > > like
>> > > > > >> > > requiring
>> > > > > >> > > > > code gen).
>> > > > > >> > > > >
>> > > > > >> > > > > The alternative would be to implement our own harness
>> to
>> > > > support
>> > > > > >> it:
>> > > > > >> > > > from a
>> > > > > >> > > > > xunit test, look for certain types and parse the
>> > > annotations,
>> > > > > and
>> > > > > >> > > execute
>> > > > > >> > > > > them using the Gherkin features.
>> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
>> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
>> > > > > >> > > > > I'll continue looking into this option and try to
>> > understand
>> > > > the
>> > > > > >> > effort
>> > > > > >> > > > > required...
>> > > > > >> > > > >
>> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
>> > > > > >> > > > > jorgebaygondra@gmail.com>
>> > > > > >> > > > > wrote:
>> > > > > >> > > > >
>> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
>> growing
>> > > > number
>> > > > > >> of
>> > > > > >> > > GLVs.
>> > > > > >> > > > > >
>> > > > > >> > > > > > We should find a way to define the different features
>> > > > > supported
>> > > > > >> by
>> > > > > >> > > each
>> > > > > >> > > > > > GLV, as it's reasonable to have different maturity
>> > levels
>> > > > per
>> > > > > >> GLV
>> > > > > >> > > (ie:
>> > > > > >> > > > > > lambdas support, traversal strategy, ...). I don't
>> know
>> > if
>> > > > it
>> > > > > >> will
>> > > > > >> > be
>> > > > > >> > > > > > beneficial to do it in the Gherkin files or within
>> each
>> > > GLV
>> > > > > >> > > > > implementation.
>> > > > > >> > > > > > Also, we should consider the process of rolling out a
>> > new
>> > > > > >> method /
>> > > > > >> > > > class
>> > > > > >> > > > > > in the java implementation, how that could affect
>> each
>> > > GLV.
>> > > > > >> > > > > >
>> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
>> > > > > >> > > > spmallette@gmail.com
>> > > > > >> > > > > >
>> > > > > >> > > > > > wrote:
>> > > > > >> > > > > >
>> > > > > >> > > > > >> that's what i meant by "reflection" or as you
>> suggest
>> > > > > eval(). I
>> > > > > >> > > guess
>> > > > > >> > > > > the
>> > > > > >> > > > > >> point is that if the language can support some way
>> of
>> > > > taking
>> > > > > >> the
>> > > > > >> > > > string
>> > > > > >> > > > > >> value and turning it automatically into a traversal
>> in
>> > > that
>> > > > > >> GLVs
>> > > > > >> > > style
>> > > > > >> > > > > >> then
>> > > > > >> > > > > >> we should do that.
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
>> > > > > >> <me...@gremlin.guru>
>> > > > > >> > > > > wrote:
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> > For unparameterized queries it can probably be as
>> > easy
>> > > > as:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > @given("the traversal of")
>> > > > > >> > > > > >> > def translate_traversal(step):
>> > > > > >> > > > > >> > g = step.context.g
>> > > > > >> > > > > >> > step.context.traversal = eval(step.text)
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > Cheers,
>> > > > > >> > > > > >> > Daniel
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
>> > > > > >> <me@gremlin.guru
>> > > > > >> > >
>> > > > > >> > > > > >> wrote:
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
>> > Gherkin
>> > > > for
>> > > > > >> > years,
>> > > > > >> > > > > but I
>> > > > > >> > > > > >> > > really like the BDD approach.
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
>> > translations
>> > > > > >> > > specifically
>> > > > > >> > > > > >> here:
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-python/src/main/jython/
>> > > > > radish/count_features_step.py#
>> > > > > >> > > L34-L46
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > This part is the only thing that looks weird to
>> me.
>> > > > > You're
>> > > > > >> > > > basically
>> > > > > >> > > > > >> > > writing every query twice; is there really no
>> > easier
>> > > > way
>> > > > > >> to do
>> > > > > >> > > > that?
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > Cheers,
>> > > > > >> > > > > >> > > Daniel
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
>> Mallette <
>> > > > > >> > > > > >> spmallette@gmail.com>
>> > > > > >> > > > > >> > > wrote:
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >> I've brought this issue up in the past and had
>> > > > suggested
>> > > > > >> some
>> > > > > >> > > > > >> options I
>> > > > > >> > > > > >> > >> had
>> > > > > >> > > > > >> > >> in mind but now I've finally put the basics of
>> > those
>> > > > > >> ideas in
>> > > > > >> > > > place
>> > > > > >> > > > > >> so I
>> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that
>> the
>> > > > issue
>> > > > > at
>> > > > > >> > hand
>> > > > > >> > > > is
>> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > we
>> > > > > >> > > > > >> > >> don't have a test suite for GLVs as
>> gremlin-test
>> > is
>> > > > > bound
>> > > > > >> to
>> > > > > >> > > the
>> > > > > >> > > > > >> JVM. We
>> > > > > >> > > > > >> > >> have some tricks that let us test
>> gremlin-python
>> > > with
>> > > > it
>> > > > > >> but
>> > > > > >> > > > those
>> > > > > >> > > > > >> > tricks
>> > > > > >> > > > > >> > >> won't work for every language and we now have
>> the
>> > > > first
>> > > > > >> > > language
>> > > > > >> > > > in
>> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript
>> > which
>> > > > > won't
>> > > > > >> > > > support
>> > > > > >> > > > > it
>> > > > > >> > > > > >> > >> (yes,
>> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on the
>> jvm
>> > > but
>> > > > > >> there
>> > > > > >> > are
>> > > > > >> > > > > >> issues
>> > > > > >> > > > > >> > >> with
>> > > > > >> > > > > >> > >> getting it all to work with the test framework
>> > that
>> > > > make
>> > > > > >> it
>> > > > > >> > > > unduly
>> > > > > >> > > > > >> > >> complicated).
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> On other threads I offered the idea that we
>> look
>> > to
>> > > > use
>> > > > > >> > Gherkin
>> > > > > >> > > > to
>> > > > > >> > > > > >> write
>> > > > > >> > > > > >> > >> general Gremlin test specifications, which then
>> > > could
>> > > > be
>> > > > > >> read
>> > > > > >> > > and
>> > > > > >> > > > > >> > >> processed
>> > > > > >> > > > > >> > >> by the wide variety of test frameworks that can
>> > read
>> > > > > that
>> > > > > >> > > format
>> > > > > >> > > > -
>> > > > > >> > > > > >> there
>> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
>> every
>> > > > > >> language -
>> > > > > >> > > for
>> > > > > >> > > > > >> > example,
>> > > > > >> > > > > >> > >> see:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://cucumber.io/
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> I just created this issue:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://issues.apache.org/
>> > > jira/browse/TINKERPOP-1784
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> and pushed this branch:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/
>> > > > tinkerpop/tree/TINKERPOP-1784
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> which demonstrates how this works with
>> > > gremlin-python.
>> > > > > The
>> > > > > >> > > basic
>> > > > > >> > > > > >> anatomy
>> > > > > >> > > > > >> > >> of
>> > > > > >> > > > > >> > >> this setup involves this new directory in
>> > > > gremlin-test:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-test/features
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> It contains the Gherkin .features files. These
>> are
>> > > the
>> > > > > >> test
>> > > > > >> > > > > >> > >> specifications.
>> > > > > >> > > > > >> > >> They are written using gremlin-java as the
>> "model"
>> > > > > >> language.
>> > > > > >> > > GLVs
>> > > > > >> > > > > >> will
>> > > > > >> > > > > >> > >> then
>> > > > > >> > > > > >> > >> need to write some infrastructure to process
>> these
>> > > > > Gherkin
>> > > > > >> > > files.
>> > > > > >> > > > > The
>> > > > > >> > > > > >> > key
>> > > > > >> > > > > >> > >> to making this "easy" to implement will lie in
>> our
>> > > > > >> abiilty to
>> > > > > >> > > > keep
>> > > > > >> > > > > >> the
>> > > > > >> > > > > >> > >> assertions we want to have relatively simple.
>> The
>> > > more
>> > > > > >> > > simplistic
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
>> easier
>> > > the
>> > > > > job
>> > > > > >> it
>> > > > > >> > > will
>> > > > > >> > > > > be
>> > > > > >> > > > > >> for
>> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of course,
>> > once
>> > > > that
>> > > > > >> > > > > >> infrastructure
>> > > > > >> > > > > >> > is
>> > > > > >> > > > > >> > >> in place, the GLV developer just has to write
>> the
>> > > GLV
>> > > > > >> version
>> > > > > >> > > of
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So you
>> can
>> > > > look
>> > > > > at
>> > > > > >> > all
>> > > > > >> > > > the
>> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
>> files:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/tree/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
>> > > translations
>> > > > > >> > > > specifically
>> > > > > >> > > > > >> here:
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> https://github.com/apache/tink
>> > > > > >> erpop/blob/TINKERPOP-1784/grem
>> > > > > >> > > > > >> > >> lin-python/src/main/jython/
>> > > > > radish/count_features_step.py#
>> > > > > >> > > L34-L46
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> I think this approach works pretty well and
>> solves
>> > > our
>> > > > > >> > general
>> > > > > >> > > > > >> problems
>> > > > > >> > > > > >> > >> for
>> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
>> > > > implementing
>> > > > > >> the
>> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
>> tests
>> > > > added
>> > > > > to
>> > > > > >> > > > .feature
>> > > > > >> > > > > >> > files
>> > > > > >> > > > > >> > >> just need to translated in the GLV. I suppose
>> we
>> > > could
>> > > > > >> > > > "automate" a
>> > > > > >> > > > > >> good
>> > > > > >> > > > > >> > >> portion of the translation with reflection of
>> some
>> > > > sort.
>> > > > > >> > > Anything
>> > > > > >> > > > > >> else
>> > > > > >> > > > > >> > >> could just be handled manually.
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> Not sure if we need to use this new model to
>> > wholly
>> > > > > >> replace
>> > > > > >> > the
>> > > > > >> > > > old
>> > > > > >> > > > > >> one.
>> > > > > >> > > > > >> > >> The process test suite has its place in helping
>> > > graph
>> > > > > >> > database
>> > > > > >> > > > > >> providers
>> > > > > >> > > > > >> > >> test their stuff. I also imagine that
>> introducing
>> > > this
>> > > > > >> > approach
>> > > > > >> > > > in
>> > > > > >> > > > > >> that
>> > > > > >> > > > > >> > >> context would create a breaking change which we
>> > > would
>> > > > > then
>> > > > > >> > need
>> > > > > >> > > > to
>> > > > > >> > > > > >> push
>> > > > > >> > > > > >> > >> off
>> > > > > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to
>> think,
>> > > but
>> > > > > for
>> > > > > >> now
>> > > > > >> > > it
>> > > > > >> > > > > >> might
>> > > > > >> > > > > >> > not
>> > > > > >> > > > > >> > >> be best to conflate the two and just treat
>> them as
>> > > > > >> separate
>> > > > > >> > > > aspects
>> > > > > >> > > > > >> of
>> > > > > >> > > > > >> > the
>> > > > > >> > > > > >> > >> test suite.
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >> Anyway - it's important we settle on an
>> approach
>> > to
>> > > > > >> testing
>> > > > > >> > as
>> > > > > >> > > we
>> > > > > >> > > > > >> really
>> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET
>> GLV
>> > > > > without
>> > > > > >> > > getting
>> > > > > >> > > > > the
>> > > > > >> > > > > >> > test
>> > > > > >> > > > > >> > >> suite solid. please yell if you have any ideas
>> or
>> > > > > >> feedback on
>> > > > > >> > > > this
>> > > > > >> > > > > >> > >> approach.
>> > > > > >> > > > > >> > >>
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> > >
>> > > > > >> > > > > >> >
>> > > > > >> > > > > >>
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > >
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [DISCUSS] GLV Test Suite
Posted by Jorge Bay Gondra <jo...@gmail.com>.
Ok.
Going back to my initial comment regarding enums: don't mind, I've included
the possibility to use enums on the C# parser.
On Mon, Oct 30, 2017 at 7:15 PM, Stephen Mallette <sp...@gmail.com>
wrote:
> I guess there wouldn't be a problem with that, though having it the way it
> works now was a nice check to ensure that the ScriptEngine was configured
> properly. I guess that shouldn't be the focus on this body of tests though.
> We should be more concerned that the elements of the Gremlin language
> actually work.
>
> On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com>
> wrote:
>
> > I see that for most languages using bytecode to native language
> translator
> > would be the easiest solution. In the case of the JavaScript GLV, it
> would
> > be fairly straightforward to implement (and I plan to do so!).
> > In the case of C#, being more strict regarding typing than java, it would
> > be an incomplete solution. For example:
> >
> > Consider the following traversal in Java: g.V().values("age")
> > The type parameter for values() method is inferred at runtime and things
> > "just works" thanks to type erasure.
> >
> > In the case of C#, it should be written as g.V().Values<int>("age") ,
> with
> > the type parameter to specify which type of traverser are you expecting
> at
> > a compile time.
> >
> > That's why it wont be possible to make a generic translator from bytecode
> > to C# code, without understanding which generic types are expected.
> >
> > In the case of C#, I was going for a translator on the C# side that
> > tokenizes the gremlin traversal. Once the traversal is tokenized, in the
> > moment of invoking the methods (with reflection), it can be made aware of
> > the modern graph data (with datatypes!), ie: For method Values<>(), if
> the
> > property key is "age" -> use int as generic type parameter.
> >
> > On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > I can't help thinking that perhaps non-JVM languages need to leverage
> > > bytecode to make traversal building easier. We already have an
> > established
> > > pattern for converting bytecode into Traversals and while it is
> different
> > > for each language, it's generally governed by the Translator interface.
> > > Note that we already do this for Java, Groovy, and Python and the code
> > > isn't too crazy - just 200 lines or so:
> > >
> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> > JavaTranslator.java
> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> > > GroovyTranslator.java
> > > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> > > PythonTranslator.java
> > >
> > > I think that non-JVM languages could send the Gremlin string from the
> > test
> > > to Gremlin Server as a script first and return the bytecode as JSON.
> Then
> > > it could use a "Translator" to then parse it into it's native language.
> > > Shouldn't we just build a CSharpTranslator similar to these? does that
> > make
> > > sense?
> > >
> > >
> > > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> > > jorgebaygondra@gmail.com
> > > > wrote:
> > >
> > > > I've been looking over the current test scenarios on the
> TINKERPOP-1784
> > > > branch, with the C# GLV test suite in mind, and oh man that's an
> > > impressive
> > > > amount of scenarios!
> > > >
> > > > So far, I have a suggestion.
> > > >
> > > > Instead of
> > > >
> > > > Scenario: g_V_fold_countXlocalX
> > > > Given the modern graph
> > > > And the traversal of
> > > > """
> > > > g.V().fold().count(Scope.local)
> > > > """
> > > >
> > > > We could use
> > > >
> > > > Scenario: g_V_fold_countXlocalX
> > > > Given the modern graph
> > > > And parameter 1 being an enum
> > > > """
> > > > Scope.local
> > > > """
> > > > And the traversal of
> > > > """
> > > > g.V().fold().count(param1)
> > > > """
> > > >
> > > > That way we can exclude enums from possible parameter values,
> narrowing
> > > > down the amount of types of parameters allowed, ie: const numeric,
> > const
> > > > strings (wrapped in double quotes), traversals (starting with `__`) ,
> > > ...,
> > > > making parsing the traversal a little easier.
> > > >
> > > >
> > > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> > spmallette@gmail.com>
> > > > wrote:
> > > >
> > > > > Just a quick update on this item - it continues........
> > > > >
> > > > > Went down a bad path earlier in the week and ended up shelving a
> lot
> > of
> > > > > work - ended up just coded into a corner. Anyway, I've not really
> > > changed
> > > > > much in the implementation, but I've still not reached a point
> where
> > > the
> > > > > addition of a new .feature files comes without meeting some new
> type
> > of
> > > > > assertion that has to be dealt with. This ends up slowing down
> > progress
> > > > on
> > > > > porting over the tests. Please feel free to peruse the latest
> changes
> > > on
> > > > > the branch and let me know if there's any feedback.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> > > spmallette@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > I did some pretty heavy refactoring to the python test logic
> > > (altered a
> > > > > > bit by some revision of the gherkin feature file language) and
> the
> > > > result
> > > > > > is a much more simplified test logic file:
> > > > > >
> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > > >
> > > > > > About 120 lines of code (down from about 170). If you include the
> > > test
> > > > > > logic "setup" file:
> > > > > >
> > > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > > gremlin-python/src/main/jython/radish/terrain.py
> > > > > >
> > > > > > we end up with about 250 lines of test logic total (take out the
> > > > > > comment/license and we're probably well under 200 total - not too
> > > > bad). I
> > > > > > think I'm closing in on the end to infrastructure building here
> so
> > > the
> > > > > > basic framework is getting close to final at this point I
> believe.
> > > I'll
> > > > > > keep scanning the tests looking for other types of assertions
> that
> > > I've
> > > > > not
> > > > > > yet covered, but it's getting pretty solid I think. Hopefully,
> > there
> > > > > won't
> > > > > > need to be too many more lines of code needed to express the test
> > > logic
> > > > > as
> > > > > > I like how things are looking right now.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > > > > jorgebaygondra@gmail.com> wrote:
> > > > > >
> > > > > >> Great progress! I like how you avoided using ids, even if it
> adds
> > > some
> > > > > >> complexity to the transformation required.
> > > > > >>
> > > > > >> The Python step definitions are still quite simple, I think it
> > would
> > > > be
> > > > > >> mostly the same on the rest of the languages.
> > > > > >>
> > > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > > > spmallette@gmail.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Just another update on progress with the test suite. The
> > language
> > > of
> > > > > the
> > > > > >> > .feature files is getting slightly more complex as I try to
> > > > translate
> > > > > >> more
> > > > > >> > and more of our Java process suite tests into the language of
> > the
> > > > > >> gherkin
> > > > > >> > files. You can see here where I've added the ability to
> include
> > > > > >> parameters:
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-test/features/filter/Has.feature#L33
> > > > > >> >
> > > > > >> > I also came up with a method of asserting edges (didn't want
> to
> > > rely
> > > > > on
> > > > > >> ids
> > > > > >> > as it makes the gherkin harder to read, plus i didn't want to
> > > assume
> > > > > >> > TinkerGraph identifiers in case these tests were every used
> with
> > > > some
> > > > > >> other
> > > > > >> > graph database that didn't use longs):
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > > > > >> >
> > > > > >> > for all those additions (and others) the logic required by the
> > GLV
> > > > to
> > > > > >> > process these tests has stayed surprisingly simple:
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > > >> >
> > > > > >> > There's a fair bit of regex/string manipulation involved
> there,
> > > but
> > > > > it's
> > > > > >> > processing strings from the feature file so that's the nature
> of
> > > it
> > > > I
> > > > > >> > suppose. I think I'm of the mind that I want to port all of
> the
> > > > tests
> > > > > to
> > > > > >> > feature files, so I wrote this unit test to help validate that
> > > none
> > > > > were
> > > > > >> > missed:
> > > > > >> >
> > > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > > > > >> > structure/FeatureCoverageTest.java
> > > > > >> >
> > > > > >> > I don't know that we have to get to 100% porting right away,
> > but I
> > > > > think
> > > > > >> > that once we have these gherkin files written they not only
> > become
> > > > the
> > > > > >> > basis for our current GLV testing work, but we might be able
> to
> > > > simply
> > > > > >> use
> > > > > >> > them for testing the Java stuff as well - that would rid us of
> > > > having
> > > > > >> the
> > > > > >> > test code duplication. It also sets us up with a portable body
> > of
> > > > > tests
> > > > > >> > that can be re-used in TinkerPop 4.x.
> > > > > >> >
> > > > > >> > I'm open to suggestions if anyone has any.
> > > > > >> >
> > > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > > > > >> > jorgebaygondra@gmail.com
> > > > > >> > > wrote:
> > > > > >> >
> > > > > >> > > I was able to build a proof of concept for a Gherkin-based
> > test
> > > > > >> runner in
> > > > > >> > > C#, that takes the proposed count and select features
> > > > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > > > >> > > gremlin-test/features/map>
> > > > > >> > > and runs them using C# step definitions.
> > > > > >> > >
> > > > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > > > > >> herkin-dotnet>
> > > > > >> > > from
> > > > > >> > > cucumber, there isn't a release of the parser with .NET Core
> > > > support
> > > > > >> so
> > > > > >> > > I've
> > > > > >> > > asked them to release one
> > > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11>
> (there
> > > is
> > > > no
> > > > > >> > > limitation from their source files). If they are not able to
> > > > release
> > > > > >> it
> > > > > >> > in
> > > > > >> > > the next few days, we can implement our own as it should be
> > > pretty
> > > > > >> > straight
> > > > > >> > > forward.
> > > > > >> > >
> > > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > > > > >> spmallette@gmail.com>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Thanks for the update. I'm trying to keep the test
> language
> > as
> > > > > >> simple
> > > > > >> > as
> > > > > >> > > > possible so that we don't need an overly complicated test
> > > > > >> > implementation.
> > > > > >> > > > Hopefully that will help make the .NET approach as easy as
> > > > > possible.
> > > > > >> > > >
> > > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > > > >> > > > jorgebaygondra@gmail.com>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > I've been looking into Gherkin support for .NET:
> SpecFlow,
> > > the
> > > > > >> > cucumber
> > > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > > > > >> > > > cucumber-implementations
> > > > > >> > > > > >,
> > > > > >> > > > > does not support .NET Core platform (we use .NET Core
> > build
> > > > > tools
> > > > > >> for
> > > > > >> > > the
> > > > > >> > > > > .NET GLV) and only supports .NET Framework.
> > > > > >> > > > >
> > > > > >> > > > > From what I can see <https://github.com/techtalk/S
> > > > > >> pecFlow/projects/2
> > > > > >> > >,
> > > > > >> > > > > .NET
> > > > > >> > > > > Core support on SpecFlow is coming at a very slow pace
> and
> > > we
> > > > > >> > shouldn't
> > > > > >> > > > > expect to land any time soon (there were some design
> > > decisions
> > > > > in
> > > > > >> > > > SpecFlow
> > > > > >> > > > > library that makes supporting other platforms
> non-trivial,
> > > > like
> > > > > >> > > requiring
> > > > > >> > > > > code gen).
> > > > > >> > > > >
> > > > > >> > > > > The alternative would be to implement our own harness to
> > > > support
> > > > > >> it:
> > > > > >> > > > from a
> > > > > >> > > > > xunit test, look for certain types and parse the
> > > annotations,
> > > > > and
> > > > > >> > > execute
> > > > > >> > > > > them using the Gherkin features.
> > > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > > > > >> > > > > I'll continue looking into this option and try to
> > understand
> > > > the
> > > > > >> > effort
> > > > > >> > > > > required...
> > > > > >> > > > >
> > > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > > > >> > > > > jorgebaygondra@gmail.com>
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Nice! Gherkin will make our lives easier with a
> growing
> > > > number
> > > > > >> of
> > > > > >> > > GLVs.
> > > > > >> > > > > >
> > > > > >> > > > > > We should find a way to define the different features
> > > > > supported
> > > > > >> by
> > > > > >> > > each
> > > > > >> > > > > > GLV, as it's reasonable to have different maturity
> > levels
> > > > per
> > > > > >> GLV
> > > > > >> > > (ie:
> > > > > >> > > > > > lambdas support, traversal strategy, ...). I don't
> know
> > if
> > > > it
> > > > > >> will
> > > > > >> > be
> > > > > >> > > > > > beneficial to do it in the Gherkin files or within
> each
> > > GLV
> > > > > >> > > > > implementation.
> > > > > >> > > > > > Also, we should consider the process of rolling out a
> > new
> > > > > >> method /
> > > > > >> > > > class
> > > > > >> > > > > > in the java implementation, how that could affect each
> > > GLV.
> > > > > >> > > > > >
> > > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > > > >> > > > spmallette@gmail.com
> > > > > >> > > > > >
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > > > > eval(). I
> > > > > >> > > guess
> > > > > >> > > > > the
> > > > > >> > > > > >> point is that if the language can support some way of
> > > > taking
> > > > > >> the
> > > > > >> > > > string
> > > > > >> > > > > >> value and turning it automatically into a traversal
> in
> > > that
> > > > > >> GLVs
> > > > > >> > > style
> > > > > >> > > > > >> then
> > > > > >> > > > > >> we should do that.
> > > > > >> > > > > >>
> > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > > > > >> <me...@gremlin.guru>
> > > > > >> > > > > wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> > For unparameterized queries it can probably be as
> > easy
> > > > as:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > @given("the traversal of")
> > > > > >> > > > > >> > def translate_traversal(step):
> > > > > >> > > > > >> > g = step.context.g
> > > > > >> > > > > >> > step.context.traversal = eval(step.text)
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > Cheers,
> > > > > >> > > > > >> > Daniel
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > > > > >> <me@gremlin.guru
> > > > > >> > >
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
> > Gherkin
> > > > for
> > > > > >> > years,
> > > > > >> > > > > but I
> > > > > >> > > > > >> > > really like the BDD approach.
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> > translations
> > > > > >> > > specifically
> > > > > >> > > > > >> here:
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > > radish/count_features_step.py#
> > > > > >> > > L34-L46
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > This part is the only thing that looks weird to
> me.
> > > > > You're
> > > > > >> > > > basically
> > > > > >> > > > > >> > > writing every query twice; is there really no
> > easier
> > > > way
> > > > > >> to do
> > > > > >> > > > that?
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > Cheers,
> > > > > >> > > > > >> > > Daniel
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen
> Mallette <
> > > > > >> > > > > >> spmallette@gmail.com>
> > > > > >> > > > > >> > > wrote:
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >> I've brought this issue up in the past and had
> > > > suggested
> > > > > >> some
> > > > > >> > > > > >> options I
> > > > > >> > > > > >> > >> had
> > > > > >> > > > > >> > >> in mind but now I've finally put the basics of
> > those
> > > > > >> ideas in
> > > > > >> > > > place
> > > > > >> > > > > >> so I
> > > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that
> the
> > > > issue
> > > > > at
> > > > > >> > hand
> > > > > >> > > > is
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > we
> > > > > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test
> > is
> > > > > bound
> > > > > >> to
> > > > > >> > > the
> > > > > >> > > > > >> JVM. We
> > > > > >> > > > > >> > >> have some tricks that let us test gremlin-python
> > > with
> > > > it
> > > > > >> but
> > > > > >> > > > those
> > > > > >> > > > > >> > tricks
> > > > > >> > > > > >> > >> won't work for every language and we now have
> the
> > > > first
> > > > > >> > > language
> > > > > >> > > > in
> > > > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript
> > which
> > > > > won't
> > > > > >> > > > support
> > > > > >> > > > > it
> > > > > >> > > > > >> > >> (yes,
> > > > > >> > > > > >> > >> i know that gremlin-javascript can run on the
> jvm
> > > but
> > > > > >> there
> > > > > >> > are
> > > > > >> > > > > >> issues
> > > > > >> > > > > >> > >> with
> > > > > >> > > > > >> > >> getting it all to work with the test framework
> > that
> > > > make
> > > > > >> it
> > > > > >> > > > unduly
> > > > > >> > > > > >> > >> complicated).
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> On other threads I offered the idea that we look
> > to
> > > > use
> > > > > >> > Gherkin
> > > > > >> > > > to
> > > > > >> > > > > >> write
> > > > > >> > > > > >> > >> general Gremlin test specifications, which then
> > > could
> > > > be
> > > > > >> read
> > > > > >> > > and
> > > > > >> > > > > >> > >> processed
> > > > > >> > > > > >> > >> by the wide variety of test frameworks that can
> > read
> > > > > that
> > > > > >> > > format
> > > > > >> > > > -
> > > > > >> > > > > >> there
> > > > > >> > > > > >> > >> tend to be Gherkin processors in just about
> every
> > > > > >> language -
> > > > > >> > > for
> > > > > >> > > > > >> > example,
> > > > > >> > > > > >> > >> see:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://cucumber.io/
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> I just created this issue:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://issues.apache.org/
> > > jira/browse/TINKERPOP-1784
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> and pushed this branch:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/
> > > > tinkerpop/tree/TINKERPOP-1784
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> which demonstrates how this works with
> > > gremlin-python.
> > > > > The
> > > > > >> > > basic
> > > > > >> > > > > >> anatomy
> > > > > >> > > > > >> > >> of
> > > > > >> > > > > >> > >> this setup involves this new directory in
> > > > gremlin-test:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-test/features
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> It contains the Gherkin .features files. These
> are
> > > the
> > > > > >> test
> > > > > >> > > > > >> > >> specifications.
> > > > > >> > > > > >> > >> They are written using gremlin-java as the
> "model"
> > > > > >> language.
> > > > > >> > > GLVs
> > > > > >> > > > > >> will
> > > > > >> > > > > >> > >> then
> > > > > >> > > > > >> > >> need to write some infrastructure to process
> these
> > > > > Gherkin
> > > > > >> > > files.
> > > > > >> > > > > The
> > > > > >> > > > > >> > key
> > > > > >> > > > > >> > >> to making this "easy" to implement will lie in
> our
> > > > > >> abiilty to
> > > > > >> > > > keep
> > > > > >> > > > > >> the
> > > > > >> > > > > >> > >> assertions we want to have relatively simple.
> The
> > > more
> > > > > >> > > simplistic
> > > > > >> > > > > the
> > > > > >> > > > > >> > >> language in the Gherkin .feature files the
> easier
> > > the
> > > > > job
> > > > > >> it
> > > > > >> > > will
> > > > > >> > > > > be
> > > > > >> > > > > >> for
> > > > > >> > > > > >> > >> GLVs to build their infrastructure. Of course,
> > once
> > > > that
> > > > > >> > > > > >> infrastructure
> > > > > >> > > > > >> > is
> > > > > >> > > > > >> > >> in place, the GLV developer just has to write
> the
> > > GLV
> > > > > >> version
> > > > > >> > > of
> > > > > >> > > > > the
> > > > > >> > > > > >> > >> Gremlin specified in the .feature file. So you
> can
> > > > look
> > > > > at
> > > > > >> > all
> > > > > >> > > > the
> > > > > >> > > > > >> > >> "infrastructure" code here in this pair of
> files:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> > > translations
> > > > > >> > > > specifically
> > > > > >> > > > > >> here:
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > > radish/count_features_step.py#
> > > > > >> > > L34-L46
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> I think this approach works pretty well and
> solves
> > > our
> > > > > >> > general
> > > > > >> > > > > >> problems
> > > > > >> > > > > >> > >> for
> > > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > > > implementing
> > > > > >> the
> > > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin
> tests
> > > > added
> > > > > to
> > > > > >> > > > .feature
> > > > > >> > > > > >> > files
> > > > > >> > > > > >> > >> just need to translated in the GLV. I suppose we
> > > could
> > > > > >> > > > "automate" a
> > > > > >> > > > > >> good
> > > > > >> > > > > >> > >> portion of the translation with reflection of
> some
> > > > sort.
> > > > > >> > > Anything
> > > > > >> > > > > >> else
> > > > > >> > > > > >> > >> could just be handled manually.
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> Not sure if we need to use this new model to
> > wholly
> > > > > >> replace
> > > > > >> > the
> > > > > >> > > > old
> > > > > >> > > > > >> one.
> > > > > >> > > > > >> > >> The process test suite has its place in helping
> > > graph
> > > > > >> > database
> > > > > >> > > > > >> providers
> > > > > >> > > > > >> > >> test their stuff. I also imagine that
> introducing
> > > this
> > > > > >> > approach
> > > > > >> > > > in
> > > > > >> > > > > >> that
> > > > > >> > > > > >> > >> context would create a breaking change which we
> > > would
> > > > > then
> > > > > >> > need
> > > > > >> > > > to
> > > > > >> > > > > >> push
> > > > > >> > > > > >> > >> off
> > > > > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to
> think,
> > > but
> > > > > for
> > > > > >> now
> > > > > >> > > it
> > > > > >> > > > > >> might
> > > > > >> > > > > >> > not
> > > > > >> > > > > >> > >> be best to conflate the two and just treat them
> as
> > > > > >> separate
> > > > > >> > > > aspects
> > > > > >> > > > > >> of
> > > > > >> > > > > >> > the
> > > > > >> > > > > >> > >> test suite.
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >> Anyway - it's important we settle on an approach
> > to
> > > > > >> testing
> > > > > >> > as
> > > > > >> > > we
> > > > > >> > > > > >> really
> > > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET
> GLV
> > > > > without
> > > > > >> > > getting
> > > > > >> > > > > the
> > > > > >> > > > > >> > test
> > > > > >> > > > > >> > >> suite solid. please yell if you have any ideas
> or
> > > > > >> feedback on
> > > > > >> > > > this
> > > > > >> > > > > >> > >> approach.
> > > > > >> > > > > >> > >>
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> > >
> > > > > >> > > > > >> >
> > > > > >> > > > > >>
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] GLV Test Suite
Posted by Stephen Mallette <sp...@gmail.com>.
I guess there wouldn't be a problem with that, though having it the way it
works now was a nice check to ensure that the ScriptEngine was configured
properly. I guess that shouldn't be the focus on this body of tests though.
We should be more concerned that the elements of the Gremlin language
actually work.
On Thu, Oct 26, 2017 at 8:47 AM, Jorge Bay Gondra <jo...@gmail.com>
wrote:
> I see that for most languages using bytecode to native language translator
> would be the easiest solution. In the case of the JavaScript GLV, it would
> be fairly straightforward to implement (and I plan to do so!).
> In the case of C#, being more strict regarding typing than java, it would
> be an incomplete solution. For example:
>
> Consider the following traversal in Java: g.V().values("age")
> The type parameter for values() method is inferred at runtime and things
> "just works" thanks to type erasure.
>
> In the case of C#, it should be written as g.V().Values<int>("age") , with
> the type parameter to specify which type of traverser are you expecting at
> a compile time.
>
> That's why it wont be possible to make a generic translator from bytecode
> to C# code, without understanding which generic types are expected.
>
> In the case of C#, I was going for a translator on the C# side that
> tokenizes the gremlin traversal. Once the traversal is tokenized, in the
> moment of invoking the methods (with reflection), it can be made aware of
> the modern graph data (with datatypes!), ie: For method Values<>(), if the
> property key is "age" -> use int as generic type parameter.
>
> On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > I can't help thinking that perhaps non-JVM languages need to leverage
> > bytecode to make traversal building easier. We already have an
> established
> > pattern for converting bytecode into Traversals and while it is different
> > for each language, it's generally governed by the Translator interface.
> > Note that we already do this for Java, Groovy, and Python and the code
> > isn't too crazy - just 200 lines or so:
> >
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/
> JavaTranslator.java
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> > GroovyTranslator.java
> > https://github.com/apache/tinkerpop/blob/master/gremlin-
> > python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> > PythonTranslator.java
> >
> > I think that non-JVM languages could send the Gremlin string from the
> test
> > to Gremlin Server as a script first and return the bytecode as JSON. Then
> > it could use a "Translator" to then parse it into it's native language.
> > Shouldn't we just build a CSharpTranslator similar to these? does that
> make
> > sense?
> >
> >
> > On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com
> > > wrote:
> >
> > > I've been looking over the current test scenarios on the TINKERPOP-1784
> > > branch, with the C# GLV test suite in mind, and oh man that's an
> > impressive
> > > amount of scenarios!
> > >
> > > So far, I have a suggestion.
> > >
> > > Instead of
> > >
> > > Scenario: g_V_fold_countXlocalX
> > > Given the modern graph
> > > And the traversal of
> > > """
> > > g.V().fold().count(Scope.local)
> > > """
> > >
> > > We could use
> > >
> > > Scenario: g_V_fold_countXlocalX
> > > Given the modern graph
> > > And parameter 1 being an enum
> > > """
> > > Scope.local
> > > """
> > > And the traversal of
> > > """
> > > g.V().fold().count(param1)
> > > """
> > >
> > > That way we can exclude enums from possible parameter values, narrowing
> > > down the amount of types of parameters allowed, ie: const numeric,
> const
> > > strings (wrapped in double quotes), traversals (starting with `__`) ,
> > ...,
> > > making parsing the traversal a little easier.
> > >
> > >
> > > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > > > Just a quick update on this item - it continues........
> > > >
> > > > Went down a bad path earlier in the week and ended up shelving a lot
> of
> > > > work - ended up just coded into a corner. Anyway, I've not really
> > changed
> > > > much in the implementation, but I've still not reached a point where
> > the
> > > > addition of a new .feature files comes without meeting some new type
> of
> > > > assertion that has to be dealt with. This ends up slowing down
> progress
> > > on
> > > > porting over the tests. Please feel free to peruse the latest changes
> > on
> > > > the branch and let me know if there's any feedback.
> > > >
> > > >
> > > >
> > > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> > spmallette@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I did some pretty heavy refactoring to the python test logic
> > (altered a
> > > > > bit by some revision of the gherkin feature file language) and the
> > > result
> > > > > is a much more simplified test logic file:
> > > > >
> > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > >
> > > > > About 120 lines of code (down from about 170). If you include the
> > test
> > > > > logic "setup" file:
> > > > >
> > > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > > gremlin-python/src/main/jython/radish/terrain.py
> > > > >
> > > > > we end up with about 250 lines of test logic total (take out the
> > > > > comment/license and we're probably well under 200 total - not too
> > > bad). I
> > > > > think I'm closing in on the end to infrastructure building here so
> > the
> > > > > basic framework is getting close to final at this point I believe.
> > I'll
> > > > > keep scanning the tests looking for other types of assertions that
> > I've
> > > > not
> > > > > yet covered, but it's getting pretty solid I think. Hopefully,
> there
> > > > won't
> > > > > need to be too many more lines of code needed to express the test
> > logic
> > > > as
> > > > > I like how things are looking right now.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > > > jorgebaygondra@gmail.com> wrote:
> > > > >
> > > > >> Great progress! I like how you avoided using ids, even if it adds
> > some
> > > > >> complexity to the transformation required.
> > > > >>
> > > > >> The Python step definitions are still quite simple, I think it
> would
> > > be
> > > > >> mostly the same on the rest of the languages.
> > > > >>
> > > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > > spmallette@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Just another update on progress with the test suite. The
> language
> > of
> > > > the
> > > > >> > .feature files is getting slightly more complex as I try to
> > > translate
> > > > >> more
> > > > >> > and more of our Java process suite tests into the language of
> the
> > > > >> gherkin
> > > > >> > files. You can see here where I've added the ability to include
> > > > >> parameters:
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-test/features/filter/Has.feature#L33
> > > > >> >
> > > > >> > I also came up with a method of asserting edges (didn't want to
> > rely
> > > > on
> > > > >> ids
> > > > >> > as it makes the gherkin harder to read, plus i didn't want to
> > assume
> > > > >> > TinkerGraph identifiers in case these tests were every used with
> > > some
> > > > >> other
> > > > >> > graph database that didn't use longs):
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > > > >> >
> > > > >> > for all those additions (and others) the logic required by the
> GLV
> > > to
> > > > >> > process these tests has stayed surprisingly simple:
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > > > >> >
> > > > >> > There's a fair bit of regex/string manipulation involved there,
> > but
> > > > it's
> > > > >> > processing strings from the feature file so that's the nature of
> > it
> > > I
> > > > >> > suppose. I think I'm of the mind that I want to port all of the
> > > tests
> > > > to
> > > > >> > feature files, so I wrote this unit test to help validate that
> > none
> > > > were
> > > > >> > missed:
> > > > >> >
> > > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > > > >> > structure/FeatureCoverageTest.java
> > > > >> >
> > > > >> > I don't know that we have to get to 100% porting right away,
> but I
> > > > think
> > > > >> > that once we have these gherkin files written they not only
> become
> > > the
> > > > >> > basis for our current GLV testing work, but we might be able to
> > > simply
> > > > >> use
> > > > >> > them for testing the Java stuff as well - that would rid us of
> > > having
> > > > >> the
> > > > >> > test code duplication. It also sets us up with a portable body
> of
> > > > tests
> > > > >> > that can be re-used in TinkerPop 4.x.
> > > > >> >
> > > > >> > I'm open to suggestions if anyone has any.
> > > > >> >
> > > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > > > >> > jorgebaygondra@gmail.com
> > > > >> > > wrote:
> > > > >> >
> > > > >> > > I was able to build a proof of concept for a Gherkin-based
> test
> > > > >> runner in
> > > > >> > > C#, that takes the proposed count and select features
> > > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > > >> > > gremlin-test/features/map>
> > > > >> > > and runs them using C# step definitions.
> > > > >> > >
> > > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > > > >> herkin-dotnet>
> > > > >> > > from
> > > > >> > > cucumber, there isn't a release of the parser with .NET Core
> > > support
> > > > >> so
> > > > >> > > I've
> > > > >> > > asked them to release one
> > > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there
> > is
> > > no
> > > > >> > > limitation from their source files). If they are not able to
> > > release
> > > > >> it
> > > > >> > in
> > > > >> > > the next few days, we can implement our own as it should be
> > pretty
> > > > >> > straight
> > > > >> > > forward.
> > > > >> > >
> > > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > > > >> spmallette@gmail.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Thanks for the update. I'm trying to keep the test language
> as
> > > > >> simple
> > > > >> > as
> > > > >> > > > possible so that we don't need an overly complicated test
> > > > >> > implementation.
> > > > >> > > > Hopefully that will help make the .NET approach as easy as
> > > > possible.
> > > > >> > > >
> > > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > > >> > > > jorgebaygondra@gmail.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > I've been looking into Gherkin support for .NET: SpecFlow,
> > the
> > > > >> > cucumber
> > > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > > > >> > > > cucumber-implementations
> > > > >> > > > > >,
> > > > >> > > > > does not support .NET Core platform (we use .NET Core
> build
> > > > tools
> > > > >> for
> > > > >> > > the
> > > > >> > > > > .NET GLV) and only supports .NET Framework.
> > > > >> > > > >
> > > > >> > > > > From what I can see <https://github.com/techtalk/S
> > > > >> pecFlow/projects/2
> > > > >> > >,
> > > > >> > > > > .NET
> > > > >> > > > > Core support on SpecFlow is coming at a very slow pace and
> > we
> > > > >> > shouldn't
> > > > >> > > > > expect to land any time soon (there were some design
> > decisions
> > > > in
> > > > >> > > > SpecFlow
> > > > >> > > > > library that makes supporting other platforms non-trivial,
> > > like
> > > > >> > > requiring
> > > > >> > > > > code gen).
> > > > >> > > > >
> > > > >> > > > > The alternative would be to implement our own harness to
> > > support
> > > > >> it:
> > > > >> > > > from a
> > > > >> > > > > xunit test, look for certain types and parse the
> > annotations,
> > > > and
> > > > >> > > execute
> > > > >> > > > > them using the Gherkin features.
> > > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > > > >> > > > > I'll continue looking into this option and try to
> understand
> > > the
> > > > >> > effort
> > > > >> > > > > required...
> > > > >> > > > >
> > > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > > >> > > > > jorgebaygondra@gmail.com>
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Nice! Gherkin will make our lives easier with a growing
> > > number
> > > > >> of
> > > > >> > > GLVs.
> > > > >> > > > > >
> > > > >> > > > > > We should find a way to define the different features
> > > > supported
> > > > >> by
> > > > >> > > each
> > > > >> > > > > > GLV, as it's reasonable to have different maturity
> levels
> > > per
> > > > >> GLV
> > > > >> > > (ie:
> > > > >> > > > > > lambdas support, traversal strategy, ...). I don't know
> if
> > > it
> > > > >> will
> > > > >> > be
> > > > >> > > > > > beneficial to do it in the Gherkin files or within each
> > GLV
> > > > >> > > > > implementation.
> > > > >> > > > > > Also, we should consider the process of rolling out a
> new
> > > > >> method /
> > > > >> > > > class
> > > > >> > > > > > in the java implementation, how that could affect each
> > GLV.
> > > > >> > > > > >
> > > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > > >> > > > spmallette@gmail.com
> > > > >> > > > > >
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > > > eval(). I
> > > > >> > > guess
> > > > >> > > > > the
> > > > >> > > > > >> point is that if the language can support some way of
> > > taking
> > > > >> the
> > > > >> > > > string
> > > > >> > > > > >> value and turning it automatically into a traversal in
> > that
> > > > >> GLVs
> > > > >> > > style
> > > > >> > > > > >> then
> > > > >> > > > > >> we should do that.
> > > > >> > > > > >>
> > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > > > >> <me...@gremlin.guru>
> > > > >> > > > > wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > For unparameterized queries it can probably be as
> easy
> > > as:
> > > > >> > > > > >> >
> > > > >> > > > > >> > @given("the traversal of")
> > > > >> > > > > >> > def translate_traversal(step):
> > > > >> > > > > >> > g = step.context.g
> > > > >> > > > > >> > step.context.traversal = eval(step.text)
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > Cheers,
> > > > >> > > > > >> > Daniel
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > > > >> <me@gremlin.guru
> > > > >> > >
> > > > >> > > > > >> wrote:
> > > > >> > > > > >> >
> > > > >> > > > > >> > > That's great stuff. I haven't used Cucumber /
> Gherkin
> > > for
> > > > >> > years,
> > > > >> > > > > but I
> > > > >> > > > > >> > > really like the BDD approach.
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > and then you can look at the GLV Gremlin
> translations
> > > > >> > > specifically
> > > > >> > > > > >> here:
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > radish/count_features_step.py#
> > > > >> > > L34-L46
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > This part is the only thing that looks weird to me.
> > > > You're
> > > > >> > > > basically
> > > > >> > > > > >> > > writing every query twice; is there really no
> easier
> > > way
> > > > >> to do
> > > > >> > > > that?
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > Cheers,
> > > > >> > > > > >> > > Daniel
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > > >> > > > > >> spmallette@gmail.com>
> > > > >> > > > > >> > > wrote:
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >> I've brought this issue up in the past and had
> > > suggested
> > > > >> some
> > > > >> > > > > >> options I
> > > > >> > > > > >> > >> had
> > > > >> > > > > >> > >> in mind but now I've finally put the basics of
> those
> > > > >> ideas in
> > > > >> > > > place
> > > > >> > > > > >> so I
> > > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that the
> > > issue
> > > > at
> > > > >> > hand
> > > > >> > > > is
> > > > >> > > > > >> that
> > > > >> > > > > >> > we
> > > > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test
> is
> > > > bound
> > > > >> to
> > > > >> > > the
> > > > >> > > > > >> JVM. We
> > > > >> > > > > >> > >> have some tricks that let us test gremlin-python
> > with
> > > it
> > > > >> but
> > > > >> > > > those
> > > > >> > > > > >> > tricks
> > > > >> > > > > >> > >> won't work for every language and we now have the
> > > first
> > > > >> > > language
> > > > >> > > > in
> > > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript
> which
> > > > won't
> > > > >> > > > support
> > > > >> > > > > it
> > > > >> > > > > >> > >> (yes,
> > > > >> > > > > >> > >> i know that gremlin-javascript can run on the jvm
> > but
> > > > >> there
> > > > >> > are
> > > > >> > > > > >> issues
> > > > >> > > > > >> > >> with
> > > > >> > > > > >> > >> getting it all to work with the test framework
> that
> > > make
> > > > >> it
> > > > >> > > > unduly
> > > > >> > > > > >> > >> complicated).
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> On other threads I offered the idea that we look
> to
> > > use
> > > > >> > Gherkin
> > > > >> > > > to
> > > > >> > > > > >> write
> > > > >> > > > > >> > >> general Gremlin test specifications, which then
> > could
> > > be
> > > > >> read
> > > > >> > > and
> > > > >> > > > > >> > >> processed
> > > > >> > > > > >> > >> by the wide variety of test frameworks that can
> read
> > > > that
> > > > >> > > format
> > > > >> > > > -
> > > > >> > > > > >> there
> > > > >> > > > > >> > >> tend to be Gherkin processors in just about every
> > > > >> language -
> > > > >> > > for
> > > > >> > > > > >> > example,
> > > > >> > > > > >> > >> see:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://cucumber.io/
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> I just created this issue:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://issues.apache.org/
> > jira/browse/TINKERPOP-1784
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> and pushed this branch:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/
> > > tinkerpop/tree/TINKERPOP-1784
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> which demonstrates how this works with
> > gremlin-python.
> > > > The
> > > > >> > > basic
> > > > >> > > > > >> anatomy
> > > > >> > > > > >> > >> of
> > > > >> > > > > >> > >> this setup involves this new directory in
> > > gremlin-test:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-test/features
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> It contains the Gherkin .features files. These are
> > the
> > > > >> test
> > > > >> > > > > >> > >> specifications.
> > > > >> > > > > >> > >> They are written using gremlin-java as the "model"
> > > > >> language.
> > > > >> > > GLVs
> > > > >> > > > > >> will
> > > > >> > > > > >> > >> then
> > > > >> > > > > >> > >> need to write some infrastructure to process these
> > > > Gherkin
> > > > >> > > files.
> > > > >> > > > > The
> > > > >> > > > > >> > key
> > > > >> > > > > >> > >> to making this "easy" to implement will lie in our
> > > > >> abiilty to
> > > > >> > > > keep
> > > > >> > > > > >> the
> > > > >> > > > > >> > >> assertions we want to have relatively simple. The
> > more
> > > > >> > > simplistic
> > > > >> > > > > the
> > > > >> > > > > >> > >> language in the Gherkin .feature files the easier
> > the
> > > > job
> > > > >> it
> > > > >> > > will
> > > > >> > > > > be
> > > > >> > > > > >> for
> > > > >> > > > > >> > >> GLVs to build their infrastructure. Of course,
> once
> > > that
> > > > >> > > > > >> infrastructure
> > > > >> > > > > >> > is
> > > > >> > > > > >> > >> in place, the GLV developer just has to write the
> > GLV
> > > > >> version
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > >> > >> Gremlin specified in the .feature file. So you can
> > > look
> > > > at
> > > > >> > all
> > > > >> > > > the
> > > > >> > > > > >> > >> "infrastructure" code here in this pair of files:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/tree/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> > translations
> > > > >> > > > specifically
> > > > >> > > > > >> here:
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> https://github.com/apache/tink
> > > > >> erpop/blob/TINKERPOP-1784/grem
> > > > >> > > > > >> > >> lin-python/src/main/jython/
> > > > radish/count_features_step.py#
> > > > >> > > L34-L46
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> I think this approach works pretty well and solves
> > our
> > > > >> > general
> > > > >> > > > > >> problems
> > > > >> > > > > >> > >> for
> > > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > > implementing
> > > > >> the
> > > > >> > > > > >> > >> "infrastructure" but after that new Gremlin tests
> > > added
> > > > to
> > > > >> > > > .feature
> > > > >> > > > > >> > files
> > > > >> > > > > >> > >> just need to translated in the GLV. I suppose we
> > could
> > > > >> > > > "automate" a
> > > > >> > > > > >> good
> > > > >> > > > > >> > >> portion of the translation with reflection of some
> > > sort.
> > > > >> > > Anything
> > > > >> > > > > >> else
> > > > >> > > > > >> > >> could just be handled manually.
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> Not sure if we need to use this new model to
> wholly
> > > > >> replace
> > > > >> > the
> > > > >> > > > old
> > > > >> > > > > >> one.
> > > > >> > > > > >> > >> The process test suite has its place in helping
> > graph
> > > > >> > database
> > > > >> > > > > >> providers
> > > > >> > > > > >> > >> test their stuff. I also imagine that introducing
> > this
> > > > >> > approach
> > > > >> > > > in
> > > > >> > > > > >> that
> > > > >> > > > > >> > >> context would create a breaking change which we
> > would
> > > > then
> > > > >> > need
> > > > >> > > > to
> > > > >> > > > > >> push
> > > > >> > > > > >> > >> off
> > > > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to think,
> > but
> > > > for
> > > > >> now
> > > > >> > > it
> > > > >> > > > > >> might
> > > > >> > > > > >> > not
> > > > >> > > > > >> > >> be best to conflate the two and just treat them as
> > > > >> separate
> > > > >> > > > aspects
> > > > >> > > > > >> of
> > > > >> > > > > >> > the
> > > > >> > > > > >> > >> test suite.
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >> Anyway - it's important we settle on an approach
> to
> > > > >> testing
> > > > >> > as
> > > > >> > > we
> > > > >> > > > > >> really
> > > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> > > > without
> > > > >> > > getting
> > > > >> > > > > the
> > > > >> > > > > >> > test
> > > > >> > > > > >> > >> suite solid. please yell if you have any ideas or
> > > > >> feedback on
> > > > >> > > > this
> > > > >> > > > > >> > >> approach.
> > > > >> > > > > >> > >>
> > > > >> > > > > >> > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] GLV Test Suite
Posted by Jorge Bay Gondra <jo...@gmail.com>.
I see that for most languages using bytecode to native language translator
would be the easiest solution. In the case of the JavaScript GLV, it would
be fairly straightforward to implement (and I plan to do so!).
In the case of C#, being more strict regarding typing than java, it would
be an incomplete solution. For example:
Consider the following traversal in Java: g.V().values("age")
The type parameter for values() method is inferred at runtime and things
"just works" thanks to type erasure.
In the case of C#, it should be written as g.V().Values<int>("age") , with
the type parameter to specify which type of traverser are you expecting at
a compile time.
That's why it wont be possible to make a generic translator from bytecode
to C# code, without understanding which generic types are expected.
In the case of C#, I was going for a translator on the C# side that
tokenizes the gremlin traversal. Once the traversal is tokenized, in the
moment of invoking the methods (with reflection), it can be made aware of
the modern graph data (with datatypes!), ie: For method Values<>(), if the
property key is "age" -> use int as generic type parameter.
On Thu, Oct 26, 2017 at 2:12 PM, Stephen Mallette <sp...@gmail.com>
wrote:
> I can't help thinking that perhaps non-JVM languages need to leverage
> bytecode to make traversal building easier. We already have an established
> pattern for converting bytecode into Traversals and while it is different
> for each language, it's generally governed by the Translator interface.
> Note that we already do this for Java, Groovy, and Python and the code
> isn't too crazy - just 200 lines or so:
>
> https://github.com/apache/tinkerpop/blob/master/gremlin-
> core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/JavaTranslator.java
> https://github.com/apache/tinkerpop/blob/master/gremlin-
> groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/
> GroovyTranslator.java
> https://github.com/apache/tinkerpop/blob/master/gremlin-
> python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/
> PythonTranslator.java
>
> I think that non-JVM languages could send the Gremlin string from the test
> to Gremlin Server as a script first and return the bytecode as JSON. Then
> it could use a "Translator" to then parse it into it's native language.
> Shouldn't we just build a CSharpTranslator similar to these? does that make
> sense?
>
>
> On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <
> jorgebaygondra@gmail.com
> > wrote:
>
> > I've been looking over the current test scenarios on the TINKERPOP-1784
> > branch, with the C# GLV test suite in mind, and oh man that's an
> impressive
> > amount of scenarios!
> >
> > So far, I have a suggestion.
> >
> > Instead of
> >
> > Scenario: g_V_fold_countXlocalX
> > Given the modern graph
> > And the traversal of
> > """
> > g.V().fold().count(Scope.local)
> > """
> >
> > We could use
> >
> > Scenario: g_V_fold_countXlocalX
> > Given the modern graph
> > And parameter 1 being an enum
> > """
> > Scope.local
> > """
> > And the traversal of
> > """
> > g.V().fold().count(param1)
> > """
> >
> > That way we can exclude enums from possible parameter values, narrowing
> > down the amount of types of parameters allowed, ie: const numeric, const
> > strings (wrapped in double quotes), traversals (starting with `__`) ,
> ...,
> > making parsing the traversal a little easier.
> >
> >
> > On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > Just a quick update on this item - it continues........
> > >
> > > Went down a bad path earlier in the week and ended up shelving a lot of
> > > work - ended up just coded into a corner. Anyway, I've not really
> changed
> > > much in the implementation, but I've still not reached a point where
> the
> > > addition of a new .feature files comes without meeting some new type of
> > > assertion that has to be dealt with. This ends up slowing down progress
> > on
> > > porting over the tests. Please feel free to peruse the latest changes
> on
> > > the branch and let me know if there's any feedback.
> > >
> > >
> > >
> > > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <
> spmallette@gmail.com
> > >
> > > wrote:
> > >
> > > > I did some pretty heavy refactoring to the python test logic
> (altered a
> > > > bit by some revision of the gherkin feature file language) and the
> > result
> > > > is a much more simplified test logic file:
> > > >
> > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > gremlin-python/src/main/jython/radish/feature_steps.py
> > > >
> > > > About 120 lines of code (down from about 170). If you include the
> test
> > > > logic "setup" file:
> > > >
> > > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > > gremlin-python/src/main/jython/radish/terrain.py
> > > >
> > > > we end up with about 250 lines of test logic total (take out the
> > > > comment/license and we're probably well under 200 total - not too
> > bad). I
> > > > think I'm closing in on the end to infrastructure building here so
> the
> > > > basic framework is getting close to final at this point I believe.
> I'll
> > > > keep scanning the tests looking for other types of assertions that
> I've
> > > not
> > > > yet covered, but it's getting pretty solid I think. Hopefully, there
> > > won't
> > > > need to be too many more lines of code needed to express the test
> logic
> > > as
> > > > I like how things are looking right now.
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > > jorgebaygondra@gmail.com> wrote:
> > > >
> > > >> Great progress! I like how you avoided using ids, even if it adds
> some
> > > >> complexity to the transformation required.
> > > >>
> > > >> The Python step definitions are still quite simple, I think it would
> > be
> > > >> mostly the same on the rest of the languages.
> > > >>
> > > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> > spmallette@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Just another update on progress with the test suite. The language
> of
> > > the
> > > >> > .feature files is getting slightly more complex as I try to
> > translate
> > > >> more
> > > >> > and more of our Java process suite tests into the language of the
> > > >> gherkin
> > > >> > files. You can see here where I've added the ability to include
> > > >> parameters:
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-test/features/filter/Has.feature#L33
> > > >> >
> > > >> > I also came up with a method of asserting edges (didn't want to
> rely
> > > on
> > > >> ids
> > > >> > as it makes the gherkin harder to read, plus i didn't want to
> assume
> > > >> > TinkerGraph identifiers in case these tests were every used with
> > some
> > > >> other
> > > >> > graph database that didn't use longs):
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > > >> >
> > > >> > for all those additions (and others) the logic required by the GLV
> > to
> > > >> > process these tests has stayed surprisingly simple:
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > > >> >
> > > >> > There's a fair bit of regex/string manipulation involved there,
> but
> > > it's
> > > >> > processing strings from the feature file so that's the nature of
> it
> > I
> > > >> > suppose. I think I'm of the mind that I want to port all of the
> > tests
> > > to
> > > >> > feature files, so I wrote this unit test to help validate that
> none
> > > were
> > > >> > missed:
> > > >> >
> > > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > > >> > structure/FeatureCoverageTest.java
> > > >> >
> > > >> > I don't know that we have to get to 100% porting right away, but I
> > > think
> > > >> > that once we have these gherkin files written they not only become
> > the
> > > >> > basis for our current GLV testing work, but we might be able to
> > simply
> > > >> use
> > > >> > them for testing the Java stuff as well - that would rid us of
> > having
> > > >> the
> > > >> > test code duplication. It also sets us up with a portable body of
> > > tests
> > > >> > that can be re-used in TinkerPop 4.x.
> > > >> >
> > > >> > I'm open to suggestions if anyone has any.
> > > >> >
> > > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > > >> > jorgebaygondra@gmail.com
> > > >> > > wrote:
> > > >> >
> > > >> > > I was able to build a proof of concept for a Gherkin-based test
> > > >> runner in
> > > >> > > C#, that takes the proposed count and select features
> > > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > > >> > > gremlin-test/features/map>
> > > >> > > and runs them using C# step definitions.
> > > >> > >
> > > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > > >> herkin-dotnet>
> > > >> > > from
> > > >> > > cucumber, there isn't a release of the parser with .NET Core
> > support
> > > >> so
> > > >> > > I've
> > > >> > > asked them to release one
> > > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there
> is
> > no
> > > >> > > limitation from their source files). If they are not able to
> > release
> > > >> it
> > > >> > in
> > > >> > > the next few days, we can implement our own as it should be
> pretty
> > > >> > straight
> > > >> > > forward.
> > > >> > >
> > > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > > >> spmallette@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Thanks for the update. I'm trying to keep the test language as
> > > >> simple
> > > >> > as
> > > >> > > > possible so that we don't need an overly complicated test
> > > >> > implementation.
> > > >> > > > Hopefully that will help make the .NET approach as easy as
> > > possible.
> > > >> > > >
> > > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > > >> > > > jorgebaygondra@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > I've been looking into Gherkin support for .NET: SpecFlow,
> the
> > > >> > cucumber
> > > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > > >> > > > cucumber-implementations
> > > >> > > > > >,
> > > >> > > > > does not support .NET Core platform (we use .NET Core build
> > > tools
> > > >> for
> > > >> > > the
> > > >> > > > > .NET GLV) and only supports .NET Framework.
> > > >> > > > >
> > > >> > > > > From what I can see <https://github.com/techtalk/S
> > > >> pecFlow/projects/2
> > > >> > >,
> > > >> > > > > .NET
> > > >> > > > > Core support on SpecFlow is coming at a very slow pace and
> we
> > > >> > shouldn't
> > > >> > > > > expect to land any time soon (there were some design
> decisions
> > > in
> > > >> > > > SpecFlow
> > > >> > > > > library that makes supporting other platforms non-trivial,
> > like
> > > >> > > requiring
> > > >> > > > > code gen).
> > > >> > > > >
> > > >> > > > > The alternative would be to implement our own harness to
> > support
> > > >> it:
> > > >> > > > from a
> > > >> > > > > xunit test, look for certain types and parse the
> annotations,
> > > and
> > > >> > > execute
> > > >> > > > > them using the Gherkin features.
> > > >> > > > > There is a .NET cross-platform Gherkin parser:
> > > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > > >> > > > > I'll continue looking into this option and try to understand
> > the
> > > >> > effort
> > > >> > > > > required...
> > > >> > > > >
> > > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > > >> > > > > jorgebaygondra@gmail.com>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Nice! Gherkin will make our lives easier with a growing
> > number
> > > >> of
> > > >> > > GLVs.
> > > >> > > > > >
> > > >> > > > > > We should find a way to define the different features
> > > supported
> > > >> by
> > > >> > > each
> > > >> > > > > > GLV, as it's reasonable to have different maturity levels
> > per
> > > >> GLV
> > > >> > > (ie:
> > > >> > > > > > lambdas support, traversal strategy, ...). I don't know if
> > it
> > > >> will
> > > >> > be
> > > >> > > > > > beneficial to do it in the Gherkin files or within each
> GLV
> > > >> > > > > implementation.
> > > >> > > > > > Also, we should consider the process of rolling out a new
> > > >> method /
> > > >> > > > class
> > > >> > > > > > in the java implementation, how that could affect each
> GLV.
> > > >> > > > > >
> > > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > > >> > > > spmallette@gmail.com
> > > >> > > > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > > eval(). I
> > > >> > > guess
> > > >> > > > > the
> > > >> > > > > >> point is that if the language can support some way of
> > taking
> > > >> the
> > > >> > > > string
> > > >> > > > > >> value and turning it automatically into a traversal in
> that
> > > >> GLVs
> > > >> > > style
> > > >> > > > > >> then
> > > >> > > > > >> we should do that.
> > > >> > > > > >>
> > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > > >> <me...@gremlin.guru>
> > > >> > > > > wrote:
> > > >> > > > > >>
> > > >> > > > > >> > For unparameterized queries it can probably be as easy
> > as:
> > > >> > > > > >> >
> > > >> > > > > >> > @given("the traversal of")
> > > >> > > > > >> > def translate_traversal(step):
> > > >> > > > > >> > g = step.context.g
> > > >> > > > > >> > step.context.traversal = eval(step.text)
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > Cheers,
> > > >> > > > > >> > Daniel
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > > >> <me@gremlin.guru
> > > >> > >
> > > >> > > > > >> wrote:
> > > >> > > > > >> >
> > > >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin
> > for
> > > >> > years,
> > > >> > > > > but I
> > > >> > > > > >> > > really like the BDD approach.
> > > >> > > > > >> > >
> > > >> > > > > >> > > and then you can look at the GLV Gremlin translations
> > > >> > > specifically
> > > >> > > > > >> here:
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/blob/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-python/src/main/jython/
> > > radish/count_features_step.py#
> > > >> > > L34-L46
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> > > This part is the only thing that looks weird to me.
> > > You're
> > > >> > > > basically
> > > >> > > > > >> > > writing every query twice; is there really no easier
> > way
> > > >> to do
> > > >> > > > that?
> > > >> > > > > >> > >
> > > >> > > > > >> > > Cheers,
> > > >> > > > > >> > > Daniel
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > > >> > > > > >> spmallette@gmail.com>
> > > >> > > > > >> > > wrote:
> > > >> > > > > >> > >
> > > >> > > > > >> > >> I've brought this issue up in the past and had
> > suggested
> > > >> some
> > > >> > > > > >> options I
> > > >> > > > > >> > >> had
> > > >> > > > > >> > >> in mind but now I've finally put the basics of those
> > > >> ideas in
> > > >> > > > place
> > > >> > > > > >> so I
> > > >> > > > > >> > >> figured I'd start a fresh thread. Recall that the
> > issue
> > > at
> > > >> > hand
> > > >> > > > is
> > > >> > > > > >> that
> > > >> > > > > >> > we
> > > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> > > bound
> > > >> to
> > > >> > > the
> > > >> > > > > >> JVM. We
> > > >> > > > > >> > >> have some tricks that let us test gremlin-python
> with
> > it
> > > >> but
> > > >> > > > those
> > > >> > > > > >> > tricks
> > > >> > > > > >> > >> won't work for every language and we now have the
> > first
> > > >> > > language
> > > >> > > > in
> > > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> > > won't
> > > >> > > > support
> > > >> > > > > it
> > > >> > > > > >> > >> (yes,
> > > >> > > > > >> > >> i know that gremlin-javascript can run on the jvm
> but
> > > >> there
> > > >> > are
> > > >> > > > > >> issues
> > > >> > > > > >> > >> with
> > > >> > > > > >> > >> getting it all to work with the test framework that
> > make
> > > >> it
> > > >> > > > unduly
> > > >> > > > > >> > >> complicated).
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> On other threads I offered the idea that we look to
> > use
> > > >> > Gherkin
> > > >> > > > to
> > > >> > > > > >> write
> > > >> > > > > >> > >> general Gremlin test specifications, which then
> could
> > be
> > > >> read
> > > >> > > and
> > > >> > > > > >> > >> processed
> > > >> > > > > >> > >> by the wide variety of test frameworks that can read
> > > that
> > > >> > > format
> > > >> > > > -
> > > >> > > > > >> there
> > > >> > > > > >> > >> tend to be Gherkin processors in just about every
> > > >> language -
> > > >> > > for
> > > >> > > > > >> > example,
> > > >> > > > > >> > >> see:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://cucumber.io/
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> I just created this issue:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://issues.apache.org/
> jira/browse/TINKERPOP-1784
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> and pushed this branch:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/
> > tinkerpop/tree/TINKERPOP-1784
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> which demonstrates how this works with
> gremlin-python.
> > > The
> > > >> > > basic
> > > >> > > > > >> anatomy
> > > >> > > > > >> > >> of
> > > >> > > > > >> > >> this setup involves this new directory in
> > gremlin-test:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/tree/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-test/features
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> It contains the Gherkin .features files. These are
> the
> > > >> test
> > > >> > > > > >> > >> specifications.
> > > >> > > > > >> > >> They are written using gremlin-java as the "model"
> > > >> language.
> > > >> > > GLVs
> > > >> > > > > >> will
> > > >> > > > > >> > >> then
> > > >> > > > > >> > >> need to write some infrastructure to process these
> > > Gherkin
> > > >> > > files.
> > > >> > > > > The
> > > >> > > > > >> > key
> > > >> > > > > >> > >> to making this "easy" to implement will lie in our
> > > >> abiilty to
> > > >> > > > keep
> > > >> > > > > >> the
> > > >> > > > > >> > >> assertions we want to have relatively simple. The
> more
> > > >> > > simplistic
> > > >> > > > > the
> > > >> > > > > >> > >> language in the Gherkin .feature files the easier
> the
> > > job
> > > >> it
> > > >> > > will
> > > >> > > > > be
> > > >> > > > > >> for
> > > >> > > > > >> > >> GLVs to build their infrastructure. Of course, once
> > that
> > > >> > > > > >> infrastructure
> > > >> > > > > >> > is
> > > >> > > > > >> > >> in place, the GLV developer just has to write the
> GLV
> > > >> version
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > >> > >> Gremlin specified in the .feature file. So you can
> > look
> > > at
> > > >> > all
> > > >> > > > the
> > > >> > > > > >> > >> "infrastructure" code here in this pair of files:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/tree/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-python/src/main/jython/radish
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> and then you can look at the GLV Gremlin
> translations
> > > >> > > > specifically
> > > >> > > > > >> here:
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> https://github.com/apache/tink
> > > >> erpop/blob/TINKERPOP-1784/grem
> > > >> > > > > >> > >> lin-python/src/main/jython/
> > > radish/count_features_step.py#
> > > >> > > L34-L46
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> I think this approach works pretty well and solves
> our
> > > >> > general
> > > >> > > > > >> problems
> > > >> > > > > >> > >> for
> > > >> > > > > >> > >> GLV testing. There is some pain up front in
> > implementing
> > > >> the
> > > >> > > > > >> > >> "infrastructure" but after that new Gremlin tests
> > added
> > > to
> > > >> > > > .feature
> > > >> > > > > >> > files
> > > >> > > > > >> > >> just need to translated in the GLV. I suppose we
> could
> > > >> > > > "automate" a
> > > >> > > > > >> good
> > > >> > > > > >> > >> portion of the translation with reflection of some
> > sort.
> > > >> > > Anything
> > > >> > > > > >> else
> > > >> > > > > >> > >> could just be handled manually.
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> Not sure if we need to use this new model to wholly
> > > >> replace
> > > >> > the
> > > >> > > > old
> > > >> > > > > >> one.
> > > >> > > > > >> > >> The process test suite has its place in helping
> graph
> > > >> > database
> > > >> > > > > >> providers
> > > >> > > > > >> > >> test their stuff. I also imagine that introducing
> this
> > > >> > approach
> > > >> > > > in
> > > >> > > > > >> that
> > > >> > > > > >> > >> context would create a breaking change which we
> would
> > > then
> > > >> > need
> > > >> > > > to
> > > >> > > > > >> push
> > > >> > > > > >> > >> off
> > > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to think,
> but
> > > for
> > > >> now
> > > >> > > it
> > > >> > > > > >> might
> > > >> > > > > >> > not
> > > >> > > > > >> > >> be best to conflate the two and just treat them as
> > > >> separate
> > > >> > > > aspects
> > > >> > > > > >> of
> > > >> > > > > >> > the
> > > >> > > > > >> > >> test suite.
> > > >> > > > > >> > >>
> > > >> > > > > >> > >> Anyway - it's important we settle on an approach to
> > > >> testing
> > > >> > as
> > > >> > > we
> > > >> > > > > >> really
> > > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> > > without
> > > >> > > getting
> > > >> > > > > the
> > > >> > > > > >> > test
> > > >> > > > > >> > >> suite solid. please yell if you have any ideas or
> > > >> feedback on
> > > >> > > > this
> > > >> > > > > >> > >> approach.
> > > >> > > > > >> > >>
> > > >> > > > > >> > >
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] GLV Test Suite
Posted by Stephen Mallette <sp...@gmail.com>.
I can't help thinking that perhaps non-JVM languages need to leverage
bytecode to make traversal building easier. We already have an established
pattern for converting bytecode into Traversals and while it is different
for each language, it's generally governed by the Translator interface.
Note that we already do this for Java, Groovy, and Python and the code
isn't too crazy - just 200 lines or so:
https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/jsr223/JavaTranslator.java
https://github.com/apache/tinkerpop/blob/master/gremlin-groovy/src/main/java/org/apache/tinkerpop/gremlin/groovy/jsr223/GroovyTranslator.java
https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/java/org/apache/tinkerpop/gremlin/python/jsr223/PythonTranslator.java
I think that non-JVM languages could send the Gremlin string from the test
to Gremlin Server as a script first and return the bytecode as JSON. Then
it could use a "Translator" to then parse it into it's native language.
Shouldn't we just build a CSharpTranslator similar to these? does that make
sense?
On Wed, Oct 25, 2017 at 11:11 AM, Jorge Bay Gondra <jorgebaygondra@gmail.com
> wrote:
> I've been looking over the current test scenarios on the TINKERPOP-1784
> branch, with the C# GLV test suite in mind, and oh man that's an impressive
> amount of scenarios!
>
> So far, I have a suggestion.
>
> Instead of
>
> Scenario: g_V_fold_countXlocalX
> Given the modern graph
> And the traversal of
> """
> g.V().fold().count(Scope.local)
> """
>
> We could use
>
> Scenario: g_V_fold_countXlocalX
> Given the modern graph
> And parameter 1 being an enum
> """
> Scope.local
> """
> And the traversal of
> """
> g.V().fold().count(param1)
> """
>
> That way we can exclude enums from possible parameter values, narrowing
> down the amount of types of parameters allowed, ie: const numeric, const
> strings (wrapped in double quotes), traversals (starting with `__`) , ...,
> making parsing the traversal a little easier.
>
>
> On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > Just a quick update on this item - it continues........
> >
> > Went down a bad path earlier in the week and ended up shelving a lot of
> > work - ended up just coded into a corner. Anyway, I've not really changed
> > much in the implementation, but I've still not reached a point where the
> > addition of a new .feature files comes without meeting some new type of
> > assertion that has to be dealt with. This ends up slowing down progress
> on
> > porting over the tests. Please feel free to peruse the latest changes on
> > the branch and let me know if there's any feedback.
> >
> >
> >
> > On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <spmallette@gmail.com
> >
> > wrote:
> >
> > > I did some pretty heavy refactoring to the python test logic (altered a
> > > bit by some revision of the gherkin feature file language) and the
> result
> > > is a much more simplified test logic file:
> > >
> > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > gremlin-python/src/main/jython/radish/feature_steps.py
> > >
> > > About 120 lines of code (down from about 170). If you include the test
> > > logic "setup" file:
> > >
> > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > > gremlin-python/src/main/jython/radish/terrain.py
> > >
> > > we end up with about 250 lines of test logic total (take out the
> > > comment/license and we're probably well under 200 total - not too
> bad). I
> > > think I'm closing in on the end to infrastructure building here so the
> > > basic framework is getting close to final at this point I believe. I'll
> > > keep scanning the tests looking for other types of assertions that I've
> > not
> > > yet covered, but it's getting pretty solid I think. Hopefully, there
> > won't
> > > need to be too many more lines of code needed to express the test logic
> > as
> > > I like how things are looking right now.
> > >
> > >
> > >
> > >
> > > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > > jorgebaygondra@gmail.com> wrote:
> > >
> > >> Great progress! I like how you avoided using ids, even if it adds some
> > >> complexity to the transformation required.
> > >>
> > >> The Python step definitions are still quite simple, I think it would
> be
> > >> mostly the same on the rest of the languages.
> > >>
> > >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <
> spmallette@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Just another update on progress with the test suite. The language of
> > the
> > >> > .feature files is getting slightly more complex as I try to
> translate
> > >> more
> > >> > and more of our Java process suite tests into the language of the
> > >> gherkin
> > >> > files. You can see here where I've added the ability to include
> > >> parameters:
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-test/features/filter/Has.feature#L33
> > >> >
> > >> > I also came up with a method of asserting edges (didn't want to rely
> > on
> > >> ids
> > >> > as it makes the gherkin harder to read, plus i didn't want to assume
> > >> > TinkerGraph identifiers in case these tests were every used with
> some
> > >> other
> > >> > graph database that didn't use longs):
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-test/features/map/Vertex.feature#L105-L111
> > >> >
> > >> > for all those additions (and others) the logic required by the GLV
> to
> > >> > process these tests has stayed surprisingly simple:
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-python/src/main/jython/radish/feature_steps.py
> > >> >
> > >> > There's a fair bit of regex/string manipulation involved there, but
> > it's
> > >> > processing strings from the feature file so that's the nature of it
> I
> > >> > suppose. I think I'm of the mind that I want to port all of the
> tests
> > to
> > >> > feature files, so I wrote this unit test to help validate that none
> > were
> > >> > missed:
> > >> >
> > >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> > >> > structure/FeatureCoverageTest.java
> > >> >
> > >> > I don't know that we have to get to 100% porting right away, but I
> > think
> > >> > that once we have these gherkin files written they not only become
> the
> > >> > basis for our current GLV testing work, but we might be able to
> simply
> > >> use
> > >> > them for testing the Java stuff as well - that would rid us of
> having
> > >> the
> > >> > test code duplication. It also sets us up with a portable body of
> > tests
> > >> > that can be re-used in TinkerPop 4.x.
> > >> >
> > >> > I'm open to suggestions if anyone has any.
> > >> >
> > >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> > >> > jorgebaygondra@gmail.com
> > >> > > wrote:
> > >> >
> > >> > > I was able to build a proof of concept for a Gherkin-based test
> > >> runner in
> > >> > > C#, that takes the proposed count and select features
> > >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> > >> > > gremlin-test/features/map>
> > >> > > and runs them using C# step definitions.
> > >> > >
> > >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> > >> herkin-dotnet>
> > >> > > from
> > >> > > cucumber, there isn't a release of the parser with .NET Core
> support
> > >> so
> > >> > > I've
> > >> > > asked them to release one
> > >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is
> no
> > >> > > limitation from their source files). If they are not able to
> release
> > >> it
> > >> > in
> > >> > > the next few days, we can implement our own as it should be pretty
> > >> > straight
> > >> > > forward.
> > >> > >
> > >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> > >> spmallette@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Thanks for the update. I'm trying to keep the test language as
> > >> simple
> > >> > as
> > >> > > > possible so that we don't need an overly complicated test
> > >> > implementation.
> > >> > > > Hopefully that will help make the .NET approach as easy as
> > possible.
> > >> > > >
> > >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> > >> > > > jorgebaygondra@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> > >> > cucumber
> > >> > > > > implementation for .NET <https://cucumber.io/docs#
> > >> > > > cucumber-implementations
> > >> > > > > >,
> > >> > > > > does not support .NET Core platform (we use .NET Core build
> > tools
> > >> for
> > >> > > the
> > >> > > > > .NET GLV) and only supports .NET Framework.
> > >> > > > >
> > >> > > > > From what I can see <https://github.com/techtalk/S
> > >> pecFlow/projects/2
> > >> > >,
> > >> > > > > .NET
> > >> > > > > Core support on SpecFlow is coming at a very slow pace and we
> > >> > shouldn't
> > >> > > > > expect to land any time soon (there were some design decisions
> > in
> > >> > > > SpecFlow
> > >> > > > > library that makes supporting other platforms non-trivial,
> like
> > >> > > requiring
> > >> > > > > code gen).
> > >> > > > >
> > >> > > > > The alternative would be to implement our own harness to
> support
> > >> it:
> > >> > > > from a
> > >> > > > > xunit test, look for certain types and parse the annotations,
> > and
> > >> > > execute
> > >> > > > > them using the Gherkin features.
> > >> > > > > There is a .NET cross-platform Gherkin parser:
> > >> > > > > https://github.com/cucumber/gherkin-dotnet
> > >> > > > > I'll continue looking into this option and try to understand
> the
> > >> > effort
> > >> > > > > required...
> > >> > > > >
> > >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> > >> > > > > jorgebaygondra@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Nice! Gherkin will make our lives easier with a growing
> number
> > >> of
> > >> > > GLVs.
> > >> > > > > >
> > >> > > > > > We should find a way to define the different features
> > supported
> > >> by
> > >> > > each
> > >> > > > > > GLV, as it's reasonable to have different maturity levels
> per
> > >> GLV
> > >> > > (ie:
> > >> > > > > > lambdas support, traversal strategy, ...). I don't know if
> it
> > >> will
> > >> > be
> > >> > > > > > beneficial to do it in the Gherkin files or within each GLV
> > >> > > > > implementation.
> > >> > > > > > Also, we should consider the process of rolling out a new
> > >> method /
> > >> > > > class
> > >> > > > > > in the java implementation, how that could affect each GLV.
> > >> > > > > >
> > >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> > >> > > > spmallette@gmail.com
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > >> that's what i meant by "reflection" or as you suggest
> > eval(). I
> > >> > > guess
> > >> > > > > the
> > >> > > > > >> point is that if the language can support some way of
> taking
> > >> the
> > >> > > > string
> > >> > > > > >> value and turning it automatically into a traversal in that
> > >> GLVs
> > >> > > style
> > >> > > > > >> then
> > >> > > > > >> we should do that.
> > >> > > > > >>
> > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> > >> <me...@gremlin.guru>
> > >> > > > > wrote:
> > >> > > > > >>
> > >> > > > > >> > For unparameterized queries it can probably be as easy
> as:
> > >> > > > > >> >
> > >> > > > > >> > @given("the traversal of")
> > >> > > > > >> > def translate_traversal(step):
> > >> > > > > >> > g = step.context.g
> > >> > > > > >> > step.context.traversal = eval(step.text)
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > Cheers,
> > >> > > > > >> > Daniel
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> > >> <me@gremlin.guru
> > >> > >
> > >> > > > > >> wrote:
> > >> > > > > >> >
> > >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin
> for
> > >> > years,
> > >> > > > > but I
> > >> > > > > >> > > really like the BDD approach.
> > >> > > > > >> > >
> > >> > > > > >> > > and then you can look at the GLV Gremlin translations
> > >> > > specifically
> > >> > > > > >> here:
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-python/src/main/jython/
> > radish/count_features_step.py#
> > >> > > L34-L46
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > This part is the only thing that looks weird to me.
> > You're
> > >> > > > basically
> > >> > > > > >> > > writing every query twice; is there really no easier
> way
> > >> to do
> > >> > > > that?
> > >> > > > > >> > >
> > >> > > > > >> > > Cheers,
> > >> > > > > >> > > Daniel
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> > >> > > > > >> spmallette@gmail.com>
> > >> > > > > >> > > wrote:
> > >> > > > > >> > >
> > >> > > > > >> > >> I've brought this issue up in the past and had
> suggested
> > >> some
> > >> > > > > >> options I
> > >> > > > > >> > >> had
> > >> > > > > >> > >> in mind but now I've finally put the basics of those
> > >> ideas in
> > >> > > > place
> > >> > > > > >> so I
> > >> > > > > >> > >> figured I'd start a fresh thread. Recall that the
> issue
> > at
> > >> > hand
> > >> > > > is
> > >> > > > > >> that
> > >> > > > > >> > we
> > >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> > bound
> > >> to
> > >> > > the
> > >> > > > > >> JVM. We
> > >> > > > > >> > >> have some tricks that let us test gremlin-python with
> it
> > >> but
> > >> > > > those
> > >> > > > > >> > tricks
> > >> > > > > >> > >> won't work for every language and we now have the
> first
> > >> > > language
> > >> > > > in
> > >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> > won't
> > >> > > > support
> > >> > > > > it
> > >> > > > > >> > >> (yes,
> > >> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
> > >> there
> > >> > are
> > >> > > > > >> issues
> > >> > > > > >> > >> with
> > >> > > > > >> > >> getting it all to work with the test framework that
> make
> > >> it
> > >> > > > unduly
> > >> > > > > >> > >> complicated).
> > >> > > > > >> > >>
> > >> > > > > >> > >> On other threads I offered the idea that we look to
> use
> > >> > Gherkin
> > >> > > > to
> > >> > > > > >> write
> > >> > > > > >> > >> general Gremlin test specifications, which then could
> be
> > >> read
> > >> > > and
> > >> > > > > >> > >> processed
> > >> > > > > >> > >> by the wide variety of test frameworks that can read
> > that
> > >> > > format
> > >> > > > -
> > >> > > > > >> there
> > >> > > > > >> > >> tend to be Gherkin processors in just about every
> > >> language -
> > >> > > for
> > >> > > > > >> > example,
> > >> > > > > >> > >> see:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://cucumber.io/
> > >> > > > > >> > >>
> > >> > > > > >> > >> I just created this issue:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> > >> > > > > >> > >>
> > >> > > > > >> > >> and pushed this branch:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/
> tinkerpop/tree/TINKERPOP-1784
> > >> > > > > >> > >>
> > >> > > > > >> > >> which demonstrates how this works with gremlin-python.
> > The
> > >> > > basic
> > >> > > > > >> anatomy
> > >> > > > > >> > >> of
> > >> > > > > >> > >> this setup involves this new directory in
> gremlin-test:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-test/features
> > >> > > > > >> > >>
> > >> > > > > >> > >> It contains the Gherkin .features files. These are the
> > >> test
> > >> > > > > >> > >> specifications.
> > >> > > > > >> > >> They are written using gremlin-java as the "model"
> > >> language.
> > >> > > GLVs
> > >> > > > > >> will
> > >> > > > > >> > >> then
> > >> > > > > >> > >> need to write some infrastructure to process these
> > Gherkin
> > >> > > files.
> > >> > > > > The
> > >> > > > > >> > key
> > >> > > > > >> > >> to making this "easy" to implement will lie in our
> > >> abiilty to
> > >> > > > keep
> > >> > > > > >> the
> > >> > > > > >> > >> assertions we want to have relatively simple. The more
> > >> > > simplistic
> > >> > > > > the
> > >> > > > > >> > >> language in the Gherkin .feature files the easier the
> > job
> > >> it
> > >> > > will
> > >> > > > > be
> > >> > > > > >> for
> > >> > > > > >> > >> GLVs to build their infrastructure. Of course, once
> that
> > >> > > > > >> infrastructure
> > >> > > > > >> > is
> > >> > > > > >> > >> in place, the GLV developer just has to write the GLV
> > >> version
> > >> > > of
> > >> > > > > the
> > >> > > > > >> > >> Gremlin specified in the .feature file. So you can
> look
> > at
> > >> > all
> > >> > > > the
> > >> > > > > >> > >> "infrastructure" code here in this pair of files:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/tree/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-python/src/main/jython/radish
> > >> > > > > >> > >>
> > >> > > > > >> > >> and then you can look at the GLV Gremlin translations
> > >> > > > specifically
> > >> > > > > >> here:
> > >> > > > > >> > >>
> > >> > > > > >> > >> https://github.com/apache/tink
> > >> erpop/blob/TINKERPOP-1784/grem
> > >> > > > > >> > >> lin-python/src/main/jython/
> > radish/count_features_step.py#
> > >> > > L34-L46
> > >> > > > > >> > >>
> > >> > > > > >> > >> I think this approach works pretty well and solves our
> > >> > general
> > >> > > > > >> problems
> > >> > > > > >> > >> for
> > >> > > > > >> > >> GLV testing. There is some pain up front in
> implementing
> > >> the
> > >> > > > > >> > >> "infrastructure" but after that new Gremlin tests
> added
> > to
> > >> > > > .feature
> > >> > > > > >> > files
> > >> > > > > >> > >> just need to translated in the GLV. I suppose we could
> > >> > > > "automate" a
> > >> > > > > >> good
> > >> > > > > >> > >> portion of the translation with reflection of some
> sort.
> > >> > > Anything
> > >> > > > > >> else
> > >> > > > > >> > >> could just be handled manually.
> > >> > > > > >> > >>
> > >> > > > > >> > >> Not sure if we need to use this new model to wholly
> > >> replace
> > >> > the
> > >> > > > old
> > >> > > > > >> one.
> > >> > > > > >> > >> The process test suite has its place in helping graph
> > >> > database
> > >> > > > > >> providers
> > >> > > > > >> > >> test their stuff. I also imagine that introducing this
> > >> > approach
> > >> > > > in
> > >> > > > > >> that
> > >> > > > > >> > >> context would create a breaking change which we would
> > then
> > >> > need
> > >> > > > to
> > >> > > > > >> push
> > >> > > > > >> > >> off
> > >> > > > > >> > >> to 3.4.0. I suppose that gives us time to think, but
> > for
> > >> now
> > >> > > it
> > >> > > > > >> might
> > >> > > > > >> > not
> > >> > > > > >> > >> be best to conflate the two and just treat them as
> > >> separate
> > >> > > > aspects
> > >> > > > > >> of
> > >> > > > > >> > the
> > >> > > > > >> > >> test suite.
> > >> > > > > >> > >>
> > >> > > > > >> > >> Anyway - it's important we settle on an approach to
> > >> testing
> > >> > as
> > >> > > we
> > >> > > > > >> really
> > >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> > without
> > >> > > getting
> > >> > > > > the
> > >> > > > > >> > test
> > >> > > > > >> > >> suite solid. please yell if you have any ideas or
> > >> feedback on
> > >> > > > this
> > >> > > > > >> > >> approach.
> > >> > > > > >> > >>
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: [DISCUSS] GLV Test Suite
Posted by Jorge Bay Gondra <jo...@gmail.com>.
I've been looking over the current test scenarios on the TINKERPOP-1784
branch, with the C# GLV test suite in mind, and oh man that's an impressive
amount of scenarios!
So far, I have a suggestion.
Instead of
Scenario: g_V_fold_countXlocalX
Given the modern graph
And the traversal of
"""
g.V().fold().count(Scope.local)
"""
We could use
Scenario: g_V_fold_countXlocalX
Given the modern graph
And parameter 1 being an enum
"""
Scope.local
"""
And the traversal of
"""
g.V().fold().count(param1)
"""
That way we can exclude enums from possible parameter values, narrowing
down the amount of types of parameters allowed, ie: const numeric, const
strings (wrapped in double quotes), traversals (starting with `__`) , ...,
making parsing the traversal a little easier.
On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <sp...@gmail.com>
wrote:
> Just a quick update on this item - it continues........
>
> Went down a bad path earlier in the week and ended up shelving a lot of
> work - ended up just coded into a corner. Anyway, I've not really changed
> much in the implementation, but I've still not reached a point where the
> addition of a new .feature files comes without meeting some new type of
> assertion that has to be dealt with. This ends up slowing down progress on
> porting over the tests. Please feel free to peruse the latest changes on
> the branch and let me know if there's any feedback.
>
>
>
> On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > I did some pretty heavy refactoring to the python test logic (altered a
> > bit by some revision of the gherkin feature file language) and the result
> > is a much more simplified test logic file:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/feature_steps.py
> >
> > About 120 lines of code (down from about 170). If you include the test
> > logic "setup" file:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/terrain.py
> >
> > we end up with about 250 lines of test logic total (take out the
> > comment/license and we're probably well under 200 total - not too bad). I
> > think I'm closing in on the end to infrastructure building here so the
> > basic framework is getting close to final at this point I believe. I'll
> > keep scanning the tests looking for other types of assertions that I've
> not
> > yet covered, but it's getting pretty solid I think. Hopefully, there
> won't
> > need to be too many more lines of code needed to express the test logic
> as
> > I like how things are looking right now.
> >
> >
> >
> >
> > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > jorgebaygondra@gmail.com> wrote:
> >
> >> Great progress! I like how you avoided using ids, even if it adds some
> >> complexity to the transformation required.
> >>
> >> The Python step definitions are still quite simple, I think it would be
> >> mostly the same on the rest of the languages.
> >>
> >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <spmallette@gmail.com
> >
> >> wrote:
> >>
> >> > Just another update on progress with the test suite. The language of
> the
> >> > .feature files is getting slightly more complex as I try to translate
> >> more
> >> > and more of our Java process suite tests into the language of the
> >> gherkin
> >> > files. You can see here where I've added the ability to include
> >> parameters:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/features/filter/Has.feature#L33
> >> >
> >> > I also came up with a method of asserting edges (didn't want to rely
> on
> >> ids
> >> > as it makes the gherkin harder to read, plus i didn't want to assume
> >> > TinkerGraph identifiers in case these tests were every used with some
> >> other
> >> > graph database that didn't use longs):
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/features/map/Vertex.feature#L105-L111
> >> >
> >> > for all those additions (and others) the logic required by the GLV to
> >> > process these tests has stayed surprisingly simple:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-python/src/main/jython/radish/feature_steps.py
> >> >
> >> > There's a fair bit of regex/string manipulation involved there, but
> it's
> >> > processing strings from the feature file so that's the nature of it I
> >> > suppose. I think I'm of the mind that I want to port all of the tests
> to
> >> > feature files, so I wrote this unit test to help validate that none
> were
> >> > missed:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> >> > structure/FeatureCoverageTest.java
> >> >
> >> > I don't know that we have to get to 100% porting right away, but I
> think
> >> > that once we have these gherkin files written they not only become the
> >> > basis for our current GLV testing work, but we might be able to simply
> >> use
> >> > them for testing the Java stuff as well - that would rid us of having
> >> the
> >> > test code duplication. It also sets us up with a portable body of
> tests
> >> > that can be re-used in TinkerPop 4.x.
> >> >
> >> > I'm open to suggestions if anyone has any.
> >> >
> >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> >> > jorgebaygondra@gmail.com
> >> > > wrote:
> >> >
> >> > > I was able to build a proof of concept for a Gherkin-based test
> >> runner in
> >> > > C#, that takes the proposed count and select features
> >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> >> > > gremlin-test/features/map>
> >> > > and runs them using C# step definitions.
> >> > >
> >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> >> herkin-dotnet>
> >> > > from
> >> > > cucumber, there isn't a release of the parser with .NET Core support
> >> so
> >> > > I've
> >> > > asked them to release one
> >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
> >> > > limitation from their source files). If they are not able to release
> >> it
> >> > in
> >> > > the next few days, we can implement our own as it should be pretty
> >> > straight
> >> > > forward.
> >> > >
> >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> >> spmallette@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Thanks for the update. I'm trying to keep the test language as
> >> simple
> >> > as
> >> > > > possible so that we don't need an overly complicated test
> >> > implementation.
> >> > > > Hopefully that will help make the .NET approach as easy as
> possible.
> >> > > >
> >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> >> > > > jorgebaygondra@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> >> > cucumber
> >> > > > > implementation for .NET <https://cucumber.io/docs#
> >> > > > cucumber-implementations
> >> > > > > >,
> >> > > > > does not support .NET Core platform (we use .NET Core build
> tools
> >> for
> >> > > the
> >> > > > > .NET GLV) and only supports .NET Framework.
> >> > > > >
> >> > > > > From what I can see <https://github.com/techtalk/S
> >> pecFlow/projects/2
> >> > >,
> >> > > > > .NET
> >> > > > > Core support on SpecFlow is coming at a very slow pace and we
> >> > shouldn't
> >> > > > > expect to land any time soon (there were some design decisions
> in
> >> > > > SpecFlow
> >> > > > > library that makes supporting other platforms non-trivial, like
> >> > > requiring
> >> > > > > code gen).
> >> > > > >
> >> > > > > The alternative would be to implement our own harness to support
> >> it:
> >> > > > from a
> >> > > > > xunit test, look for certain types and parse the annotations,
> and
> >> > > execute
> >> > > > > them using the Gherkin features.
> >> > > > > There is a .NET cross-platform Gherkin parser:
> >> > > > > https://github.com/cucumber/gherkin-dotnet
> >> > > > > I'll continue looking into this option and try to understand the
> >> > effort
> >> > > > > required...
> >> > > > >
> >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> >> > > > > jorgebaygondra@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Nice! Gherkin will make our lives easier with a growing number
> >> of
> >> > > GLVs.
> >> > > > > >
> >> > > > > > We should find a way to define the different features
> supported
> >> by
> >> > > each
> >> > > > > > GLV, as it's reasonable to have different maturity levels per
> >> GLV
> >> > > (ie:
> >> > > > > > lambdas support, traversal strategy, ...). I don't know if it
> >> will
> >> > be
> >> > > > > > beneficial to do it in the Gherkin files or within each GLV
> >> > > > > implementation.
> >> > > > > > Also, we should consider the process of rolling out a new
> >> method /
> >> > > > class
> >> > > > > > in the java implementation, how that could affect each GLV.
> >> > > > > >
> >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> >> > > > spmallette@gmail.com
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> that's what i meant by "reflection" or as you suggest
> eval(). I
> >> > > guess
> >> > > > > the
> >> > > > > >> point is that if the language can support some way of taking
> >> the
> >> > > > string
> >> > > > > >> value and turning it automatically into a traversal in that
> >> GLVs
> >> > > style
> >> > > > > >> then
> >> > > > > >> we should do that.
> >> > > > > >>
> >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> >> <me...@gremlin.guru>
> >> > > > > wrote:
> >> > > > > >>
> >> > > > > >> > For unparameterized queries it can probably be as easy as:
> >> > > > > >> >
> >> > > > > >> > @given("the traversal of")
> >> > > > > >> > def translate_traversal(step):
> >> > > > > >> > g = step.context.g
> >> > > > > >> > step.context.traversal = eval(step.text)
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > Cheers,
> >> > > > > >> > Daniel
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> >> <me@gremlin.guru
> >> > >
> >> > > > > >> wrote:
> >> > > > > >> >
> >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
> >> > years,
> >> > > > > but I
> >> > > > > >> > > really like the BDD approach.
> >> > > > > >> > >
> >> > > > > >> > > and then you can look at the GLV Gremlin translations
> >> > > specifically
> >> > > > > >> here:
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/
> radish/count_features_step.py#
> >> > > L34-L46
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > This part is the only thing that looks weird to me.
> You're
> >> > > > basically
> >> > > > > >> > > writing every query twice; is there really no easier way
> >> to do
> >> > > > that?
> >> > > > > >> > >
> >> > > > > >> > > Cheers,
> >> > > > > >> > > Daniel
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> >> > > > > >> spmallette@gmail.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > >
> >> > > > > >> > >> I've brought this issue up in the past and had suggested
> >> some
> >> > > > > >> options I
> >> > > > > >> > >> had
> >> > > > > >> > >> in mind but now I've finally put the basics of those
> >> ideas in
> >> > > > place
> >> > > > > >> so I
> >> > > > > >> > >> figured I'd start a fresh thread. Recall that the issue
> at
> >> > hand
> >> > > > is
> >> > > > > >> that
> >> > > > > >> > we
> >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> bound
> >> to
> >> > > the
> >> > > > > >> JVM. We
> >> > > > > >> > >> have some tricks that let us test gremlin-python with it
> >> but
> >> > > > those
> >> > > > > >> > tricks
> >> > > > > >> > >> won't work for every language and we now have the first
> >> > > language
> >> > > > in
> >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> won't
> >> > > > support
> >> > > > > it
> >> > > > > >> > >> (yes,
> >> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
> >> there
> >> > are
> >> > > > > >> issues
> >> > > > > >> > >> with
> >> > > > > >> > >> getting it all to work with the test framework that make
> >> it
> >> > > > unduly
> >> > > > > >> > >> complicated).
> >> > > > > >> > >>
> >> > > > > >> > >> On other threads I offered the idea that we look to use
> >> > Gherkin
> >> > > > to
> >> > > > > >> write
> >> > > > > >> > >> general Gremlin test specifications, which then could be
> >> read
> >> > > and
> >> > > > > >> > >> processed
> >> > > > > >> > >> by the wide variety of test frameworks that can read
> that
> >> > > format
> >> > > > -
> >> > > > > >> there
> >> > > > > >> > >> tend to be Gherkin processors in just about every
> >> language -
> >> > > for
> >> > > > > >> > example,
> >> > > > > >> > >> see:
> >> > > > > >> > >>
> >> > > > > >> > >> https://cucumber.io/
> >> > > > > >> > >>
> >> > > > > >> > >> I just created this issue:
> >> > > > > >> > >>
> >> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> >> > > > > >> > >>
> >> > > > > >> > >> and pushed this branch:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> >> > > > > >> > >>
> >> > > > > >> > >> which demonstrates how this works with gremlin-python.
> The
> >> > > basic
> >> > > > > >> anatomy
> >> > > > > >> > >> of
> >> > > > > >> > >> this setup involves this new directory in gremlin-test:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-test/features
> >> > > > > >> > >>
> >> > > > > >> > >> It contains the Gherkin .features files. These are the
> >> test
> >> > > > > >> > >> specifications.
> >> > > > > >> > >> They are written using gremlin-java as the "model"
> >> language.
> >> > > GLVs
> >> > > > > >> will
> >> > > > > >> > >> then
> >> > > > > >> > >> need to write some infrastructure to process these
> Gherkin
> >> > > files.
> >> > > > > The
> >> > > > > >> > key
> >> > > > > >> > >> to making this "easy" to implement will lie in our
> >> abiilty to
> >> > > > keep
> >> > > > > >> the
> >> > > > > >> > >> assertions we want to have relatively simple. The more
> >> > > simplistic
> >> > > > > the
> >> > > > > >> > >> language in the Gherkin .feature files the easier the
> job
> >> it
> >> > > will
> >> > > > > be
> >> > > > > >> for
> >> > > > > >> > >> GLVs to build their infrastructure. Of course, once that
> >> > > > > >> infrastructure
> >> > > > > >> > is
> >> > > > > >> > >> in place, the GLV developer just has to write the GLV
> >> version
> >> > > of
> >> > > > > the
> >> > > > > >> > >> Gremlin specified in the .feature file. So you can look
> at
> >> > all
> >> > > > the
> >> > > > > >> > >> "infrastructure" code here in this pair of files:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/radish
> >> > > > > >> > >>
> >> > > > > >> > >> and then you can look at the GLV Gremlin translations
> >> > > > specifically
> >> > > > > >> here:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/
> radish/count_features_step.py#
> >> > > L34-L46
> >> > > > > >> > >>
> >> > > > > >> > >> I think this approach works pretty well and solves our
> >> > general
> >> > > > > >> problems
> >> > > > > >> > >> for
> >> > > > > >> > >> GLV testing. There is some pain up front in implementing
> >> the
> >> > > > > >> > >> "infrastructure" but after that new Gremlin tests added
> to
> >> > > > .feature
> >> > > > > >> > files
> >> > > > > >> > >> just need to translated in the GLV. I suppose we could
> >> > > > "automate" a
> >> > > > > >> good
> >> > > > > >> > >> portion of the translation with reflection of some sort.
> >> > > Anything
> >> > > > > >> else
> >> > > > > >> > >> could just be handled manually.
> >> > > > > >> > >>
> >> > > > > >> > >> Not sure if we need to use this new model to wholly
> >> replace
> >> > the
> >> > > > old
> >> > > > > >> one.
> >> > > > > >> > >> The process test suite has its place in helping graph
> >> > database
> >> > > > > >> providers
> >> > > > > >> > >> test their stuff. I also imagine that introducing this
> >> > approach
> >> > > > in
> >> > > > > >> that
> >> > > > > >> > >> context would create a breaking change which we would
> then
> >> > need
> >> > > > to
> >> > > > > >> push
> >> > > > > >> > >> off
> >> > > > > >> > >> to 3.4.0. I suppose that gives us time to think, but
> for
> >> now
> >> > > it
> >> > > > > >> might
> >> > > > > >> > not
> >> > > > > >> > >> be best to conflate the two and just treat them as
> >> separate
> >> > > > aspects
> >> > > > > >> of
> >> > > > > >> > the
> >> > > > > >> > >> test suite.
> >> > > > > >> > >>
> >> > > > > >> > >> Anyway - it's important we settle on an approach to
> >> testing
> >> > as
> >> > > we
> >> > > > > >> really
> >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> without
> >> > > getting
> >> > > > > the
> >> > > > > >> > test
> >> > > > > >> > >> suite solid. please yell if you have any ideas or
> >> feedback on
> >> > > > this
> >> > > > > >> > >> approach.
> >> > > > > >> > >>
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>