You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by Marko Rodriguez <ok...@gmail.com> on 2015/10/05 16:45:51 UTC

[DISCUSS] How to handle backwards compatibility with new GroupStep?

Hello R-Pop-Tik-e,

Matt Frantz and I have been going back and forth on the following ticket:

	https://issues.apache.org/jira/browse/TINKERPOP3-866

This idea was spawned because Kuppitz and I were trying to figure out if we could get rid of GroupCountStep and replace it with a parameterized version of GroupStep.

A short review of the argument is provided below (MENTORS: the full argument is in JIRA in the URL above (which also went to dev@ as JIRA comments/tickets email dev@)).
	1. GroupStep (3.0.1) uses a collection prior to reduction.
	2. This is inefficient as we waste memory holding data in a collection.
	3. This gets especially nasty in OLAP when you can get some massive data growth.
	4. Lets make GroupStep use lazy evaluators.
	5. Done. Damn -- its not backwards compatible.
	6. More sucky, we can't just have a @Deprecated method.

Matt had a nice idea (MENTORS: again this idea is in JIRA). What about making GraphTraversal30 be a thing that always exists for each version. Thus, people who can't so easily upgrade to can still stick with GraphTraversal30.

The question becomes, how is this expressed (MENTORS: This is lightly in JIRA but I bring it up here in more depth):

g = graph.traversal(GraphTraversal30.build().create())
		OR
gremlin.graph.traversal.default=org.apache.tinkerpop.process.traversal.dsl.graph30.GraphTraversal30

Please reply to JIRA with your thoughts on how we can accomplish this (MENTORS: people will reply to JIRA with their thoughts, however, fortunately all JIRA comments/tickets go to dev@).
		https://issues.apache.org/jira/browse/TINKERPOP3-866

Thank you very much for your time,
Marko.

http://markorodriguez.com

Re: [DISCUSS] How to handle backwards compatibility with new GroupStep?

Posted by Daniel Gruno <hu...@apache.org>.
Hi Marko,
Replying on JIRA with ideas is fine (albeit a bit cumbersome for those
that aren't registered there yet), and I appreciate that you started the
discussion on the ML. I would prefer that we also allow people to chime
in with ideas here (someone can always copy it into JIRA if need be),
but it's a good first step to start off by clearly saying "here's
something we need to discuss, and here's where we'll do it".

While we do prefer async forums such as email or JIRA for this, you are
of course allowed to use more direct methods such as slack, hipchat,
IRC, IRL meetings etc, as long as the outcome of that is brought back to
the list (and those who did not attend have a chance to chime in).

As an aside, not specifically related to this email, you can think of
mentors as teachers at a university or school. While we will help you as
much as we can during incubation, once the big exam comes (and we'll
have a few of those even before that), it is up to _you_ to prove to us
that you are ready and prepared. Pointing to something and saying "oh
just read that other book over there" is hardly going to persuade us,
what matters is what is presented to us, not what might lie hidden in a
JIRA somewhere. The dev and private lists are our primary avenue for
evaluating your progress, and if we're not satisfied with what we see
there, we will act (grade) accordingly, regardless of what else may be
in a footnote or a compendium somewhere else. It's the _presentation_
(i.e. how you progress as a community, not as a JIRA group) that matters
first and foremost.

We frankly don't care much about the programming side of Tinkerpop, we
care about the community aspect, and you cannot foster and grow a
community via JIRA.

With regards,
Daniel

On 10/05/2015 04:45 PM, Marko Rodriguez wrote:
> Hello R-Pop-Tik-e,
> 
> Matt Frantz and I have been going back and forth on the following ticket:
> 
> 	https://issues.apache.org/jira/browse/TINKERPOP3-866
> 
> This idea was spawned because Kuppitz and I were trying to figure out if we could get rid of GroupCountStep and replace it with a parameterized version of GroupStep.
> 
> A short review of the argument is provided below (MENTORS: the full argument is in JIRA in the URL above (which also went to dev@ as JIRA comments/tickets email dev@)).
> 	1. GroupStep (3.0.1) uses a collection prior to reduction.
> 	2. This is inefficient as we waste memory holding data in a collection.
> 	3. This gets especially nasty in OLAP when you can get some massive data growth.
> 	4. Lets make GroupStep use lazy evaluators.
> 	5. Done. Damn -- its not backwards compatible.
> 	6. More sucky, we can't just have a @Deprecated method.
> 
> Matt had a nice idea (MENTORS: again this idea is in JIRA). What about making GraphTraversal30 be a thing that always exists for each version. Thus, people who can't so easily upgrade to can still stick with GraphTraversal30.
> 
> The question becomes, how is this expressed (MENTORS: This is lightly in JIRA but I bring it up here in more depth):
> 
> g = graph.traversal(GraphTraversal30.build().create())
> 		OR
> gremlin.graph.traversal.default=org.apache.tinkerpop.process.traversal.dsl.graph30.GraphTraversal30
> 
> Please reply to JIRA with your thoughts on how we can accomplish this (MENTORS: people will reply to JIRA with their thoughts, however, fortunately all JIRA comments/tickets go to dev@).
> 		https://issues.apache.org/jira/browse/TINKERPOP3-866
> 
> Thank you very much for your time,
> Marko.
> 
> http://markorodriguez.com
> 
>

Re: [DISCUSS] How to handle backwards compatibility with new GroupStep?

Posted by Marko Rodriguez <ok...@gmail.com>.
Hi Matt,

There are a few issues raised with the parallel versioned GraphTraversal models. Unfortunately, we now have our discussion split between dev@ and JIRA.
	https://issues.apache.org/jira/browse/TINKERPOP3-866

Thus, please read the discussion here and then add your email as a comment. I will then reply there.

OTHERS: If you care about this discussion and DO NOT have JIRA access, please comment using this dev@ thread and I will put your comment in JIRA and provide proper credit (though it will be my account the adds the comment).

Thanks,
Marko.

http://markorodriguez.com

On Oct 5, 2015, at 11:35 AM, Matt Frantz <ma...@gmail.com> wrote:

> Yes, test support will be a fair amount of work.  I assume we want to be
> able to test multiple versions of the DSL with the same test code, but that
> might be impossible to do without rewriting a lot of tests, since they
> refer to specific DSL interface types.  Since the various versions are not
> going to share an inheritance relationship, we need a form of structural
> typing or duck typing.  Yikes.
> 
> What about refactoring gremlin-test so that we maintain N copies of the
> tests for N different GraphTraversal versions?  When we freeze a version,
> we freeze the tests by copying them.
> 
> On Mon, Oct 5, 2015 at 8:08 AM, Stephen Mallette <sp...@gmail.com>
> wrote:
> 
>> [snip]
>> 
>> What are the implications for tests if we look to support this backward
>> compatibility (which, on the surface sounds good to me)?
>>

Re: [DISCUSS] How to handle backwards compatibility with new GroupStep?

Posted by Matt Frantz <ma...@gmail.com>.
Yes, test support will be a fair amount of work.  I assume we want to be
able to test multiple versions of the DSL with the same test code, but that
might be impossible to do without rewriting a lot of tests, since they
refer to specific DSL interface types.  Since the various versions are not
going to share an inheritance relationship, we need a form of structural
typing or duck typing.  Yikes.

What about refactoring gremlin-test so that we maintain N copies of the
tests for N different GraphTraversal versions?  When we freeze a version,
we freeze the tests by copying them.

On Mon, Oct 5, 2015 at 8:08 AM, Stephen Mallette <sp...@gmail.com>
wrote:

> [snip]
>
> What are the implications for tests if we look to support this backward
> compatibility (which, on the surface sounds good to me)?
>

Re: [DISCUSS] How to handle backwards compatibility with new GroupStep?

Posted by Stephen Mallette <sp...@gmail.com>.
The message serializer classes for Gremlin Server already have a notion
like this, so the pattern has already been explored a bit in that context
within the TinkerPop code base.

https://github.com/apache/incubator-tinkerpop/blob/809da22648067d178b114d9afbdeb5ffbf44ed9e/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ser/GraphSONMessageSerializerV1d0.java

Note that it uses V1d0 as a naming pattern, so to apply that to
GraphTraversal, you'd end up with GraphTraversalV3d0 (the assumption here
is that we would see breaks in the patch release so no need for an
additional "d0").

What are the implications for tests if we look to support this backward
compatibility (which, on the surface sounds good to me)?



On Mon, Oct 5, 2015 at 10:45 AM, Marko Rodriguez <ok...@gmail.com>
wrote:

> Hello R-Pop-Tik-e,
>
> Matt Frantz and I have been going back and forth on the following ticket:
>
>         https://issues.apache.org/jira/browse/TINKERPOP3-866
>
> This idea was spawned because Kuppitz and I were trying to figure out if
> we could get rid of GroupCountStep and replace it with a parameterized
> version of GroupStep.
>
> A short review of the argument is provided below (MENTORS: the full
> argument is in JIRA in the URL above (which also went to dev@ as JIRA
> comments/tickets email dev@)).
>         1. GroupStep (3.0.1) uses a collection prior to reduction.
>         2. This is inefficient as we waste memory holding data in a
> collection.
>         3. This gets especially nasty in OLAP when you can get some
> massive data growth.
>         4. Lets make GroupStep use lazy evaluators.
>         5. Done. Damn -- its not backwards compatible.
>         6. More sucky, we can't just have a @Deprecated method.
>
> Matt had a nice idea (MENTORS: again this idea is in JIRA). What about
> making GraphTraversal30 be a thing that always exists for each version.
> Thus, people who can't so easily upgrade to can still stick with
> GraphTraversal30.
>
> The question becomes, how is this expressed (MENTORS: This is lightly in
> JIRA but I bring it up here in more depth):
>
> g = graph.traversal(GraphTraversal30.build().create())
>                 OR
>
> gremlin.graph.traversal.default=org.apache.tinkerpop.process.traversal.dsl.graph30.GraphTraversal30
>
> Please reply to JIRA with your thoughts on how we can accomplish this
> (MENTORS: people will reply to JIRA with their thoughts, however,
> fortunately all JIRA comments/tickets go to dev@).
>                 https://issues.apache.org/jira/browse/TINKERPOP3-866
>
> Thank you very much for your time,
> Marko.
>
> http://markorodriguez.com
>
>