Support for osgi in tinkerpop

[Łukasz Dywicki <lu...@code-house.org>]

Hey all,
At the begining I would love to thank you all tinkerpop developers for your amazing work. I have found tinkerpop ecosystem couple months ago and I loved your logos since begining. I did not use tinkerpop initially but a month ago I started playing around with some basics. Yesterday I started deploying my application to Apache Karaf (which is my default "server", I am also commiter/PMC in this project) and did patch tinkerpop 2.7.0-SNAPSHOT to run smoothly in this environment.

This morning I had small discussion with Stephen about support for OSGi. Here I would like continue and explain what are basic steps to start supporting OSGi. I will also try to explain a bit why it is not worst idea on the globe. I suppose most of you have heard about OSGi here or there. Majority of people remembers extremes so each of you probably recorded at least one comment on this - love or hate. Till some degree it comes to same discussion like with Maven. One side says - it is so limited that making something non trivial is real pain. Another will tell you it is not allowing you to complicate build too much. Both sides are right till some degree. Since you know my root project in ASF you also know I can not be impassive/dispassionate/objective in such discussion. :-) 

Let’s start from defining levels of OSGI support:
1) OSGi metadata in manifest entries
2) Taking care about class loading 
3) Pluggable extensions

First steep is intended to add couple of headers into JAR - like Import-Package, Export-Package. First header is used to compute dependency graph for a module in runtime and allows to make sure that all requirements are satisfied. Second header declares what should be visible to other bundles - in other words what can be referenced in other module via Import-Package. In addition these headers might contain version information which provides an easy way to avoid common class loading problems such ClassNotFoundException or NoMethodDefError. Tinkerpop is ready to take this step since each JAR you produce have it’s own packaging root and you do not have split packages.  

Second step is bit more tricky but not that hard afterwards. Each class initialization should use appropriate class loader. In traditional environment such Tomcat/J2SE all JARs are collected into application classloader which consisting all entries from WEB-INF/lib or lib and shared around with everyone. Under OSGi this assumption is not valid any longer since every module can see only what it declared and have own classloader. Here framework is similar to Maven - what is not explicity declared as dependency is not visible. For example when tinkerpop-core will try to initialize class from package which is not declared in manifest entries (ie. Class.forName(com.acme.Graph) result will be ClassNotFoundException, even if com.acme.Graph is already exported by another module. To avoid troubles in most of cases it is sufficient to pass for Class.forName invocation also classloader which should be used. The same goes to serialization.

Third step is related to bringing extensions. As already described above - dynamic Class.forName constructions are not working by default under OSGi. Provisioning of extensions might be done in many different ways once library provides pluggable strategy for it. Usage of service loader is fine, however it is also bound to classloader which is used to locate META-INF entries. 

Now let’s take a further look on what are common problems with OSGi and how to avoid it. First of all OSGi forces clear separation of API and implementation packages. It’s not possible to have circular dependencies (another similarity to maven). Again Tinkerpop is ready since package org.apache.tinkerpop.gremlin.structure contains all what is needed to operate with most of graphs. It also contains IO package which could be succesfully split into tinkerpop-graphson, tinkerpop-gyro and optionally tinkerpop-graphml. Last separation is optional cause it is dependant only on JDK classes and does not bring any external libs into classpath.
I believe you might be a bit sceptic of ending up with hundred of JARs containing two classes. In case described above it is 100% right to devide since you are providing GraphWriter/Reader implementations based on 3rd party API which is not part of your public API itself - it’s integration. Also it might be benefitiary for your users once moving this dependencies out of core cause they won’t get dozen of unecessary dependencies. Someone might be interested in graphml but might be not interested in graphson at all. I suppose most of users are interested in one storage implementation at time.
Last thing which I would like to touch is configuration. For example now gremlin server depens on metrics which is fine, but also does depend on speciffic reportes which is not so good. This happens because the server module does not contain only a server related classes but also builds a default distribution which should be ready to run out of the box. After separing these two things running server under OSGi will be possible and extensible as well. After all op processors are interested only in reporting values, not in how they are being reported.

At very end of this mail - I would like also to state that I am able to help you with providing support for at least OSGi metadata, if you are interested. I am also open to continue this discussion to see if solving potential class loading issues is possible. I haven’t digg into tinkerpop 3 code except dependencies so I might miss some important bits and pieces thus assuming or making things simpler than they really are.

Kind regards,
Łukasz Dywicki
—
Apache Karaf commiter & PMC
Twitter: ldywicki
Blog: http://dywicki.pl
Code-House - http://code-house.org

Re: Support for osgi in tinkerpop

[James Carman <ja...@carmanconsulting.com>]

You have people offering to help and you may be able to get it "for free."
 Why not welcome the contributions?  That's called community building.
That's why you're in the incubator

On Thursday, March 26, 2015, Stephen Mallette <sp...@gmail.com> wrote:

> OSGi support is not high on the priority list for us right now.  It is
> something I think we are open to considering in the future depending on the
> impact it has to our project but it is not currently a roadmap item.
>
> On Wed, Mar 25, 2015 at 8:58 AM, James Carman <james@carmanconsulting.com
> <javascript:;>>
> wrote:
>
> > Many times, the default behavior of the bundle plugin is all that is
> > needed to get started.  It couldn't hurt to try that.  You can take a
> > look at a nice parent pom.xml file that does this for you:
> >
> > https://github.com/savoirtech/osgi-parent/blob/master/pom.xml
> >
> >
> > On Tue, Mar 24, 2015 at 6:55 PM, Łukasz Dywicki <luke@code-house.org
> <javascript:;>>
> > wrote:
> > > Hey Stephen,
> > > Thank you for your reply. I believe that making libraries supporting
> > OSGi directly is simply the best option in longer run and re-packaging
> just
> > to add manifest entries is a bit PITA. Repackager is forced to work on
> > every major and minor release and after all it is just double work.
> > >
> > > Since I am not a graph expert and not ready to make changes in
> tinkerpop
> > core I can support you in infrastructure tasks which are related to osgi
> > stuff. Sometimes there is a need for code relocations, maven maintenance,
> > version updates and such. Hopefully, I will be able to do that without
> > breaking your release plans.
> > >
> > > I think that now is a perfect time to make code relocations and tweaks
> > into SPI since API shape is close to finalize so very small troup of
> people
> > will suffer from changes if any. Some of these changes will not be
> possible
> > after delivering 3.0 and will be for sure delayed until 3.1 to don’t
> break
> > SPI compatibility and let companies have easier transition between 3.0
> and
> > 3.1 later on.
> > >
> > > Kind regards,
> > > Lukasz
> > >
> > >
> > >> Wiadomość napisana przez Stephen Mallette <spmallette@gmail.com
> <javascript:;>> w
> > dniu 24 mar 2015, o godz. 23:31:
> > >>
> > >> Hi Łukasz,
> > >>
> > >> Thanks for the detailed explanation about OSGi.  It was probably a
> good
> > >> introduction to some and a refresher to others.  I mentioned to Marko
> > that
> > >> if we were going to accept a pull request from someone to OSGi-enable
> > >> TinkerPop, it may as well come from a committer on Karaf.  :)  I think
> > that
> > >> most individuals who have brought up OSGi in the past have simply
> tried
> > to
> > >> throw us a pull request with some pom.xml changes in it for OSGi
> > metadata,
> > >> without any analysis done as to the other two levels of OSGi support
> > that
> > >> you outlined. It was helpful to have that additional information. I'm
> > sure
> > >> that we likely break that model in various places in our code base,
> so I
> > >> would expect that some changes would be required in addition to just
> > some
> > >> pom.xml entries.
> > >>
> > >> That said, I think that I have to agree with Matt that this is an
> issue
> > >> best left until after we get through an initial release or two.  Let's
> > >> revisit this issue when we get ourselves a bit more settled into
> Apache.
> > >>
> > >> Thanks again,
> > >>
> > >> Stephen
> > >>
> > >> On Tue, Mar 24, 2015 at 11:28 AM, Hadrian Zbarcea <hzbarcea@gmail.com
> <javascript:;>>
> > >> wrote:
> > >>
> > >>> Oh, absolutely. In case that wasn't clear :).
> > >>> Hadrian
> > >>>
> > >>>
> > >>> On 03/24/2015 11:05 AM, Matt Franklin wrote:
> > >>>
> > >>>> IMO, the community should focus on getting the release process down
> > first.
> > >>>> Could this be a feature added to a later milestone or 3.x point
> > release?
> > >>>>
> > >>>>
> > >>>>
> > >
> >
>

Re: Support for osgi in tinkerpop

[James Carman <ja...@carmanconsulting.com>]

By "free" I was referring to my previous comment that a lot of
projects can get away with merely pasting what I showed you into their
pom files and they're done.  The OSGification doesn't necessarily have
to modify any code.  It merely consists of some added entries in the
MANIFEST.MF file.  Doing so, can open up your project to a whole new
realm of possibilities from a deployment standpoint.  If that can get
you started, and you have people like Lukasz (and me, and Hadrian)
willing to help iron out any issues that might come up, then why not
do it?

As far as this particular conversation is concerned, this is all about
perception.  You have people offering to help out with TinkerPop and
you are seemingly being dismissive about it ("not on our roadmap").
That doesn't look good from a community-building standpoint.  The "we"
or "us" in TinkerPop is not merely the committers or PMC members.
It's the entire TinkerPop community and everyone should be able to
feel like they can provide input without being shooed away.

On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <sp...@gmail.com> wrote:
> First of all, apologies to Lukasz for hijacking his thread as it now takes
> an administrative turn away from OSGi discussion.
>
> James, I never said that we don't want help in this area - I only ask that
> we delay such things and discuss it further down the line.  I don't see
> anything as coming "for free" btw.  Every contribution comes with a cost
> (e.g. maintenance, complexity, etc).
>
> David, this is the general roadmap for 3.0:
>
> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversion/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel
>
> Marko posted it as such and requested feedback:
>
> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
>
> but i don't think that was what you were really getting at.  So that, plus
> James' response about community building, leads to a question that I
> thought I understood the answer to, but maybe i don't.  When someone makes
> a suggestion for some feature on this dev mailing list, offers a pull
> request, creates a jira issue, etc, what process does that suggestion need
> to go through?  Is there something official that we need to adhere to?  I
> assume there aren't hard and fast rules, and that this being one of the
> first external contribution offers of significance you guys are just being
> watchful in how it is handled.  However, if there are "rules", I'd like to
> know what they are.
>
> Thanks,
>
> Stephen
>
>
>
>
>
> On Thu, Mar 26, 2015 at 7:21 AM, David Nalley <da...@gnsa.us> wrote:
>
>> On Thu, Mar 26, 2015 at 6:31 AM, Stephen Mallette <sp...@gmail.com>
>> wrote:
>> > OSGi support is not high on the priority list for us right now.  It is
>> > something I think we are open to considering in the future depending on
>> the
>> > impact it has to our project but it is not currently a roadmap item.
>> >
>>
>> Who is the 'us' and where was this roadmap decided? Link?
>>

Re: Support for osgi in tinkerpop

[Matt Franklin <m....@gmail.com>]

On Fri, Mar 27, 2015 at 1:25 PM Hadrian Zbarcea <hz...@gmail.com> wrote:

> Luke,
>
> Leave AMQ out this. Totally different animals. As a mentor, I have very
> good vibes about the Tinkerpop project. State your case and keep the
> contributions coming.
>

Bit +1


>
> Cheers,
> Hadrian
>
> On 03/27/2015 12:09 PM, Łukasz Dywicki wrote:
> > Hey Matt & others,
> > please find my replies inline.
> >
> >
> >>
> >>> I didn’t want jump into topic before having anything in my hands, so
> here
> >>> is pull request:
> >>> https://github.com/apache/incubator-tinkerpop/pull/43 <
> >>> https://github.com/apache/incubator-tinkerpop/pull/43>
> >>>
> >>> And corresponding JIRA issue
> >>> https://issues.apache.org/jira/browse/TINKERPOP3-602 <
> >>> https://issues.apache.org/jira/browse/TINKERPOP3-602>
> >>> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
> >>> this flag 3 minutes after issue was created. It may be open for 3.1
> and so
> >>> on untill is fixed in 3.2 for example, as long as you are not ready to
> >>> accept pull request.
> >>>
> >>
> >> Unless I am missing something, it looks like this PR just adds the OSGi
> >> metadata for existing modules, correct?  This does not include the SPI
> >> changes you discuss below.
> > This PR does not change public API. SPI changes is big topic and I
> didn't have a good feeling coming out from discussion. I work on these
> things in my spare time for "fun” project so I don’t really want to spend
> lots of time on thing which will be put on the shelf for half year,
> especially if there is no clear statement from PMC. Yesterday I got my
> chance to cleanup maven setup a bit (https://github.com/Code-
> House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811
> <https://github.com/Code-House/incubator-tinkerpop/commit/
> 0f6ff034124fd83409f6b278d4f93b5fbbdfa811> - will submit pull request
> later).
> >
> >
> >
> >>> Now more concrete things. As every software project tinkerpop is also
> >>> focused on stability. My main concern is that if some actions will not
> be
> >>> taken before GA they will be extremely hard to push over later on.
> Roadmap
> >>> is clearly stating that general will is to have no major API changes
> for
> >>> 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid
> >>> extra work and efforts for migrations in future for project community.
> >>> Question if incubating project is ready to graduate if people see on
> >>> horizon code reorganization in it?
> >>>
> >>
> >> Apache incubation is about ensuring the community is healthy and able to
> >> self-govern under the Apache Way, not necessarily the code.
> > It is clear to me that community is 1st priority and code is 2nd, but
> one can not live without another. Relation between both is very strong.
> There are some examples of projects in Apache community which have been
> present in it for a long time but because of the way how PMC have worked
> they never established development community around project, except people
> coming from one firm. People from this could be promoted to become
> committers, other people didn't get contributions accepted or they were not
> supported to do such. For years people who was willing to use OSS release
> of ActiveMQ  were suffering from permanent lack of mainanance releases.
> Community in this case couldn’t do much with it but asking. Hopefuly AMQ
> will get straight some day.
> > Here I’m not saying tinkerpop is the same, just giving example how
> communities might not evolve if there is no will in project itself.
> >
> >
> >>> I raised a discussion of splitting some of IO packages out of core
> >>> interfaces. It’s wrong design decision and makes public API dependant
> on
> >>> concrete graph serialization techniques. This argument is based on
> >>> technical merit. OSGification just enforces this separation upfront.
> Almost
> >>> every project which starts with similar as tinkerpop approach ends up
> >>> moving this kind of dependencies out "core" soon or later. After all,
> can
> >>> you imagine all codecs embedded inside netty core?
> >>>
> >>
> >> Personally, my initial reaction to the Graph.Io interface is similar to
> >> yours.  However, I have not spent a lot of time in that area of the
> >> codebase, so I am reserving some judgement until I have more context.
> >>
> >> As you know, it is difficult to completely envision the effect of such a
> >> proposed change from just an e-mail.  Perhaps you could provide a
> working
> >> example (PR) of what you propose from an interface change that the
> >> community can evaluate.
> >>> Roadmap is in the hands of project PMC and nobody can influence it,
> >>> however there was already comment from Hadrian, who is tinkerpop mentor
> >>> saying:
> >>> "there is a large user bases running in OSGi who will greatly benefit
> from
> >>> Tinkerpop"
> >>>
> >>
> >> While the PMC is ultimately responsible for the success of the project,
> the
> >> entire community should feel like they can voice their opinion and drive
> >> the roadmap through contributions.
> >>
> >> This is not an oligarchy.  It is a meritocracy.
> >>
> >> Of course, this does not mean that every proposed change will be
> included
> >> in the code base, but individuals should not feel dismissed when
> presenting
> >> a rational argument.
> > Matt, cause of that I did not burn more power on SPI work - it makes
> sense only if idea gets acceptance cause as Stephen pointed earlier, this
> stuff is not PMC concern at this moment (but only mine and few other people
> asking before). It’s not on roadmap. There is no point to make
> forks/feature branches living too long cause they will generate huge effort
> to mainain.
> > How I am supposed to feel not dismissed, if such simple thing as extra
> manifest entries can’t get accepted? When issues gets "refused" before pull
> request is even reviewed and never been verified? Will it be easier with
> more serious things? I don't think so.
> >
> > Kind regards,
> > Lukasz
> > —
> > luke@code-house.org
> > Twitter: ldywicki
> > Blog: http://dywicki.pl
> > Code-House - http://code-house.org
> >
> >
> >
> >>
> >>>
> >>> According to mvnrepository.com <http://mvnrepository.com/> osgi.core
> JAR
> >>> is used in over 2000 places (slf4j-api in 9300) while tinkerpop
> artifacts
> >>> distributed among different groups have 30 usages at maximum. Lack of
> >>> support for it in modern libs when jigsaw is on horizon is rather
> uncommon.
> >>>
> >>> Kind regards,
> >>> Lukasz
> >>>
> >>>
> >>>
> >>>> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w
> dniu
> >>> 26 mar 2015, o godz. 13:39:
> >>>>
> >>>> Thanks James/David for the clarification. I was mostly trying to
> convey
> >>> my
> >>>> feeling that we should delay such work, but cast in your light I can
> see
> >>>> how else it could be read.   I'll be more conscious of word choice in
> the
> >>>> future.
> >>>>
> >>>> On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
> >>>>
> >>>>> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <
> spmallette@gmail.com
> >>>>
> >>>>> wrote:
> >>>>>> First of all, apologies to Lukasz for hijacking his thread as it now
> >>>>> takes
> >>>>>> an administrative turn away from OSGi discussion.
> >>>>>>
> >>>>>> James, I never said that we don't want help in this area - I only
> ask
> >>>>> that
> >>>>>> we delay such things and discuss it further down the line.  I don't
> see
> >>>>>> anything as coming "for free" btw.  Every contribution comes with a
> >>> cost
> >>>>>> (e.g. maintenance, complexity, etc).
> >>>>>>
> >>>>>> David, this is the general roadmap for 3.0:
> >>>>>>
> >>>>>>
> >>>>> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversi
> >>> on/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:
> >>> version-issues-panel
> >>>>>>
> >>>>>> Marko posted it as such and requested feedback:
> >>>>>>
> >>>>>>
> >>>>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop
> >>> -dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com
> %3E
> >>>>>>
> >>>>>> but i don't think that was what you were really getting at.
> >>>>>
> >>>>> Indeed :)
> >>>>>
> >>>>>> So that, plus
> >>>>>> James' response about community building, leads to a question that I
> >>>>>> thought I understood the answer to, but maybe i don't.  When someone
> >>>>> makes
> >>>>>> a suggestion for some feature on this dev mailing list, offers a
> pull
> >>>>>> request, creates a jira issue, etc, what process does that
> suggestion
> >>>>> need
> >>>>>> to go through?  Is there something official that we need to adhere
> >>> to?  I
> >>>>>> assume there aren't hard and fast rules, and that this being one of
> the
> >>>>>> first external contribution offers of significance you guys are just
> >>>>> being
> >>>>>> watchful in how it is handled.  However, if there are "rules", I'd
> like
> >>>>> to
> >>>>>> know what they are.
> >>>>>>
> >>>>>
> >>>>> So what process that goes through is really for the project to
> decide.
> >>>>> (And yes, that's a terrible answer to your question - but it's also
> >>>>> why you see folks piling up here; because it's easy to get wrong, and
> >>>>> it's very important).
> >>>>>
> >>>>> The things that raised the red flag for me in the conversation was:
> >>>>> * One person says it's not on the roadmap; but from tone and word
> >>>>> choice is sounds like a group decision. Specifically this makes me
> >>>>> wonder if decision making is happening somewhere other than the list.
> >>>>> * No link to the roadmap (Lukasz may or may not be following closely
> >>>>> enough to have seen that; so roadmap may be elusive)
> >>>>>
> >>>>> There's also the issue that development doesn't have to be linear,
> >>>>> someone could work in a feature branch, submit a pull request,
> >>>>> assuming they had proper expectations. Dismissing new folks with 'not
> >>>>> on our roadmap', with seemingly no alternative doesn't help grow your
> >>>>> community.
> >>>>>
> >>>>> --David
> >>>>>
> >>>
> >>>
> >
> >
>

Re: Support for osgi in tinkerpop

[Hadrian Zbarcea <hz...@gmail.com>]

Luke,

Leave AMQ out this. Totally different animals. As a mentor, I have very 
good vibes about the Tinkerpop project. State your case and keep the 
contributions coming.

Cheers,
Hadrian

On 03/27/2015 12:09 PM, Łukasz Dywicki wrote:
> Hey Matt & others,
> please find my replies inline.
>
>
>>
>>> I didn’t want jump into topic before having anything in my hands, so here
>>> is pull request:
>>> https://github.com/apache/incubator-tinkerpop/pull/43 <
>>> https://github.com/apache/incubator-tinkerpop/pull/43>
>>>
>>> And corresponding JIRA issue
>>> https://issues.apache.org/jira/browse/TINKERPOP3-602 <
>>> https://issues.apache.org/jira/browse/TINKERPOP3-602>
>>> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
>>> this flag 3 minutes after issue was created. It may be open for 3.1 and so
>>> on untill is fixed in 3.2 for example, as long as you are not ready to
>>> accept pull request.
>>>
>>
>> Unless I am missing something, it looks like this PR just adds the OSGi
>> metadata for existing modules, correct?  This does not include the SPI
>> changes you discuss below.
> This PR does not change public API. SPI changes is big topic and I didn't have a good feeling coming out from discussion. I work on these things in my spare time for "fun” project so I don’t really want to spend lots of time on thing which will be put on the shelf for half year, especially if there is no clear statement from PMC. Yesterday I got my chance to cleanup maven setup a bit (https://github.com/Code-House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811 <https://github.com/Code-House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811> - will submit pull request later).
>
>
>
>>> Now more concrete things. As every software project tinkerpop is also
>>> focused on stability. My main concern is that if some actions will not be
>>> taken before GA they will be extremely hard to push over later on. Roadmap
>>> is clearly stating that general will is to have no major API changes for
>>> 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid
>>> extra work and efforts for migrations in future for project community.
>>> Question if incubating project is ready to graduate if people see on
>>> horizon code reorganization in it?
>>>
>>
>> Apache incubation is about ensuring the community is healthy and able to
>> self-govern under the Apache Way, not necessarily the code.
> It is clear to me that community is 1st priority and code is 2nd, but one can not live without another. Relation between both is very strong. There are some examples of projects in Apache community which have been present in it for a long time but because of the way how PMC have worked they never established development community around project, except people coming from one firm. People from this could be promoted to become committers, other people didn't get contributions accepted or they were not supported to do such. For years people who was willing to use OSS release of ActiveMQ  were suffering from permanent lack of mainanance releases. Community in this case couldn’t do much with it but asking. Hopefuly AMQ will get straight some day.
> Here I’m not saying tinkerpop is the same, just giving example how communities might not evolve if there is no will in project itself.
>
>
>>> I raised a discussion of splitting some of IO packages out of core
>>> interfaces. It’s wrong design decision and makes public API dependant on
>>> concrete graph serialization techniques. This argument is based on
>>> technical merit. OSGification just enforces this separation upfront. Almost
>>> every project which starts with similar as tinkerpop approach ends up
>>> moving this kind of dependencies out "core" soon or later. After all, can
>>> you imagine all codecs embedded inside netty core?
>>>
>>
>> Personally, my initial reaction to the Graph.Io interface is similar to
>> yours.  However, I have not spent a lot of time in that area of the
>> codebase, so I am reserving some judgement until I have more context.
>>
>> As you know, it is difficult to completely envision the effect of such a
>> proposed change from just an e-mail.  Perhaps you could provide a working
>> example (PR) of what you propose from an interface change that the
>> community can evaluate.
>>> Roadmap is in the hands of project PMC and nobody can influence it,
>>> however there was already comment from Hadrian, who is tinkerpop mentor
>>> saying:
>>> "there is a large user bases running in OSGi who will greatly benefit from
>>> Tinkerpop"
>>>
>>
>> While the PMC is ultimately responsible for the success of the project, the
>> entire community should feel like they can voice their opinion and drive
>> the roadmap through contributions.
>>
>> This is not an oligarchy.  It is a meritocracy.
>>
>> Of course, this does not mean that every proposed change will be included
>> in the code base, but individuals should not feel dismissed when presenting
>> a rational argument.
> Matt, cause of that I did not burn more power on SPI work - it makes sense only if idea gets acceptance cause as Stephen pointed earlier, this stuff is not PMC concern at this moment (but only mine and few other people asking before). It’s not on roadmap. There is no point to make forks/feature branches living too long cause they will generate huge effort to mainain.
> How I am supposed to feel not dismissed, if such simple thing as extra manifest entries can’t get accepted? When issues gets "refused" before pull request is even reviewed and never been verified? Will it be easier with more serious things? I don't think so.
>
> Kind regards,
> Lukasz
> —
> luke@code-house.org
> Twitter: ldywicki
> Blog: http://dywicki.pl
> Code-House - http://code-house.org
>
>
>
>>
>>>
>>> According to mvnrepository.com <http://mvnrepository.com/> osgi.core JAR
>>> is used in over 2000 places (slf4j-api in 9300) while tinkerpop artifacts
>>> distributed among different groups have 30 usages at maximum. Lack of
>>> support for it in modern libs when jigsaw is on horizon is rather uncommon.
>>>
>>> Kind regards,
>>> Lukasz
>>>
>>>
>>>
>>>> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w dniu
>>> 26 mar 2015, o godz. 13:39:
>>>>
>>>> Thanks James/David for the clarification. I was mostly trying to convey
>>> my
>>>> feeling that we should delay such work, but cast in your light I can see
>>>> how else it could be read.   I'll be more conscious of word choice in the
>>>> future.
>>>>
>>>> On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
>>>>
>>>>> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <spmallette@gmail.com
>>>>
>>>>> wrote:
>>>>>> First of all, apologies to Lukasz for hijacking his thread as it now
>>>>> takes
>>>>>> an administrative turn away from OSGi discussion.
>>>>>>
>>>>>> James, I never said that we don't want help in this area - I only ask
>>>>> that
>>>>>> we delay such things and discuss it further down the line.  I don't see
>>>>>> anything as coming "for free" btw.  Every contribution comes with a
>>> cost
>>>>>> (e.g. maintenance, complexity, etc).
>>>>>>
>>>>>> David, this is the general roadmap for 3.0:
>>>>>>
>>>>>>
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversi
>>> on/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:
>>> version-issues-panel
>>>>>>
>>>>>> Marko posted it as such and requested feedback:
>>>>>>
>>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop
>>> -dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
>>>>>>
>>>>>> but i don't think that was what you were really getting at.
>>>>>
>>>>> Indeed :)
>>>>>
>>>>>> So that, plus
>>>>>> James' response about community building, leads to a question that I
>>>>>> thought I understood the answer to, but maybe i don't.  When someone
>>>>> makes
>>>>>> a suggestion for some feature on this dev mailing list, offers a pull
>>>>>> request, creates a jira issue, etc, what process does that suggestion
>>>>> need
>>>>>> to go through?  Is there something official that we need to adhere
>>> to?  I
>>>>>> assume there aren't hard and fast rules, and that this being one of the
>>>>>> first external contribution offers of significance you guys are just
>>>>> being
>>>>>> watchful in how it is handled.  However, if there are "rules", I'd like
>>>>> to
>>>>>> know what they are.
>>>>>>
>>>>>
>>>>> So what process that goes through is really for the project to decide.
>>>>> (And yes, that's a terrible answer to your question - but it's also
>>>>> why you see folks piling up here; because it's easy to get wrong, and
>>>>> it's very important).
>>>>>
>>>>> The things that raised the red flag for me in the conversation was:
>>>>> * One person says it's not on the roadmap; but from tone and word
>>>>> choice is sounds like a group decision. Specifically this makes me
>>>>> wonder if decision making is happening somewhere other than the list.
>>>>> * No link to the roadmap (Lukasz may or may not be following closely
>>>>> enough to have seen that; so roadmap may be elusive)
>>>>>
>>>>> There's also the issue that development doesn't have to be linear,
>>>>> someone could work in a feature branch, submit a pull request,
>>>>> assuming they had proper expectations. Dismissing new folks with 'not
>>>>> on our roadmap', with seemingly no alternative doesn't help grow your
>>>>> community.
>>>>>
>>>>> --David
>>>>>
>>>
>>>
>
>

Re: Support for osgi in tinkerpop

[James Carman <ja...@carmanconsulting.com>]

On Mon, Mar 30, 2015 at 10:20 AM, Stephen Mallette <sp...@gmail.com> wrote:
>
> As for the Io interface, I'll restate some of what I said before on this
> topic and add a bit more.  I understand that the approach defies a common
> implementation pattern and reduces some flexibility as it pertains to
> including additional IO implementations in the future.  From what I know of
> real-world usage of TinkerPop, the requirement for additional IO
> implementations beyond the three we have is unlikely, so sacrifice of that
> flexibility in exchange for a straightforward API, direct implementation
> for vendors, etc. is a worthwhile trade-off.  If I thought that TinkerPop
> needed future support for tons of codecs (tulip, gml, gexf, etc), then this
> approach would not be so good - no doubt - but imo that is not a great
> direction because it simply represents more code to manage that doesn't
> have a lot of value.  The notion of lots of codecs also worries me as it
> will further complicate vendor implementations - they have enough to worry
> about with three IO implementations that they have to get their graphs to
> work with.  Adding more IO stuffs, that don't have real value to users,
> seems like an undue burden discouraging more Graph implementations which
> are far more valuable to the community than support for more IO.
>

Remember, that we're trying to grow this TinkerPop community.  By
providing that "hook", it makes it easier for outside folks to
contribute other approaches without having to modify the "core."
Otherwise, it's more likely that their contribution would be rejected
because we wouldn't want to add in a new dependency.

Re: Support for osgi in tinkerpop

[Stephen Mallette <sp...@gmail.com>]

> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
this flag 3 minutes after issue was created.

Not a big deal and didn't mean anything by it.  I've been thinking about
getting some basic conventions organized in Jira as we've just started
using it.  I had it in my mind that since we don't have an official release
under apache, that there is no specific version an issue can really
affect.  As such, it made sense in my mind that all issues should remain
blanked for that field until we reach M8 at which point open issues as of
that time will flip to that version in that field.  I've been keeping notes
on this in the project README:

https://github.com/apache/incubator-tinkerpop#issue-tracker-conventions

and expected to solicit some feedback once we'd played with jira a bit
longer.

As for the Io interface, I'll restate some of what I said before on this
topic and add a bit more.  I understand that the approach defies a common
implementation pattern and reduces some flexibility as it pertains to
including additional IO implementations in the future.  From what I know of
real-world usage of TinkerPop, the requirement for additional IO
implementations beyond the three we have is unlikely, so sacrifice of that
flexibility in exchange for a straightforward API, direct implementation
for vendors, etc. is a worthwhile trade-off.  If I thought that TinkerPop
needed future support for tons of codecs (tulip, gml, gexf, etc), then this
approach would not be so good - no doubt - but imo that is not a great
direction because it simply represents more code to manage that doesn't
have a lot of value.  The notion of lots of codecs also worries me as it
will further complicate vendor implementations - they have enough to worry
about with three IO implementations that they have to get their graphs to
work with.  Adding more IO stuffs, that don't have real value to users,
seems like an undue burden discouraging more Graph implementations which
are far more valuable to the community than support for more IO.

There are some more details here in this issue where Graph.Io() was first
conceived:

https://issues.apache.org/jira/browse/TINKERPOP3-376

It outlines some of the problems we hoped to address, but not all (might be
some other issues in jira, discussions in gremlin-users, etc.).
Anyway, Lukasz,
you had indicated in this post:

http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C3D4AE343-2B49-409B-A724-1BF4F81FF404%40code-house.org%3E

that the design you had in mind would make vendor implementations "easier,
not harder".  A working solution that incorporated flexibility and better
decoupling without compromise to current capabilities and what we have
working now in the vendor test-suite, hadoop, gremlin server/driver, spark,
etc. would be compelling, which leads to matt's question:

> I think the right question to ask to the Tinkerpop community should be,
if a fully working solution is presented, would the community consider
including it in 3.0.0 GA?

Given Marko's proposal for getting to GA:

http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E

feedback for core interfaces is expected over the course of M8 so I would
say that the answer to that question is "yes" assuming that the solution is
considerate of the context supplied above.


On Fri, Mar 27, 2015 at 8:15 PM, Matt Franklin <m....@gmail.com>
wrote:

> On Fri, Mar 27, 2015 at 12:10 PM Łukasz Dywicki <lu...@code-house.org>
> wrote:
>
> > Hey Matt & others,
> > please find my replies inline.
> >
> >
> > >
> > >> I didn’t want jump into topic before having anything in my hands, so
> > here
> > >> is pull request:
> > >> https://github.com/apache/incubator-tinkerpop/pull/43 <
> > >> https://github.com/apache/incubator-tinkerpop/pull/43>
> > >>
> > >> And corresponding JIRA issue
> > >> https://issues.apache.org/jira/browse/TINKERPOP3-602 <
> > >> https://issues.apache.org/jira/browse/TINKERPOP3-602>
> > >> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
> > >> this flag 3 minutes after issue was created. It may be open for 3.1
> and
> > so
> > >> on untill is fixed in 3.2 for example, as long as you are not ready to
> > >> accept pull request.
> > >>
> > >
> > > Unless I am missing something, it looks like this PR just adds the OSGi
> > > metadata for existing modules, correct?  This does not include the SPI
> > > changes you discuss below.
> > This PR does not change public API. SPI changes is big topic and I didn't
> > have a good feeling coming out from discussion. I work on these things in
> > my spare time for "fun” project so I don’t really want to spend lots of
> > time on thing which will be put on the shelf for half year, especially if
> > there is no clear statement from PMC. Yesterday I got my chance to
> cleanup
> > maven setup a bit (https://github.com/Code-House/incubator-tinkerpop/
> > commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811 <
> https://github.com/Code-
> >
> House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811>
> > - will submit pull request later).
> >
> >
> >
> > >> Now more concrete things. As every software project tinkerpop is also
> > >> focused on stability. My main concern is that if some actions will not
> > be
> > >> taken before GA they will be extremely hard to push over later on.
> > Roadmap
> > >> is clearly stating that general will is to have no major API changes
> for
> > >> 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid
> > >> extra work and efforts for migrations in future for project community.
> > >> Question if incubating project is ready to graduate if people see on
> > >> horizon code reorganization in it?
> > >>
> > >
> > > Apache incubation is about ensuring the community is healthy and able
> to
> > > self-govern under the Apache Way, not necessarily the code.
> > It is clear to me that community is 1st priority and code is 2nd, but one
> > can not live without another. Relation between both is very strong. There
> > are some examples of projects in Apache community which have been present
> > in it for a long time but because of the way how PMC have worked they
> never
> > established development community around project, except people coming
> from
> > one firm. People from this could be promoted to become committers, other
> > people didn't get contributions accepted or they were not supported to do
> > such. For years people who was willing to use OSS release of ActiveMQ
> were
> > suffering from permanent lack of mainanance releases. Community in this
> > case couldn’t do much with it but asking. Hopefuly AMQ will get straight
> > some day.
> > Here I’m not saying tinkerpop is the same, just giving example how
> > communities might not evolve if there is no will in project itself.
> >
> >
> > >> I raised a discussion of splitting some of IO packages out of core
> > >> interfaces. It’s wrong design decision and makes public API dependant
> on
> > >> concrete graph serialization techniques. This argument is based on
> > >> technical merit. OSGification just enforces this separation upfront.
> > Almost
> > >> every project which starts with similar as tinkerpop approach ends up
> > >> moving this kind of dependencies out "core" soon or later. After all,
> > can
> > >> you imagine all codecs embedded inside netty core?
> > >>
> > >
> > > Personally, my initial reaction to the Graph.Io interface is similar to
> > > yours.  However, I have not spent a lot of time in that area of the
> > > codebase, so I am reserving some judgement until I have more context.
> > >
> > > As you know, it is difficult to completely envision the effect of such
> a
> > > proposed change from just an e-mail.  Perhaps you could provide a
> working
> > > example (PR) of what you propose from an interface change that the
> > > community can evaluate.
> > >> Roadmap is in the hands of project PMC and nobody can influence it,
> > >> however there was already comment from Hadrian, who is tinkerpop
> mentor
> > >> saying:
> > >> "there is a large user bases running in OSGi who will greatly benefit
> > from
> > >> Tinkerpop"
> > >>
> > >
> > > While the PMC is ultimately responsible for the success of the project,
> > the
> > > entire community should feel like they can voice their opinion and
> drive
> > > the roadmap through contributions.
> > >
> > > This is not an oligarchy.  It is a meritocracy.
> > >
> > > Of course, this does not mean that every proposed change will be
> included
> > > in the code base, but individuals should not feel dismissed when
> > presenting
> > > a rational argument.
> > Matt, cause of that I did not burn more power on SPI work - it makes
> sense
> > only if idea gets acceptance cause as Stephen pointed earlier, this stuff
> > is not PMC concern at this moment (but only mine and few other people
> > asking before). It’s not on roadmap. There is no point to make
> > forks/feature branches living too long cause they will generate huge
> effort
> > to mainain.
> >
>
> Clearly, the community does not benefit long-running branches that go
> unmerged.  However, I think you will recognize that while there is a bad
> design pattern in place, it likely emerged for a reason.  I assume the
> resistance you feel comes from the fact that the current solution is
> "working" for all current parties.  Until an implementation is presented
> that demonstrates a superior and fully integrated pattern, with no
> regression issues, I think the resistance is somewhat natural.
>
> Rather than get bogged down in bikeshedding about a design, I think the
> right question to ask to the Tinkerpop community should be, if a fully
> working solution is presented, would the community consider including it in
> 3.0.0 GA?
>
> For my personal perspective, not speaking as a mentor, if you built
> something that was more loosely coupled, but did not increase the pain of
> working with the IO layer, I would like to see it integrated into 3.0.0GA.
> I have my reservations that the activity won't snowball though.
>
>
> > How I am supposed to feel not dismissed, if such simple thing as extra
> > manifest entries can’t get accepted? When issues gets "refused" before
> pull
> > request is even reviewed and never been verified? Will it be easier with
> > more serious things? I don't think so.
>
>
> I understand why you might feel this way.  However, I see that the ticket
> is still open and so is the pull request.  My guess is the version was
> changed without realizing that the corresponding pull request has no code
> changes, just pom modifications to support OSGi metadata creation; but, I
> will let Stephen voice his own reasons for changing the version.
>
>
> >
>
>
> > Kind regards,
> > Lukasz
> > —
> > luke@code-house.org
> > Twitter: ldywicki
> > Blog: http://dywicki.pl
> > Code-House - http://code-house.org
> >
> >
> >
> > >
> > >>
> > >> According to mvnrepository.com <http://mvnrepository.com/> osgi.core
> > JAR
> > >> is used in over 2000 places (slf4j-api in 9300) while tinkerpop
> > artifacts
> > >> distributed among different groups have 30 usages at maximum. Lack of
> > >> support for it in modern libs when jigsaw is on horizon is rather
> > uncommon.
> > >>
> > >> Kind regards,
> > >> Lukasz
> > >>
> > >>
> > >>
> > >>> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w
> > dniu
> > >> 26 mar 2015, o godz. 13:39:
> > >>>
> > >>> Thanks James/David for the clarification. I was mostly trying to
> convey
> > >> my
> > >>> feeling that we should delay such work, but cast in your light I can
> > see
> > >>> how else it could be read.   I'll be more conscious of word choice in
> > the
> > >>> future.
> > >>>
> > >>> On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
> > >>>
> > >>>> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <
> > spmallette@gmail.com
> > >>>
> > >>>> wrote:
> > >>>>> First of all, apologies to Lukasz for hijacking his thread as it
> now
> > >>>> takes
> > >>>>> an administrative turn away from OSGi discussion.
> > >>>>>
> > >>>>> James, I never said that we don't want help in this area - I only
> ask
> > >>>> that
> > >>>>> we delay such things and discuss it further down the line.  I don't
> > see
> > >>>>> anything as coming "for free" btw.  Every contribution comes with a
> > >> cost
> > >>>>> (e.g. maintenance, complexity, etc).
> > >>>>>
> > >>>>> David, this is the general roadmap for 3.0:
> > >>>>>
> > >>>>>
> > >>>> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversi
> > >> on/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:
> > >> version-issues-panel
> > >>>>>
> > >>>>> Marko posted it as such and requested feedback:
> > >>>>>
> > >>>>>
> > >>>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop
> > >> -dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com
> %3E
> > >>>>>
> > >>>>> but i don't think that was what you were really getting at.
> > >>>>
> > >>>> Indeed :)
> > >>>>
> > >>>>> So that, plus
> > >>>>> James' response about community building, leads to a question that
> I
> > >>>>> thought I understood the answer to, but maybe i don't.  When
> someone
> > >>>> makes
> > >>>>> a suggestion for some feature on this dev mailing list, offers a
> pull
> > >>>>> request, creates a jira issue, etc, what process does that
> suggestion
> > >>>> need
> > >>>>> to go through?  Is there something official that we need to adhere
> > >> to?  I
> > >>>>> assume there aren't hard and fast rules, and that this being one of
> > the
> > >>>>> first external contribution offers of significance you guys are
> just
> > >>>> being
> > >>>>> watchful in how it is handled.  However, if there are "rules", I'd
> > like
> > >>>> to
> > >>>>> know what they are.
> > >>>>>
> > >>>>
> > >>>> So what process that goes through is really for the project to
> decide.
> > >>>> (And yes, that's a terrible answer to your question - but it's also
> > >>>> why you see folks piling up here; because it's easy to get wrong,
> and
> > >>>> it's very important).
> > >>>>
> > >>>> The things that raised the red flag for me in the conversation was:
> > >>>> * One person says it's not on the roadmap; but from tone and word
> > >>>> choice is sounds like a group decision. Specifically this makes me
> > >>>> wonder if decision making is happening somewhere other than the
> list.
> > >>>> * No link to the roadmap (Lukasz may or may not be following closely
> > >>>> enough to have seen that; so roadmap may be elusive)
> > >>>>
> > >>>> There's also the issue that development doesn't have to be linear,
> > >>>> someone could work in a feature branch, submit a pull request,
> > >>>> assuming they had proper expectations. Dismissing new folks with
> 'not
> > >>>> on our roadmap', with seemingly no alternative doesn't help grow
> your
> > >>>> community.
> > >>>>
> > >>>> --David
> > >>>>
> > >>
> > >>
> >
> >
>

Re: Support for osgi in tinkerpop

[Matt Franklin <m....@gmail.com>]

On Fri, Mar 27, 2015 at 12:10 PM Łukasz Dywicki <lu...@code-house.org> wrote:

> Hey Matt & others,
> please find my replies inline.
>
>
> >
> >> I didn’t want jump into topic before having anything in my hands, so
> here
> >> is pull request:
> >> https://github.com/apache/incubator-tinkerpop/pull/43 <
> >> https://github.com/apache/incubator-tinkerpop/pull/43>
> >>
> >> And corresponding JIRA issue
> >> https://issues.apache.org/jira/browse/TINKERPOP3-602 <
> >> https://issues.apache.org/jira/browse/TINKERPOP3-602>
> >> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
> >> this flag 3 minutes after issue was created. It may be open for 3.1 and
> so
> >> on untill is fixed in 3.2 for example, as long as you are not ready to
> >> accept pull request.
> >>
> >
> > Unless I am missing something, it looks like this PR just adds the OSGi
> > metadata for existing modules, correct?  This does not include the SPI
> > changes you discuss below.
> This PR does not change public API. SPI changes is big topic and I didn't
> have a good feeling coming out from discussion. I work on these things in
> my spare time for "fun” project so I don’t really want to spend lots of
> time on thing which will be put on the shelf for half year, especially if
> there is no clear statement from PMC. Yesterday I got my chance to cleanup
> maven setup a bit (https://github.com/Code-House/incubator-tinkerpop/
> commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811 <https://github.com/Code-
> House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811>
> - will submit pull request later).
>
>
>
> >> Now more concrete things. As every software project tinkerpop is also
> >> focused on stability. My main concern is that if some actions will not
> be
> >> taken before GA they will be extremely hard to push over later on.
> Roadmap
> >> is clearly stating that general will is to have no major API changes for
> >> 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid
> >> extra work and efforts for migrations in future for project community.
> >> Question if incubating project is ready to graduate if people see on
> >> horizon code reorganization in it?
> >>
> >
> > Apache incubation is about ensuring the community is healthy and able to
> > self-govern under the Apache Way, not necessarily the code.
> It is clear to me that community is 1st priority and code is 2nd, but one
> can not live without another. Relation between both is very strong. There
> are some examples of projects in Apache community which have been present
> in it for a long time but because of the way how PMC have worked they never
> established development community around project, except people coming from
> one firm. People from this could be promoted to become committers, other
> people didn't get contributions accepted or they were not supported to do
> such. For years people who was willing to use OSS release of ActiveMQ  were
> suffering from permanent lack of mainanance releases. Community in this
> case couldn’t do much with it but asking. Hopefuly AMQ will get straight
> some day.
> Here I’m not saying tinkerpop is the same, just giving example how
> communities might not evolve if there is no will in project itself.
>
>
> >> I raised a discussion of splitting some of IO packages out of core
> >> interfaces. It’s wrong design decision and makes public API dependant on
> >> concrete graph serialization techniques. This argument is based on
> >> technical merit. OSGification just enforces this separation upfront.
> Almost
> >> every project which starts with similar as tinkerpop approach ends up
> >> moving this kind of dependencies out "core" soon or later. After all,
> can
> >> you imagine all codecs embedded inside netty core?
> >>
> >
> > Personally, my initial reaction to the Graph.Io interface is similar to
> > yours.  However, I have not spent a lot of time in that area of the
> > codebase, so I am reserving some judgement until I have more context.
> >
> > As you know, it is difficult to completely envision the effect of such a
> > proposed change from just an e-mail.  Perhaps you could provide a working
> > example (PR) of what you propose from an interface change that the
> > community can evaluate.
> >> Roadmap is in the hands of project PMC and nobody can influence it,
> >> however there was already comment from Hadrian, who is tinkerpop mentor
> >> saying:
> >> "there is a large user bases running in OSGi who will greatly benefit
> from
> >> Tinkerpop"
> >>
> >
> > While the PMC is ultimately responsible for the success of the project,
> the
> > entire community should feel like they can voice their opinion and drive
> > the roadmap through contributions.
> >
> > This is not an oligarchy.  It is a meritocracy.
> >
> > Of course, this does not mean that every proposed change will be included
> > in the code base, but individuals should not feel dismissed when
> presenting
> > a rational argument.
> Matt, cause of that I did not burn more power on SPI work - it makes sense
> only if idea gets acceptance cause as Stephen pointed earlier, this stuff
> is not PMC concern at this moment (but only mine and few other people
> asking before). It’s not on roadmap. There is no point to make
> forks/feature branches living too long cause they will generate huge effort
> to mainain.
>

Clearly, the community does not benefit long-running branches that go
unmerged.  However, I think you will recognize that while there is a bad
design pattern in place, it likely emerged for a reason.  I assume the
resistance you feel comes from the fact that the current solution is
"working" for all current parties.  Until an implementation is presented
that demonstrates a superior and fully integrated pattern, with no
regression issues, I think the resistance is somewhat natural.

Rather than get bogged down in bikeshedding about a design, I think the
right question to ask to the Tinkerpop community should be, if a fully
working solution is presented, would the community consider including it in
3.0.0 GA?

For my personal perspective, not speaking as a mentor, if you built
something that was more loosely coupled, but did not increase the pain of
working with the IO layer, I would like to see it integrated into 3.0.0GA.
I have my reservations that the activity won't snowball though.


> How I am supposed to feel not dismissed, if such simple thing as extra
> manifest entries can’t get accepted? When issues gets "refused" before pull
> request is even reviewed and never been verified? Will it be easier with
> more serious things? I don't think so.


I understand why you might feel this way.  However, I see that the ticket
is still open and so is the pull request.  My guess is the version was
changed without realizing that the corresponding pull request has no code
changes, just pom modifications to support OSGi metadata creation; but, I
will let Stephen voice his own reasons for changing the version.


>


> Kind regards,
> Lukasz
> —
> luke@code-house.org
> Twitter: ldywicki
> Blog: http://dywicki.pl
> Code-House - http://code-house.org
>
>
>
> >
> >>
> >> According to mvnrepository.com <http://mvnrepository.com/> osgi.core
> JAR
> >> is used in over 2000 places (slf4j-api in 9300) while tinkerpop
> artifacts
> >> distributed among different groups have 30 usages at maximum. Lack of
> >> support for it in modern libs when jigsaw is on horizon is rather
> uncommon.
> >>
> >> Kind regards,
> >> Lukasz
> >>
> >>
> >>
> >>> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w
> dniu
> >> 26 mar 2015, o godz. 13:39:
> >>>
> >>> Thanks James/David for the clarification. I was mostly trying to convey
> >> my
> >>> feeling that we should delay such work, but cast in your light I can
> see
> >>> how else it could be read.   I'll be more conscious of word choice in
> the
> >>> future.
> >>>
> >>> On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
> >>>
> >>>> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <
> spmallette@gmail.com
> >>>
> >>>> wrote:
> >>>>> First of all, apologies to Lukasz for hijacking his thread as it now
> >>>> takes
> >>>>> an administrative turn away from OSGi discussion.
> >>>>>
> >>>>> James, I never said that we don't want help in this area - I only ask
> >>>> that
> >>>>> we delay such things and discuss it further down the line.  I don't
> see
> >>>>> anything as coming "for free" btw.  Every contribution comes with a
> >> cost
> >>>>> (e.g. maintenance, complexity, etc).
> >>>>>
> >>>>> David, this is the general roadmap for 3.0:
> >>>>>
> >>>>>
> >>>> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversi
> >> on/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:
> >> version-issues-panel
> >>>>>
> >>>>> Marko posted it as such and requested feedback:
> >>>>>
> >>>>>
> >>>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop
> >> -dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
> >>>>>
> >>>>> but i don't think that was what you were really getting at.
> >>>>
> >>>> Indeed :)
> >>>>
> >>>>> So that, plus
> >>>>> James' response about community building, leads to a question that I
> >>>>> thought I understood the answer to, but maybe i don't.  When someone
> >>>> makes
> >>>>> a suggestion for some feature on this dev mailing list, offers a pull
> >>>>> request, creates a jira issue, etc, what process does that suggestion
> >>>> need
> >>>>> to go through?  Is there something official that we need to adhere
> >> to?  I
> >>>>> assume there aren't hard and fast rules, and that this being one of
> the
> >>>>> first external contribution offers of significance you guys are just
> >>>> being
> >>>>> watchful in how it is handled.  However, if there are "rules", I'd
> like
> >>>> to
> >>>>> know what they are.
> >>>>>
> >>>>
> >>>> So what process that goes through is really for the project to decide.
> >>>> (And yes, that's a terrible answer to your question - but it's also
> >>>> why you see folks piling up here; because it's easy to get wrong, and
> >>>> it's very important).
> >>>>
> >>>> The things that raised the red flag for me in the conversation was:
> >>>> * One person says it's not on the roadmap; but from tone and word
> >>>> choice is sounds like a group decision. Specifically this makes me
> >>>> wonder if decision making is happening somewhere other than the list.
> >>>> * No link to the roadmap (Lukasz may or may not be following closely
> >>>> enough to have seen that; so roadmap may be elusive)
> >>>>
> >>>> There's also the issue that development doesn't have to be linear,
> >>>> someone could work in a feature branch, submit a pull request,
> >>>> assuming they had proper expectations. Dismissing new folks with 'not
> >>>> on our roadmap', with seemingly no alternative doesn't help grow your
> >>>> community.
> >>>>
> >>>> --David
> >>>>
> >>
> >>
>
>

Re: Support for osgi in tinkerpop

[Łukasz Dywicki <lu...@code-house.org>]

Hey Matt & others,
please find my replies inline.


> 
>> I didn’t want jump into topic before having anything in my hands, so here
>> is pull request:
>> https://github.com/apache/incubator-tinkerpop/pull/43 <
>> https://github.com/apache/incubator-tinkerpop/pull/43>
>> 
>> And corresponding JIRA issue
>> https://issues.apache.org/jira/browse/TINKERPOP3-602 <
>> https://issues.apache.org/jira/browse/TINKERPOP3-602>
>> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
>> this flag 3 minutes after issue was created. It may be open for 3.1 and so
>> on untill is fixed in 3.2 for example, as long as you are not ready to
>> accept pull request.
>> 
> 
> Unless I am missing something, it looks like this PR just adds the OSGi
> metadata for existing modules, correct?  This does not include the SPI
> changes you discuss below.
This PR does not change public API. SPI changes is big topic and I didn't have a good feeling coming out from discussion. I work on these things in my spare time for "fun” project so I don’t really want to spend lots of time on thing which will be put on the shelf for half year, especially if there is no clear statement from PMC. Yesterday I got my chance to cleanup maven setup a bit (https://github.com/Code-House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811 <https://github.com/Code-House/incubator-tinkerpop/commit/0f6ff034124fd83409f6b278d4f93b5fbbdfa811> - will submit pull request later).



>> Now more concrete things. As every software project tinkerpop is also
>> focused on stability. My main concern is that if some actions will not be
>> taken before GA they will be extremely hard to push over later on. Roadmap
>> is clearly stating that general will is to have no major API changes for
>> 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid
>> extra work and efforts for migrations in future for project community.
>> Question if incubating project is ready to graduate if people see on
>> horizon code reorganization in it?
>> 
> 
> Apache incubation is about ensuring the community is healthy and able to
> self-govern under the Apache Way, not necessarily the code.
It is clear to me that community is 1st priority and code is 2nd, but one can not live without another. Relation between both is very strong. There are some examples of projects in Apache community which have been present in it for a long time but because of the way how PMC have worked they never established development community around project, except people coming from one firm. People from this could be promoted to become committers, other people didn't get contributions accepted or they were not supported to do such. For years people who was willing to use OSS release of ActiveMQ  were suffering from permanent lack of mainanance releases. Community in this case couldn’t do much with it but asking. Hopefuly AMQ will get straight some day.
Here I’m not saying tinkerpop is the same, just giving example how communities might not evolve if there is no will in project itself.


>> I raised a discussion of splitting some of IO packages out of core
>> interfaces. It’s wrong design decision and makes public API dependant on
>> concrete graph serialization techniques. This argument is based on
>> technical merit. OSGification just enforces this separation upfront. Almost
>> every project which starts with similar as tinkerpop approach ends up
>> moving this kind of dependencies out "core" soon or later. After all, can
>> you imagine all codecs embedded inside netty core?
>> 
> 
> Personally, my initial reaction to the Graph.Io interface is similar to
> yours.  However, I have not spent a lot of time in that area of the
> codebase, so I am reserving some judgement until I have more context.
> 
> As you know, it is difficult to completely envision the effect of such a
> proposed change from just an e-mail.  Perhaps you could provide a working
> example (PR) of what you propose from an interface change that the
> community can evaluate.
>> Roadmap is in the hands of project PMC and nobody can influence it,
>> however there was already comment from Hadrian, who is tinkerpop mentor
>> saying:
>> "there is a large user bases running in OSGi who will greatly benefit from
>> Tinkerpop"
>> 
> 
> While the PMC is ultimately responsible for the success of the project, the
> entire community should feel like they can voice their opinion and drive
> the roadmap through contributions.
> 
> This is not an oligarchy.  It is a meritocracy.
> 
> Of course, this does not mean that every proposed change will be included
> in the code base, but individuals should not feel dismissed when presenting
> a rational argument.
Matt, cause of that I did not burn more power on SPI work - it makes sense only if idea gets acceptance cause as Stephen pointed earlier, this stuff is not PMC concern at this moment (but only mine and few other people asking before). It’s not on roadmap. There is no point to make forks/feature branches living too long cause they will generate huge effort to mainain.
How I am supposed to feel not dismissed, if such simple thing as extra manifest entries can’t get accepted? When issues gets "refused" before pull request is even reviewed and never been verified? Will it be easier with more serious things? I don't think so.

Kind regards,
Lukasz
—
luke@code-house.org
Twitter: ldywicki
Blog: http://dywicki.pl
Code-House - http://code-house.org



> 
>> 
>> According to mvnrepository.com <http://mvnrepository.com/> osgi.core JAR
>> is used in over 2000 places (slf4j-api in 9300) while tinkerpop artifacts
>> distributed among different groups have 30 usages at maximum. Lack of
>> support for it in modern libs when jigsaw is on horizon is rather uncommon.
>> 
>> Kind regards,
>> Lukasz
>> 
>> 
>> 
>>> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w dniu
>> 26 mar 2015, o godz. 13:39:
>>> 
>>> Thanks James/David for the clarification. I was mostly trying to convey
>> my
>>> feeling that we should delay such work, but cast in your light I can see
>>> how else it could be read.   I'll be more conscious of word choice in the
>>> future.
>>> 
>>> On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
>>> 
>>>> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <spmallette@gmail.com
>>> 
>>>> wrote:
>>>>> First of all, apologies to Lukasz for hijacking his thread as it now
>>>> takes
>>>>> an administrative turn away from OSGi discussion.
>>>>> 
>>>>> James, I never said that we don't want help in this area - I only ask
>>>> that
>>>>> we delay such things and discuss it further down the line.  I don't see
>>>>> anything as coming "for free" btw.  Every contribution comes with a
>> cost
>>>>> (e.g. maintenance, complexity, etc).
>>>>> 
>>>>> David, this is the general roadmap for 3.0:
>>>>> 
>>>>> 
>>>> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversi
>> on/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:
>> version-issues-panel
>>>>> 
>>>>> Marko posted it as such and requested feedback:
>>>>> 
>>>>> 
>>>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop
>> -dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
>>>>> 
>>>>> but i don't think that was what you were really getting at.
>>>> 
>>>> Indeed :)
>>>> 
>>>>> So that, plus
>>>>> James' response about community building, leads to a question that I
>>>>> thought I understood the answer to, but maybe i don't.  When someone
>>>> makes
>>>>> a suggestion for some feature on this dev mailing list, offers a pull
>>>>> request, creates a jira issue, etc, what process does that suggestion
>>>> need
>>>>> to go through?  Is there something official that we need to adhere
>> to?  I
>>>>> assume there aren't hard and fast rules, and that this being one of the
>>>>> first external contribution offers of significance you guys are just
>>>> being
>>>>> watchful in how it is handled.  However, if there are "rules", I'd like
>>>> to
>>>>> know what they are.
>>>>> 
>>>> 
>>>> So what process that goes through is really for the project to decide.
>>>> (And yes, that's a terrible answer to your question - but it's also
>>>> why you see folks piling up here; because it's easy to get wrong, and
>>>> it's very important).
>>>> 
>>>> The things that raised the red flag for me in the conversation was:
>>>> * One person says it's not on the roadmap; but from tone and word
>>>> choice is sounds like a group decision. Specifically this makes me
>>>> wonder if decision making is happening somewhere other than the list.
>>>> * No link to the roadmap (Lukasz may or may not be following closely
>>>> enough to have seen that; so roadmap may be elusive)
>>>> 
>>>> There's also the issue that development doesn't have to be linear,
>>>> someone could work in a feature branch, submit a pull request,
>>>> assuming they had proper expectations. Dismissing new folks with 'not
>>>> on our roadmap', with seemingly no alternative doesn't help grow your
>>>> community.
>>>> 
>>>> --David
>>>> 
>> 
>> 

Re: Support for osgi in tinkerpop

[Matt Franklin <m....@gmail.com>]

On Thu, Mar 26, 2015 at 7:08 PM Łukasz Dywicki <lu...@code-house.org> wrote:

> I didn’t want jump into topic before having anything in my hands, so here
> is pull request:
> https://github.com/apache/incubator-tinkerpop/pull/43 <
> https://github.com/apache/incubator-tinkerpop/pull/43>
>
> And corresponding JIRA issue
> https://issues.apache.org/jira/browse/TINKERPOP3-602 <
> https://issues.apache.org/jira/browse/TINKERPOP3-602>
> @Stephen I marked this issue affects 3.0.0GA, not sure why you removed
> this flag 3 minutes after issue was created. It may be open for 3.1 and so
> on untill is fixed in 3.2 for example, as long as you are not ready to
> accept pull request.
>

Unless I am missing something, it looks like this PR just adds the OSGi
metadata for existing modules, correct?  This does not include the SPI
changes you discuss below.


>
> Now more concrete things. As every software project tinkerpop is also
> focused on stability. My main concern is that if some actions will not be
> taken before GA they will be extremely hard to push over later on. Roadmap
> is clearly stating that general will is to have no major API changes for
> 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid
> extra work and efforts for migrations in future for project community.
> Question if incubating project is ready to graduate if people see on
> horizon code reorganization in it?
>

Apache incubation is about ensuring the community is healthy and able to
self-govern under the Apache Way, not necessarily the code.


>
> I raised a discussion of splitting some of IO packages out of core
> interfaces. It’s wrong design decision and makes public API dependant on
> concrete graph serialization techniques. This argument is based on
> technical merit. OSGification just enforces this separation upfront. Almost
> every project which starts with similar as tinkerpop approach ends up
> moving this kind of dependencies out "core" soon or later. After all, can
> you imagine all codecs embedded inside netty core?
>

Personally, my initial reaction to the Graph.Io interface is similar to
yours.  However, I have not spent a lot of time in that area of the
codebase, so I am reserving some judgement until I have more context.

As you know, it is difficult to completely envision the effect of such a
proposed change from just an e-mail.  Perhaps you could provide a working
example (PR) of what you propose from an interface change that the
community can evaluate.


>
> Roadmap is in the hands of project PMC and nobody can influence it,
> however there was already comment from Hadrian, who is tinkerpop mentor
> saying:
> "there is a large user bases running in OSGi who will greatly benefit from
> Tinkerpop"
>

While the PMC is ultimately responsible for the success of the project, the
entire community should feel like they can voice their opinion and drive
the roadmap through contributions.

This is not an oligarchy.  It is a meritocracy.

Of course, this does not mean that every proposed change will be included
in the code base, but individuals should not feel dismissed when presenting
a rational argument.


>
> According to mvnrepository.com <http://mvnrepository.com/> osgi.core JAR
> is used in over 2000 places (slf4j-api in 9300) while tinkerpop artifacts
> distributed among different groups have 30 usages at maximum. Lack of
> support for it in modern libs when jigsaw is on horizon is rather uncommon.
>
> Kind regards,
> Lukasz
>
>
>
> > Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w dniu
> 26 mar 2015, o godz. 13:39:
> >
> > Thanks James/David for the clarification. I was mostly trying to convey
> my
> > feeling that we should delay such work, but cast in your light I can see
> > how else it could be read.   I'll be more conscious of word choice in the
> > future.
> >
> > On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
> >
> >> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <spmallette@gmail.com
> >
> >> wrote:
> >>> First of all, apologies to Lukasz for hijacking his thread as it now
> >> takes
> >>> an administrative turn away from OSGi discussion.
> >>>
> >>> James, I never said that we don't want help in this area - I only ask
> >> that
> >>> we delay such things and discuss it further down the line.  I don't see
> >>> anything as coming "for free" btw.  Every contribution comes with a
> cost
> >>> (e.g. maintenance, complexity, etc).
> >>>
> >>> David, this is the general roadmap for 3.0:
> >>>
> >>>
> >> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversi
> on/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:
> version-issues-panel
> >>>
> >>> Marko posted it as such and requested feedback:
> >>>
> >>>
> >> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop
> -dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
> >>>
> >>> but i don't think that was what you were really getting at.
> >>
> >> Indeed :)
> >>
> >>> So that, plus
> >>> James' response about community building, leads to a question that I
> >>> thought I understood the answer to, but maybe i don't.  When someone
> >> makes
> >>> a suggestion for some feature on this dev mailing list, offers a pull
> >>> request, creates a jira issue, etc, what process does that suggestion
> >> need
> >>> to go through?  Is there something official that we need to adhere
> to?  I
> >>> assume there aren't hard and fast rules, and that this being one of the
> >>> first external contribution offers of significance you guys are just
> >> being
> >>> watchful in how it is handled.  However, if there are "rules", I'd like
> >> to
> >>> know what they are.
> >>>
> >>
> >> So what process that goes through is really for the project to decide.
> >> (And yes, that's a terrible answer to your question - but it's also
> >> why you see folks piling up here; because it's easy to get wrong, and
> >> it's very important).
> >>
> >> The things that raised the red flag for me in the conversation was:
> >> * One person says it's not on the roadmap; but from tone and word
> >> choice is sounds like a group decision. Specifically this makes me
> >> wonder if decision making is happening somewhere other than the list.
> >> * No link to the roadmap (Lukasz may or may not be following closely
> >> enough to have seen that; so roadmap may be elusive)
> >>
> >> There's also the issue that development doesn't have to be linear,
> >> someone could work in a feature branch, submit a pull request,
> >> assuming they had proper expectations. Dismissing new folks with 'not
> >> on our roadmap', with seemingly no alternative doesn't help grow your
> >> community.
> >>
> >> --David
> >>
>
>

Re: Support for osgi in tinkerpop

[Łukasz Dywicki <lu...@code-house.org>]

I didn’t want jump into topic before having anything in my hands, so here is pull request:
https://github.com/apache/incubator-tinkerpop/pull/43 <https://github.com/apache/incubator-tinkerpop/pull/43>

And corresponding JIRA issue
https://issues.apache.org/jira/browse/TINKERPOP3-602 <https://issues.apache.org/jira/browse/TINKERPOP3-602>
@Stephen I marked this issue affects 3.0.0GA, not sure why you removed this flag 3 minutes after issue was created. It may be open for 3.1 and so on untill is fixed in 3.2 for example, as long as you are not ready to accept pull request.

Now more concrete things. As every software project tinkerpop is also focused on stability. My main concern is that if some actions will not be taken before GA they will be extremely hard to push over later on. Roadmap is clearly stating that general will is to have no major API changes for 3.1 and 3.2 as well. So that’s why I was willing to help now, to avoid extra work and efforts for migrations in future for project community. Question if incubating project is ready to graduate if people see on horizon code reorganization in it?

I raised a discussion of splitting some of IO packages out of core interfaces. It’s wrong design decision and makes public API dependant on concrete graph serialization techniques. This argument is based on technical merit. OSGification just enforces this separation upfront. Almost every project which starts with similar as tinkerpop approach ends up moving this kind of dependencies out "core" soon or later. After all, can you imagine all codecs embedded inside netty core?

Roadmap is in the hands of project PMC and nobody can influence it, however there was already comment from Hadrian, who is tinkerpop mentor saying:
"there is a large user bases running in OSGi who will greatly benefit from Tinkerpop"

According to mvnrepository.com <http://mvnrepository.com/> osgi.core JAR is used in over 2000 places (slf4j-api in 9300) while tinkerpop artifacts distributed among different groups have 30 usages at maximum. Lack of support for it in modern libs when jigsaw is on horizon is rather uncommon.

Kind regards,
Lukasz



> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w dniu 26 mar 2015, o godz. 13:39:
> 
> Thanks James/David for the clarification. I was mostly trying to convey my
> feeling that we should delay such work, but cast in your light I can see
> how else it could be read.   I'll be more conscious of word choice in the
> future.
> 
> On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:
> 
>> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <sp...@gmail.com>
>> wrote:
>>> First of all, apologies to Lukasz for hijacking his thread as it now
>> takes
>>> an administrative turn away from OSGi discussion.
>>> 
>>> James, I never said that we don't want help in this area - I only ask
>> that
>>> we delay such things and discuss it further down the line.  I don't see
>>> anything as coming "for free" btw.  Every contribution comes with a cost
>>> (e.g. maintenance, complexity, etc).
>>> 
>>> David, this is the general roadmap for 3.0:
>>> 
>>> 
>> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversion/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel
>>> 
>>> Marko posted it as such and requested feedback:
>>> 
>>> 
>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
>>> 
>>> but i don't think that was what you were really getting at.
>> 
>> Indeed :)
>> 
>>> So that, plus
>>> James' response about community building, leads to a question that I
>>> thought I understood the answer to, but maybe i don't.  When someone
>> makes
>>> a suggestion for some feature on this dev mailing list, offers a pull
>>> request, creates a jira issue, etc, what process does that suggestion
>> need
>>> to go through?  Is there something official that we need to adhere to?  I
>>> assume there aren't hard and fast rules, and that this being one of the
>>> first external contribution offers of significance you guys are just
>> being
>>> watchful in how it is handled.  However, if there are "rules", I'd like
>> to
>>> know what they are.
>>> 
>> 
>> So what process that goes through is really for the project to decide.
>> (And yes, that's a terrible answer to your question - but it's also
>> why you see folks piling up here; because it's easy to get wrong, and
>> it's very important).
>> 
>> The things that raised the red flag for me in the conversation was:
>> * One person says it's not on the roadmap; but from tone and word
>> choice is sounds like a group decision. Specifically this makes me
>> wonder if decision making is happening somewhere other than the list.
>> * No link to the roadmap (Lukasz may or may not be following closely
>> enough to have seen that; so roadmap may be elusive)
>> 
>> There's also the issue that development doesn't have to be linear,
>> someone could work in a feature branch, submit a pull request,
>> assuming they had proper expectations. Dismissing new folks with 'not
>> on our roadmap', with seemingly no alternative doesn't help grow your
>> community.
>> 
>> --David
>> 

Re: Support for osgi in tinkerpop

[Stephen Mallette <sp...@gmail.com>]

Thanks James/David for the clarification. I was mostly trying to convey my
feeling that we should delay such work, but cast in your light I can see
how else it could be read.   I'll be more conscious of word choice in the
future.

On Thu, Mar 26, 2015 at 8:31 AM, David Nalley <da...@gnsa.us> wrote:

> On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <sp...@gmail.com>
> wrote:
> > First of all, apologies to Lukasz for hijacking his thread as it now
> takes
> > an administrative turn away from OSGi discussion.
> >
> > James, I never said that we don't want help in this area - I only ask
> that
> > we delay such things and discuss it further down the line.  I don't see
> > anything as coming "for free" btw.  Every contribution comes with a cost
> > (e.g. maintenance, complexity, etc).
> >
> > David, this is the general roadmap for 3.0:
> >
> >
> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversion/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel
> >
> > Marko posted it as such and requested feedback:
> >
> >
> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
> >
> > but i don't think that was what you were really getting at.
>
> Indeed :)
>
> > So that, plus
> > James' response about community building, leads to a question that I
> > thought I understood the answer to, but maybe i don't.  When someone
> makes
> > a suggestion for some feature on this dev mailing list, offers a pull
> > request, creates a jira issue, etc, what process does that suggestion
> need
> > to go through?  Is there something official that we need to adhere to?  I
> > assume there aren't hard and fast rules, and that this being one of the
> > first external contribution offers of significance you guys are just
> being
> > watchful in how it is handled.  However, if there are "rules", I'd like
> to
> > know what they are.
> >
>
> So what process that goes through is really for the project to decide.
> (And yes, that's a terrible answer to your question - but it's also
> why you see folks piling up here; because it's easy to get wrong, and
> it's very important).
>
> The things that raised the red flag for me in the conversation was:
> * One person says it's not on the roadmap; but from tone and word
> choice is sounds like a group decision. Specifically this makes me
> wonder if decision making is happening somewhere other than the list.
> * No link to the roadmap (Lukasz may or may not be following closely
> enough to have seen that; so roadmap may be elusive)
>
> There's also the issue that development doesn't have to be linear,
> someone could work in a feature branch, submit a pull request,
> assuming they had proper expectations. Dismissing new folks with 'not
> on our roadmap', with seemingly no alternative doesn't help grow your
> community.
>
> --David
>

Re: Support for osgi in tinkerpop

[David Nalley <da...@gnsa.us>]

On Thu, Mar 26, 2015 at 8:07 AM, Stephen Mallette <sp...@gmail.com> wrote:
> First of all, apologies to Lukasz for hijacking his thread as it now takes
> an administrative turn away from OSGi discussion.
>
> James, I never said that we don't want help in this area - I only ask that
> we delay such things and discuss it further down the line.  I don't see
> anything as coming "for free" btw.  Every contribution comes with a cost
> (e.g. maintenance, complexity, etc).
>
> David, this is the general roadmap for 3.0:
>
> https://issues.apache.org/jira/browse/TINKERPOP3/fixforversion/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel
>
> Marko posted it as such and requested feedback:
>
> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E
>
> but i don't think that was what you were really getting at.

Indeed :)

> So that, plus
> James' response about community building, leads to a question that I
> thought I understood the answer to, but maybe i don't.  When someone makes
> a suggestion for some feature on this dev mailing list, offers a pull
> request, creates a jira issue, etc, what process does that suggestion need
> to go through?  Is there something official that we need to adhere to?  I
> assume there aren't hard and fast rules, and that this being one of the
> first external contribution offers of significance you guys are just being
> watchful in how it is handled.  However, if there are "rules", I'd like to
> know what they are.
>

So what process that goes through is really for the project to decide.
(And yes, that's a terrible answer to your question - but it's also
why you see folks piling up here; because it's easy to get wrong, and
it's very important).

The things that raised the red flag for me in the conversation was:
* One person says it's not on the roadmap; but from tone and word
choice is sounds like a group decision. Specifically this makes me
wonder if decision making is happening somewhere other than the list.
* No link to the roadmap (Lukasz may or may not be following closely
enough to have seen that; so roadmap may be elusive)

There's also the issue that development doesn't have to be linear,
someone could work in a feature branch, submit a pull request,
assuming they had proper expectations. Dismissing new folks with 'not
on our roadmap', with seemingly no alternative doesn't help grow your
community.

--David

Re: Support for osgi in tinkerpop

[Stephen Mallette <sp...@gmail.com>]

First of all, apologies to Lukasz for hijacking his thread as it now takes
an administrative turn away from OSGi discussion.

James, I never said that we don't want help in this area - I only ask that
we delay such things and discuss it further down the line.  I don't see
anything as coming "for free" btw.  Every contribution comes with a cost
(e.g. maintenance, complexity, etc).

David, this is the general roadmap for 3.0:

https://issues.apache.org/jira/browse/TINKERPOP3/fixforversion/12329740/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel

Marko posted it as such and requested feedback:

http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201503.mbox/%3C8E1590FC-FC5C-4AC2-9C74-4290143DCC2F%40gmail.com%3E

but i don't think that was what you were really getting at.  So that, plus
James' response about community building, leads to a question that I
thought I understood the answer to, but maybe i don't.  When someone makes
a suggestion for some feature on this dev mailing list, offers a pull
request, creates a jira issue, etc, what process does that suggestion need
to go through?  Is there something official that we need to adhere to?  I
assume there aren't hard and fast rules, and that this being one of the
first external contribution offers of significance you guys are just being
watchful in how it is handled.  However, if there are "rules", I'd like to
know what they are.

Thanks,

Stephen





On Thu, Mar 26, 2015 at 7:21 AM, David Nalley <da...@gnsa.us> wrote:

> On Thu, Mar 26, 2015 at 6:31 AM, Stephen Mallette <sp...@gmail.com>
> wrote:
> > OSGi support is not high on the priority list for us right now.  It is
> > something I think we are open to considering in the future depending on
> the
> > impact it has to our project but it is not currently a roadmap item.
> >
>
> Who is the 'us' and where was this roadmap decided? Link?
>

Re: Support for osgi in tinkerpop

[David Nalley <da...@gnsa.us>]

On Thu, Mar 26, 2015 at 6:31 AM, Stephen Mallette <sp...@gmail.com> wrote:
> OSGi support is not high on the priority list for us right now.  It is
> something I think we are open to considering in the future depending on the
> impact it has to our project but it is not currently a roadmap item.
>

Who is the 'us' and where was this roadmap decided? Link?

Re: Support for osgi in tinkerpop

[Stephen Mallette <sp...@gmail.com>]

OSGi support is not high on the priority list for us right now.  It is
something I think we are open to considering in the future depending on the
impact it has to our project but it is not currently a roadmap item.

On Wed, Mar 25, 2015 at 8:58 AM, James Carman <ja...@carmanconsulting.com>
wrote:

> Many times, the default behavior of the bundle plugin is all that is
> needed to get started.  It couldn't hurt to try that.  You can take a
> look at a nice parent pom.xml file that does this for you:
>
> https://github.com/savoirtech/osgi-parent/blob/master/pom.xml
>
>
> On Tue, Mar 24, 2015 at 6:55 PM, Łukasz Dywicki <lu...@code-house.org>
> wrote:
> > Hey Stephen,
> > Thank you for your reply. I believe that making libraries supporting
> OSGi directly is simply the best option in longer run and re-packaging just
> to add manifest entries is a bit PITA. Repackager is forced to work on
> every major and minor release and after all it is just double work.
> >
> > Since I am not a graph expert and not ready to make changes in tinkerpop
> core I can support you in infrastructure tasks which are related to osgi
> stuff. Sometimes there is a need for code relocations, maven maintenance,
> version updates and such. Hopefully, I will be able to do that without
> breaking your release plans.
> >
> > I think that now is a perfect time to make code relocations and tweaks
> into SPI since API shape is close to finalize so very small troup of people
> will suffer from changes if any. Some of these changes will not be possible
> after delivering 3.0 and will be for sure delayed until 3.1 to don’t break
> SPI compatibility and let companies have easier transition between 3.0 and
> 3.1 later on.
> >
> > Kind regards,
> > Lukasz
> >
> >
> >> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w
> dniu 24 mar 2015, o godz. 23:31:
> >>
> >> Hi Łukasz,
> >>
> >> Thanks for the detailed explanation about OSGi.  It was probably a good
> >> introduction to some and a refresher to others.  I mentioned to Marko
> that
> >> if we were going to accept a pull request from someone to OSGi-enable
> >> TinkerPop, it may as well come from a committer on Karaf.  :)  I think
> that
> >> most individuals who have brought up OSGi in the past have simply tried
> to
> >> throw us a pull request with some pom.xml changes in it for OSGi
> metadata,
> >> without any analysis done as to the other two levels of OSGi support
> that
> >> you outlined. It was helpful to have that additional information. I'm
> sure
> >> that we likely break that model in various places in our code base, so I
> >> would expect that some changes would be required in addition to just
> some
> >> pom.xml entries.
> >>
> >> That said, I think that I have to agree with Matt that this is an issue
> >> best left until after we get through an initial release or two.  Let's
> >> revisit this issue when we get ourselves a bit more settled into Apache.
> >>
> >> Thanks again,
> >>
> >> Stephen
> >>
> >> On Tue, Mar 24, 2015 at 11:28 AM, Hadrian Zbarcea <hz...@gmail.com>
> >> wrote:
> >>
> >>> Oh, absolutely. In case that wasn't clear :).
> >>> Hadrian
> >>>
> >>>
> >>> On 03/24/2015 11:05 AM, Matt Franklin wrote:
> >>>
> >>>> IMO, the community should focus on getting the release process down
> first.
> >>>> Could this be a feature added to a later milestone or 3.x point
> release?
> >>>>
> >>>>
> >>>>
> >
>

Re: Support for osgi in tinkerpop

[James Carman <ja...@carmanconsulting.com>]

Many times, the default behavior of the bundle plugin is all that is
needed to get started.  It couldn't hurt to try that.  You can take a
look at a nice parent pom.xml file that does this for you:

https://github.com/savoirtech/osgi-parent/blob/master/pom.xml


On Tue, Mar 24, 2015 at 6:55 PM, Łukasz Dywicki <lu...@code-house.org> wrote:
> Hey Stephen,
> Thank you for your reply. I believe that making libraries supporting OSGi directly is simply the best option in longer run and re-packaging just to add manifest entries is a bit PITA. Repackager is forced to work on every major and minor release and after all it is just double work.
>
> Since I am not a graph expert and not ready to make changes in tinkerpop core I can support you in infrastructure tasks which are related to osgi stuff. Sometimes there is a need for code relocations, maven maintenance, version updates and such. Hopefully, I will be able to do that without breaking your release plans.
>
> I think that now is a perfect time to make code relocations and tweaks into SPI since API shape is close to finalize so very small troup of people will suffer from changes if any. Some of these changes will not be possible after delivering 3.0 and will be for sure delayed until 3.1 to don’t break SPI compatibility and let companies have easier transition between 3.0 and 3.1 later on.
>
> Kind regards,
> Lukasz
>
>
>> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w dniu 24 mar 2015, o godz. 23:31:
>>
>> Hi Łukasz,
>>
>> Thanks for the detailed explanation about OSGi.  It was probably a good
>> introduction to some and a refresher to others.  I mentioned to Marko that
>> if we were going to accept a pull request from someone to OSGi-enable
>> TinkerPop, it may as well come from a committer on Karaf.  :)  I think that
>> most individuals who have brought up OSGi in the past have simply tried to
>> throw us a pull request with some pom.xml changes in it for OSGi metadata,
>> without any analysis done as to the other two levels of OSGi support that
>> you outlined. It was helpful to have that additional information. I'm sure
>> that we likely break that model in various places in our code base, so I
>> would expect that some changes would be required in addition to just some
>> pom.xml entries.
>>
>> That said, I think that I have to agree with Matt that this is an issue
>> best left until after we get through an initial release or two.  Let's
>> revisit this issue when we get ourselves a bit more settled into Apache.
>>
>> Thanks again,
>>
>> Stephen
>>
>> On Tue, Mar 24, 2015 at 11:28 AM, Hadrian Zbarcea <hz...@gmail.com>
>> wrote:
>>
>>> Oh, absolutely. In case that wasn't clear :).
>>> Hadrian
>>>
>>>
>>> On 03/24/2015 11:05 AM, Matt Franklin wrote:
>>>
>>>> IMO, the community should focus on getting the release process down first.
>>>> Could this be a feature added to a later milestone or 3.x point release?
>>>>
>>>>
>>>>
>

Re: Support for osgi in tinkerpop

[Łukasz Dywicki <lu...@code-house.org>]

Hey Stephen,
Thank you for your reply. I believe that making libraries supporting OSGi directly is simply the best option in longer run and re-packaging just to add manifest entries is a bit PITA. Repackager is forced to work on every major and minor release and after all it is just double work.

Since I am not a graph expert and not ready to make changes in tinkerpop core I can support you in infrastructure tasks which are related to osgi stuff. Sometimes there is a need for code relocations, maven maintenance, version updates and such. Hopefully, I will be able to do that without breaking your release plans.

I think that now is a perfect time to make code relocations and tweaks into SPI since API shape is close to finalize so very small troup of people will suffer from changes if any. Some of these changes will not be possible after delivering 3.0 and will be for sure delayed until 3.1 to don’t break SPI compatibility and let companies have easier transition between 3.0 and 3.1 later on.

Kind regards,
Lukasz


> Wiadomość napisana przez Stephen Mallette <sp...@gmail.com> w dniu 24 mar 2015, o godz. 23:31:
> 
> Hi Łukasz,
> 
> Thanks for the detailed explanation about OSGi.  It was probably a good
> introduction to some and a refresher to others.  I mentioned to Marko that
> if we were going to accept a pull request from someone to OSGi-enable
> TinkerPop, it may as well come from a committer on Karaf.  :)  I think that
> most individuals who have brought up OSGi in the past have simply tried to
> throw us a pull request with some pom.xml changes in it for OSGi metadata,
> without any analysis done as to the other two levels of OSGi support that
> you outlined. It was helpful to have that additional information. I'm sure
> that we likely break that model in various places in our code base, so I
> would expect that some changes would be required in addition to just some
> pom.xml entries.
> 
> That said, I think that I have to agree with Matt that this is an issue
> best left until after we get through an initial release or two.  Let's
> revisit this issue when we get ourselves a bit more settled into Apache.
> 
> Thanks again,
> 
> Stephen
> 
> On Tue, Mar 24, 2015 at 11:28 AM, Hadrian Zbarcea <hz...@gmail.com>
> wrote:
> 
>> Oh, absolutely. In case that wasn't clear :).
>> Hadrian
>> 
>> 
>> On 03/24/2015 11:05 AM, Matt Franklin wrote:
>> 
>>> IMO, the community should focus on getting the release process down first.
>>> Could this be a feature added to a later milestone or 3.x point release?
>>> 
>>> 
>>> 

Re: Support for osgi in tinkerpop

[Stephen Mallette <sp...@gmail.com>]

Hi Łukasz,

Thanks for the detailed explanation about OSGi.  It was probably a good
introduction to some and a refresher to others.  I mentioned to Marko that
if we were going to accept a pull request from someone to OSGi-enable
TinkerPop, it may as well come from a committer on Karaf.  :)  I think that
most individuals who have brought up OSGi in the past have simply tried to
throw us a pull request with some pom.xml changes in it for OSGi metadata,
without any analysis done as to the other two levels of OSGi support that
you outlined. It was helpful to have that additional information. I'm sure
that we likely break that model in various places in our code base, so I
would expect that some changes would be required in addition to just some
pom.xml entries.

That said, I think that I have to agree with Matt that this is an issue
best left until after we get through an initial release or two.  Let's
revisit this issue when we get ourselves a bit more settled into Apache.

Thanks again,

Stephen

On Tue, Mar 24, 2015 at 11:28 AM, Hadrian Zbarcea <hz...@gmail.com>
wrote:

> Oh, absolutely. In case that wasn't clear :).
> Hadrian
>
>
> On 03/24/2015 11:05 AM, Matt Franklin wrote:
>
>> IMO, the community should focus on getting the release process down first.
>> Could this be a feature added to a later milestone or 3.x point release?
>>
>>
>>

Re: Support for osgi in tinkerpop

[Hadrian Zbarcea <hz...@gmail.com>]

Oh, absolutely. In case that wasn't clear :).
Hadrian

On 03/24/2015 11:05 AM, Matt Franklin wrote:
> IMO, the community should focus on getting the release process down first.
> Could this be a feature added to a later milestone or 3.x point release?
>
>

Re: Support for osgi in tinkerpop

[Matt Franklin <m....@gmail.com>]

On Mon, Mar 23, 2015 at 9:35 PM Hadrian Zbarcea <hz...@gmail.com> wrote:

> Luke,
>
> The only version that is relevant right now is Tinkerpop 3. It is
> unclear if 2.x versions will be released at the ASF. Others will correct
> me if I am wrong.
>
> Personally, I strongly encourage OSGi support for Tinkerpop, for many
> reasons, including the fact that there is a large user bases running in
> OSGi who will greatly benefit from Tinkerpop.


+1


> I can also make myself
> available to contribute to such a noble cause.


IMO, the community should focus on getting the release process down first.
Could this be a feature added to a later milestone or 3.x point release?


>
>
Tinkerpop black belts, wdyt?
>
> Cheers,
> Hadrian
>
>
> On 03/23/2015 08:52 PM, Łukasz Dywicki wrote:
> > Hey all,
> > At the begining I would love to thank you all tinkerpop developers for
> your amazing work. I have found tinkerpop ecosystem couple months ago and I
> loved your logos since begining. I did not use tinkerpop initially but a
> month ago I started playing around with some basics. Yesterday I started
> deploying my application to Apache Karaf (which is my default "server", I
> am also commiter/PMC in this project) and did patch tinkerpop
> 2.7.0-SNAPSHOT to run smoothly in this environment.
> >
> > This morning I had small discussion with Stephen about support for OSGi.
> Here I would like continue and explain what are basic steps to start
> supporting OSGi. I will also try to explain a bit why it is not worst idea
> on the globe. I suppose most of you have heard about OSGi here or there.
> Majority of people remembers extremes so each of you probably recorded at
> least one comment on this - love or hate. Till some degree it comes to same
> discussion like with Maven. One side says - it is so limited that making
> something non trivial is real pain. Another will tell you it is not
> allowing you to complicate build too much. Both sides are right till some
> degree. Since you know my root project in ASF you also know I can not be
> impassive/dispassionate/objective in such discussion. :-)
> >
> > Let’s start from defining levels of OSGI support:
> > 1) OSGi metadata in manifest entries
> > 2) Taking care about class loading
> > 3) Pluggable extensions
> >
> > First steep is intended to add couple of headers into JAR - like
> Import-Package, Export-Package. First header is used to compute dependency
> graph for a module in runtime and allows to make sure that all requirements
> are satisfied. Second header declares what should be visible to other
> bundles - in other words what can be referenced in other module via
> Import-Package. In addition these headers might contain version information
> which provides an easy way to avoid common class loading problems such
> ClassNotFoundException or NoMethodDefError. Tinkerpop is ready to take this
> step since each JAR you produce have it’s own packaging root and you do not
> have split packages.
> >
> > Second step is bit more tricky but not that hard afterwards. Each class
> initialization should use appropriate class loader. In traditional
> environment such Tomcat/J2SE all JARs are collected into application
> classloader which consisting all entries from WEB-INF/lib or lib and shared
> around with everyone. Under OSGi this assumption is not valid any longer
> since every module can see only what it declared and have own classloader.
> Here framework is similar to Maven - what is not explicity declared as
> dependency is not visible. For example when tinkerpop-core will try to
> initialize class from package which is not declared in manifest entries
> (ie. Class.forName(com.acme.Graph) result will be ClassNotFoundException,
> even if com.acme.Graph is already exported by another module. To avoid
> troubles in most of cases it is sufficient to pass for Class.forName
> invocation also classloader which should be used. The same goes to
> serialization.
> >
> > Third step is related to bringing extensions. As already described above
> - dynamic Class.forName constructions are not working by default under
> OSGi. Provisioning of extensions might be done in many different ways once
> library provides pluggable strategy for it. Usage of service loader is
> fine, however it is also bound to classloader which is used to locate
> META-INF entries.
> >
> > Now let’s take a further look on what are common problems with OSGi and
> how to avoid it. First of all OSGi forces clear separation of API and
> implementation packages. It’s not possible to have circular dependencies
> (another similarity to maven). Again Tinkerpop is ready since package
> org.apache.tinkerpop.gremlin.structure contains all what is needed to
> operate with most of graphs. It also contains IO package which could be
> succesfully split into tinkerpop-graphson, tinkerpop-gyro and optionally
> tinkerpop-graphml. Last separation is optional cause it is dependant only
> on JDK classes and does not bring any external libs into classpath.
> > I believe you might be a bit sceptic of ending up with hundred of JARs
> containing two classes. In case described above it is 100% right to devide
> since you are providing GraphWriter/Reader implementations based on 3rd
> party API which is not part of your public API itself - it’s integration.
> Also it might be benefitiary for your users once moving this dependencies
> out of core cause they won’t get dozen of unecessary dependencies. Someone
> might be interested in graphml but might be not interested in graphson at
> all. I suppose most of users are interested in one storage implementation
> at time.
> > Last thing which I would like to touch is configuration. For example now
> gremlin server depens on metrics which is fine, but also does depend on
> speciffic reportes which is not so good. This happens because the server
> module does not contain only a server related classes but also builds a
> default distribution which should be ready to run out of the box. After
> separing these two things running server under OSGi will be possible and
> extensible as well. After all op processors are interested only in
> reporting values, not in how they are being reported.
> >
> > At very end of this mail - I would like also to state that I am able to
> help you with providing support for at least OSGi metadata, if you are
> interested. I am also open to continue this discussion to see if solving
> potential class loading issues is possible. I haven’t digg into tinkerpop 3
> code except dependencies so I might miss some important bits and pieces
> thus assuming or making things simpler than they really are.
> >
> > Kind regards,
> > Łukasz Dywicki
> > —
> > Apache Karaf commiter & PMC
> > Twitter: ldywicki
> > Blog: http://dywicki.pl
> > Code-House - http://code-house.org
> >
>

Re: Support for osgi in tinkerpop

[Hadrian Zbarcea <hz...@gmail.com>]

Luke,

The only version that is relevant right now is Tinkerpop 3. It is 
unclear if 2.x versions will be released at the ASF. Others will correct 
me if I am wrong.

Personally, I strongly encourage OSGi support for Tinkerpop, for many 
reasons, including the fact that there is a large user bases running in 
OSGi who will greatly benefit from Tinkerpop. I can also make myself 
available to contribute to such a noble cause.

Tinkerpop black belts, wdyt?

Cheers,
Hadrian


On 03/23/2015 08:52 PM, Łukasz Dywicki wrote:
> Hey all,
> At the begining I would love to thank you all tinkerpop developers for your amazing work. I have found tinkerpop ecosystem couple months ago and I loved your logos since begining. I did not use tinkerpop initially but a month ago I started playing around with some basics. Yesterday I started deploying my application to Apache Karaf (which is my default "server", I am also commiter/PMC in this project) and did patch tinkerpop 2.7.0-SNAPSHOT to run smoothly in this environment.
>
> This morning I had small discussion with Stephen about support for OSGi. Here I would like continue and explain what are basic steps to start supporting OSGi. I will also try to explain a bit why it is not worst idea on the globe. I suppose most of you have heard about OSGi here or there. Majority of people remembers extremes so each of you probably recorded at least one comment on this - love or hate. Till some degree it comes to same discussion like with Maven. One side says - it is so limited that making something non trivial is real pain. Another will tell you it is not allowing you to complicate build too much. Both sides are right till some degree. Since you know my root project in ASF you also know I can not be impassive/dispassionate/objective in such discussion. :-)
>
> Let’s start from defining levels of OSGI support:
> 1) OSGi metadata in manifest entries
> 2) Taking care about class loading
> 3) Pluggable extensions
>
> First steep is intended to add couple of headers into JAR - like Import-Package, Export-Package. First header is used to compute dependency graph for a module in runtime and allows to make sure that all requirements are satisfied. Second header declares what should be visible to other bundles - in other words what can be referenced in other module via Import-Package. In addition these headers might contain version information which provides an easy way to avoid common class loading problems such ClassNotFoundException or NoMethodDefError. Tinkerpop is ready to take this step since each JAR you produce have it’s own packaging root and you do not have split packages.
>
> Second step is bit more tricky but not that hard afterwards. Each class initialization should use appropriate class loader. In traditional environment such Tomcat/J2SE all JARs are collected into application classloader which consisting all entries from WEB-INF/lib or lib and shared around with everyone. Under OSGi this assumption is not valid any longer since every module can see only what it declared and have own classloader. Here framework is similar to Maven - what is not explicity declared as dependency is not visible. For example when tinkerpop-core will try to initialize class from package which is not declared in manifest entries (ie. Class.forName(com.acme.Graph) result will be ClassNotFoundException, even if com.acme.Graph is already exported by another module. To avoid troubles in most of cases it is sufficient to pass for Class.forName invocation also classloader which should be used. The same goes to serialization.
>
> Third step is related to bringing extensions. As already described above - dynamic Class.forName constructions are not working by default under OSGi. Provisioning of extensions might be done in many different ways once library provides pluggable strategy for it. Usage of service loader is fine, however it is also bound to classloader which is used to locate META-INF entries.
>
> Now let’s take a further look on what are common problems with OSGi and how to avoid it. First of all OSGi forces clear separation of API and implementation packages. It’s not possible to have circular dependencies (another similarity to maven). Again Tinkerpop is ready since package org.apache.tinkerpop.gremlin.structure contains all what is needed to operate with most of graphs. It also contains IO package which could be succesfully split into tinkerpop-graphson, tinkerpop-gyro and optionally tinkerpop-graphml. Last separation is optional cause it is dependant only on JDK classes and does not bring any external libs into classpath.
> I believe you might be a bit sceptic of ending up with hundred of JARs containing two classes. In case described above it is 100% right to devide since you are providing GraphWriter/Reader implementations based on 3rd party API which is not part of your public API itself - it’s integration. Also it might be benefitiary for your users once moving this dependencies out of core cause they won’t get dozen of unecessary dependencies. Someone might be interested in graphml but might be not interested in graphson at all. I suppose most of users are interested in one storage implementation at time.
> Last thing which I would like to touch is configuration. For example now gremlin server depens on metrics which is fine, but also does depend on speciffic reportes which is not so good. This happens because the server module does not contain only a server related classes but also builds a default distribution which should be ready to run out of the box. After separing these two things running server under OSGi will be possible and extensible as well. After all op processors are interested only in reporting values, not in how they are being reported.
>
> At very end of this mail - I would like also to state that I am able to help you with providing support for at least OSGi metadata, if you are interested. I am also open to continue this discussion to see if solving potential class loading issues is possible. I haven’t digg into tinkerpop 3 code except dependencies so I might miss some important bits and pieces thus assuming or making things simpler than they really are.
>
> Kind regards,
> Łukasz Dywicki
> —
> Apache Karaf commiter & PMC
> Twitter: ldywicki
> Blog: http://dywicki.pl
> Code-House - http://code-house.org
>