Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

Hey all,

There are many forms of offlist development.  One form of offlist
development is working on large code drops in private and then
contributing them all at once.  Threshold size is probably arguable,
and varies by project; put that aside for the moment.  I've been
working on an explanation of how large code drops damage community and
code.  I'd love to hear your feedback.  I'm including my project and
the dev@community list in the hopes that people from other projects
also have a perspective.  Here it goes:


Imagine you are an individual contributor on a project.  You would
like to contribute something.  You see a feature you'd like to add or
a bug you'd like to fix, a user you would like to support, or a
release you'd like to test.  You start on it.  You submit your pull
request, you answer the user's question, you test the release.  You
continue doing this at a low level for a few months.  You see other
people starting to contribute too.  This is nice.  You're working
together with others towards a common goal.  Then, out of the blue a
company with multiple paid contributors shows up.  Let's name them
Acme. Acme drops a year of code on the project.  They could do this
many ways.  For example:  A.) Acme could develop in the repository you
were working in, or B.) Acme could create a project-internal fork and
create a new repository. C.) Acme could even telegraph months in
advance that they intend to do this, by posting to the dev list or
contacting key contributors offlist, or just by having done it a few
times already.


A.) First let's imagine that Acme made massive changes in the
repository you were working in.  Perhaps they already solved the
problem you solved, but in a different way.  Perhaps, they deleted
functions you made changes in.  Perhaps they added significant
functionality you would have liked to help with.  What good were your
efforts?  Wouldn't you find this discouraging?

And now you want to continue to make changes, but the code you want to
change has commit messages referencing tickets which you have no
access to.  Or it has no reference to tickets at all.  You find an
area that seems to be needlessly complex: can you remove the
complexity?  You have no way of knowing what you'd be breaking.

Perhaps you have a proprietary UI which depends on a behavior which
was removed or changed.  Now your UI is broken.  Because the code drop
is so large, you have no way to reasonably review it for
incompatibilities.  It is not possible to review a year of development
all at once.  And if your review turns up problems?  Do you accept the
entire pull request or decline the whole thing?  Putting all the code
into one pull request is a form of blackmail (commonly used in the
formulation of bills for Congress).  If you want the good you have to
take the bad.


B.) Now let's imagine that Acme forked the code and created a new
repository which they then added to the project.  None of the work you
did is in this new repository.  If those features you implemented were
important to you, you will have to re-introduce them into the new
repository.

You'll have to start from zero learning to work in the new repository.
You also had no say in how that code was developed, so maybe the
feature that you need is unnecessarily difficult to implement in that
repository.   You don't know why things are the way they are there, so
you're walking through a mine field without a map when you're making
changes.

And anyways, why is Acme Corp so certain you had nothing of value to add?

Releasing this code also becomes contentious. Which of the two
competing repositories gets released?  Both of them? How does the
project communicate to users about how these pieces fit together.


C.) Imagine Acme gave you lots of forewarning that this was coming.
You still have no say in how the code is developed.  You know that
anything you might contribute could be obsoleted.  You can't tell
users whether the up-and-coming release will be compatible.  And
what's the point in testing that release?  You don't know how to check
that your needs are being considered in the architecture of the new
code base.

You have no sense of ownership over what comes out of that process.

You see that nobody else outside of Acme is working on the project
either, for the same reasons.


Most contributors would get discouraged and prefer not to participate
if those were the conditions.  If contributors didn't get discouraged,
they would fairly quickly be taking orders from the employees of Acme
Corp.  Acme Corp has all the inside information about what's coming in
a year in the next code dump.  Information is power.  Contributors who
are also users may also chose to stop contributing and become free
riders.  Why not just depend on Acme Corp for all of the development?

What Acme seems to be getting out of this scenario is an Apache
feather.  It's a form of free-riding on Apache's reputation.


Now let's imagine that you are the CTO of another company, let's call
them Kaushal.  Kaushal is considering taking part in this open source
project, but they are a competitor to Acme.  As Kaushal's CTO, you can
see, based on commit history, and participation that Acme is
dominating the project.  You would be smart to expect that Acme would
take advantage of their dominant position in the project.  Acme could
deliberately sabotage Kaushal's use cases, or simply 'starve' them by
convincing people not to help Kaushal.  Kaushal's CTO could respond to
this threat in one of two ways: 1.) Simply not take part on the open
source project.  Create their own closed source thing, or their own
open source project, and not engage.  This is the most likely
response.  2.) Try to dominate the project themselves.  Kaushal has
the same tools available that Acme has. Kaushal's CTO could tell his
employees to do long-interval code drops just like Acme is doing.  Now
with two corporations doing long-interval code drops on a project,
merging the code becomes very very difficult.  Fights about who gets
to decide what could eventually cause a complete cessation of release
activity.


So imagine that all competitors chose to remain just users, and Acme
remains in control.  Now imagine Acme loses interest in the project.
Acme found something that will make them more money, or Acme's
business fails.  Or Acme gets tired of offering their development
resources to the free riders.  Acme stops contributing to the project.
But the project has become so dependent on Acme that it cannot exist
without Acme.  When Acme exits, project activity could end.


Open source projects require transparency, not just as a moral value,
but as a pragmatic prerequisite for collaboration.  Offlist
development damages the community *and* the code.

Best Regards,
Myrle

P.S.  Some very interesting research on the game-theoretical aspects
of modularization in open source:
http://people.hbs.edu/cbaldwin/DR2/BaldwinClark.ArchOS.Jun03.pdf
"Does Code Architecture Mitigate Free Riding in the Open Source
Development Model?"

I would argue that the information divisibility being applied here at
the code modularity dimension also applies to the time dimension.  So,
it seems likely the argument against large code drops can be made
mathematically. Which really tickles the geek in me. : o)

Re: Why are large code drops damaging to a community?

[zeljko leskur <zz...@gmail.com>]

Hi again ! Yes endeed ! As I have spoken I Am Always on dispositon gladly
could help if I would be able to do that. Of course without any obligation.
I could have oportunity learning from peoples who are highly, educated and
skillful on their Job. You can contact me whenever you want. Thank you very
much for paying your attention. Almost forgotten to Said important thing
that you are very pleasant,very smart and  by the way very cute and that is
a great combination.Thanks agan and we Will talking soon again. Wish you
very well.
Dana 18. 10. 2018. 14:25 osoba "Shane Curcuru" <as...@shanecurcuru.org>
napisala je:

> Myrle Krantz wrote on 10/18/18 7:18 AM:
> > Hey all,
> >
> > There are many forms of offlist development.  One form of offlist
> > development is working on large code drops in private and then
> > contributing them all at once.  Threshold size is probably arguable,
> > and varies by project; put that aside for the moment.  I've been
> > working on an explanation of how large code drops damage community and
> > code.  I'd love to hear your feedback.  I'm including my project and
> > the dev@community list in the hopes that people from other projects
> > also have a perspective.  Here it goes:
>
> Thanks Myrle for an excellent writeup, including details of how large
> code drops harm Apache style open communities.  This is a great set of
> examples showing the "why" behind the ASF's requirement that the whole
> community be allowed to participate in *all* parts of a project's
> technical development.
>
> The requirement for an open and consensus-driven development process at
> the ASF doesn't just impact the individual perspective - which you've
> covered in detail in your post.  More importantly, it impacts the whole
> *community* ownership feeling of an Apache project.
>
> This is a key difference I see between Apache projects and
> maintainer-led projects.  There are many examples of maintainer projects
> with issues, successes, whatever.  But they primarily focus on how a
> handful of maintainers are struggling to keep their project growing for
> the benefit of all their users.
>
> The issue is the mindset of "maintainers".  An Apache project should
> have the entire community of contributors feeling like they have a stake
> in the project, that they can help build new ideas, and that they
> *share* as a whole group the direction of the project.  While much of
> the actions are the same, my point is about the feeling of ownership.
> It's not just a few "maintainers" driving things; it's *everyone* who's
> a committer (and hopefully soon, many of the contributors who get voted
> in).
>
> Offlist development prevents this truly shared sense of ownership from
> developing, which is why it's prohibited for Apache projects.
>
> ...snip...
> > Open source projects require transparency, not just as a moral value,
> > but as a pragmatic prerequisite for collaboration.  Offlist
> > development damages the community *and* the code.
>
> More to the point, repeated significant offlist development will attract
> the attention of the ASF board, which may well take direct action to
> prevent that kind of behavior from happening going forward.  One
> advantage of the ASF's independent governance is that it can enforce the
> core open development model that's a key part of the Apache Way.
>
> Note that I'm struggling to find where "offlist development is
> forbidden" is explicitly called out on the apache.org website, so
> pointers to existing places that *explain* this core concept and why
> it's important are appreciated.
>
> My attempt to explain how open development works is here:
>
>   http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24
>
> - Telegraph your intent: email dev@ with your *ideas* ahead of time.
> This allows feedback, encouragement, someone else to point out similar
> code is already over there, etc.
>
> - Draft designs openly.  Put the rough first draft on the
> wiki/website/dev@ list, and then do your edits *in public*.  This allows
> feedback on the architecture as it's being built, and again, gets better
> ideas.  It also allows a sense of community ownership.
>
> - Submit work in chunks (add: on a regular and frequent basis).  Checkin
> the shell of the API.  Then checkin each section of implementation.  If
> you're waiting for your code to look perfect before showing anyone else,
> you're not really helping the community.  Doing the development in the
> open allows for... you guessed it, feedback.
>
> - Welcome feedback along the way.  This doesn't mean you need to accept
> every change request or suggestion.  But it does mean you can take the
> best ideas from the whole community to add them easily, as the
> individual bits of work are being done.
>
> --
>
> - Shane
>   ComDev PMC
>   The Apache Software Foundation
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

Thank you for sharing your experiences with offlist development Alex.

Even single-developer large code drops can be damaging to the
community in some cases.  Malcolm Upayavira gave a good example.  But
single developers may find it easier to remedy the situation in the
same way Upayavira did: by cutting the code up into reviewable-sized
pieces, and courageously accepting community feedback to improve the
code.  Responding constructively to criticism is how we grow, right?
By developing alone for a long period, we deprive ourselves of that
criticism.  Criticism outside of our in-group is all the better
because it raises issues we wouldn't discover on our own.  That's one
of the major strengths of open source.

My goal here is *not* to find the exact shade of grey at which we
prohibit off-list development.  You're right, Alex, almost all
decisions on this topic should be made by the project community and
should be focused on the effect on that project community.  If people
who want to participate are being excluded by a particular development
practice (even if it's a cool hackathon at ApacheCon), that is
something that a community should at very least be aware of and be
ready to mitigate.  Each community can weigh downsides and upsides for
themselves.

My goal here is to spark awareness, by helping to discover and
communicate the spectrum.  An inventory can be a diagnostic tool by
which communities can determine where they are on the spectrum and
decide whether they are comfortable with their current shade of grey.

Best Regards,
Myrle



On Wed, Oct 24, 2018 at 6:42 PM Alex Harui <ah...@adobe.com.invalid> wrote:
>
> This is just a nitpick, but it is the subject line that is bothering me.  Having a "no large code drops" mantra is not the same as a "limit off-list collaboration" mantra which is different from a "no off-list development" mantra.
>
> For Flex, Adobe made something like 5 large code drops.  It simply took so long to clear the various chunks of code being donated to Apache through Adobe's approval process that it was done in stages.  In fact, if I had time, the Flex/Royale community wanted it, and I could get Adobe to support it, I'd bestow a couple of other large code drops to Apache.  It is all pre-existing code, but again, the subject line makes it sound like any large code drop is bad, which is not true.
>
> To say "no off-list development" could be construed as some limit on how many lines of code you can write as an individual before making it available to others to review.
>
> I think the real key here is "off-list collaboration".  There will always be off-list collaboration, and I'll bet some really significant things can happen at a hackathon.  That should not be prohibited.  I think you are simply trying to express the notion that groups, especially groups defined by having a common employer, don't do too much collaboration off-list before inviting others into the conversation.
>
> My 2 cents,
> -Alex
>
> On 10/24/18, 7:17 AM, "Myrle Krantz" <my...@apache.org> wrote:
>
>     Hey all,
>
>     I'd like to invite anyone with relevant positive or negative
>     experiences with off-list development or large code drops to share
>     those experiences.  The ASF policy of "no off-list development" is
>     implemented in a wide variety of ways in various communities, but
>     there are still may be some things that we can agree are absolute
>     no-goes.  I'd like to figure out what things we see as:
>
>     * You might get away with that once, but don't try it twice.  That's
>     damaging to the community.
>     * Avoid doing that unless you have a really good reason that the
>     community has accepted.
>     * That has a bit of a smell to it.  Have you discussed that with your community?
>     * That's fine.  You're helping your community.
>     * What a wonderful idea! Absolutely do that if you and your community want to!
>
>     I'm hoping to put together a diagnostic tool for offlist development
>     that can help communities help themselves.  Something similar to
>     Beck's Depression Inventory.  Because like mental health, community
>     health is complex, and sometimes it is not clearly 'good' or 'bad'.
>
>     In order to do that though, I'd like to read your stories.  Especially
>     from the people who've been around a few projects and seen a few
>     things.  The stories Malcolm, Chris, and Jim already shared are
>     exactly the sort of thing I'm looking for, so if y'all would like to
>     elaborate that'd be really cool too.
>
>     Best Regards,
>     Myrle

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Christofer Dutz <ch...@c-ware.de>]

Hi all,

well of course there will be off-list development, if something new is donated to the ASF. 
I guess Myrle's point is, that for an existing project, features that are communicated and implemented off-list and are brought back after some time is damaging. 
I fully agree with that assumption.

Of course, there is off-list discussion, but the important parts of these events, discussions whatsoever always have to find their way back to the list.
I was driving my co-workers nuts in the PLC4X project, as one of them was sitting directly next to me. When we discussed things I asked him to post a summaries to the list. 
As he didn't do it I usually wrote them for him and after a while started responding with things like: "I will gladly discuss this with you, if you ask it on the list" ... 
I think in the meanwhile he got the hang of it and things are going smoothly.

Also whenever I got important feedback or discussed things outside the project itself, I wrote summaries and posted them on the list, just to keep the others on the loop.

The important thing is: Someone interested in a project should always be able to follow the major directional discussion. 
Off-list collaboration without timely feedback to the list, cuts off others and that's not the Apache way.

Chris



Am 24.10.18, 17:43 schrieb "Alex Harui" <ah...@adobe.com.INVALID>:

    This is just a nitpick, but it is the subject line that is bothering me.  Having a "no large code drops" mantra is not the same as a "limit off-list collaboration" mantra which is different from a "no off-list development" mantra.
    
    For Flex, Adobe made something like 5 large code drops.  It simply took so long to clear the various chunks of code being donated to Apache through Adobe's approval process that it was done in stages.  In fact, if I had time, the Flex/Royale community wanted it, and I could get Adobe to support it, I'd bestow a couple of other large code drops to Apache.  It is all pre-existing code, but again, the subject line makes it sound like any large code drop is bad, which is not true.
    
    To say "no off-list development" could be construed as some limit on how many lines of code you can write as an individual before making it available to others to review.
    
    I think the real key here is "off-list collaboration".  There will always be off-list collaboration, and I'll bet some really significant things can happen at a hackathon.  That should not be prohibited.  I think you are simply trying to express the notion that groups, especially groups defined by having a common employer, don't do too much collaboration off-list before inviting others into the conversation.
    
    My 2 cents,
    -Alex
    
    On 10/24/18, 7:17 AM, "Myrle Krantz" <my...@apache.org> wrote:
    
        Hey all,
        
        I'd like to invite anyone with relevant positive or negative
        experiences with off-list development or large code drops to share
        those experiences.  The ASF policy of "no off-list development" is
        implemented in a wide variety of ways in various communities, but
        there are still may be some things that we can agree are absolute
        no-goes.  I'd like to figure out what things we see as:
        
        * You might get away with that once, but don't try it twice.  That's
        damaging to the community.
        * Avoid doing that unless you have a really good reason that the
        community has accepted.
        * That has a bit of a smell to it.  Have you discussed that with your community?
        * That's fine.  You're helping your community.
        * What a wonderful idea! Absolutely do that if you and your community want to!
        
        I'm hoping to put together a diagnostic tool for offlist development
        that can help communities help themselves.  Something similar to
        Beck's Depression Inventory.  Because like mental health, community
        health is complex, and sometimes it is not clearly 'good' or 'bad'.
        
        In order to do that though, I'd like to read your stories.  Especially
        from the people who've been around a few projects and seen a few
        things.  The stories Malcolm, Chris, and Jim already shared are
        exactly the sort of thing I'm looking for, so if y'all would like to
        elaborate that'd be really cool too.
        
        Best Regards,
        Myrle
        
        On Sat, Oct 20, 2018 at 6:41 AM Myrle Krantz <my...@apache.org> wrote:
        >
        > Hey Jim,
        >
        > I’d say they are a symptom *and* a problem. But putting that aside, can you unroll what you mean please?
        >
        > What was that code drop from SGI a symptom of?
        >
        > What did Robert Thau do (or not do), before during or after to ensure the success of httpd?
        >
        > Best Regards,
        > Myrle
        >
        > On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:
        >>
        >> I would say that, in general, large code drops are more a *symptom* of a problem, rather than a problem, in and of itself...
        >>
        >> > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID> wrote:
        >> >
        >> > IMO, the issue isn't about large code drops.  Some will be ok.
        >> >
        >> > The issue is about significant collaboration off-list about anything, not just code.
        >> >
        >> > My 2 cents,
        >> > -Alex
        >> >
        >> > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
        >> >
        >> >    +1 on this civil discourse.
        >> >
        >> >    I would like to offer that sometimes large code drops are unavoidable and
        >> >    necessary.  Jim's explanation of httpd contribution of type 1 is a good
        >> >    example.
        >> >
        >> >    I think we would find that many projects started with a large code drop
        >> >    (maybe more than one) - a sufficient amount of code - to get a project
        >> >    started.  When projects are young it would be normal and expected for this
        >> >    to happen. It quickly gets a community to a "thing" that can be added to.
        >> >
        >> >    It obviously depends on the kinds of components, tools, frameworks, etc
        >> >    that are being developed. Game theory is quite apropos - you need a
        >> >    sufficient incentive for *timely* collaboration, of hanging together.
        >> >
        >> >    Further, if your "thing" is going to be used directly in market (i.e. with
        >> >    very little of a product wrapper ), then there is a strong *disincentive*
        >> >    to share back the latest and greatest. The further from market immediacy
        >> >    the easier it is to contribute. Both the Collaboration space and
        >> >    Competitive space are clearly delineated, whereas in a close to market
        >> >    immediacy situation you have too much overlap and therefore a built in
        >> >    delay of code contribution to preserve market competitiveness.
        >> >
        >> >    So, combining the "sufficient code to attract contribution" metric with the
        >> >    market-immediacy metric and you can predict engagement by outside vendors
        >> >    (or their contributors) in a project. In such a situation, it is better, in
        >> >    my view, to accept any and all branched code even if it is dev'd off-list.
        >> >    This allows for inspection/ code examination and further exploration - at a
        >> >    minimum.  Accepting on a branch is neither the same as accepting for
        >> >    release, nor merging to master branch.
        >> >
        >> >    Now, the assumption that the code is better than what the community has
        >> >    developed has to be challenged.  It could be that the branched code should
        >> >    be judged only on the merits of the code (is it better and more complete),
        >> >    or it could be judged on the basis that it "breaks the current build".
        >> >    There can be a culture of a project to accept such code drops with the
        >> >    caveat that if the merges cannot be done by the submitting group, then the
        >> >    project will have a resistance to such submissions (you break it, you fix
        >> >    it), or alternatively that there will be a small group of people that are
        >> >    sourced from such delayed-contribution types - that work on doing the
        >> >    merges.  The key seems to be to create the incentive to share code before
        >> >    others do, to avoid being the one that breaks the build.
        >> >
        >> >    ~jdailey67
        >> >
        >> >
        >> >
        >> >
        >> >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
        >> >
        >> >> Large code drops are almost always damaging, since inherent in that
        >> >> process is the concept of "throwing the code over a wall". But sometimes it
        >> >> does work out, assuming that continuity and "good intentions" are followed.
        >> >>
        >> >> To show this, join me in the Wayback Machine as Sherman and I travel to
        >> >> the year 1995...
        >> >>
        >> >> This is right around the start of Apache, back when Apache meant the web
        >> >> server, and at the time, the project was basically what was left of the
        >> >> NCSA web server plus some patches and bug fixes... Around this time, one of
        >> >> the core group, Robert Thau, started independent work on a re-architecture
        >> >> of the server, which he code-named "Shambala". It was basically a single
        >> >> contributor effort (himself). One day he simply said to the group, "Here, I
        >> >> have this new design and architecture for Apache. It adds a lot of
        >> >> features." So much of what defines httpd today can find its origin right
        >> >> there: modular framework, pools, preforking (and, as such, the initial
        >> >> gleaming towards MPMs), extendable API, etc...
        >> >>
        >> >> In many ways, this was a large code drop. What made it different is that
        >> >> there was *support* by the author and the community to work on integrating
        >> >> it into the whole. It became, basically, a community effort.
        >> >>
        >> >> Now compare that with a different scenario... Once httpd had picked up
        >> >> steam, and making sure that it was ported to everyone's favorite *nix
        >> >> flavor was important, SGI had done work on a set of patches that ported
        >> >> httpd to their OS and provided these patches (a set of 10 very large
        >> >> patch-files, iirc) to the group. What was clear in those patches is that
        >> >> there was no consideration at all on how those patches affected or broke
        >> >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
        >> >> totally destroying any sort of portability for anyone else. And when we
        >> >> responded by, asking for more information, help with chatting with their
        >> >> developers to try to figure things out, and basically trying to figure out
        >> >> how to use and merge this stuff, SGI was basically just silent. They sent
        >> >> it to us and that was the beginning and the end of their involvement as far
        >> >> as they were concerned.[1]
        >> >>
        >> >> Way, way too many large code drops are the latter. Hardly any are the
        >> >> former.
        >> >>
        >> >>
        >> >> 1. I have paraphrased both the Shambala and SGI events
        >> >
        >> >
        >>
        >>
        >> ---------------------------------------------------------------------
        >> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
        >> For additional commands, e-mail: dev-help@community.apache.org
        >>
        
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
        For additional commands, e-mail: dev-help@community.apache.org
        
        
    
    ?B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB�?�?[��X��ܚX�K??K[XZ[?�??]�][��X��ܚX�P?��[][�]?K�\?X�?K�ܙ�B��܈?Y??]?[ۘ[??��[X[�?�??K[XZ[?�??]�Z?[???��[][�]?K�\?X�?K�ܙ�B�B


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Alex Harui <ah...@adobe.com.INVALID>]

This is just a nitpick, but it is the subject line that is bothering me.  Having a "no large code drops" mantra is not the same as a "limit off-list collaboration" mantra which is different from a "no off-list development" mantra.

For Flex, Adobe made something like 5 large code drops.  It simply took so long to clear the various chunks of code being donated to Apache through Adobe's approval process that it was done in stages.  In fact, if I had time, the Flex/Royale community wanted it, and I could get Adobe to support it, I'd bestow a couple of other large code drops to Apache.  It is all pre-existing code, but again, the subject line makes it sound like any large code drop is bad, which is not true.

To say "no off-list development" could be construed as some limit on how many lines of code you can write as an individual before making it available to others to review.

I think the real key here is "off-list collaboration".  There will always be off-list collaboration, and I'll bet some really significant things can happen at a hackathon.  That should not be prohibited.  I think you are simply trying to express the notion that groups, especially groups defined by having a common employer, don't do too much collaboration off-list before inviting others into the conversation.

My 2 cents,
-Alex

On 10/24/18, 7:17 AM, "Myrle Krantz" <my...@apache.org> wrote:

    Hey all,
    
    I'd like to invite anyone with relevant positive or negative
    experiences with off-list development or large code drops to share
    those experiences.  The ASF policy of "no off-list development" is
    implemented in a wide variety of ways in various communities, but
    there are still may be some things that we can agree are absolute
    no-goes.  I'd like to figure out what things we see as:
    
    * You might get away with that once, but don't try it twice.  That's
    damaging to the community.
    * Avoid doing that unless you have a really good reason that the
    community has accepted.
    * That has a bit of a smell to it.  Have you discussed that with your community?
    * That's fine.  You're helping your community.
    * What a wonderful idea! Absolutely do that if you and your community want to!
    
    I'm hoping to put together a diagnostic tool for offlist development
    that can help communities help themselves.  Something similar to
    Beck's Depression Inventory.  Because like mental health, community
    health is complex, and sometimes it is not clearly 'good' or 'bad'.
    
    In order to do that though, I'd like to read your stories.  Especially
    from the people who've been around a few projects and seen a few
    things.  The stories Malcolm, Chris, and Jim already shared are
    exactly the sort of thing I'm looking for, so if y'all would like to
    elaborate that'd be really cool too.
    
    Best Regards,
    Myrle
    
    On Sat, Oct 20, 2018 at 6:41 AM Myrle Krantz <my...@apache.org> wrote:
    >
    > Hey Jim,
    >
    > I’d say they are a symptom *and* a problem. But putting that aside, can you unroll what you mean please?
    >
    > What was that code drop from SGI a symptom of?
    >
    > What did Robert Thau do (or not do), before during or after to ensure the success of httpd?
    >
    > Best Regards,
    > Myrle
    >
    > On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:
    >>
    >> I would say that, in general, large code drops are more a *symptom* of a problem, rather than a problem, in and of itself...
    >>
    >> > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID> wrote:
    >> >
    >> > IMO, the issue isn't about large code drops.  Some will be ok.
    >> >
    >> > The issue is about significant collaboration off-list about anything, not just code.
    >> >
    >> > My 2 cents,
    >> > -Alex
    >> >
    >> > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
    >> >
    >> >    +1 on this civil discourse.
    >> >
    >> >    I would like to offer that sometimes large code drops are unavoidable and
    >> >    necessary.  Jim's explanation of httpd contribution of type 1 is a good
    >> >    example.
    >> >
    >> >    I think we would find that many projects started with a large code drop
    >> >    (maybe more than one) - a sufficient amount of code - to get a project
    >> >    started.  When projects are young it would be normal and expected for this
    >> >    to happen. It quickly gets a community to a "thing" that can be added to.
    >> >
    >> >    It obviously depends on the kinds of components, tools, frameworks, etc
    >> >    that are being developed. Game theory is quite apropos - you need a
    >> >    sufficient incentive for *timely* collaboration, of hanging together.
    >> >
    >> >    Further, if your "thing" is going to be used directly in market (i.e. with
    >> >    very little of a product wrapper ), then there is a strong *disincentive*
    >> >    to share back the latest and greatest. The further from market immediacy
    >> >    the easier it is to contribute. Both the Collaboration space and
    >> >    Competitive space are clearly delineated, whereas in a close to market
    >> >    immediacy situation you have too much overlap and therefore a built in
    >> >    delay of code contribution to preserve market competitiveness.
    >> >
    >> >    So, combining the "sufficient code to attract contribution" metric with the
    >> >    market-immediacy metric and you can predict engagement by outside vendors
    >> >    (or their contributors) in a project. In such a situation, it is better, in
    >> >    my view, to accept any and all branched code even if it is dev'd off-list.
    >> >    This allows for inspection/ code examination and further exploration - at a
    >> >    minimum.  Accepting on a branch is neither the same as accepting for
    >> >    release, nor merging to master branch.
    >> >
    >> >    Now, the assumption that the code is better than what the community has
    >> >    developed has to be challenged.  It could be that the branched code should
    >> >    be judged only on the merits of the code (is it better and more complete),
    >> >    or it could be judged on the basis that it "breaks the current build".
    >> >    There can be a culture of a project to accept such code drops with the
    >> >    caveat that if the merges cannot be done by the submitting group, then the
    >> >    project will have a resistance to such submissions (you break it, you fix
    >> >    it), or alternatively that there will be a small group of people that are
    >> >    sourced from such delayed-contribution types - that work on doing the
    >> >    merges.  The key seems to be to create the incentive to share code before
    >> >    others do, to avoid being the one that breaks the build.
    >> >
    >> >    ~jdailey67
    >> >
    >> >
    >> >
    >> >
    >> >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
    >> >
    >> >> Large code drops are almost always damaging, since inherent in that
    >> >> process is the concept of "throwing the code over a wall". But sometimes it
    >> >> does work out, assuming that continuity and "good intentions" are followed.
    >> >>
    >> >> To show this, join me in the Wayback Machine as Sherman and I travel to
    >> >> the year 1995...
    >> >>
    >> >> This is right around the start of Apache, back when Apache meant the web
    >> >> server, and at the time, the project was basically what was left of the
    >> >> NCSA web server plus some patches and bug fixes... Around this time, one of
    >> >> the core group, Robert Thau, started independent work on a re-architecture
    >> >> of the server, which he code-named "Shambala". It was basically a single
    >> >> contributor effort (himself). One day he simply said to the group, "Here, I
    >> >> have this new design and architecture for Apache. It adds a lot of
    >> >> features." So much of what defines httpd today can find its origin right
    >> >> there: modular framework, pools, preforking (and, as such, the initial
    >> >> gleaming towards MPMs), extendable API, etc...
    >> >>
    >> >> In many ways, this was a large code drop. What made it different is that
    >> >> there was *support* by the author and the community to work on integrating
    >> >> it into the whole. It became, basically, a community effort.
    >> >>
    >> >> Now compare that with a different scenario... Once httpd had picked up
    >> >> steam, and making sure that it was ported to everyone's favorite *nix
    >> >> flavor was important, SGI had done work on a set of patches that ported
    >> >> httpd to their OS and provided these patches (a set of 10 very large
    >> >> patch-files, iirc) to the group. What was clear in those patches is that
    >> >> there was no consideration at all on how those patches affected or broke
    >> >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
    >> >> totally destroying any sort of portability for anyone else. And when we
    >> >> responded by, asking for more information, help with chatting with their
    >> >> developers to try to figure things out, and basically trying to figure out
    >> >> how to use and merge this stuff, SGI was basically just silent. They sent
    >> >> it to us and that was the beginning and the end of their involvement as far
    >> >> as they were concerned.[1]
    >> >>
    >> >> Way, way too many large code drops are the latter. Hardly any are the
    >> >> former.
    >> >>
    >> >>
    >> >> 1. I have paraphrased both the Shambala and SGI events
    >> >
    >> >
    >>
    >>
    >> ---------------------------------------------------------------------
    >> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
    >> For additional commands, e-mail: dev-help@community.apache.org
    >>
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
    For additional commands, e-mail: dev-help@community.apache.org
    
    

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

Hey all,

I'd like to invite anyone with relevant positive or negative
experiences with off-list development or large code drops to share
those experiences.  The ASF policy of "no off-list development" is
implemented in a wide variety of ways in various communities, but
there are still may be some things that we can agree are absolute
no-goes.  I'd like to figure out what things we see as:

* You might get away with that once, but don't try it twice.  That's
damaging to the community.
* Avoid doing that unless you have a really good reason that the
community has accepted.
* That has a bit of a smell to it.  Have you discussed that with your community?
* That's fine.  You're helping your community.
* What a wonderful idea! Absolutely do that if you and your community want to!

I'm hoping to put together a diagnostic tool for offlist development
that can help communities help themselves.  Something similar to
Beck's Depression Inventory.  Because like mental health, community
health is complex, and sometimes it is not clearly 'good' or 'bad'.

In order to do that though, I'd like to read your stories.  Especially
from the people who've been around a few projects and seen a few
things.  The stories Malcolm, Chris, and Jim already shared are
exactly the sort of thing I'm looking for, so if y'all would like to
elaborate that'd be really cool too.

Best Regards,
Myrle

On Sat, Oct 20, 2018 at 6:41 AM Myrle Krantz <my...@apache.org> wrote:
>
> Hey Jim,
>
> I’d say they are a symptom *and* a problem. But putting that aside, can you unroll what you mean please?
>
> What was that code drop from SGI a symptom of?
>
> What did Robert Thau do (or not do), before during or after to ensure the success of httpd?
>
> Best Regards,
> Myrle
>
> On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:
>>
>> I would say that, in general, large code drops are more a *symptom* of a problem, rather than a problem, in and of itself...
>>
>> > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID> wrote:
>> >
>> > IMO, the issue isn't about large code drops.  Some will be ok.
>> >
>> > The issue is about significant collaboration off-list about anything, not just code.
>> >
>> > My 2 cents,
>> > -Alex
>> >
>> > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
>> >
>> >    +1 on this civil discourse.
>> >
>> >    I would like to offer that sometimes large code drops are unavoidable and
>> >    necessary.  Jim's explanation of httpd contribution of type 1 is a good
>> >    example.
>> >
>> >    I think we would find that many projects started with a large code drop
>> >    (maybe more than one) - a sufficient amount of code - to get a project
>> >    started.  When projects are young it would be normal and expected for this
>> >    to happen. It quickly gets a community to a "thing" that can be added to.
>> >
>> >    It obviously depends on the kinds of components, tools, frameworks, etc
>> >    that are being developed. Game theory is quite apropos - you need a
>> >    sufficient incentive for *timely* collaboration, of hanging together.
>> >
>> >    Further, if your "thing" is going to be used directly in market (i.e. with
>> >    very little of a product wrapper ), then there is a strong *disincentive*
>> >    to share back the latest and greatest. The further from market immediacy
>> >    the easier it is to contribute. Both the Collaboration space and
>> >    Competitive space are clearly delineated, whereas in a close to market
>> >    immediacy situation you have too much overlap and therefore a built in
>> >    delay of code contribution to preserve market competitiveness.
>> >
>> >    So, combining the "sufficient code to attract contribution" metric with the
>> >    market-immediacy metric and you can predict engagement by outside vendors
>> >    (or their contributors) in a project. In such a situation, it is better, in
>> >    my view, to accept any and all branched code even if it is dev'd off-list.
>> >    This allows for inspection/ code examination and further exploration - at a
>> >    minimum.  Accepting on a branch is neither the same as accepting for
>> >    release, nor merging to master branch.
>> >
>> >    Now, the assumption that the code is better than what the community has
>> >    developed has to be challenged.  It could be that the branched code should
>> >    be judged only on the merits of the code (is it better and more complete),
>> >    or it could be judged on the basis that it "breaks the current build".
>> >    There can be a culture of a project to accept such code drops with the
>> >    caveat that if the merges cannot be done by the submitting group, then the
>> >    project will have a resistance to such submissions (you break it, you fix
>> >    it), or alternatively that there will be a small group of people that are
>> >    sourced from such delayed-contribution types - that work on doing the
>> >    merges.  The key seems to be to create the incentive to share code before
>> >    others do, to avoid being the one that breaks the build.
>> >
>> >    ~jdailey67
>> >
>> >
>> >
>> >
>> >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
>> >
>> >> Large code drops are almost always damaging, since inherent in that
>> >> process is the concept of "throwing the code over a wall". But sometimes it
>> >> does work out, assuming that continuity and "good intentions" are followed.
>> >>
>> >> To show this, join me in the Wayback Machine as Sherman and I travel to
>> >> the year 1995...
>> >>
>> >> This is right around the start of Apache, back when Apache meant the web
>> >> server, and at the time, the project was basically what was left of the
>> >> NCSA web server plus some patches and bug fixes... Around this time, one of
>> >> the core group, Robert Thau, started independent work on a re-architecture
>> >> of the server, which he code-named "Shambala". It was basically a single
>> >> contributor effort (himself). One day he simply said to the group, "Here, I
>> >> have this new design and architecture for Apache. It adds a lot of
>> >> features." So much of what defines httpd today can find its origin right
>> >> there: modular framework, pools, preforking (and, as such, the initial
>> >> gleaming towards MPMs), extendable API, etc...
>> >>
>> >> In many ways, this was a large code drop. What made it different is that
>> >> there was *support* by the author and the community to work on integrating
>> >> it into the whole. It became, basically, a community effort.
>> >>
>> >> Now compare that with a different scenario... Once httpd had picked up
>> >> steam, and making sure that it was ported to everyone's favorite *nix
>> >> flavor was important, SGI had done work on a set of patches that ported
>> >> httpd to their OS and provided these patches (a set of 10 very large
>> >> patch-files, iirc) to the group. What was clear in those patches is that
>> >> there was no consideration at all on how those patches affected or broke
>> >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
>> >> totally destroying any sort of portability for anyone else. And when we
>> >> responded by, asking for more information, help with chatting with their
>> >> developers to try to figure things out, and basically trying to figure out
>> >> how to use and merge this stuff, SGI was basically just silent. They sent
>> >> it to us and that was the beginning and the end of their involvement as far
>> >> as they were concerned.[1]
>> >>
>> >> Way, way too many large code drops are the latter. Hardly any are the
>> >> former.
>> >>
>> >>
>> >> 1. I have paraphrased both the Shambala and SGI events
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
>> For additional commands, e-mail: dev-help@community.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Fwd: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

(Chris accidentally left out the fineract dev list here, but this is
definitely relevant to the topic at hand.)

Best Regards,
Myrle

---------- Forwarded message ---------
From: Christofer Dutz <ch...@c-ware.de>
Date: Sat, Oct 20, 2018 at 8:17 PM
Subject: Re: Why are large code drops damaging to a community?
To: dev@community.apache.org <de...@community.apache.org>


Trying to keep this short ...

Well I think it's important to keep the others in the loop no matter
what it's about. Especially if it's about code.

The probably most extreme example is the Edgent incubating project.
After IBM pulling the plug, it's sort of only me still being involved.
So right now I have a project, I know 100% of the build, but about 10%
of the code. Now people are asking me questions, I just can't answer,
which is damaging. From my point of view, it feels like a huge code
drop (even if I know it wasn't).

My 2,5 cents,

Chris

Outlook for Android<https://aka.ms/ghei36> herunterladen

From: mike peoni <ma...@gmail.com>
Sent: Saturday, October 20, 2018 7:01:47 AM
To: dev@community.apache.org
Subject: Re: Why are large code drops damaging to a community?

In talking about https  MyOpj.VisibleEh would this be appropriate for a
large Apl so you wouldn't raise flags and further more i put 2 Apl on face
book just anonymized on my page then they put up a developers page and i
liked it and i went to close my
account and my Apl where there
In the dev, page and i was told
I would forfit my data I'm deeply
concerned because they tried to issue a visa for a substantial amount but
not what the Apl
worth they have been negotiating as of now i don't
know where i stand "please comment as to the dev, page
Apache is neg, on fees. As far
as big data goes if th framework was done right there shouldn't be a
problem with it intergrating unless it's branched invasively.

On Sat, Oct 20, 2018, 12:41 AM Myrle Krantz <my...@apache.org> wrote:

> Hey Jim,
>
> I’d say they are a symptom *and* a problem. But putting that aside, can you
> unroll what you mean please?
>
> What was that code drop from SGI a symptom of?
>
> What did Robert Thau do (or not do), before during or after to ensure the
> success of httpd?
>
> Best Regards,
> Myrle
>
> On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:
>
> > I would say that, in general, large code drops are more a *symptom* of a
> > problem, rather than a problem, in and of itself...
> >
> > > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID>
> > wrote:
> > >
> > > IMO, the issue isn't about large code drops.  Some will be ok.
> > >
> > > The issue is about significant collaboration off-list about anything,
> > not just code.
> > >
> > > My 2 cents,
> > > -Alex
> > >
> > > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> > >
> > >    +1 on this civil discourse.
> > >
> > >    I would like to offer that sometimes large code drops are
> unavoidable
> > and
> > >    necessary.  Jim's explanation of httpd contribution of type 1 is a
> > good
> > >    example.
> > >
> > >    I think we would find that many projects started with a large code
> > drop
> > >    (maybe more than one) - a sufficient amount of code - to get a
> project
> > >    started.  When projects are young it would be normal and expected
> for
> > this
> > >    to happen. It quickly gets a community to a "thing" that can be
> added
> > to.
> > >
> > >    It obviously depends on the kinds of components, tools, frameworks,
> > etc
> > >    that are being developed. Game theory is quite apropos - you need a
> > >    sufficient incentive for *timely* collaboration, of hanging
> together.
> > >
> > >    Further, if your "thing" is going to be used directly in market
> (i.e.
> > with
> > >    very little of a product wrapper ), then there is a strong
> > *disincentive*
> > >    to share back the latest and greatest. The further from market
> > immediacy
> > >    the easier it is to contribute. Both the Collaboration space and
> > >    Competitive space are clearly delineated, whereas in a close to
> market
> > >    immediacy situation you have too much overlap and therefore a built
> in
> > >    delay of code contribution to preserve market competitiveness.
> > >
> > >    So, combining the "sufficient code to attract contribution" metric
> > with the
> > >    market-immediacy metric and you can predict engagement by outside
> > vendors
> > >    (or their contributors) in a project. In such a situation, it is
> > better, in
> > >    my view, to accept any and all branched code even if it is dev'd
> > off-list.
> > >    This allows for inspection/ code examination and further exploration
> > - at a
> > >    minimum.  Accepting on a branch is neither the same as accepting for
> > >    release, nor merging to master branch.
> > >
> > >    Now, the assumption that the code is better than what the community
> > has
> > >    developed has to be challenged.  It could be that the branched code
> > should
> > >    be judged only on the merits of the code (is it better and more
> > complete),
> > >    or it could be judged on the basis that it "breaks the current
> build".
> > >    There can be a culture of a project to accept such code drops with
> the
> > >    caveat that if the merges cannot be done by the submitting group,
> > then the
> > >    project will have a resistance to such submissions (you break it,
> you
> > fix
> > >    it), or alternatively that there will be a small group of people
> that
> > are
> > >    sourced from such delayed-contribution types - that work on doing
> the
> > >    merges.  The key seems to be to create the incentive to share code
> > before
> > >    others do, to avoid being the one that breaks the build.
> > >
> > >    ~jdailey67
> > >
> > >
> > >
> > >
> > >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com>
> > wrote:
> > >
> > >> Large code drops are almost always damaging, since inherent in that
> > >> process is the concept of "throwing the code over a wall". But
> > sometimes it
> > >> does work out, assuming that continuity and "good intentions" are
> > followed.
> > >>
> > >> To show this, join me in the Wayback Machine as Sherman and I travel
> to
> > >> the year 1995...
> > >>
> > >> This is right around the start of Apache, back when Apache meant the
> web
> > >> server, and at the time, the project was basically what was left of
> the
> > >> NCSA web server plus some patches and bug fixes... Around this time,
> > one of
> > >> the core group, Robert Thau, started independent work on a
> > re-architecture
> > >> of the server, which he code-named "Shambala". It was basically a
> single
> > >> contributor effort (himself). One day he simply said to the group,
> > "Here, I
> > >> have this new design and architecture for Apache. It adds a lot of
> > >> features." So much of what defines httpd today can find its origin
> right
> > >> there: modular framework, pools, preforking (and, as such, the initial
> > >> gleaming towards MPMs), extendable API, etc...
> > >>
> > >> In many ways, this was a large code drop. What made it different is
> that
> > >> there was *support* by the author and the community to work on
> > integrating
> > >> it into the whole. It became, basically, a community effort.
> > >>
> > >> Now compare that with a different scenario... Once httpd had picked up
> > >> steam, and making sure that it was ported to everyone's favorite *nix
> > >> flavor was important, SGI had done work on a set of patches that
> ported
> > >> httpd to their OS and provided these patches (a set of 10 very large
> > >> patch-files, iirc) to the group. What was clear in those patches is
> that
> > >> there was no consideration at all on how those patches affected or
> broke
> > >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> > >> totally destroying any sort of portability for anyone else. And when
> we
> > >> responded by, asking for more information, help with chatting with
> their
> > >> developers to try to figure things out, and basically trying to figure
> > out
> > >> how to use and merge this stuff, SGI was basically just silent. They
> > sent
> > >> it to us and that was the beginning and the end of their involvement
> as
> > far
> > >> as they were concerned.[1]
> > >>
> > >> Way, way too many large code drops are the latter. Hardly any are the
> > >> former.
> > >>
> > >>
> > >> 1. I have paraphrased both the Shambala and SGI events
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> > For additional commands, e-mail: dev-help@community.apache.org
> >
> >
>

Outlook for Android<https://aka.ms/ghei36> herunterladen

Re: Why are large code drops damaging to a community?

[Christofer Dutz <ch...@c-ware.de>]

Trying to keep this short ...

Well I think it's important to keep the others in the loop no matter what it's about. Especially if it's about code.

The probably most extreme example is the Edgent incubating project. After IBM pulling the plug, it's sort of only me still being involved. So right now I have a project, I know 100% of the build, but about 10% of the code. Now people are asking me questions, I just can't answer, which is damaging. From my point of view, it feels like a huge code drop (even if I know it wasn't).

My 2,5 cents,

Chris

Outlook for Android<https://aka.ms/ghei36> herunterladen

From: mike peoni <ma...@gmail.com>
Sent: Saturday, October 20, 2018 7:01:47 AM
To: dev@community.apache.org
Subject: Re: Why are large code drops damaging to a community?

In talking about https  MyOpj.VisibleEh would this be appropriate for a
large Apl so you wouldn't raise flags and further more i put 2 Apl on face
book just anonymized on my page then they put up a developers page and i
liked it and i went to close my
account and my Apl where there
In the dev, page and i was told
I would forfit my data I'm deeply
concerned because they tried to issue a visa for a substantial amount but
not what the Apl
worth they have been negotiating as of now i don't
know where i stand "please comment as to the dev, page
Apache is neg, on fees. As far
as big data goes if th framework was done right there shouldn't be a
problem with it intergrating unless it's branched invasively.

On Sat, Oct 20, 2018, 12:41 AM Myrle Krantz <my...@apache.org> wrote:

> Hey Jim,
>
> I’d say they are a symptom *and* a problem. But putting that aside, can you
> unroll what you mean please?
>
> What was that code drop from SGI a symptom of?
>
> What did Robert Thau do (or not do), before during or after to ensure the
> success of httpd?
>
> Best Regards,
> Myrle
>
> On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:
>
> > I would say that, in general, large code drops are more a *symptom* of a
> > problem, rather than a problem, in and of itself...
> >
> > > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID>
> > wrote:
> > >
> > > IMO, the issue isn't about large code drops.  Some will be ok.
> > >
> > > The issue is about significant collaboration off-list about anything,
> > not just code.
> > >
> > > My 2 cents,
> > > -Alex
> > >
> > > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> > >
> > >    +1 on this civil discourse.
> > >
> > >    I would like to offer that sometimes large code drops are
> unavoidable
> > and
> > >    necessary.  Jim's explanation of httpd contribution of type 1 is a
> > good
> > >    example.
> > >
> > >    I think we would find that many projects started with a large code
> > drop
> > >    (maybe more than one) - a sufficient amount of code - to get a
> project
> > >    started.  When projects are young it would be normal and expected
> for
> > this
> > >    to happen. It quickly gets a community to a "thing" that can be
> added
> > to.
> > >
> > >    It obviously depends on the kinds of components, tools, frameworks,
> > etc
> > >    that are being developed. Game theory is quite apropos - you need a
> > >    sufficient incentive for *timely* collaboration, of hanging
> together.
> > >
> > >    Further, if your "thing" is going to be used directly in market
> (i.e.
> > with
> > >    very little of a product wrapper ), then there is a strong
> > *disincentive*
> > >    to share back the latest and greatest. The further from market
> > immediacy
> > >    the easier it is to contribute. Both the Collaboration space and
> > >    Competitive space are clearly delineated, whereas in a close to
> market
> > >    immediacy situation you have too much overlap and therefore a built
> in
> > >    delay of code contribution to preserve market competitiveness.
> > >
> > >    So, combining the "sufficient code to attract contribution" metric
> > with the
> > >    market-immediacy metric and you can predict engagement by outside
> > vendors
> > >    (or their contributors) in a project. In such a situation, it is
> > better, in
> > >    my view, to accept any and all branched code even if it is dev'd
> > off-list.
> > >    This allows for inspection/ code examination and further exploration
> > - at a
> > >    minimum.  Accepting on a branch is neither the same as accepting for
> > >    release, nor merging to master branch.
> > >
> > >    Now, the assumption that the code is better than what the community
> > has
> > >    developed has to be challenged.  It could be that the branched code
> > should
> > >    be judged only on the merits of the code (is it better and more
> > complete),
> > >    or it could be judged on the basis that it "breaks the current
> build".
> > >    There can be a culture of a project to accept such code drops with
> the
> > >    caveat that if the merges cannot be done by the submitting group,
> > then the
> > >    project will have a resistance to such submissions (you break it,
> you
> > fix
> > >    it), or alternatively that there will be a small group of people
> that
> > are
> > >    sourced from such delayed-contribution types - that work on doing
> the
> > >    merges.  The key seems to be to create the incentive to share code
> > before
> > >    others do, to avoid being the one that breaks the build.
> > >
> > >    ~jdailey67
> > >
> > >
> > >
> > >
> > >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com>
> > wrote:
> > >
> > >> Large code drops are almost always damaging, since inherent in that
> > >> process is the concept of "throwing the code over a wall". But
> > sometimes it
> > >> does work out, assuming that continuity and "good intentions" are
> > followed.
> > >>
> > >> To show this, join me in the Wayback Machine as Sherman and I travel
> to
> > >> the year 1995...
> > >>
> > >> This is right around the start of Apache, back when Apache meant the
> web
> > >> server, and at the time, the project was basically what was left of
> the
> > >> NCSA web server plus some patches and bug fixes... Around this time,
> > one of
> > >> the core group, Robert Thau, started independent work on a
> > re-architecture
> > >> of the server, which he code-named "Shambala". It was basically a
> single
> > >> contributor effort (himself). One day he simply said to the group,
> > "Here, I
> > >> have this new design and architecture for Apache. It adds a lot of
> > >> features." So much of what defines httpd today can find its origin
> right
> > >> there: modular framework, pools, preforking (and, as such, the initial
> > >> gleaming towards MPMs), extendable API, etc...
> > >>
> > >> In many ways, this was a large code drop. What made it different is
> that
> > >> there was *support* by the author and the community to work on
> > integrating
> > >> it into the whole. It became, basically, a community effort.
> > >>
> > >> Now compare that with a different scenario... Once httpd had picked up
> > >> steam, and making sure that it was ported to everyone's favorite *nix
> > >> flavor was important, SGI had done work on a set of patches that
> ported
> > >> httpd to their OS and provided these patches (a set of 10 very large
> > >> patch-files, iirc) to the group. What was clear in those patches is
> that
> > >> there was no consideration at all on how those patches affected or
> broke
> > >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> > >> totally destroying any sort of portability for anyone else. And when
> we
> > >> responded by, asking for more information, help with chatting with
> their
> > >> developers to try to figure things out, and basically trying to figure
> > out
> > >> how to use and merge this stuff, SGI was basically just silent. They
> > sent
> > >> it to us and that was the beginning and the end of their involvement
> as
> > far
> > >> as they were concerned.[1]
> > >>
> > >> Way, way too many large code drops are the latter. Hardly any are the
> > >> former.
> > >>
> > >>
> > >> 1. I have paraphrased both the Shambala and SGI events
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> > For additional commands, e-mail: dev-help@community.apache.org
> >
> >
>

Outlook for Android<https://aka.ms/ghei36> herunterladen

Re: Why are large code drops damaging to a community?

[mike peoni <ma...@gmail.com>]

In talking about https  MyOpj.VisibleEh would this be appropriate for a
large Apl so you wouldn't raise flags and further more i put 2 Apl on face
book just anonymized on my page then they put up a developers page and i
liked it and i went to close my
account and my Apl where there
In the dev, page and i was told
I would forfit my data I'm deeply
concerned because they tried to issue a visa for a substantial amount but
not what the Apl
worth they have been negotiating as of now i don't
know where i stand "please comment as to the dev, page
Apache is neg, on fees. As far
as big data goes if th framework was done right there shouldn't be a
problem with it intergrating unless it's branched invasively.

On Sat, Oct 20, 2018, 12:41 AM Myrle Krantz <my...@apache.org> wrote:

> Hey Jim,
>
> I’d say they are a symptom *and* a problem. But putting that aside, can you
> unroll what you mean please?
>
> What was that code drop from SGI a symptom of?
>
> What did Robert Thau do (or not do), before during or after to ensure the
> success of httpd?
>
> Best Regards,
> Myrle
>
> On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:
>
> > I would say that, in general, large code drops are more a *symptom* of a
> > problem, rather than a problem, in and of itself...
> >
> > > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID>
> > wrote:
> > >
> > > IMO, the issue isn't about large code drops.  Some will be ok.
> > >
> > > The issue is about significant collaboration off-list about anything,
> > not just code.
> > >
> > > My 2 cents,
> > > -Alex
> > >
> > > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> > >
> > >    +1 on this civil discourse.
> > >
> > >    I would like to offer that sometimes large code drops are
> unavoidable
> > and
> > >    necessary.  Jim's explanation of httpd contribution of type 1 is a
> > good
> > >    example.
> > >
> > >    I think we would find that many projects started with a large code
> > drop
> > >    (maybe more than one) - a sufficient amount of code - to get a
> project
> > >    started.  When projects are young it would be normal and expected
> for
> > this
> > >    to happen. It quickly gets a community to a "thing" that can be
> added
> > to.
> > >
> > >    It obviously depends on the kinds of components, tools, frameworks,
> > etc
> > >    that are being developed. Game theory is quite apropos - you need a
> > >    sufficient incentive for *timely* collaboration, of hanging
> together.
> > >
> > >    Further, if your "thing" is going to be used directly in market
> (i.e.
> > with
> > >    very little of a product wrapper ), then there is a strong
> > *disincentive*
> > >    to share back the latest and greatest. The further from market
> > immediacy
> > >    the easier it is to contribute. Both the Collaboration space and
> > >    Competitive space are clearly delineated, whereas in a close to
> market
> > >    immediacy situation you have too much overlap and therefore a built
> in
> > >    delay of code contribution to preserve market competitiveness.
> > >
> > >    So, combining the "sufficient code to attract contribution" metric
> > with the
> > >    market-immediacy metric and you can predict engagement by outside
> > vendors
> > >    (or their contributors) in a project. In such a situation, it is
> > better, in
> > >    my view, to accept any and all branched code even if it is dev'd
> > off-list.
> > >    This allows for inspection/ code examination and further exploration
> > - at a
> > >    minimum.  Accepting on a branch is neither the same as accepting for
> > >    release, nor merging to master branch.
> > >
> > >    Now, the assumption that the code is better than what the community
> > has
> > >    developed has to be challenged.  It could be that the branched code
> > should
> > >    be judged only on the merits of the code (is it better and more
> > complete),
> > >    or it could be judged on the basis that it "breaks the current
> build".
> > >    There can be a culture of a project to accept such code drops with
> the
> > >    caveat that if the merges cannot be done by the submitting group,
> > then the
> > >    project will have a resistance to such submissions (you break it,
> you
> > fix
> > >    it), or alternatively that there will be a small group of people
> that
> > are
> > >    sourced from such delayed-contribution types - that work on doing
> the
> > >    merges.  The key seems to be to create the incentive to share code
> > before
> > >    others do, to avoid being the one that breaks the build.
> > >
> > >    ~jdailey67
> > >
> > >
> > >
> > >
> > >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com>
> > wrote:
> > >
> > >> Large code drops are almost always damaging, since inherent in that
> > >> process is the concept of "throwing the code over a wall". But
> > sometimes it
> > >> does work out, assuming that continuity and "good intentions" are
> > followed.
> > >>
> > >> To show this, join me in the Wayback Machine as Sherman and I travel
> to
> > >> the year 1995...
> > >>
> > >> This is right around the start of Apache, back when Apache meant the
> web
> > >> server, and at the time, the project was basically what was left of
> the
> > >> NCSA web server plus some patches and bug fixes... Around this time,
> > one of
> > >> the core group, Robert Thau, started independent work on a
> > re-architecture
> > >> of the server, which he code-named "Shambala". It was basically a
> single
> > >> contributor effort (himself). One day he simply said to the group,
> > "Here, I
> > >> have this new design and architecture for Apache. It adds a lot of
> > >> features." So much of what defines httpd today can find its origin
> right
> > >> there: modular framework, pools, preforking (and, as such, the initial
> > >> gleaming towards MPMs), extendable API, etc...
> > >>
> > >> In many ways, this was a large code drop. What made it different is
> that
> > >> there was *support* by the author and the community to work on
> > integrating
> > >> it into the whole. It became, basically, a community effort.
> > >>
> > >> Now compare that with a different scenario... Once httpd had picked up
> > >> steam, and making sure that it was ported to everyone's favorite *nix
> > >> flavor was important, SGI had done work on a set of patches that
> ported
> > >> httpd to their OS and provided these patches (a set of 10 very large
> > >> patch-files, iirc) to the group. What was clear in those patches is
> that
> > >> there was no consideration at all on how those patches affected or
> broke
> > >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> > >> totally destroying any sort of portability for anyone else. And when
> we
> > >> responded by, asking for more information, help with chatting with
> their
> > >> developers to try to figure things out, and basically trying to figure
> > out
> > >> how to use and merge this stuff, SGI was basically just silent. They
> > sent
> > >> it to us and that was the beginning and the end of their involvement
> as
> > far
> > >> as they were concerned.[1]
> > >>
> > >> Way, way too many large code drops are the latter. Hardly any are the
> > >> former.
> > >>
> > >>
> > >> 1. I have paraphrased both the Shambala and SGI events
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> > For additional commands, e-mail: dev-help@community.apache.org
> >
> >
>

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

Hey Jim,

I’d say they are a symptom *and* a problem. But putting that aside, can you
unroll what you mean please?

What was that code drop from SGI a symptom of?

What did Robert Thau do (or not do), before during or after to ensure the
success of httpd?

Best Regards,
Myrle

On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:

> I would say that, in general, large code drops are more a *symptom* of a
> problem, rather than a problem, in and of itself...
>
> > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID>
> wrote:
> >
> > IMO, the issue isn't about large code drops.  Some will be ok.
> >
> > The issue is about significant collaboration off-list about anything,
> not just code.
> >
> > My 2 cents,
> > -Alex
> >
> > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> >
> >    +1 on this civil discourse.
> >
> >    I would like to offer that sometimes large code drops are unavoidable
> and
> >    necessary.  Jim's explanation of httpd contribution of type 1 is a
> good
> >    example.
> >
> >    I think we would find that many projects started with a large code
> drop
> >    (maybe more than one) - a sufficient amount of code - to get a project
> >    started.  When projects are young it would be normal and expected for
> this
> >    to happen. It quickly gets a community to a "thing" that can be added
> to.
> >
> >    It obviously depends on the kinds of components, tools, frameworks,
> etc
> >    that are being developed. Game theory is quite apropos - you need a
> >    sufficient incentive for *timely* collaboration, of hanging together.
> >
> >    Further, if your "thing" is going to be used directly in market (i.e.
> with
> >    very little of a product wrapper ), then there is a strong
> *disincentive*
> >    to share back the latest and greatest. The further from market
> immediacy
> >    the easier it is to contribute. Both the Collaboration space and
> >    Competitive space are clearly delineated, whereas in a close to market
> >    immediacy situation you have too much overlap and therefore a built in
> >    delay of code contribution to preserve market competitiveness.
> >
> >    So, combining the "sufficient code to attract contribution" metric
> with the
> >    market-immediacy metric and you can predict engagement by outside
> vendors
> >    (or their contributors) in a project. In such a situation, it is
> better, in
> >    my view, to accept any and all branched code even if it is dev'd
> off-list.
> >    This allows for inspection/ code examination and further exploration
> - at a
> >    minimum.  Accepting on a branch is neither the same as accepting for
> >    release, nor merging to master branch.
> >
> >    Now, the assumption that the code is better than what the community
> has
> >    developed has to be challenged.  It could be that the branched code
> should
> >    be judged only on the merits of the code (is it better and more
> complete),
> >    or it could be judged on the basis that it "breaks the current build".
> >    There can be a culture of a project to accept such code drops with the
> >    caveat that if the merges cannot be done by the submitting group,
> then the
> >    project will have a resistance to such submissions (you break it, you
> fix
> >    it), or alternatively that there will be a small group of people that
> are
> >    sourced from such delayed-contribution types - that work on doing the
> >    merges.  The key seems to be to create the incentive to share code
> before
> >    others do, to avoid being the one that breaks the build.
> >
> >    ~jdailey67
> >
> >
> >
> >
> >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com>
> wrote:
> >
> >> Large code drops are almost always damaging, since inherent in that
> >> process is the concept of "throwing the code over a wall". But
> sometimes it
> >> does work out, assuming that continuity and "good intentions" are
> followed.
> >>
> >> To show this, join me in the Wayback Machine as Sherman and I travel to
> >> the year 1995...
> >>
> >> This is right around the start of Apache, back when Apache meant the web
> >> server, and at the time, the project was basically what was left of the
> >> NCSA web server plus some patches and bug fixes... Around this time,
> one of
> >> the core group, Robert Thau, started independent work on a
> re-architecture
> >> of the server, which he code-named "Shambala". It was basically a single
> >> contributor effort (himself). One day he simply said to the group,
> "Here, I
> >> have this new design and architecture for Apache. It adds a lot of
> >> features." So much of what defines httpd today can find its origin right
> >> there: modular framework, pools, preforking (and, as such, the initial
> >> gleaming towards MPMs), extendable API, etc...
> >>
> >> In many ways, this was a large code drop. What made it different is that
> >> there was *support* by the author and the community to work on
> integrating
> >> it into the whole. It became, basically, a community effort.
> >>
> >> Now compare that with a different scenario... Once httpd had picked up
> >> steam, and making sure that it was ported to everyone's favorite *nix
> >> flavor was important, SGI had done work on a set of patches that ported
> >> httpd to their OS and provided these patches (a set of 10 very large
> >> patch-files, iirc) to the group. What was clear in those patches is that
> >> there was no consideration at all on how those patches affected or broke
> >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> >> totally destroying any sort of portability for anyone else. And when we
> >> responded by, asking for more information, help with chatting with their
> >> developers to try to figure things out, and basically trying to figure
> out
> >> how to use and merge this stuff, SGI was basically just silent. They
> sent
> >> it to us and that was the beginning and the end of their involvement as
> far
> >> as they were concerned.[1]
> >>
> >> Way, way too many large code drops are the latter. Hardly any are the
> >> former.
> >>
> >>
> >> 1. I have paraphrased both the Shambala and SGI events
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

Hey Jim,

I’d say they are a symptom *and* a problem. But putting that aside, can you
unroll what you mean please?

What was that code drop from SGI a symptom of?

What did Robert Thau do (or not do), before during or after to ensure the
success of httpd?

Best Regards,
Myrle

On Sat 20. Oct 2018 at 00:28 Jim Jagielski <ji...@jagunet.com> wrote:

> I would say that, in general, large code drops are more a *symptom* of a
> problem, rather than a problem, in and of itself...
>
> > On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID>
> wrote:
> >
> > IMO, the issue isn't about large code drops.  Some will be ok.
> >
> > The issue is about significant collaboration off-list about anything,
> not just code.
> >
> > My 2 cents,
> > -Alex
> >
> > On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> >
> >    +1 on this civil discourse.
> >
> >    I would like to offer that sometimes large code drops are unavoidable
> and
> >    necessary.  Jim's explanation of httpd contribution of type 1 is a
> good
> >    example.
> >
> >    I think we would find that many projects started with a large code
> drop
> >    (maybe more than one) - a sufficient amount of code - to get a project
> >    started.  When projects are young it would be normal and expected for
> this
> >    to happen. It quickly gets a community to a "thing" that can be added
> to.
> >
> >    It obviously depends on the kinds of components, tools, frameworks,
> etc
> >    that are being developed. Game theory is quite apropos - you need a
> >    sufficient incentive for *timely* collaboration, of hanging together.
> >
> >    Further, if your "thing" is going to be used directly in market (i.e.
> with
> >    very little of a product wrapper ), then there is a strong
> *disincentive*
> >    to share back the latest and greatest. The further from market
> immediacy
> >    the easier it is to contribute. Both the Collaboration space and
> >    Competitive space are clearly delineated, whereas in a close to market
> >    immediacy situation you have too much overlap and therefore a built in
> >    delay of code contribution to preserve market competitiveness.
> >
> >    So, combining the "sufficient code to attract contribution" metric
> with the
> >    market-immediacy metric and you can predict engagement by outside
> vendors
> >    (or their contributors) in a project. In such a situation, it is
> better, in
> >    my view, to accept any and all branched code even if it is dev'd
> off-list.
> >    This allows for inspection/ code examination and further exploration
> - at a
> >    minimum.  Accepting on a branch is neither the same as accepting for
> >    release, nor merging to master branch.
> >
> >    Now, the assumption that the code is better than what the community
> has
> >    developed has to be challenged.  It could be that the branched code
> should
> >    be judged only on the merits of the code (is it better and more
> complete),
> >    or it could be judged on the basis that it "breaks the current build".
> >    There can be a culture of a project to accept such code drops with the
> >    caveat that if the merges cannot be done by the submitting group,
> then the
> >    project will have a resistance to such submissions (you break it, you
> fix
> >    it), or alternatively that there will be a small group of people that
> are
> >    sourced from such delayed-contribution types - that work on doing the
> >    merges.  The key seems to be to create the incentive to share code
> before
> >    others do, to avoid being the one that breaks the build.
> >
> >    ~jdailey67
> >
> >
> >
> >
> >    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com>
> wrote:
> >
> >> Large code drops are almost always damaging, since inherent in that
> >> process is the concept of "throwing the code over a wall". But
> sometimes it
> >> does work out, assuming that continuity and "good intentions" are
> followed.
> >>
> >> To show this, join me in the Wayback Machine as Sherman and I travel to
> >> the year 1995...
> >>
> >> This is right around the start of Apache, back when Apache meant the web
> >> server, and at the time, the project was basically what was left of the
> >> NCSA web server plus some patches and bug fixes... Around this time,
> one of
> >> the core group, Robert Thau, started independent work on a
> re-architecture
> >> of the server, which he code-named "Shambala". It was basically a single
> >> contributor effort (himself). One day he simply said to the group,
> "Here, I
> >> have this new design and architecture for Apache. It adds a lot of
> >> features." So much of what defines httpd today can find its origin right
> >> there: modular framework, pools, preforking (and, as such, the initial
> >> gleaming towards MPMs), extendable API, etc...
> >>
> >> In many ways, this was a large code drop. What made it different is that
> >> there was *support* by the author and the community to work on
> integrating
> >> it into the whole. It became, basically, a community effort.
> >>
> >> Now compare that with a different scenario... Once httpd had picked up
> >> steam, and making sure that it was ported to everyone's favorite *nix
> >> flavor was important, SGI had done work on a set of patches that ported
> >> httpd to their OS and provided these patches (a set of 10 very large
> >> patch-files, iirc) to the group. What was clear in those patches is that
> >> there was no consideration at all on how those patches affected or broke
> >> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> >> totally destroying any sort of portability for anyone else. And when we
> >> responded by, asking for more information, help with chatting with their
> >> developers to try to figure things out, and basically trying to figure
> out
> >> how to use and merge this stuff, SGI was basically just silent. They
> sent
> >> it to us and that was the beginning and the end of their involvement as
> far
> >> as they were concerned.[1]
> >>
> >> Way, way too many large code drops are the latter. Hardly any are the
> >> former.
> >>
> >>
> >> 1. I have paraphrased both the Shambala and SGI events
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Jim Jagielski <ji...@jaguNET.com>]

I would say that, in general, large code drops are more a *symptom* of a problem, rather than a problem, in and of itself...

> On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID> wrote:
> 
> IMO, the issue isn't about large code drops.  Some will be ok.
> 
> The issue is about significant collaboration off-list about anything, not just code.
> 
> My 2 cents,
> -Alex
> 
> On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> 
>    +1 on this civil discourse.
> 
>    I would like to offer that sometimes large code drops are unavoidable and
>    necessary.  Jim's explanation of httpd contribution of type 1 is a good
>    example.
> 
>    I think we would find that many projects started with a large code drop
>    (maybe more than one) - a sufficient amount of code - to get a project
>    started.  When projects are young it would be normal and expected for this
>    to happen. It quickly gets a community to a "thing" that can be added to.
> 
>    It obviously depends on the kinds of components, tools, frameworks, etc
>    that are being developed. Game theory is quite apropos - you need a
>    sufficient incentive for *timely* collaboration, of hanging together.
> 
>    Further, if your "thing" is going to be used directly in market (i.e. with
>    very little of a product wrapper ), then there is a strong *disincentive*
>    to share back the latest and greatest. The further from market immediacy
>    the easier it is to contribute. Both the Collaboration space and
>    Competitive space are clearly delineated, whereas in a close to market
>    immediacy situation you have too much overlap and therefore a built in
>    delay of code contribution to preserve market competitiveness.
> 
>    So, combining the "sufficient code to attract contribution" metric with the
>    market-immediacy metric and you can predict engagement by outside vendors
>    (or their contributors) in a project. In such a situation, it is better, in
>    my view, to accept any and all branched code even if it is dev'd off-list.
>    This allows for inspection/ code examination and further exploration - at a
>    minimum.  Accepting on a branch is neither the same as accepting for
>    release, nor merging to master branch.
> 
>    Now, the assumption that the code is better than what the community has
>    developed has to be challenged.  It could be that the branched code should
>    be judged only on the merits of the code (is it better and more complete),
>    or it could be judged on the basis that it "breaks the current build".
>    There can be a culture of a project to accept such code drops with the
>    caveat that if the merges cannot be done by the submitting group, then the
>    project will have a resistance to such submissions (you break it, you fix
>    it), or alternatively that there will be a small group of people that are
>    sourced from such delayed-contribution types - that work on doing the
>    merges.  The key seems to be to create the incentive to share code before
>    others do, to avoid being the one that breaks the build.
> 
>    ~jdailey67
> 
> 
> 
> 
>    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
> 
>> Large code drops are almost always damaging, since inherent in that
>> process is the concept of "throwing the code over a wall". But sometimes it
>> does work out, assuming that continuity and "good intentions" are followed.
>> 
>> To show this, join me in the Wayback Machine as Sherman and I travel to
>> the year 1995...
>> 
>> This is right around the start of Apache, back when Apache meant the web
>> server, and at the time, the project was basically what was left of the
>> NCSA web server plus some patches and bug fixes... Around this time, one of
>> the core group, Robert Thau, started independent work on a re-architecture
>> of the server, which he code-named "Shambala". It was basically a single
>> contributor effort (himself). One day he simply said to the group, "Here, I
>> have this new design and architecture for Apache. It adds a lot of
>> features." So much of what defines httpd today can find its origin right
>> there: modular framework, pools, preforking (and, as such, the initial
>> gleaming towards MPMs), extendable API, etc...
>> 
>> In many ways, this was a large code drop. What made it different is that
>> there was *support* by the author and the community to work on integrating
>> it into the whole. It became, basically, a community effort.
>> 
>> Now compare that with a different scenario... Once httpd had picked up
>> steam, and making sure that it was ported to everyone's favorite *nix
>> flavor was important, SGI had done work on a set of patches that ported
>> httpd to their OS and provided these patches (a set of 10 very large
>> patch-files, iirc) to the group. What was clear in those patches is that
>> there was no consideration at all on how those patches affected or broke
>> anyone else. They rewrote huge swaths of code, optimizing for SGI and
>> totally destroying any sort of portability for anyone else. And when we
>> responded by, asking for more information, help with chatting with their
>> developers to try to figure things out, and basically trying to figure out
>> how to use and merge this stuff, SGI was basically just silent. They sent
>> it to us and that was the beginning and the end of their involvement as far
>> as they were concerned.[1]
>> 
>> Way, way too many large code drops are the latter. Hardly any are the
>> former.
>> 
>> 
>> 1. I have paraphrased both the Shambala and SGI events
> 
> 

Re: Why are large code drops damaging to a community?

[Jim Jagielski <ji...@jaguNET.com>]

I would say that, in general, large code drops are more a *symptom* of a problem, rather than a problem, in and of itself...

> On Oct 19, 2018, at 5:12 PM, Alex Harui <ah...@adobe.com.INVALID> wrote:
> 
> IMO, the issue isn't about large code drops.  Some will be ok.
> 
> The issue is about significant collaboration off-list about anything, not just code.
> 
> My 2 cents,
> -Alex
> 
> On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:
> 
>    +1 on this civil discourse.
> 
>    I would like to offer that sometimes large code drops are unavoidable and
>    necessary.  Jim's explanation of httpd contribution of type 1 is a good
>    example.
> 
>    I think we would find that many projects started with a large code drop
>    (maybe more than one) - a sufficient amount of code - to get a project
>    started.  When projects are young it would be normal and expected for this
>    to happen. It quickly gets a community to a "thing" that can be added to.
> 
>    It obviously depends on the kinds of components, tools, frameworks, etc
>    that are being developed. Game theory is quite apropos - you need a
>    sufficient incentive for *timely* collaboration, of hanging together.
> 
>    Further, if your "thing" is going to be used directly in market (i.e. with
>    very little of a product wrapper ), then there is a strong *disincentive*
>    to share back the latest and greatest. The further from market immediacy
>    the easier it is to contribute. Both the Collaboration space and
>    Competitive space are clearly delineated, whereas in a close to market
>    immediacy situation you have too much overlap and therefore a built in
>    delay of code contribution to preserve market competitiveness.
> 
>    So, combining the "sufficient code to attract contribution" metric with the
>    market-immediacy metric and you can predict engagement by outside vendors
>    (or their contributors) in a project. In such a situation, it is better, in
>    my view, to accept any and all branched code even if it is dev'd off-list.
>    This allows for inspection/ code examination and further exploration - at a
>    minimum.  Accepting on a branch is neither the same as accepting for
>    release, nor merging to master branch.
> 
>    Now, the assumption that the code is better than what the community has
>    developed has to be challenged.  It could be that the branched code should
>    be judged only on the merits of the code (is it better and more complete),
>    or it could be judged on the basis that it "breaks the current build".
>    There can be a culture of a project to accept such code drops with the
>    caveat that if the merges cannot be done by the submitting group, then the
>    project will have a resistance to such submissions (you break it, you fix
>    it), or alternatively that there will be a small group of people that are
>    sourced from such delayed-contribution types - that work on doing the
>    merges.  The key seems to be to create the incentive to share code before
>    others do, to avoid being the one that breaks the build.
> 
>    ~jdailey67
> 
> 
> 
> 
>    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
> 
>> Large code drops are almost always damaging, since inherent in that
>> process is the concept of "throwing the code over a wall". But sometimes it
>> does work out, assuming that continuity and "good intentions" are followed.
>> 
>> To show this, join me in the Wayback Machine as Sherman and I travel to
>> the year 1995...
>> 
>> This is right around the start of Apache, back when Apache meant the web
>> server, and at the time, the project was basically what was left of the
>> NCSA web server plus some patches and bug fixes... Around this time, one of
>> the core group, Robert Thau, started independent work on a re-architecture
>> of the server, which he code-named "Shambala". It was basically a single
>> contributor effort (himself). One day he simply said to the group, "Here, I
>> have this new design and architecture for Apache. It adds a lot of
>> features." So much of what defines httpd today can find its origin right
>> there: modular framework, pools, preforking (and, as such, the initial
>> gleaming towards MPMs), extendable API, etc...
>> 
>> In many ways, this was a large code drop. What made it different is that
>> there was *support* by the author and the community to work on integrating
>> it into the whole. It became, basically, a community effort.
>> 
>> Now compare that with a different scenario... Once httpd had picked up
>> steam, and making sure that it was ported to everyone's favorite *nix
>> flavor was important, SGI had done work on a set of patches that ported
>> httpd to their OS and provided these patches (a set of 10 very large
>> patch-files, iirc) to the group. What was clear in those patches is that
>> there was no consideration at all on how those patches affected or broke
>> anyone else. They rewrote huge swaths of code, optimizing for SGI and
>> totally destroying any sort of portability for anyone else. And when we
>> responded by, asking for more information, help with chatting with their
>> developers to try to figure things out, and basically trying to figure out
>> how to use and merge this stuff, SGI was basically just silent. They sent
>> it to us and that was the beginning and the end of their involvement as far
>> as they were concerned.[1]
>> 
>> Way, way too many large code drops are the latter. Hardly any are the
>> former.
>> 
>> 
>> 1. I have paraphrased both the Shambala and SGI events
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Alex Harui <ah...@adobe.com.INVALID>]

IMO, the issue isn't about large code drops.  Some will be ok.

The issue is about significant collaboration off-list about anything, not just code.

My 2 cents,
-Alex

On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:

    +1 on this civil discourse.
    
    I would like to offer that sometimes large code drops are unavoidable and
    necessary.  Jim's explanation of httpd contribution of type 1 is a good
    example.
    
    I think we would find that many projects started with a large code drop
    (maybe more than one) - a sufficient amount of code - to get a project
    started.  When projects are young it would be normal and expected for this
    to happen. It quickly gets a community to a "thing" that can be added to.
    
    It obviously depends on the kinds of components, tools, frameworks, etc
    that are being developed. Game theory is quite apropos - you need a
    sufficient incentive for *timely* collaboration, of hanging together.
    
    Further, if your "thing" is going to be used directly in market (i.e. with
    very little of a product wrapper ), then there is a strong *disincentive*
    to share back the latest and greatest. The further from market immediacy
    the easier it is to contribute. Both the Collaboration space and
    Competitive space are clearly delineated, whereas in a close to market
    immediacy situation you have too much overlap and therefore a built in
    delay of code contribution to preserve market competitiveness.
    
    So, combining the "sufficient code to attract contribution" metric with the
    market-immediacy metric and you can predict engagement by outside vendors
    (or their contributors) in a project. In such a situation, it is better, in
    my view, to accept any and all branched code even if it is dev'd off-list.
    This allows for inspection/ code examination and further exploration - at a
    minimum.  Accepting on a branch is neither the same as accepting for
    release, nor merging to master branch.
    
    Now, the assumption that the code is better than what the community has
    developed has to be challenged.  It could be that the branched code should
    be judged only on the merits of the code (is it better and more complete),
    or it could be judged on the basis that it "breaks the current build".
    There can be a culture of a project to accept such code drops with the
    caveat that if the merges cannot be done by the submitting group, then the
    project will have a resistance to such submissions (you break it, you fix
    it), or alternatively that there will be a small group of people that are
    sourced from such delayed-contribution types - that work on doing the
    merges.  The key seems to be to create the incentive to share code before
    others do, to avoid being the one that breaks the build.
    
    ~jdailey67
    
    
    
    
    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
    
    > Large code drops are almost always damaging, since inherent in that
    > process is the concept of "throwing the code over a wall". But sometimes it
    > does work out, assuming that continuity and "good intentions" are followed.
    >
    > To show this, join me in the Wayback Machine as Sherman and I travel to
    > the year 1995...
    >
    > This is right around the start of Apache, back when Apache meant the web
    > server, and at the time, the project was basically what was left of the
    > NCSA web server plus some patches and bug fixes... Around this time, one of
    > the core group, Robert Thau, started independent work on a re-architecture
    > of the server, which he code-named "Shambala". It was basically a single
    > contributor effort (himself). One day he simply said to the group, "Here, I
    > have this new design and architecture for Apache. It adds a lot of
    > features." So much of what defines httpd today can find its origin right
    > there: modular framework, pools, preforking (and, as such, the initial
    > gleaming towards MPMs), extendable API, etc...
    >
    > In many ways, this was a large code drop. What made it different is that
    > there was *support* by the author and the community to work on integrating
    > it into the whole. It became, basically, a community effort.
    >
    > Now compare that with a different scenario... Once httpd had picked up
    > steam, and making sure that it was ported to everyone's favorite *nix
    > flavor was important, SGI had done work on a set of patches that ported
    > httpd to their OS and provided these patches (a set of 10 very large
    > patch-files, iirc) to the group. What was clear in those patches is that
    > there was no consideration at all on how those patches affected or broke
    > anyone else. They rewrote huge swaths of code, optimizing for SGI and
    > totally destroying any sort of portability for anyone else. And when we
    > responded by, asking for more information, help with chatting with their
    > developers to try to figure things out, and basically trying to figure out
    > how to use and merge this stuff, SGI was basically just silent. They sent
    > it to us and that was the beginning and the end of their involvement as far
    > as they were concerned.[1]
    >
    > Way, way too many large code drops are the latter. Hardly any are the
    > former.
    >
    >
    > 1. I have paraphrased both the Shambala and SGI events
    


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Alex Harui <ah...@adobe.com.INVALID>]

IMO, the issue isn't about large code drops.  Some will be ok.

The issue is about significant collaboration off-list about anything, not just code.

My 2 cents,
-Alex

On 10/19/18, 1:32 PM, "James Dailey" <ja...@gmail.com> wrote:

    +1 on this civil discourse.
    
    I would like to offer that sometimes large code drops are unavoidable and
    necessary.  Jim's explanation of httpd contribution of type 1 is a good
    example.
    
    I think we would find that many projects started with a large code drop
    (maybe more than one) - a sufficient amount of code - to get a project
    started.  When projects are young it would be normal and expected for this
    to happen. It quickly gets a community to a "thing" that can be added to.
    
    It obviously depends on the kinds of components, tools, frameworks, etc
    that are being developed. Game theory is quite apropos - you need a
    sufficient incentive for *timely* collaboration, of hanging together.
    
    Further, if your "thing" is going to be used directly in market (i.e. with
    very little of a product wrapper ), then there is a strong *disincentive*
    to share back the latest and greatest. The further from market immediacy
    the easier it is to contribute. Both the Collaboration space and
    Competitive space are clearly delineated, whereas in a close to market
    immediacy situation you have too much overlap and therefore a built in
    delay of code contribution to preserve market competitiveness.
    
    So, combining the "sufficient code to attract contribution" metric with the
    market-immediacy metric and you can predict engagement by outside vendors
    (or their contributors) in a project. In such a situation, it is better, in
    my view, to accept any and all branched code even if it is dev'd off-list.
    This allows for inspection/ code examination and further exploration - at a
    minimum.  Accepting on a branch is neither the same as accepting for
    release, nor merging to master branch.
    
    Now, the assumption that the code is better than what the community has
    developed has to be challenged.  It could be that the branched code should
    be judged only on the merits of the code (is it better and more complete),
    or it could be judged on the basis that it "breaks the current build".
    There can be a culture of a project to accept such code drops with the
    caveat that if the merges cannot be done by the submitting group, then the
    project will have a resistance to such submissions (you break it, you fix
    it), or alternatively that there will be a small group of people that are
    sourced from such delayed-contribution types - that work on doing the
    merges.  The key seems to be to create the incentive to share code before
    others do, to avoid being the one that breaks the build.
    
    ~jdailey67
    
    
    
    
    On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:
    
    > Large code drops are almost always damaging, since inherent in that
    > process is the concept of "throwing the code over a wall". But sometimes it
    > does work out, assuming that continuity and "good intentions" are followed.
    >
    > To show this, join me in the Wayback Machine as Sherman and I travel to
    > the year 1995...
    >
    > This is right around the start of Apache, back when Apache meant the web
    > server, and at the time, the project was basically what was left of the
    > NCSA web server plus some patches and bug fixes... Around this time, one of
    > the core group, Robert Thau, started independent work on a re-architecture
    > of the server, which he code-named "Shambala". It was basically a single
    > contributor effort (himself). One day he simply said to the group, "Here, I
    > have this new design and architecture for Apache. It adds a lot of
    > features." So much of what defines httpd today can find its origin right
    > there: modular framework, pools, preforking (and, as such, the initial
    > gleaming towards MPMs), extendable API, etc...
    >
    > In many ways, this was a large code drop. What made it different is that
    > there was *support* by the author and the community to work on integrating
    > it into the whole. It became, basically, a community effort.
    >
    > Now compare that with a different scenario... Once httpd had picked up
    > steam, and making sure that it was ported to everyone's favorite *nix
    > flavor was important, SGI had done work on a set of patches that ported
    > httpd to their OS and provided these patches (a set of 10 very large
    > patch-files, iirc) to the group. What was clear in those patches is that
    > there was no consideration at all on how those patches affected or broke
    > anyone else. They rewrote huge swaths of code, optimizing for SGI and
    > totally destroying any sort of portability for anyone else. And when we
    > responded by, asking for more information, help with chatting with their
    > developers to try to figure things out, and basically trying to figure out
    > how to use and merge this stuff, SGI was basically just silent. They sent
    > it to us and that was the beginning and the end of their involvement as far
    > as they were concerned.[1]
    >
    > Way, way too many large code drops are the latter. Hardly any are the
    > former.
    >
    >
    > 1. I have paraphrased both the Shambala and SGI events
    

Re: Why are large code drops damaging to a community?

[sebb <se...@gmail.com>]

On Wed, 31 Oct 2018 at 08:44, Myrle Krantz <my...@apache.org> wrote:
>
> On Fri, Oct 19, 2018 at 10:32 PM James Dailey <ja...@gmail.com> wrote:
> >
> > +1 on this civil discourse.
> >
> > I would like to offer that sometimes large code drops are unavoidable and
> > necessary.  Jim's explanation of httpd contribution of type 1 is a good
> > example.
>
> I suspect that Jim's example of Robert Thau's httpd contribution may
> not have been necessary were he working with the source control tools
> and methods available today.  That was in 1995.  Even when we assume
> it was necessary in 1995, it was risky and probably only succeeded
> because of an abundance of communication and real good will with the
> community.  I'm guessing though.  I haven't given up hope that Jim
> will provide more details.
>
> > I think we would find that many projects started with a large code drop
> > (maybe more than one) - a sufficient amount of code - to get a project
> > started.  When projects are young it would be normal and expected for this
> > to happen. It quickly gets a community to a "thing" that can be added to.
>
> Many do start with a large code drop.  They go through incubation, and
> sometimes fail to attract communities.  If they fail to attract
> communities, they eventually reach the attic.

FTR:

Podlings which fail to attract a community do not graduate - they are retired.
They do not go to the Attic, which is reserved for TLPs.

> Many projects also
> succeed in starting at the ASF with an empty repository and building
> from zero.  These are often very successful at attracting communities.
> People like to work on things in which they are empowered to
> participate in decision making.  People like the feeling of trust and
> community which results.  PLC4X which Chris Dutz is working on is an
> excellent example.
>
> > It obviously depends on the kinds of components, tools, frameworks, etc
> > that are being developed. Game theory is quite apropos - you need a
> > sufficient incentive for *timely* collaboration, of hanging together.
> >
> > Further, if your "thing" is going to be used directly in market (i.e. with
> > very little of a product wrapper ), then there is a strong *disincentive*
> > to share back the latest and greatest. The further from market immediacy
> > the easier it is to contribute. Both the Collaboration space and
> > Competitive space are clearly delineated, whereas in a close to market
> > immediacy situation you have too much overlap and therefore a built in
> > delay of code contribution to preserve market competitiveness.
>
> This is one important reason that there are very few full applications
> at the ASF.  Fortunately for Fineract, we don't have to function as a
> full application.  This might be an argument for our community to stay
> away from customer-facing front-ends.
>
> If working together with others on your code would cause you important
> market disadvantages, then you probably don't want to take part in
> open source as it is conceived at the Apache Software Foundation.  If
> a vendor's goal is simply to publish their source, then a plain old
> github account is probably the least expensive method available to
> that vendor.  If a vendor's goal is to dominate a project, then there
> are 501(c)6's out there which have the latitude to make that sort of
> collaboration possible.  Those are valid and reasonable approaches.
> The ASF isn't trying to compete in those spaces.  The ASF wants its
> projects to be built as a community of equals.
>
> > So, combining the "sufficient code to attract contribution" metric with the
> > market-immediacy metric and you can predict engagement by outside vendors
> > (or their contributors) in a project.
>
> A "Sufficient code" metric isn't one I've ever personally used to
> decide which project to contribute to.  I don't believe vendors use
> this either.  I think some developers are even turned off by the
> existence of massive amounts of code.  It triggers the "not invented
> here" complex many developers have. : o)  But I'm happy to look at
> data that shows I'm wrong.
>
> Perhaps  you mean a "sufficient business functionality" metric?  But
> even then, what I've seen more often is a "sufficiently healthy
> community" metric.  Ie: how long has the project existed, how long is
> it likely to continue to exist?  These are the questions I get asked
> by people outside of our project.
>
> > In such a situation, it is better, in
> > my view, to accept any and all branched code even if it is dev'd off-list.
>
> The Apache Software Foundation does not "accept any and all" off-list
> development.  There are reasonable arguments to be had about the grey
> areas and how far they extend, but there is *never* carte blanche.
>
> > This allows for inspection/ code examination and further exploration - at a
> > minimum.  Accepting on a branch is neither the same as accepting for
> > release, nor merging to master branch.
>
> Inspection/examination can also be accomplished on github.  We don't
> need to accept code into an Apache project in order to be able to see
> it.  If a vendor wishes to propose merging a branch, we can look at
> that code and determine whether it is inside or outside our project's
> grey area *before* we accept the code.  If code is large, that
> proposal can be made available in reviewable-sized chunks by the
> vendor, as Upayavira described doing.  This not only has the advantage
> of improving code quality, it also means community members have a
> chance to participate meaningfully in the decision-making process and
> learn how to work within the new code.  Learning by doing is always
> the deepest learning.
>
> > Now, the assumption that the code is better than what the community has
> > developed has to be challenged.  It could be that the branched code should
> > be judged only on the merits of the code (is it better and more complete),
> > or it could be judged on the basis that it "breaks the current build".
> > There can be a culture of a project to accept such code drops with the
> > caveat that if the merges cannot be done by the submitting group, then the
> > project will have a resistance to such submissions (you break it, you fix
> > it), or alternatively that there will be a small group of people that are
> > sourced from such delayed-contribution types - that work on doing the
> > merges.  The key seems to be to create the incentive to share code before
> > others do, to avoid being the one that breaks the build.
>
> Shall we call this the "preemptive architecture" approach?  Certainly
> programmers sometimes engage in this kind of adversarial behavior.
> I've seen it.  I've never been happy with the consequences.  I would
> not wish to work for a company which was racing to get there first
> with the largest possible chunk of code, with the consequence of
> failure being resolving months of merge conflicts.  Not to mention
> that there's more to software engineering than just lines of code to
> merge.  Entire problem-solving approaches can be in conflict.  And
> resolutions of these will involve more than just staring at a
> three-way merge of files all day.
>
> The consequences for employees aside, it doesn't seem likely that
> anybody in the scenario you describe actually has time to test the new
> code or its merge, nor to participate in the community aspects of
> teaching others about the code being contributed and collaborating
> with people outside their company.  I'd be very concerned about the
> quality of code produced using this approach.
>
> I hope we can avoid incentivizing vendors to treat their employees and
> our projects this way.
>
> Best Regards,
> Myrle
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Akramulkarim Karim <ak...@gmail.com>]

On 31 Oct 2018 17:03, "Jim Jagielski" <ji...@jagunet.com> wrote:

Ahh... I missed the "so if y'all would like to
elaborate" part of your email... Let me pull
together some thoughts and expand/extend the story :)


> On Oct 31, 2018, at 4:44 AM, Myrle Krantz <my...@apache.org> wrote:
>
> I haven't given up hope that Jim
> will provide more details.
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Jim Jagielski <ji...@jaguNET.com>]

Ahh... I missed the "so if y'all would like to
elaborate" part of your email... Let me pull
together some thoughts and expand/extend the story :)

> On Oct 31, 2018, at 4:44 AM, Myrle Krantz <my...@apache.org> wrote:
> 
> I haven't given up hope that Jim
> will provide more details.
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[sebb <se...@gmail.com>]

On Wed, 31 Oct 2018 at 08:44, Myrle Krantz <my...@apache.org> wrote:
>
> On Fri, Oct 19, 2018 at 10:32 PM James Dailey <ja...@gmail.com> wrote:
> >
> > +1 on this civil discourse.
> >
> > I would like to offer that sometimes large code drops are unavoidable and
> > necessary.  Jim's explanation of httpd contribution of type 1 is a good
> > example.
>
> I suspect that Jim's example of Robert Thau's httpd contribution may
> not have been necessary were he working with the source control tools
> and methods available today.  That was in 1995.  Even when we assume
> it was necessary in 1995, it was risky and probably only succeeded
> because of an abundance of communication and real good will with the
> community.  I'm guessing though.  I haven't given up hope that Jim
> will provide more details.
>
> > I think we would find that many projects started with a large code drop
> > (maybe more than one) - a sufficient amount of code - to get a project
> > started.  When projects are young it would be normal and expected for this
> > to happen. It quickly gets a community to a "thing" that can be added to.
>
> Many do start with a large code drop.  They go through incubation, and
> sometimes fail to attract communities.  If they fail to attract
> communities, they eventually reach the attic.

FTR:

Podlings which fail to attract a community do not graduate - they are retired.
They do not go to the Attic, which is reserved for TLPs.

> Many projects also
> succeed in starting at the ASF with an empty repository and building
> from zero.  These are often very successful at attracting communities.
> People like to work on things in which they are empowered to
> participate in decision making.  People like the feeling of trust and
> community which results.  PLC4X which Chris Dutz is working on is an
> excellent example.
>
> > It obviously depends on the kinds of components, tools, frameworks, etc
> > that are being developed. Game theory is quite apropos - you need a
> > sufficient incentive for *timely* collaboration, of hanging together.
> >
> > Further, if your "thing" is going to be used directly in market (i.e. with
> > very little of a product wrapper ), then there is a strong *disincentive*
> > to share back the latest and greatest. The further from market immediacy
> > the easier it is to contribute. Both the Collaboration space and
> > Competitive space are clearly delineated, whereas in a close to market
> > immediacy situation you have too much overlap and therefore a built in
> > delay of code contribution to preserve market competitiveness.
>
> This is one important reason that there are very few full applications
> at the ASF.  Fortunately for Fineract, we don't have to function as a
> full application.  This might be an argument for our community to stay
> away from customer-facing front-ends.
>
> If working together with others on your code would cause you important
> market disadvantages, then you probably don't want to take part in
> open source as it is conceived at the Apache Software Foundation.  If
> a vendor's goal is simply to publish their source, then a plain old
> github account is probably the least expensive method available to
> that vendor.  If a vendor's goal is to dominate a project, then there
> are 501(c)6's out there which have the latitude to make that sort of
> collaboration possible.  Those are valid and reasonable approaches.
> The ASF isn't trying to compete in those spaces.  The ASF wants its
> projects to be built as a community of equals.
>
> > So, combining the "sufficient code to attract contribution" metric with the
> > market-immediacy metric and you can predict engagement by outside vendors
> > (or their contributors) in a project.
>
> A "Sufficient code" metric isn't one I've ever personally used to
> decide which project to contribute to.  I don't believe vendors use
> this either.  I think some developers are even turned off by the
> existence of massive amounts of code.  It triggers the "not invented
> here" complex many developers have. : o)  But I'm happy to look at
> data that shows I'm wrong.
>
> Perhaps  you mean a "sufficient business functionality" metric?  But
> even then, what I've seen more often is a "sufficiently healthy
> community" metric.  Ie: how long has the project existed, how long is
> it likely to continue to exist?  These are the questions I get asked
> by people outside of our project.
>
> > In such a situation, it is better, in
> > my view, to accept any and all branched code even if it is dev'd off-list.
>
> The Apache Software Foundation does not "accept any and all" off-list
> development.  There are reasonable arguments to be had about the grey
> areas and how far they extend, but there is *never* carte blanche.
>
> > This allows for inspection/ code examination and further exploration - at a
> > minimum.  Accepting on a branch is neither the same as accepting for
> > release, nor merging to master branch.
>
> Inspection/examination can also be accomplished on github.  We don't
> need to accept code into an Apache project in order to be able to see
> it.  If a vendor wishes to propose merging a branch, we can look at
> that code and determine whether it is inside or outside our project's
> grey area *before* we accept the code.  If code is large, that
> proposal can be made available in reviewable-sized chunks by the
> vendor, as Upayavira described doing.  This not only has the advantage
> of improving code quality, it also means community members have a
> chance to participate meaningfully in the decision-making process and
> learn how to work within the new code.  Learning by doing is always
> the deepest learning.
>
> > Now, the assumption that the code is better than what the community has
> > developed has to be challenged.  It could be that the branched code should
> > be judged only on the merits of the code (is it better and more complete),
> > or it could be judged on the basis that it "breaks the current build".
> > There can be a culture of a project to accept such code drops with the
> > caveat that if the merges cannot be done by the submitting group, then the
> > project will have a resistance to such submissions (you break it, you fix
> > it), or alternatively that there will be a small group of people that are
> > sourced from such delayed-contribution types - that work on doing the
> > merges.  The key seems to be to create the incentive to share code before
> > others do, to avoid being the one that breaks the build.
>
> Shall we call this the "preemptive architecture" approach?  Certainly
> programmers sometimes engage in this kind of adversarial behavior.
> I've seen it.  I've never been happy with the consequences.  I would
> not wish to work for a company which was racing to get there first
> with the largest possible chunk of code, with the consequence of
> failure being resolving months of merge conflicts.  Not to mention
> that there's more to software engineering than just lines of code to
> merge.  Entire problem-solving approaches can be in conflict.  And
> resolutions of these will involve more than just staring at a
> three-way merge of files all day.
>
> The consequences for employees aside, it doesn't seem likely that
> anybody in the scenario you describe actually has time to test the new
> code or its merge, nor to participate in the community aspects of
> teaching others about the code being contributed and collaborating
> with people outside their company.  I'd be very concerned about the
> quality of code produced using this approach.
>
> I hope we can avoid incentivizing vendors to treat their employees and
> our projects this way.
>
> Best Regards,
> Myrle
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>

Re: Why are large code drops damaging to a community?

[Jim Jagielski <ji...@jaguNET.com>]

Ahh... I missed the "so if y'all would like to
elaborate" part of your email... Let me pull
together some thoughts and expand/extend the story :)

> On Oct 31, 2018, at 4:44 AM, Myrle Krantz <my...@apache.org> wrote:
> 
> I haven't given up hope that Jim
> will provide more details.
> 

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

On Fri, Oct 19, 2018 at 10:32 PM James Dailey <ja...@gmail.com> wrote:
>
> +1 on this civil discourse.
>
> I would like to offer that sometimes large code drops are unavoidable and
> necessary.  Jim's explanation of httpd contribution of type 1 is a good
> example.

I suspect that Jim's example of Robert Thau's httpd contribution may
not have been necessary were he working with the source control tools
and methods available today.  That was in 1995.  Even when we assume
it was necessary in 1995, it was risky and probably only succeeded
because of an abundance of communication and real good will with the
community.  I'm guessing though.  I haven't given up hope that Jim
will provide more details.

> I think we would find that many projects started with a large code drop
> (maybe more than one) - a sufficient amount of code - to get a project
> started.  When projects are young it would be normal and expected for this
> to happen. It quickly gets a community to a "thing" that can be added to.

Many do start with a large code drop.  They go through incubation, and
sometimes fail to attract communities.  If they fail to attract
communities, they eventually reach the attic.  Many projects also
succeed in starting at the ASF with an empty repository and building
from zero.  These are often very successful at attracting communities.
People like to work on things in which they are empowered to
participate in decision making.  People like the feeling of trust and
community which results.  PLC4X which Chris Dutz is working on is an
excellent example.

> It obviously depends on the kinds of components, tools, frameworks, etc
> that are being developed. Game theory is quite apropos - you need a
> sufficient incentive for *timely* collaboration, of hanging together.
>
> Further, if your "thing" is going to be used directly in market (i.e. with
> very little of a product wrapper ), then there is a strong *disincentive*
> to share back the latest and greatest. The further from market immediacy
> the easier it is to contribute. Both the Collaboration space and
> Competitive space are clearly delineated, whereas in a close to market
> immediacy situation you have too much overlap and therefore a built in
> delay of code contribution to preserve market competitiveness.

This is one important reason that there are very few full applications
at the ASF.  Fortunately for Fineract, we don't have to function as a
full application.  This might be an argument for our community to stay
away from customer-facing front-ends.

If working together with others on your code would cause you important
market disadvantages, then you probably don't want to take part in
open source as it is conceived at the Apache Software Foundation.  If
a vendor's goal is simply to publish their source, then a plain old
github account is probably the least expensive method available to
that vendor.  If a vendor's goal is to dominate a project, then there
are 501(c)6's out there which have the latitude to make that sort of
collaboration possible.  Those are valid and reasonable approaches.
The ASF isn't trying to compete in those spaces.  The ASF wants its
projects to be built as a community of equals.

> So, combining the "sufficient code to attract contribution" metric with the
> market-immediacy metric and you can predict engagement by outside vendors
> (or their contributors) in a project.

A "Sufficient code" metric isn't one I've ever personally used to
decide which project to contribute to.  I don't believe vendors use
this either.  I think some developers are even turned off by the
existence of massive amounts of code.  It triggers the "not invented
here" complex many developers have. : o)  But I'm happy to look at
data that shows I'm wrong.

Perhaps  you mean a "sufficient business functionality" metric?  But
even then, what I've seen more often is a "sufficiently healthy
community" metric.  Ie: how long has the project existed, how long is
it likely to continue to exist?  These are the questions I get asked
by people outside of our project.

> In such a situation, it is better, in
> my view, to accept any and all branched code even if it is dev'd off-list.

The Apache Software Foundation does not "accept any and all" off-list
development.  There are reasonable arguments to be had about the grey
areas and how far they extend, but there is *never* carte blanche.

> This allows for inspection/ code examination and further exploration - at a
> minimum.  Accepting on a branch is neither the same as accepting for
> release, nor merging to master branch.

Inspection/examination can also be accomplished on github.  We don't
need to accept code into an Apache project in order to be able to see
it.  If a vendor wishes to propose merging a branch, we can look at
that code and determine whether it is inside or outside our project's
grey area *before* we accept the code.  If code is large, that
proposal can be made available in reviewable-sized chunks by the
vendor, as Upayavira described doing.  This not only has the advantage
of improving code quality, it also means community members have a
chance to participate meaningfully in the decision-making process and
learn how to work within the new code.  Learning by doing is always
the deepest learning.

> Now, the assumption that the code is better than what the community has
> developed has to be challenged.  It could be that the branched code should
> be judged only on the merits of the code (is it better and more complete),
> or it could be judged on the basis that it "breaks the current build".
> There can be a culture of a project to accept such code drops with the
> caveat that if the merges cannot be done by the submitting group, then the
> project will have a resistance to such submissions (you break it, you fix
> it), or alternatively that there will be a small group of people that are
> sourced from such delayed-contribution types - that work on doing the
> merges.  The key seems to be to create the incentive to share code before
> others do, to avoid being the one that breaks the build.

Shall we call this the "preemptive architecture" approach?  Certainly
programmers sometimes engage in this kind of adversarial behavior.
I've seen it.  I've never been happy with the consequences.  I would
not wish to work for a company which was racing to get there first
with the largest possible chunk of code, with the consequence of
failure being resolving months of merge conflicts.  Not to mention
that there's more to software engineering than just lines of code to
merge.  Entire problem-solving approaches can be in conflict.  And
resolutions of these will involve more than just staring at a
three-way merge of files all day.

The consequences for employees aside, it doesn't seem likely that
anybody in the scenario you describe actually has time to test the new
code or its merge, nor to participate in the community aspects of
teaching others about the code being contributed and collaborating
with people outside their company.  I'd be very concerned about the
quality of code produced using this approach.

I hope we can avoid incentivizing vendors to treat their employees and
our projects this way.

Best Regards,
Myrle

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

On Fri, Oct 19, 2018 at 10:32 PM James Dailey <ja...@gmail.com> wrote:
>
> +1 on this civil discourse.
>
> I would like to offer that sometimes large code drops are unavoidable and
> necessary.  Jim's explanation of httpd contribution of type 1 is a good
> example.

I suspect that Jim's example of Robert Thau's httpd contribution may
not have been necessary were he working with the source control tools
and methods available today.  That was in 1995.  Even when we assume
it was necessary in 1995, it was risky and probably only succeeded
because of an abundance of communication and real good will with the
community.  I'm guessing though.  I haven't given up hope that Jim
will provide more details.

> I think we would find that many projects started with a large code drop
> (maybe more than one) - a sufficient amount of code - to get a project
> started.  When projects are young it would be normal and expected for this
> to happen. It quickly gets a community to a "thing" that can be added to.

Many do start with a large code drop.  They go through incubation, and
sometimes fail to attract communities.  If they fail to attract
communities, they eventually reach the attic.  Many projects also
succeed in starting at the ASF with an empty repository and building
from zero.  These are often very successful at attracting communities.
People like to work on things in which they are empowered to
participate in decision making.  People like the feeling of trust and
community which results.  PLC4X which Chris Dutz is working on is an
excellent example.

> It obviously depends on the kinds of components, tools, frameworks, etc
> that are being developed. Game theory is quite apropos - you need a
> sufficient incentive for *timely* collaboration, of hanging together.
>
> Further, if your "thing" is going to be used directly in market (i.e. with
> very little of a product wrapper ), then there is a strong *disincentive*
> to share back the latest and greatest. The further from market immediacy
> the easier it is to contribute. Both the Collaboration space and
> Competitive space are clearly delineated, whereas in a close to market
> immediacy situation you have too much overlap and therefore a built in
> delay of code contribution to preserve market competitiveness.

This is one important reason that there are very few full applications
at the ASF.  Fortunately for Fineract, we don't have to function as a
full application.  This might be an argument for our community to stay
away from customer-facing front-ends.

If working together with others on your code would cause you important
market disadvantages, then you probably don't want to take part in
open source as it is conceived at the Apache Software Foundation.  If
a vendor's goal is simply to publish their source, then a plain old
github account is probably the least expensive method available to
that vendor.  If a vendor's goal is to dominate a project, then there
are 501(c)6's out there which have the latitude to make that sort of
collaboration possible.  Those are valid and reasonable approaches.
The ASF isn't trying to compete in those spaces.  The ASF wants its
projects to be built as a community of equals.

> So, combining the "sufficient code to attract contribution" metric with the
> market-immediacy metric and you can predict engagement by outside vendors
> (or their contributors) in a project.

A "Sufficient code" metric isn't one I've ever personally used to
decide which project to contribute to.  I don't believe vendors use
this either.  I think some developers are even turned off by the
existence of massive amounts of code.  It triggers the "not invented
here" complex many developers have. : o)  But I'm happy to look at
data that shows I'm wrong.

Perhaps  you mean a "sufficient business functionality" metric?  But
even then, what I've seen more often is a "sufficiently healthy
community" metric.  Ie: how long has the project existed, how long is
it likely to continue to exist?  These are the questions I get asked
by people outside of our project.

> In such a situation, it is better, in
> my view, to accept any and all branched code even if it is dev'd off-list.

The Apache Software Foundation does not "accept any and all" off-list
development.  There are reasonable arguments to be had about the grey
areas and how far they extend, but there is *never* carte blanche.

> This allows for inspection/ code examination and further exploration - at a
> minimum.  Accepting on a branch is neither the same as accepting for
> release, nor merging to master branch.

Inspection/examination can also be accomplished on github.  We don't
need to accept code into an Apache project in order to be able to see
it.  If a vendor wishes to propose merging a branch, we can look at
that code and determine whether it is inside or outside our project's
grey area *before* we accept the code.  If code is large, that
proposal can be made available in reviewable-sized chunks by the
vendor, as Upayavira described doing.  This not only has the advantage
of improving code quality, it also means community members have a
chance to participate meaningfully in the decision-making process and
learn how to work within the new code.  Learning by doing is always
the deepest learning.

> Now, the assumption that the code is better than what the community has
> developed has to be challenged.  It could be that the branched code should
> be judged only on the merits of the code (is it better and more complete),
> or it could be judged on the basis that it "breaks the current build".
> There can be a culture of a project to accept such code drops with the
> caveat that if the merges cannot be done by the submitting group, then the
> project will have a resistance to such submissions (you break it, you fix
> it), or alternatively that there will be a small group of people that are
> sourced from such delayed-contribution types - that work on doing the
> merges.  The key seems to be to create the incentive to share code before
> others do, to avoid being the one that breaks the build.

Shall we call this the "preemptive architecture" approach?  Certainly
programmers sometimes engage in this kind of adversarial behavior.
I've seen it.  I've never been happy with the consequences.  I would
not wish to work for a company which was racing to get there first
with the largest possible chunk of code, with the consequence of
failure being resolving months of merge conflicts.  Not to mention
that there's more to software engineering than just lines of code to
merge.  Entire problem-solving approaches can be in conflict.  And
resolutions of these will involve more than just staring at a
three-way merge of files all day.

The consequences for employees aside, it doesn't seem likely that
anybody in the scenario you describe actually has time to test the new
code or its merge, nor to participate in the community aspects of
teaching others about the code being contributed and collaborating
with people outside their company.  I'd be very concerned about the
quality of code produced using this approach.

I hope we can avoid incentivizing vendors to treat their employees and
our projects this way.

Best Regards,
Myrle

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[James Dailey <ja...@gmail.com>]

+1 on this civil discourse.

I would like to offer that sometimes large code drops are unavoidable and
necessary.  Jim's explanation of httpd contribution of type 1 is a good
example.

I think we would find that many projects started with a large code drop
(maybe more than one) - a sufficient amount of code - to get a project
started.  When projects are young it would be normal and expected for this
to happen. It quickly gets a community to a "thing" that can be added to.

It obviously depends on the kinds of components, tools, frameworks, etc
that are being developed. Game theory is quite apropos - you need a
sufficient incentive for *timely* collaboration, of hanging together.

Further, if your "thing" is going to be used directly in market (i.e. with
very little of a product wrapper ), then there is a strong *disincentive*
to share back the latest and greatest. The further from market immediacy
the easier it is to contribute. Both the Collaboration space and
Competitive space are clearly delineated, whereas in a close to market
immediacy situation you have too much overlap and therefore a built in
delay of code contribution to preserve market competitiveness.

So, combining the "sufficient code to attract contribution" metric with the
market-immediacy metric and you can predict engagement by outside vendors
(or their contributors) in a project. In such a situation, it is better, in
my view, to accept any and all branched code even if it is dev'd off-list.
This allows for inspection/ code examination and further exploration - at a
minimum.  Accepting on a branch is neither the same as accepting for
release, nor merging to master branch.

Now, the assumption that the code is better than what the community has
developed has to be challenged.  It could be that the branched code should
be judged only on the merits of the code (is it better and more complete),
or it could be judged on the basis that it "breaks the current build".
There can be a culture of a project to accept such code drops with the
caveat that if the merges cannot be done by the submitting group, then the
project will have a resistance to such submissions (you break it, you fix
it), or alternatively that there will be a small group of people that are
sourced from such delayed-contribution types - that work on doing the
merges.  The key seems to be to create the incentive to share code before
others do, to avoid being the one that breaks the build.

~jdailey67




On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:

> Large code drops are almost always damaging, since inherent in that
> process is the concept of "throwing the code over a wall". But sometimes it
> does work out, assuming that continuity and "good intentions" are followed.
>
> To show this, join me in the Wayback Machine as Sherman and I travel to
> the year 1995...
>
> This is right around the start of Apache, back when Apache meant the web
> server, and at the time, the project was basically what was left of the
> NCSA web server plus some patches and bug fixes... Around this time, one of
> the core group, Robert Thau, started independent work on a re-architecture
> of the server, which he code-named "Shambala". It was basically a single
> contributor effort (himself). One day he simply said to the group, "Here, I
> have this new design and architecture for Apache. It adds a lot of
> features." So much of what defines httpd today can find its origin right
> there: modular framework, pools, preforking (and, as such, the initial
> gleaming towards MPMs), extendable API, etc...
>
> In many ways, this was a large code drop. What made it different is that
> there was *support* by the author and the community to work on integrating
> it into the whole. It became, basically, a community effort.
>
> Now compare that with a different scenario... Once httpd had picked up
> steam, and making sure that it was ported to everyone's favorite *nix
> flavor was important, SGI had done work on a set of patches that ported
> httpd to their OS and provided these patches (a set of 10 very large
> patch-files, iirc) to the group. What was clear in those patches is that
> there was no consideration at all on how those patches affected or broke
> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> totally destroying any sort of portability for anyone else. And when we
> responded by, asking for more information, help with chatting with their
> developers to try to figure things out, and basically trying to figure out
> how to use and merge this stuff, SGI was basically just silent. They sent
> it to us and that was the beginning and the end of their involvement as far
> as they were concerned.[1]
>
> Way, way too many large code drops are the latter. Hardly any are the
> former.
>
>
> 1. I have paraphrased both the Shambala and SGI events

Re: Why are large code drops damaging to a community?

[James Dailey <ja...@gmail.com>]

+1 on this civil discourse.

I would like to offer that sometimes large code drops are unavoidable and
necessary.  Jim's explanation of httpd contribution of type 1 is a good
example.

I think we would find that many projects started with a large code drop
(maybe more than one) - a sufficient amount of code - to get a project
started.  When projects are young it would be normal and expected for this
to happen. It quickly gets a community to a "thing" that can be added to.

It obviously depends on the kinds of components, tools, frameworks, etc
that are being developed. Game theory is quite apropos - you need a
sufficient incentive for *timely* collaboration, of hanging together.

Further, if your "thing" is going to be used directly in market (i.e. with
very little of a product wrapper ), then there is a strong *disincentive*
to share back the latest and greatest. The further from market immediacy
the easier it is to contribute. Both the Collaboration space and
Competitive space are clearly delineated, whereas in a close to market
immediacy situation you have too much overlap and therefore a built in
delay of code contribution to preserve market competitiveness.

So, combining the "sufficient code to attract contribution" metric with the
market-immediacy metric and you can predict engagement by outside vendors
(or their contributors) in a project. In such a situation, it is better, in
my view, to accept any and all branched code even if it is dev'd off-list.
This allows for inspection/ code examination and further exploration - at a
minimum.  Accepting on a branch is neither the same as accepting for
release, nor merging to master branch.

Now, the assumption that the code is better than what the community has
developed has to be challenged.  It could be that the branched code should
be judged only on the merits of the code (is it better and more complete),
or it could be judged on the basis that it "breaks the current build".
There can be a culture of a project to accept such code drops with the
caveat that if the merges cannot be done by the submitting group, then the
project will have a resistance to such submissions (you break it, you fix
it), or alternatively that there will be a small group of people that are
sourced from such delayed-contribution types - that work on doing the
merges.  The key seems to be to create the incentive to share code before
others do, to avoid being the one that breaks the build.

~jdailey67




On Fri, Oct 19, 2018 at 6:10 AM Jim Jagielski <ji...@jagunet.com> wrote:

> Large code drops are almost always damaging, since inherent in that
> process is the concept of "throwing the code over a wall". But sometimes it
> does work out, assuming that continuity and "good intentions" are followed.
>
> To show this, join me in the Wayback Machine as Sherman and I travel to
> the year 1995...
>
> This is right around the start of Apache, back when Apache meant the web
> server, and at the time, the project was basically what was left of the
> NCSA web server plus some patches and bug fixes... Around this time, one of
> the core group, Robert Thau, started independent work on a re-architecture
> of the server, which he code-named "Shambala". It was basically a single
> contributor effort (himself). One day he simply said to the group, "Here, I
> have this new design and architecture for Apache. It adds a lot of
> features." So much of what defines httpd today can find its origin right
> there: modular framework, pools, preforking (and, as such, the initial
> gleaming towards MPMs), extendable API, etc...
>
> In many ways, this was a large code drop. What made it different is that
> there was *support* by the author and the community to work on integrating
> it into the whole. It became, basically, a community effort.
>
> Now compare that with a different scenario... Once httpd had picked up
> steam, and making sure that it was ported to everyone's favorite *nix
> flavor was important, SGI had done work on a set of patches that ported
> httpd to their OS and provided these patches (a set of 10 very large
> patch-files, iirc) to the group. What was clear in those patches is that
> there was no consideration at all on how those patches affected or broke
> anyone else. They rewrote huge swaths of code, optimizing for SGI and
> totally destroying any sort of portability for anyone else. And when we
> responded by, asking for more information, help with chatting with their
> developers to try to figure things out, and basically trying to figure out
> how to use and merge this stuff, SGI was basically just silent. They sent
> it to us and that was the beginning and the end of their involvement as far
> as they were concerned.[1]
>
> Way, way too many large code drops are the latter. Hardly any are the
> former.
>
>
> 1. I have paraphrased both the Shambala and SGI events

Re: Why are large code drops damaging to a community?

[Steph van Schalkwyk <sv...@gmail.com>]

It be disheartening if one has pulled to fix one thing and another
committer fixes it under another huge fix without collaborating with one.
This happens occasionally with superstars and it is enough to stop new
committers contributing at all.
Steph


On Fri, Oct 19, 2018 at 9:44 AM Malcolm Upayavira Holmes <uv...@odoko.co.uk>
wrote:

> My wayback was to 2002/3, when I was playing with Cocoon. Their CLI was
> implemented as a large monolithic Java class, and quite impenetrable. I had
> spent an age working on a large refactoring that made it much clearer and
> more usable. Unfortunately, it also broke backwards compatibility.
>
> Vadim Gritsenko, who was mentoring me refused to accept this patch. He
> told me to do it in small reversible changes. This was really annoying at
> the time, as I had spent so much time on it. Accepting his feedback, I
> started again. This time, I presented the same change, as a sequence of
> smaller changes, each of which he accepted. The net result was the same
> refactoring, in a way that he, and the community could absorb. Oh, and the
> new version was back compatible.
>
> That, for me, was a very powerful lesson in how open development can work
> effectively.
>
> Upayavira
>
> On Fri, 19 Oct 2018, at 2:10 PM, Jim Jagielski wrote:
> > Large code drops are almost always damaging, since inherent in that
> > process is the concept of "throwing the code over a wall". But sometimes
> > it does work out, assuming that continuity and "good intentions" are
> > followed.
> >
> > To show this, join me in the Wayback Machine as Sherman and I travel to
> > the year 1995...
> >
> > This is right around the start of Apache, back when Apache meant the web
> > server, and at the time, the project was basically what was left of the
> > NCSA web server plus some patches and bug fixes... Around this time, one
> > of the core group, Robert Thau, started independent work on a re-
> > architecture of the server, which he code-named "Shambala". It was
> > basically a single contributor effort (himself). One day he simply said
> > to the group, "Here, I have this new design and architecture for Apache.
> > It adds a lot of features." So much of what defines httpd today can find
> > its origin right there: modular framework, pools, preforking (and, as
> > such, the initial gleaming towards MPMs), extendable API, etc...
> >
> > In many ways, this was a large code drop. What made it different is that
> > there was *support* by the author and the community to work on
> > integrating it into the whole. It became, basically, a community effort.
> >
> > Now compare that with a different scenario... Once httpd had picked up
> > steam, and making sure that it was ported to everyone's favorite *nix
> > flavor was important, SGI had done work on a set of patches that ported
> > httpd to their OS and provided these patches (a set of 10 very large
> > patch-files, iirc) to the group. What was clear in those patches is that
> > there was no consideration at all on how those patches affected or broke
> > anyone else. They rewrote huge swaths of code, optimizing for SGI and
> > totally destroying any sort of portability for anyone else. And when we
> > responded by, asking for more information, help with chatting with their
> > developers to try to figure things out, and basically trying to figure
> > out how to use and merge this stuff, SGI was basically just silent. They
> > sent it to us and that was the beginning and the end of their
> > involvement as far as they were concerned.[1]
> >
> > Way, way too many large code drops are the latter. Hardly any are the
> former.
> >
> >
> > 1. I have paraphrased both the Shambala and SGI events
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> > For additional commands, e-mail: dev-help@community.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Malcolm Upayavira Holmes <uv...@odoko.co.uk>]

My wayback was to 2002/3, when I was playing with Cocoon. Their CLI was implemented as a large monolithic Java class, and quite impenetrable. I had spent an age working on a large refactoring that made it much clearer and more usable. Unfortunately, it also broke backwards compatibility.

Vadim Gritsenko, who was mentoring me refused to accept this patch. He told me to do it in small reversible changes. This was really annoying at the time, as I had spent so much time on it. Accepting his feedback, I started again. This time, I presented the same change, as a sequence of smaller changes, each of which he accepted. The net result was the same refactoring, in a way that he, and the community could absorb. Oh, and the new version was back compatible.

That, for me, was a very powerful lesson in how open development can work effectively.

Upayavira

On Fri, 19 Oct 2018, at 2:10 PM, Jim Jagielski wrote:
> Large code drops are almost always damaging, since inherent in that 
> process is the concept of "throwing the code over a wall". But sometimes 
> it does work out, assuming that continuity and "good intentions" are 
> followed.
> 
> To show this, join me in the Wayback Machine as Sherman and I travel to 
> the year 1995...
> 
> This is right around the start of Apache, back when Apache meant the web 
> server, and at the time, the project was basically what was left of the 
> NCSA web server plus some patches and bug fixes... Around this time, one 
> of the core group, Robert Thau, started independent work on a re-
> architecture of the server, which he code-named "Shambala". It was 
> basically a single contributor effort (himself). One day he simply said 
> to the group, "Here, I have this new design and architecture for Apache. 
> It adds a lot of features." So much of what defines httpd today can find 
> its origin right there: modular framework, pools, preforking (and, as 
> such, the initial gleaming towards MPMs), extendable API, etc...
> 
> In many ways, this was a large code drop. What made it different is that 
> there was *support* by the author and the community to work on 
> integrating it into the whole. It became, basically, a community effort.
> 
> Now compare that with a different scenario... Once httpd had picked up 
> steam, and making sure that it was ported to everyone's favorite *nix 
> flavor was important, SGI had done work on a set of patches that ported 
> httpd to their OS and provided these patches (a set of 10 very large 
> patch-files, iirc) to the group. What was clear in those patches is that 
> there was no consideration at all on how those patches affected or broke 
> anyone else. They rewrote huge swaths of code, optimizing for SGI and 
> totally destroying any sort of portability for anyone else. And when we 
> responded by, asking for more information, help with chatting with their 
> developers to try to figure things out, and basically trying to figure 
> out how to use and merge this stuff, SGI was basically just silent. They 
> sent it to us and that was the beginning and the end of their 
> involvement as far as they were concerned.[1]
> 
> Way, way too many large code drops are the latter. Hardly any are the former.
> 
> 
> 1. I have paraphrased both the Shambala and SGI events
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Jim Jagielski <ji...@jaguNET.com>]

Large code drops are almost always damaging, since inherent in that process is the concept of "throwing the code over a wall". But sometimes it does work out, assuming that continuity and "good intentions" are followed.

To show this, join me in the Wayback Machine as Sherman and I travel to the year 1995...

This is right around the start of Apache, back when Apache meant the web server, and at the time, the project was basically what was left of the NCSA web server plus some patches and bug fixes... Around this time, one of the core group, Robert Thau, started independent work on a re-architecture of the server, which he code-named "Shambala". It was basically a single contributor effort (himself). One day he simply said to the group, "Here, I have this new design and architecture for Apache. It adds a lot of features." So much of what defines httpd today can find its origin right there: modular framework, pools, preforking (and, as such, the initial gleaming towards MPMs), extendable API, etc...

In many ways, this was a large code drop. What made it different is that there was *support* by the author and the community to work on integrating it into the whole. It became, basically, a community effort.

Now compare that with a different scenario... Once httpd had picked up steam, and making sure that it was ported to everyone's favorite *nix flavor was important, SGI had done work on a set of patches that ported httpd to their OS and provided these patches (a set of 10 very large patch-files, iirc) to the group. What was clear in those patches is that there was no consideration at all on how those patches affected or broke anyone else. They rewrote huge swaths of code, optimizing for SGI and totally destroying any sort of portability for anyone else. And when we responded by, asking for more information, help with chatting with their developers to try to figure things out, and basically trying to figure out how to use and merge this stuff, SGI was basically just silent. They sent it to us and that was the beginning and the end of their involvement as far as they were concerned.[1]

Way, way too many large code drops are the latter. Hardly any are the former.


1. I have paraphrased both the Shambala and SGI events
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Jim Jagielski <ji...@jaguNET.com>]

Large code drops are almost always damaging, since inherent in that process is the concept of "throwing the code over a wall". But sometimes it does work out, assuming that continuity and "good intentions" are followed.

To show this, join me in the Wayback Machine as Sherman and I travel to the year 1995...

This is right around the start of Apache, back when Apache meant the web server, and at the time, the project was basically what was left of the NCSA web server plus some patches and bug fixes... Around this time, one of the core group, Robert Thau, started independent work on a re-architecture of the server, which he code-named "Shambala". It was basically a single contributor effort (himself). One day he simply said to the group, "Here, I have this new design and architecture for Apache. It adds a lot of features." So much of what defines httpd today can find its origin right there: modular framework, pools, preforking (and, as such, the initial gleaming towards MPMs), extendable API, etc...

In many ways, this was a large code drop. What made it different is that there was *support* by the author and the community to work on integrating it into the whole. It became, basically, a community effort.

Now compare that with a different scenario... Once httpd had picked up steam, and making sure that it was ported to everyone's favorite *nix flavor was important, SGI had done work on a set of patches that ported httpd to their OS and provided these patches (a set of 10 very large patch-files, iirc) to the group. What was clear in those patches is that there was no consideration at all on how those patches affected or broke anyone else. They rewrote huge swaths of code, optimizing for SGI and totally destroying any sort of portability for anyone else. And when we responded by, asking for more information, help with chatting with their developers to try to figure things out, and basically trying to figure out how to use and merge this stuff, SGI was basically just silent. They sent it to us and that was the beginning and the end of their involvement as far as they were concerned.[1]

Way, way too many large code drops are the latter. Hardly any are the former.


1. I have paraphrased both the Shambala and SGI events

RE: Why are large code drops damaging to a community?

[Sendoro Juma <se...@singo.africa>]

+1

> 
>     On October 19, 2018 at 4:18 AM ross@gardler.org wrote:
> 
>     +1 to both Shane and Myrle's input here.
> 
>     The Apache Way is all about consensus building in order to maximize the potential for collaboration between partners. It is impossible to drive consensus within a community of individuals without enabling them to be a part of the whole process. Large code drops equate to a statement of "this is the way it is - my way or the highway". It is not a model for consensus building.
> 
>     However, there are other models in which vendors define direction. Shane refers to these as "maintainer led" but I think that is too general, I prefer the more specific vendor led, because it is possible to have a consensus driven maintainer led project.
> 
>     Vendor led and the Apache Way are different models. One scales the community very well (Apache Way) and is ideal for building frameworks and/or components from which products are built. The other (vendor led) doesn't scale so well but is ideal for building highly focused products. The Apache Way maximizes the opportunity for cross-organizational collaboration and thus drives combinatorial innovation. Vendor led limits the scope of the collaboration but allows one to target a more clearly defined customer base.
> 
>     The trick to success is ensuring that you are using the right model for the right parts of the open source ecosystem. There is no single model for governance of open source, success comes from understand when and how to apply different models to different parts of your software solution.
> 
>     Ross
> 
>     -----Original Message-----
>     From: Shane Curcuru <as...@shanecurcuru.org>
>     Sent: Thursday, October 18, 2018 8:26 PM
>     To: Apache Community Development <de...@community.apache.org>; Apache Fineract Dev <de...@fineract.apache.org>
>     Subject: Re: Why are large code drops damaging to a community?
> 
>     Myrle Krantz wrote on 10/18/18 7:18 AM:
> 
>         > > 
> >         Hey all,
> > 
> >         There are many forms of offlist development. One form of offlist
> >         development is working on large code drops in private and then
> >         contributing them all at once. Threshold size is probably arguable,
> >         and varies by project; put that aside for the moment. I've been
> >         working on an explanation of how large code drops damage community and
> >         code. I'd love to hear your feedback. I'm including my project and
> >         the dev@community list in the hopes that people from other projects
> >         also have a perspective. Here it goes:
> > 
> >     > 
>     Thanks Myrle for an excellent writeup, including details of how large code drops harm Apache style open communities. This is a great set of examples showing the "why" behind the ASF's requirement that the whole community be allowed to participate in *all* parts of a project's technical development.
> 
>     The requirement for an open and consensus-driven development process at the ASF doesn't just impact the individual perspective - which you've covered in detail in your post. More importantly, it impacts the whole
>     *community* ownership feeling of an Apache project.
> 
>     This is a key difference I see between Apache projects and maintainer-led projects. There are many examples of maintainer projects with issues, successes, whatever. But they primarily focus on how a handful of maintainers are struggling to keep their project growing for the benefit of all their users.
> 
>     The issue is the mindset of "maintainers". An Apache project should have the entire community of contributors feeling like they have a stake in the project, that they can help build new ideas, and that they
>     *share* as a whole group the direction of the project. While much of the actions are the same, my point is about the feeling of ownership.
>     It's not just a few "maintainers" driving things; it's *everyone* who's a committer (and hopefully soon, many of the contributors who get voted in).
> 
>     Offlist development prevents this truly shared sense of ownership from developing, which is why it's prohibited for Apache projects.
> 
>     ...snip...
> 
>         > > 
> >         Open source projects require transparency, not just as a moral value,
> >         but as a pragmatic prerequisite for collaboration. Offlist
> >         development damages the community *and* the code.
> > 
> >     > 
>     More to the point, repeated significant offlist development will attract the attention of the ASF board, which may well take direct action to prevent that kind of behavior from happening going forward. One advantage of the ASF's independent governance is that it can enforce the core open development model that's a key part of the Apache Way.
> 
>     Note that I'm struggling to find where "offlist development is forbidden" is explicitly called out on the apache.org website, so pointers to existing places that *explain* this core concept and why it's important are appreciated.
> 
>     My attempt to explain how open development works is here:
> 
>     http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24
> 
>         * Telegraph your intent: email dev@ with your *ideas* ahead of time.
>           This allows feedback, encouragement, someone else to point out similar code is already over there, etc.
> 
>         * Draft designs openly. Put the rough first draft on the wiki/website/dev@ list, and then do your edits *in public*. This allows feedback on the architecture as it's being built, and again, gets better ideas. It also allows a sense of community ownership.
> 
>         * Submit work in chunks (add: on a regular and frequent basis). Checkin the shell of the API. Then checkin each section of implementation. If you're waiting for your code to look perfect before showing anyone else, you're not really helping the community. Doing the development in the open allows for... you guessed it, feedback.
> 
>         * Welcome feedback along the way. This doesn't mean you need to accept every change request or suggestion. But it does mean you can take the best ideas from the whole community to add them easily, as the individual bits of work are being done.
> 
>     --
> 
>         * Shane
>           ComDev PMC
>           The Apache Software Foundation
> 
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
>     For additional commands, e-mail: dev-help@community.apache.org
> 


 

RE: Why are large code drops damaging to a community?

[Sendoro Juma <se...@singo.africa>]

+1

> 
>     On October 19, 2018 at 4:18 AM ross@gardler.org wrote:
> 
>     +1 to both Shane and Myrle's input here.
> 
>     The Apache Way is all about consensus building in order to maximize the potential for collaboration between partners. It is impossible to drive consensus within a community of individuals without enabling them to be a part of the whole process. Large code drops equate to a statement of "this is the way it is - my way or the highway". It is not a model for consensus building.
> 
>     However, there are other models in which vendors define direction. Shane refers to these as "maintainer led" but I think that is too general, I prefer the more specific vendor led, because it is possible to have a consensus driven maintainer led project.
> 
>     Vendor led and the Apache Way are different models. One scales the community very well (Apache Way) and is ideal for building frameworks and/or components from which products are built. The other (vendor led) doesn't scale so well but is ideal for building highly focused products. The Apache Way maximizes the opportunity for cross-organizational collaboration and thus drives combinatorial innovation. Vendor led limits the scope of the collaboration but allows one to target a more clearly defined customer base.
> 
>     The trick to success is ensuring that you are using the right model for the right parts of the open source ecosystem. There is no single model for governance of open source, success comes from understand when and how to apply different models to different parts of your software solution.
> 
>     Ross
> 
>     -----Original Message-----
>     From: Shane Curcuru <as...@shanecurcuru.org>
>     Sent: Thursday, October 18, 2018 8:26 PM
>     To: Apache Community Development <de...@community.apache.org>; Apache Fineract Dev <de...@fineract.apache.org>
>     Subject: Re: Why are large code drops damaging to a community?
> 
>     Myrle Krantz wrote on 10/18/18 7:18 AM:
> 
>         > > 
> >         Hey all,
> > 
> >         There are many forms of offlist development. One form of offlist
> >         development is working on large code drops in private and then
> >         contributing them all at once. Threshold size is probably arguable,
> >         and varies by project; put that aside for the moment. I've been
> >         working on an explanation of how large code drops damage community and
> >         code. I'd love to hear your feedback. I'm including my project and
> >         the dev@community list in the hopes that people from other projects
> >         also have a perspective. Here it goes:
> > 
> >     > 
>     Thanks Myrle for an excellent writeup, including details of how large code drops harm Apache style open communities. This is a great set of examples showing the "why" behind the ASF's requirement that the whole community be allowed to participate in *all* parts of a project's technical development.
> 
>     The requirement for an open and consensus-driven development process at the ASF doesn't just impact the individual perspective - which you've covered in detail in your post. More importantly, it impacts the whole
>     *community* ownership feeling of an Apache project.
> 
>     This is a key difference I see between Apache projects and maintainer-led projects. There are many examples of maintainer projects with issues, successes, whatever. But they primarily focus on how a handful of maintainers are struggling to keep their project growing for the benefit of all their users.
> 
>     The issue is the mindset of "maintainers". An Apache project should have the entire community of contributors feeling like they have a stake in the project, that they can help build new ideas, and that they
>     *share* as a whole group the direction of the project. While much of the actions are the same, my point is about the feeling of ownership.
>     It's not just a few "maintainers" driving things; it's *everyone* who's a committer (and hopefully soon, many of the contributors who get voted in).
> 
>     Offlist development prevents this truly shared sense of ownership from developing, which is why it's prohibited for Apache projects.
> 
>     ...snip...
> 
>         > > 
> >         Open source projects require transparency, not just as a moral value,
> >         but as a pragmatic prerequisite for collaboration. Offlist
> >         development damages the community *and* the code.
> > 
> >     > 
>     More to the point, repeated significant offlist development will attract the attention of the ASF board, which may well take direct action to prevent that kind of behavior from happening going forward. One advantage of the ASF's independent governance is that it can enforce the core open development model that's a key part of the Apache Way.
> 
>     Note that I'm struggling to find where "offlist development is forbidden" is explicitly called out on the apache.org website, so pointers to existing places that *explain* this core concept and why it's important are appreciated.
> 
>     My attempt to explain how open development works is here:
> 
>     http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24
> 
>         * Telegraph your intent: email dev@ with your *ideas* ahead of time.
>           This allows feedback, encouragement, someone else to point out similar code is already over there, etc.
> 
>         * Draft designs openly. Put the rough first draft on the wiki/website/dev@ list, and then do your edits *in public*. This allows feedback on the architecture as it's being built, and again, gets better ideas. It also allows a sense of community ownership.
> 
>         * Submit work in chunks (add: on a regular and frequent basis). Checkin the shell of the API. Then checkin each section of implementation. If you're waiting for your code to look perfect before showing anyone else, you're not really helping the community. Doing the development in the open allows for... you guessed it, feedback.
> 
>         * Welcome feedback along the way. This doesn't mean you need to accept every change request or suggestion. But it does mean you can take the best ideas from the whole community to add them easily, as the individual bits of work are being done.
> 
>     --
> 
>         * Shane
>           ComDev PMC
>           The Apache Software Foundation
> 
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
>     For additional commands, e-mail: dev-help@community.apache.org
> 


 

RE: Why are large code drops damaging to a community?

[ro...@gardler.org]

+1 to both Shane and Myrle's input here.

The Apache Way is all about consensus building in order to maximize the potential for collaboration between partners. It is impossible to drive consensus within a community of individuals without enabling them to be a part of the whole process. Large code drops equate to a statement of "this is the way it is - my way or the highway". It is not a model for consensus building.

However, there are other models in which vendors define direction. Shane refers to these as "maintainer led" but I think that is too general, I prefer the more specific vendor led, because it is possible to have a consensus driven maintainer led project. 

Vendor led and the Apache Way are different models. One scales the community very well (Apache Way) and is ideal for building frameworks and/or components from which products are built. The other (vendor led) doesn't scale so well but is ideal for building highly focused products. The Apache Way maximizes the opportunity for cross-organizational collaboration and thus drives combinatorial innovation. Vendor led limits the scope of the collaboration but allows one to target a more clearly defined customer base.

The trick to success is ensuring that you are using the right model for the right parts of the open source ecosystem. There is no single model for governance of open source, success comes from understand when and how to apply different models to different parts of your software solution.

Ross



-----Original Message-----
From: Shane Curcuru <as...@shanecurcuru.org> 
Sent: Thursday, October 18, 2018 8:26 PM
To: Apache Community Development <de...@community.apache.org>; Apache Fineract Dev <de...@fineract.apache.org>
Subject: Re: Why are large code drops damaging to a community?

Myrle Krantz wrote on 10/18/18 7:18 AM:
> Hey all,
> 
> There are many forms of offlist development.  One form of offlist 
> development is working on large code drops in private and then 
> contributing them all at once.  Threshold size is probably arguable, 
> and varies by project; put that aside for the moment.  I've been 
> working on an explanation of how large code drops damage community and 
> code.  I'd love to hear your feedback.  I'm including my project and 
> the dev@community list in the hopes that people from other projects 
> also have a perspective.  Here it goes:

Thanks Myrle for an excellent writeup, including details of how large code drops harm Apache style open communities.  This is a great set of examples showing the "why" behind the ASF's requirement that the whole community be allowed to participate in *all* parts of a project's technical development.

The requirement for an open and consensus-driven development process at the ASF doesn't just impact the individual perspective - which you've covered in detail in your post.  More importantly, it impacts the whole
*community* ownership feeling of an Apache project.

This is a key difference I see between Apache projects and maintainer-led projects.  There are many examples of maintainer projects with issues, successes, whatever.  But they primarily focus on how a handful of maintainers are struggling to keep their project growing for the benefit of all their users.

The issue is the mindset of "maintainers".  An Apache project should have the entire community of contributors feeling like they have a stake in the project, that they can help build new ideas, and that they
*share* as a whole group the direction of the project.  While much of the actions are the same, my point is about the feeling of ownership.
It's not just a few "maintainers" driving things; it's *everyone* who's a committer (and hopefully soon, many of the contributors who get voted in).

Offlist development prevents this truly shared sense of ownership from developing, which is why it's prohibited for Apache projects.

...snip...
> Open source projects require transparency, not just as a moral value, 
> but as a pragmatic prerequisite for collaboration.  Offlist 
> development damages the community *and* the code.

More to the point, repeated significant offlist development will attract the attention of the ASF board, which may well take direct action to prevent that kind of behavior from happening going forward.  One advantage of the ASF's independent governance is that it can enforce the core open development model that's a key part of the Apache Way.

Note that I'm struggling to find where "offlist development is forbidden" is explicitly called out on the apache.org website, so pointers to existing places that *explain* this core concept and why it's important are appreciated.

My attempt to explain how open development works is here:

  http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24

- Telegraph your intent: email dev@ with your *ideas* ahead of time.
This allows feedback, encouragement, someone else to point out similar code is already over there, etc.

- Draft designs openly.  Put the rough first draft on the wiki/website/dev@ list, and then do your edits *in public*.  This allows feedback on the architecture as it's being built, and again, gets better ideas.  It also allows a sense of community ownership.

- Submit work in chunks (add: on a regular and frequent basis).  Checkin the shell of the API.  Then checkin each section of implementation.  If you're waiting for your code to look perfect before showing anyone else, you're not really helping the community.  Doing the development in the open allows for... you guessed it, feedback.

- Welcome feedback along the way.  This doesn't mean you need to accept every change request or suggestion.  But it does mean you can take the best ideas from the whole community to add them easily, as the individual bits of work are being done.

-- 

- Shane
  ComDev PMC
  The Apache Software Foundation

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

RE: Why are large code drops damaging to a community?

[ro...@gardler.org]

+1 to both Shane and Myrle's input here.

The Apache Way is all about consensus building in order to maximize the potential for collaboration between partners. It is impossible to drive consensus within a community of individuals without enabling them to be a part of the whole process. Large code drops equate to a statement of "this is the way it is - my way or the highway". It is not a model for consensus building.

However, there are other models in which vendors define direction. Shane refers to these as "maintainer led" but I think that is too general, I prefer the more specific vendor led, because it is possible to have a consensus driven maintainer led project. 

Vendor led and the Apache Way are different models. One scales the community very well (Apache Way) and is ideal for building frameworks and/or components from which products are built. The other (vendor led) doesn't scale so well but is ideal for building highly focused products. The Apache Way maximizes the opportunity for cross-organizational collaboration and thus drives combinatorial innovation. Vendor led limits the scope of the collaboration but allows one to target a more clearly defined customer base.

The trick to success is ensuring that you are using the right model for the right parts of the open source ecosystem. There is no single model for governance of open source, success comes from understand when and how to apply different models to different parts of your software solution.

Ross



-----Original Message-----
From: Shane Curcuru <as...@shanecurcuru.org> 
Sent: Thursday, October 18, 2018 8:26 PM
To: Apache Community Development <de...@community.apache.org>; Apache Fineract Dev <de...@fineract.apache.org>
Subject: Re: Why are large code drops damaging to a community?

Myrle Krantz wrote on 10/18/18 7:18 AM:
> Hey all,
> 
> There are many forms of offlist development.  One form of offlist 
> development is working on large code drops in private and then 
> contributing them all at once.  Threshold size is probably arguable, 
> and varies by project; put that aside for the moment.  I've been 
> working on an explanation of how large code drops damage community and 
> code.  I'd love to hear your feedback.  I'm including my project and 
> the dev@community list in the hopes that people from other projects 
> also have a perspective.  Here it goes:

Thanks Myrle for an excellent writeup, including details of how large code drops harm Apache style open communities.  This is a great set of examples showing the "why" behind the ASF's requirement that the whole community be allowed to participate in *all* parts of a project's technical development.

The requirement for an open and consensus-driven development process at the ASF doesn't just impact the individual perspective - which you've covered in detail in your post.  More importantly, it impacts the whole
*community* ownership feeling of an Apache project.

This is a key difference I see between Apache projects and maintainer-led projects.  There are many examples of maintainer projects with issues, successes, whatever.  But they primarily focus on how a handful of maintainers are struggling to keep their project growing for the benefit of all their users.

The issue is the mindset of "maintainers".  An Apache project should have the entire community of contributors feeling like they have a stake in the project, that they can help build new ideas, and that they
*share* as a whole group the direction of the project.  While much of the actions are the same, my point is about the feeling of ownership.
It's not just a few "maintainers" driving things; it's *everyone* who's a committer (and hopefully soon, many of the contributors who get voted in).

Offlist development prevents this truly shared sense of ownership from developing, which is why it's prohibited for Apache projects.

...snip...
> Open source projects require transparency, not just as a moral value, 
> but as a pragmatic prerequisite for collaboration.  Offlist 
> development damages the community *and* the code.

More to the point, repeated significant offlist development will attract the attention of the ASF board, which may well take direct action to prevent that kind of behavior from happening going forward.  One advantage of the ASF's independent governance is that it can enforce the core open development model that's a key part of the Apache Way.

Note that I'm struggling to find where "offlist development is forbidden" is explicitly called out on the apache.org website, so pointers to existing places that *explain* this core concept and why it's important are appreciated.

My attempt to explain how open development works is here:

  http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24

- Telegraph your intent: email dev@ with your *ideas* ahead of time.
This allows feedback, encouragement, someone else to point out similar code is already over there, etc.

- Draft designs openly.  Put the rough first draft on the wiki/website/dev@ list, and then do your edits *in public*.  This allows feedback on the architecture as it's being built, and again, gets better ideas.  It also allows a sense of community ownership.

- Submit work in chunks (add: on a regular and frequent basis).  Checkin the shell of the API.  Then checkin each section of implementation.  If you're waiting for your code to look perfect before showing anyone else, you're not really helping the community.  Doing the development in the open allows for... you guessed it, feedback.

- Welcome feedback along the way.  This doesn't mean you need to accept every change request or suggestion.  But it does mean you can take the best ideas from the whole community to add them easily, as the individual bits of work are being done.

-- 

- Shane
  ComDev PMC
  The Apache Software Foundation

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Shane Curcuru <as...@shanecurcuru.org>]

Myrle Krantz wrote on 10/18/18 7:18 AM:
> Hey all,
> 
> There are many forms of offlist development.  One form of offlist
> development is working on large code drops in private and then
> contributing them all at once.  Threshold size is probably arguable,
> and varies by project; put that aside for the moment.  I've been
> working on an explanation of how large code drops damage community and
> code.  I'd love to hear your feedback.  I'm including my project and
> the dev@community list in the hopes that people from other projects
> also have a perspective.  Here it goes:

Thanks Myrle for an excellent writeup, including details of how large
code drops harm Apache style open communities.  This is a great set of
examples showing the "why" behind the ASF's requirement that the whole
community be allowed to participate in *all* parts of a project's
technical development.

The requirement for an open and consensus-driven development process at
the ASF doesn't just impact the individual perspective - which you've
covered in detail in your post.  More importantly, it impacts the whole
*community* ownership feeling of an Apache project.

This is a key difference I see between Apache projects and
maintainer-led projects.  There are many examples of maintainer projects
with issues, successes, whatever.  But they primarily focus on how a
handful of maintainers are struggling to keep their project growing for
the benefit of all their users.

The issue is the mindset of "maintainers".  An Apache project should
have the entire community of contributors feeling like they have a stake
in the project, that they can help build new ideas, and that they
*share* as a whole group the direction of the project.  While much of
the actions are the same, my point is about the feeling of ownership.
It's not just a few "maintainers" driving things; it's *everyone* who's
a committer (and hopefully soon, many of the contributors who get voted in).

Offlist development prevents this truly shared sense of ownership from
developing, which is why it's prohibited for Apache projects.

...snip...
> Open source projects require transparency, not just as a moral value,
> but as a pragmatic prerequisite for collaboration.  Offlist
> development damages the community *and* the code.

More to the point, repeated significant offlist development will attract
the attention of the ASF board, which may well take direct action to
prevent that kind of behavior from happening going forward.  One
advantage of the ASF's independent governance is that it can enforce the
core open development model that's a key part of the Apache Way.

Note that I'm struggling to find where "offlist development is
forbidden" is explicitly called out on the apache.org website, so
pointers to existing places that *explain* this core concept and why
it's important are appreciated.

My attempt to explain how open development works is here:

  http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24

- Telegraph your intent: email dev@ with your *ideas* ahead of time.
This allows feedback, encouragement, someone else to point out similar
code is already over there, etc.

- Draft designs openly.  Put the rough first draft on the
wiki/website/dev@ list, and then do your edits *in public*.  This allows
feedback on the architecture as it's being built, and again, gets better
ideas.  It also allows a sense of community ownership.

- Submit work in chunks (add: on a regular and frequent basis).  Checkin
the shell of the API.  Then checkin each section of implementation.  If
you're waiting for your code to look perfect before showing anyone else,
you're not really helping the community.  Doing the development in the
open allows for... you guessed it, feedback.

- Welcome feedback along the way.  This doesn't mean you need to accept
every change request or suggestion.  But it does mean you can take the
best ideas from the whole community to add them easily, as the
individual bits of work are being done.

-- 

- Shane
  ComDev PMC
  The Apache Software Foundation

Re: Why are large code drops damaging to a community?

[zeljko leskur <zz...@gmail.com>]

Hi ! Thank you very much for your mail and friendly and very I might Said
expert explanation . I do know that you're doing your Job but anyway I Am
highly respecting you and your knowledge. So I'll try to explain my
position and  at the same time descrobe my position. I will do that
honestly and real statement. So, I do love on Croatia or Hrvatska. My
formal education is economy high school . IT is likely half of university.
I'm fifty five now and I Am a retired Sergeant of Croatoan army . IT was
war Here but IT doesn't relating about our convetsation. I was started with
computers and net before ten years. I am working at laptops strictly and
this one is third since 2009.yrs. I was passed through all Windows
configurations and just before two months started with Linux. I did not
looked myself likely informatic knowledgeable person.I am just user who
knows
Dana 18. 10. 2018. 13:18 osoba "Myrle Krantz" <my...@apache.org> napisala
je:

> Hey all,
>
> There are many forms of offlist development.  One form of offlist
> development is working on large code drops in private and then
> contributing them all at once.  Threshold size is probably arguable,
> and varies by project; put that aside for the moment.  I've been
> working on an explanation of how large code drops damage community and
> code.  I'd love to hear your feedback.  I'm including my project and
> the dev@community list in the hopes that people from other projects
> also have a perspective.  Here it goes:
>
>
> Imagine you are an individual contributor on a project.  You would
> like to contribute something.  You see a feature you'd like to add or
> a bug you'd like to fix, a user you would like to support, or a
> release you'd like to test.  You start on it.  You submit your pull
> request, you answer the user's question, you test the release.  You
> continue doing this at a low level for a few months.  You see other
> people starting to contribute too.  This is nice.  You're working
> together with others towards a common goal.  Then, out of the blue a
> company with multiple paid contributors shows up.  Let's name them
> Acme. Acme drops a year of code on the project.  They could do this
> many ways.  For example:  A.) Acme could develop in the repository you
> were working in, or B.) Acme could create a project-internal fork and
> create a new repository. C.) Acme could even telegraph months in
> advance that they intend to do this, by posting to the dev list or
> contacting key contributors offlist, or just by having done it a few
> times already.
>
>
> A.) First let's imagine that Acme made massive changes in the
> repository you were working in.  Perhaps they already solved the
> problem you solved, but in a different way.  Perhaps, they deleted
> functions you made changes in.  Perhaps they added significant
> functionality you would have liked to help with.  What good were your
> efforts?  Wouldn't you find this discouraging?
>
> And now you want to continue to make changes, but the code you want to
> change has commit messages referencing tickets which you have no
> access to.  Or it has no reference to tickets at all.  You find an
> area that seems to be needlessly complex: can you remove the
> complexity?  You have no way of knowing what you'd be breaking.
>
> Perhaps you have a proprietary UI which depends on a behavior which
> was removed or changed.  Now your UI is broken.  Because the code drop
> is so large, you have no way to reasonably review it for
> incompatibilities.  It is not possible to review a year of development
> all at once.  And if your review turns up problems?  Do you accept the
> entire pull request or decline the whole thing?  Putting all the code
> into one pull request is a form of blackmail (commonly used in the
> formulation of bills for Congress).  If you want the good you have to
> take the bad.
>
>
> B.) Now let's imagine that Acme forked the code and created a new
> repository which they then added to the project.  None of the work you
> did is in this new repository.  If those features you implemented were
> important to you, you will have to re-introduce them into the new
> repository.
>
> You'll have to start from zero learning to work in the new repository.
> You also had no say in how that code was developed, so maybe the
> feature that you need is unnecessarily difficult to implement in that
> repository.   You don't know why things are the way they are there, so
> you're walking through a mine field without a map when you're making
> changes.
>
> And anyways, why is Acme Corp so certain you had nothing of value to add?
>
> Releasing this code also becomes contentious. Which of the two
> competing repositories gets released?  Both of them? How does the
> project communicate to users about how these pieces fit together.
>
>
> C.) Imagine Acme gave you lots of forewarning that this was coming.
> You still have no say in how the code is developed.  You know that
> anything you might contribute could be obsoleted.  You can't tell
> users whether the up-and-coming release will be compatible.  And
> what's the point in testing that release?  You don't know how to check
> that your needs are being considered in the architecture of the new
> code base.
>
> You have no sense of ownership over what comes out of that process.
>
> You see that nobody else outside of Acme is working on the project
> either, for the same reasons.
>
>
> Most contributors would get discouraged and prefer not to participate
> if those were the conditions.  If contributors didn't get discouraged,
> they would fairly quickly be taking orders from the employees of Acme
> Corp.  Acme Corp has all the inside information about what's coming in
> a year in the next code dump.  Information is power.  Contributors who
> are also users may also chose to stop contributing and become free
> riders.  Why not just depend on Acme Corp for all of the development?
>
> What Acme seems to be getting out of this scenario is an Apache
> feather.  It's a form of free-riding on Apache's reputation.
>
>
> Now let's imagine that you are the CTO of another company, let's call
> them Kaushal.  Kaushal is considering taking part in this open source
> project, but they are a competitor to Acme.  As Kaushal's CTO, you can
> see, based on commit history, and participation that Acme is
> dominating the project.  You would be smart to expect that Acme would
> take advantage of their dominant position in the project.  Acme could
> deliberately sabotage Kaushal's use cases, or simply 'starve' them by
> convincing people not to help Kaushal.  Kaushal's CTO could respond to
> this threat in one of two ways: 1.) Simply not take part on the open
> source project.  Create their own closed source thing, or their own
> open source project, and not engage.  This is the most likely
> response.  2.) Try to dominate the project themselves.  Kaushal has
> the same tools available that Acme has. Kaushal's CTO could tell his
> employees to do long-interval code drops just like Acme is doing.  Now
> with two corporations doing long-interval code drops on a project,
> merging the code becomes very very difficult.  Fights about who gets
> to decide what could eventually cause a complete cessation of release
> activity.
>
>
> So imagine that all competitors chose to remain just users, and Acme
> remains in control.  Now imagine Acme loses interest in the project.
> Acme found something that will make them more money, or Acme's
> business fails.  Or Acme gets tired of offering their development
> resources to the free riders.  Acme stops contributing to the project.
> But the project has become so dependent on Acme that it cannot exist
> without Acme.  When Acme exits, project activity could end.
>
>
> Open source projects require transparency, not just as a moral value,
> but as a pragmatic prerequisite for collaboration.  Offlist
> development damages the community *and* the code.
>
> Best Regards,
> Myrle
>
> P.S.  Some very interesting research on the game-theoretical aspects
> of modularization in open source:
> http://people.hbs.edu/cbaldwin/DR2/BaldwinClark.ArchOS.Jun03.pdf
> "Does Code Architecture Mitigate Free Riding in the Open Source
> Development Model?"
>
> I would argue that the information divisibility being applied here at
> the code modularity dimension also applies to the time dimension.  So,
> it seems likely the argument against large code drops can be made
> mathematically. Which really tickles the geek in me. : o)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Shane Curcuru <as...@shanecurcuru.org>]

Myrle Krantz wrote on 10/18/18 7:18 AM:
> Hey all,
> 
> There are many forms of offlist development.  One form of offlist
> development is working on large code drops in private and then
> contributing them all at once.  Threshold size is probably arguable,
> and varies by project; put that aside for the moment.  I've been
> working on an explanation of how large code drops damage community and
> code.  I'd love to hear your feedback.  I'm including my project and
> the dev@community list in the hopes that people from other projects
> also have a perspective.  Here it goes:

Thanks Myrle for an excellent writeup, including details of how large
code drops harm Apache style open communities.  This is a great set of
examples showing the "why" behind the ASF's requirement that the whole
community be allowed to participate in *all* parts of a project's
technical development.

The requirement for an open and consensus-driven development process at
the ASF doesn't just impact the individual perspective - which you've
covered in detail in your post.  More importantly, it impacts the whole
*community* ownership feeling of an Apache project.

This is a key difference I see between Apache projects and
maintainer-led projects.  There are many examples of maintainer projects
with issues, successes, whatever.  But they primarily focus on how a
handful of maintainers are struggling to keep their project growing for
the benefit of all their users.

The issue is the mindset of "maintainers".  An Apache project should
have the entire community of contributors feeling like they have a stake
in the project, that they can help build new ideas, and that they
*share* as a whole group the direction of the project.  While much of
the actions are the same, my point is about the feeling of ownership.
It's not just a few "maintainers" driving things; it's *everyone* who's
a committer (and hopefully soon, many of the contributors who get voted in).

Offlist development prevents this truly shared sense of ownership from
developing, which is why it's prohibited for Apache projects.

...snip...
> Open source projects require transparency, not just as a moral value,
> but as a pragmatic prerequisite for collaboration.  Offlist
> development damages the community *and* the code.

More to the point, repeated significant offlist development will attract
the attention of the ASF board, which may well take direct action to
prevent that kind of behavior from happening going forward.  One
advantage of the ASF's independent governance is that it can enforce the
core open development model that's a key part of the Apache Way.

Note that I'm struggling to find where "offlist development is
forbidden" is explicitly called out on the apache.org website, so
pointers to existing places that *explain* this core concept and why
it's important are appreciated.

My attempt to explain how open development works is here:

  http://shaneslides.com/apachecon/TheApacheWay-ApacheConNA2018.html#24

- Telegraph your intent: email dev@ with your *ideas* ahead of time.
This allows feedback, encouragement, someone else to point out similar
code is already over there, etc.

- Draft designs openly.  Put the rough first draft on the
wiki/website/dev@ list, and then do your edits *in public*.  This allows
feedback on the architecture as it's being built, and again, gets better
ideas.  It also allows a sense of community ownership.

- Submit work in chunks (add: on a regular and frequent basis).  Checkin
the shell of the API.  Then checkin each section of implementation.  If
you're waiting for your code to look perfect before showing anyone else,
you're not really helping the community.  Doing the development in the
open allows for... you guessed it, feedback.

- Welcome feedback along the way.  This doesn't mean you need to accept
every change request or suggestion.  But it does mean you can take the
best ideas from the whole community to add them easily, as the
individual bits of work are being done.

-- 

- Shane
  ComDev PMC
  The Apache Software Foundation

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[zeljko leskur <zz...@gmail.com>]

Something about this topic Firstly cause of practical reasons. As I have
noticed I Am opened to every colaboration and I would Like to give hand if
I could help withouth any obligations. Just friendly and freely. I do
appreciate you and your working. Sincerely yours.--Zeljko Leskur.
Dana 18. 10. 2018. 13:18 osoba "Myrle Krantz" <my...@apache.org> napisala
je:

> Hey all,
>
> There are many forms of offlist development.  One form of offlist
> development is working on large code drops in private and then
> contributing them all at once.  Threshold size is probably arguable,
> and varies by project; put that aside for the moment.  I've been
> working on an explanation of how large code drops damage community and
> code.  I'd love to hear your feedback.  I'm including my project and
> the dev@community list in the hopes that people from other projects
> also have a perspective.  Here it goes:
>
>
> Imagine you are an individual contributor on a project.  You would
> like to contribute something.  You see a feature you'd like to add or
> a bug you'd like to fix, a user you would like to support, or a
> release you'd like to test.  You start on it.  You submit your pull
> request, you answer the user's question, you test the release.  You
> continue doing this at a low level for a few months.  You see other
> people starting to contribute too.  This is nice.  You're working
> together with others towards a common goal.  Then, out of the blue a
> company with multiple paid contributors shows up.  Let's name them
> Acme. Acme drops a year of code on the project.  They could do this
> many ways.  For example:  A.) Acme could develop in the repository you
> were working in, or B.) Acme could create a project-internal fork and
> create a new repository. C.) Acme could even telegraph months in
> advance that they intend to do this, by posting to the dev list or
> contacting key contributors offlist, or just by having done it a few
> times already.
>
>
> A.) First let's imagine that Acme made massive changes in the
> repository you were working in.  Perhaps they already solved the
> problem you solved, but in a different way.  Perhaps, they deleted
> functions you made changes in.  Perhaps they added significant
> functionality you would have liked to help with.  What good were your
> efforts?  Wouldn't you find this discouraging?
>
> And now you want to continue to make changes, but the code you want to
> change has commit messages referencing tickets which you have no
> access to.  Or it has no reference to tickets at all.  You find an
> area that seems to be needlessly complex: can you remove the
> complexity?  You have no way of knowing what you'd be breaking.
>
> Perhaps you have a proprietary UI which depends on a behavior which
> was removed or changed.  Now your UI is broken.  Because the code drop
> is so large, you have no way to reasonably review it for
> incompatibilities.  It is not possible to review a year of development
> all at once.  And if your review turns up problems?  Do you accept the
> entire pull request or decline the whole thing?  Putting all the code
> into one pull request is a form of blackmail (commonly used in the
> formulation of bills for Congress).  If you want the good you have to
> take the bad.
>
>
> B.) Now let's imagine that Acme forked the code and created a new
> repository which they then added to the project.  None of the work you
> did is in this new repository.  If those features you implemented were
> important to you, you will have to re-introduce them into the new
> repository.
>
> You'll have to start from zero learning to work in the new repository.
> You also had no say in how that code was developed, so maybe the
> feature that you need is unnecessarily difficult to implement in that
> repository.   You don't know why things are the way they are there, so
> you're walking through a mine field without a map when you're making
> changes.
>
> And anyways, why is Acme Corp so certain you had nothing of value to add?
>
> Releasing this code also becomes contentious. Which of the two
> competing repositories gets released?  Both of them? How does the
> project communicate to users about how these pieces fit together.
>
>
> C.) Imagine Acme gave you lots of forewarning that this was coming.
> You still have no say in how the code is developed.  You know that
> anything you might contribute could be obsoleted.  You can't tell
> users whether the up-and-coming release will be compatible.  And
> what's the point in testing that release?  You don't know how to check
> that your needs are being considered in the architecture of the new
> code base.
>
> You have no sense of ownership over what comes out of that process.
>
> You see that nobody else outside of Acme is working on the project
> either, for the same reasons.
>
>
> Most contributors would get discouraged and prefer not to participate
> if those were the conditions.  If contributors didn't get discouraged,
> they would fairly quickly be taking orders from the employees of Acme
> Corp.  Acme Corp has all the inside information about what's coming in
> a year in the next code dump.  Information is power.  Contributors who
> are also users may also chose to stop contributing and become free
> riders.  Why not just depend on Acme Corp for all of the development?
>
> What Acme seems to be getting out of this scenario is an Apache
> feather.  It's a form of free-riding on Apache's reputation.
>
>
> Now let's imagine that you are the CTO of another company, let's call
> them Kaushal.  Kaushal is considering taking part in this open source
> project, but they are a competitor to Acme.  As Kaushal's CTO, you can
> see, based on commit history, and participation that Acme is
> dominating the project.  You would be smart to expect that Acme would
> take advantage of their dominant position in the project.  Acme could
> deliberately sabotage Kaushal's use cases, or simply 'starve' them by
> convincing people not to help Kaushal.  Kaushal's CTO could respond to
> this threat in one of two ways: 1.) Simply not take part on the open
> source project.  Create their own closed source thing, or their own
> open source project, and not engage.  This is the most likely
> response.  2.) Try to dominate the project themselves.  Kaushal has
> the same tools available that Acme has. Kaushal's CTO could tell his
> employees to do long-interval code drops just like Acme is doing.  Now
> with two corporations doing long-interval code drops on a project,
> merging the code becomes very very difficult.  Fights about who gets
> to decide what could eventually cause a complete cessation of release
> activity.
>
>
> So imagine that all competitors chose to remain just users, and Acme
> remains in control.  Now imagine Acme loses interest in the project.
> Acme found something that will make them more money, or Acme's
> business fails.  Or Acme gets tired of offering their development
> resources to the free riders.  Acme stops contributing to the project.
> But the project has become so dependent on Acme that it cannot exist
> without Acme.  When Acme exits, project activity could end.
>
>
> Open source projects require transparency, not just as a moral value,
> but as a pragmatic prerequisite for collaboration.  Offlist
> development damages the community *and* the code.
>
> Best Regards,
> Myrle
>
> P.S.  Some very interesting research on the game-theoretical aspects
> of modularization in open source:
> http://people.hbs.edu/cbaldwin/DR2/BaldwinClark.ArchOS.Jun03.pdf
> "Does Code Architecture Mitigate Free Riding in the Open Source
> Development Model?"
>
> I would argue that the information divisibility being applied here at
> the code modularity dimension also applies to the time dimension.  So,
> it seems likely the argument against large code drops can be made
> mathematically. Which really tickles the geek in me. : o)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Myrle Krantz <my...@apache.org>]

I'm very grateful to Joan for sharing this.  It must have been difficult
and time consuming to write this experience down.  It required courage to
put this information in the public, in the context of my original post
which was very critical of offlist development.  I learned a lot from your
story Joan.

I agree with only part of Bertrand's assessment however.  Here we need to
separate the question about whether the large code drop was damaging for
the community from the value judgement about whether it was *right* for the
community.

It's fairly clear that the code drop Joan described *was* damaging to the
community.  They lost contributions, and disrupted people's work.  The
Nebraska committer put in a great deal of time.

But it's also fairly clear that that the code drop Joan described was
*right* for her community anyways.  The "strong will of the contributors to
help the community in the long term, as opposed to just dropping code and
moving on." that Bertrand cites were important mitigating factors which
balanced out those effects.

I'd also add: IBM/Cloudant seemed to have shown considerable compassion for
the community members in the process.  IBM/Cloudant answered the question,
that I posed "why is Acme Corp so certain you had nothing of value to add?"
in a way that redirected the insult inherent in offlist development into
self-criticism.

Best Regards,
Myrle

By the way, the framework I'm applying here is something I learned from
reading "Winning Decisions" by J. Edward Russo.  p. 176 - 180. The basic
idea is that you separate facts from value judgments.  Facts may be fairly
easy to agree on.  And differing value judgments may be fairly easy to
accept.  Given a separation of the two, it can sometimes be easier to find
appropriate compromises and appropriate boundaries.



On Mon, Nov 5, 2018 at 9:55 AM Bertrand Delacretaz <bd...@apache.org>
wrote:

> Hi,
>
> On Sat, Nov 3, 2018 at 8:37 PM Joan Touzet <wo...@apache.org> wrote:
> > ...In a way, it feels a bit like having had bypass surgery, I
> > guess :)
>
> Indeed, thank you very much for sharing this!
>
> I think the conclusion at this point is that large code drops are not
> *necessarily* damaging to a community but handling them requires a lot
> of attention and work. Along with a strong will of the contributors to
> help the community in the long term, as opposed to just dropping code
> and moving on.
>
> -Bertrand
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Re: Why are large code drops damaging to a community?

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Sat, Nov 3, 2018 at 8:37 PM Joan Touzet <wo...@apache.org> wrote:
> ...In a way, it feels a bit like having had bypass surgery, I
> guess :)

Indeed, thank you very much for sharing this!

I think the conclusion at this point is that large code drops are not
*necessarily* damaging to a community but handling them requires a lot
of attention and work. Along with a strong will of the contributors to
help the community in the long term, as opposed to just dropping code
and moving on.

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org

Re: Why are large code drops damaging to a community?

[Joan Touzet <wo...@apache.org>]

Hi Myrle,

Thanks for starting this topic, and thanks to everyone else who has
shared their stories.

I'd like to bring up another scenario that is all-too-common in the
larger Apache projects: commercial "value-added" versions that find
their way back to the ASF after some time. I know this has happened for
CouchDB. I believe it's also happened for Hadoop, for OpenOffice, and
for other projects.

In the case of CouchDB, we did this precisely once, with Cloudant's (now
IBM's) bigcouch fork. CouchDB 1.x was a single-server, standalone
database solution. You could set up replication between multiple 1.x
installations and call it "a cluster," but to each server, there was no
special understanding about those other servers.  They were treated
exactly the same as someone replicating data to a laptop, to a phone
running PouchDB, or some other 3rd party software that was not
understood.

The Cloudant bigcouch fork (OSS itself) overlaid on top of CouchDB
proper clustering semantics. Now, 3 servers would work in tandem to
store and index data, and the key principles of distributed computing
were adhered to. There was considerable interest in bigcouch in the
CouchDB community, with the main question always being: when will this
get merged back into CouchDB itself? It was a major reworking of the
internals of the project, it couldn't be done in piecemeal, and
would certainly result in a version 2.0 if it happened.

On top of this, Cloudant then forked CouchDB again, applied the changes
in their bigcouch fork, then applied proprietary changes for their own
web service offering. This version was known as "dbcore" internal to
Cloudant. Most of the changes were specific to running a business on
hosted CouchDB, but as dbcore started, bigcouch was effectively 
abandoned, bugfixes and patches only landing on the private dbcore
repository.

Internal to Cloudant (disclaimer: I was working there at the time, I am
no longer an employee), there were repeated calls for most of dbcore to
be merged back into Apache CouchDB. There was always interest and
desire, but business imperatives kept getting in the way. We knew it
was a major undertaking.

In 2012, shortly after IBM acquired Cloudant, I lead the dev team in the
"big merge" effort to get the code that was deemed acceptable to OSS
back into CouchDB. There were multiple public blog posts, public pull
requests (on GitHub) showing the progress of the merge, and various
awareness campaigns on the mailing list.  It was not a trivial task; it
involved flying one key developer to spend a week with another key
developer to go through the change history commit by commit. (We joking
referred to this as the "Nebraska Merge.")

All of this was done with the express approval of the project, its PMC,
and the large majority of the contributors to the project. I wouldn't
have agreed to lead the charge if that wasn't the case. CouchDB 2.0.0
released with the new code in September 2016.

That's not to say it wasn't disruptive to the community.  At least one
pending set of changes, adding a new feature (that had also been
developed on someone's private fork!), had to be discarded because it was
not compatible with distributed computing considerations post-merge.
Similarly, many new requests for changes or features have had to be sent
back for redesign because they assumed a single-server model. However,
the flipside is also true: we were able to ditch our older web-based UI
(which was un-maintained for many years) and replace it with a modern
HTML5 interface that has brought many, *many* more JavaScript developers
into the community with its ease of development and maintenance.

One key point is that IBM/Cloudant changed their development model as it
related to CouchDB + proprietary features rapidly after this release.
They moved any development that required changes to the OSS code
directly to the main Apache repository, so that no massive code dumps
would happen in the future. In fact, their continued releases of new
OSS-available functionality - like improved indexing, user-partitioned
databases, clustered purge, and the vastly improved UI - all happen out
in the open, through the same ASF process by which we accept changes
from smaller companies and individual contributors. IBM/Cloudant has
done an outstanding job in this, and we're very happy that they do so.
Without this sort of goodwill and cooperation, as a PMC member, I'd be
nervous about another massive code drop disrupting the CouchDB community
again.

Overall, I'd say the community is stronger post-merge than pre-merge,
but the actual merge itself was very disruptive. I'm also glad it
happened. In a way, it feels a bit like having had bypass surgery, I
guess :)

-Joan


----- Original Message -----
> From: "Myrle Krantz" <my...@apache.org>
> To: dev@community.apache.org, "dev" <de...@fineract.apache.org>
> Sent: Thursday, October 18, 2018 7:18:07 AM
> Subject: Why are large code drops damaging to a community?
> 
> Hey all,
> 
> There are many forms of offlist development.  One form of offlist
> development is working on large code drops in private and then
> contributing them all at once.  Threshold size is probably arguable,
> and varies by project; put that aside for the moment.  I've been
> working on an explanation of how large code drops damage community
> and
> code.  I'd love to hear your feedback.  I'm including my project and
> the dev@community list in the hopes that people from other projects
> also have a perspective.  Here it goes:
> 
> 
> Imagine you are an individual contributor on a project.  You would
> like to contribute something.  You see a feature you'd like to add or
> a bug you'd like to fix, a user you would like to support, or a
> release you'd like to test.  You start on it.  You submit your pull
> request, you answer the user's question, you test the release.  You
> continue doing this at a low level for a few months.  You see other
> people starting to contribute too.  This is nice.  You're working
> together with others towards a common goal.  Then, out of the blue a
> company with multiple paid contributors shows up.  Let's name them
> Acme. Acme drops a year of code on the project.  They could do this
> many ways.  For example:  A.) Acme could develop in the repository
> you
> were working in, or B.) Acme could create a project-internal fork and
> create a new repository. C.) Acme could even telegraph months in
> advance that they intend to do this, by posting to the dev list or
> contacting key contributors offlist, or just by having done it a few
> times already.
> 
> 
> A.) First let's imagine that Acme made massive changes in the
> repository you were working in.  Perhaps they already solved the
> problem you solved, but in a different way.  Perhaps, they deleted
> functions you made changes in.  Perhaps they added significant
> functionality you would have liked to help with.  What good were your
> efforts?  Wouldn't you find this discouraging?
> 
> And now you want to continue to make changes, but the code you want
> to
> change has commit messages referencing tickets which you have no
> access to.  Or it has no reference to tickets at all.  You find an
> area that seems to be needlessly complex: can you remove the
> complexity?  You have no way of knowing what you'd be breaking.
> 
> Perhaps you have a proprietary UI which depends on a behavior which
> was removed or changed.  Now your UI is broken.  Because the code
> drop
> is so large, you have no way to reasonably review it for
> incompatibilities.  It is not possible to review a year of
> development
> all at once.  And if your review turns up problems?  Do you accept
> the
> entire pull request or decline the whole thing?  Putting all the code
> into one pull request is a form of blackmail (commonly used in the
> formulation of bills for Congress).  If you want the good you have to
> take the bad.
> 
> 
> B.) Now let's imagine that Acme forked the code and created a new
> repository which they then added to the project.  None of the work
> you
> did is in this new repository.  If those features you implemented
> were
> important to you, you will have to re-introduce them into the new
> repository.
> 
> You'll have to start from zero learning to work in the new
> repository.
> You also had no say in how that code was developed, so maybe the
> feature that you need is unnecessarily difficult to implement in that
> repository.   You don't know why things are the way they are there,
> so
> you're walking through a mine field without a map when you're making
> changes.
> 
> And anyways, why is Acme Corp so certain you had nothing of value to
> add?
> 
> Releasing this code also becomes contentious. Which of the two
> competing repositories gets released?  Both of them? How does the
> project communicate to users about how these pieces fit together.
> 
> 
> C.) Imagine Acme gave you lots of forewarning that this was coming.
> You still have no say in how the code is developed.  You know that
> anything you might contribute could be obsoleted.  You can't tell
> users whether the up-and-coming release will be compatible.  And
> what's the point in testing that release?  You don't know how to
> check
> that your needs are being considered in the architecture of the new
> code base.
> 
> You have no sense of ownership over what comes out of that process.
> 
> You see that nobody else outside of Acme is working on the project
> either, for the same reasons.
> 
> 
> Most contributors would get discouraged and prefer not to participate
> if those were the conditions.  If contributors didn't get
> discouraged,
> they would fairly quickly be taking orders from the employees of Acme
> Corp.  Acme Corp has all the inside information about what's coming
> in
> a year in the next code dump.  Information is power.  Contributors
> who
> are also users may also chose to stop contributing and become free
> riders.  Why not just depend on Acme Corp for all of the development?
> 
> What Acme seems to be getting out of this scenario is an Apache
> feather.  It's a form of free-riding on Apache's reputation.
> 
> 
> Now let's imagine that you are the CTO of another company, let's call
> them Kaushal.  Kaushal is considering taking part in this open source
> project, but they are a competitor to Acme.  As Kaushal's CTO, you
> can
> see, based on commit history, and participation that Acme is
> dominating the project.  You would be smart to expect that Acme would
> take advantage of their dominant position in the project.  Acme could
> deliberately sabotage Kaushal's use cases, or simply 'starve' them by
> convincing people not to help Kaushal.  Kaushal's CTO could respond
> to
> this threat in one of two ways: 1.) Simply not take part on the open
> source project.  Create their own closed source thing, or their own
> open source project, and not engage.  This is the most likely
> response.  2.) Try to dominate the project themselves.  Kaushal has
> the same tools available that Acme has. Kaushal's CTO could tell his
> employees to do long-interval code drops just like Acme is doing.
>  Now
> with two corporations doing long-interval code drops on a project,
> merging the code becomes very very difficult.  Fights about who gets
> to decide what could eventually cause a complete cessation of release
> activity.
> 
> 
> So imagine that all competitors chose to remain just users, and Acme
> remains in control.  Now imagine Acme loses interest in the project.
> Acme found something that will make them more money, or Acme's
> business fails.  Or Acme gets tired of offering their development
> resources to the free riders.  Acme stops contributing to the
> project.
> But the project has become so dependent on Acme that it cannot exist
> without Acme.  When Acme exits, project activity could end.
> 
> 
> Open source projects require transparency, not just as a moral value,
> but as a pragmatic prerequisite for collaboration.  Offlist
> development damages the community *and* the code.
> 
> Best Regards,
> Myrle
> 
> P.S.  Some very interesting research on the game-theoretical aspects
> of modularization in open source:
> http://people.hbs.edu/cbaldwin/DR2/BaldwinClark.ArchOS.Jun03.pdf
> "Does Code Architecture Mitigate Free Riding in the Open Source
> Development Model?"
> 
> I would argue that the information divisibility being applied here at
> the code modularity dimension also applies to the time dimension.
>  So,
> it seems likely the argument against large code drops can be made
> mathematically. Which really tickles the geek in me. : o)
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org