Re: On wearing multiple hats

[Phil Steitz <ph...@gmail.com>]

On 10/26/16 7:06 AM, Isabel Drost-Fromm wrote:

Thanks for sharing this, Isabel.  I have a similar kind of personal
comment below, but I have to first say that my immediate reaction
reading about your experience was how lucky we are that you managed
to make it to that Acon :)  Please thank that person for us!

> On Tue, Oct 25, 2016 at 02:19:06PM +0000, Ross Gardler wrote:
>> > First, I'm tired of hearing it too but let's not be fooled, most of the time it comes from people ill informed about how the ASF works.
> I have no doubt about that. Maybe we can do a better job telling people how we
> work then?
>
>> > We use social controls within the projects and we have a fully independent board to handle escalations should a community member feel that their (or anothers) merit not be recognized.
> Let me tell you a story to illustrate what kind of problem I see with that (and
> I'm happy to be told, that this is just me having that problem or not reading
> the docs):
>
> I came to the ASF when nutch went from sourceforge through the incubator to be a
> sub project of Lucene. Being the naive student I was I had no idea what this
> whole ASF thingy was - all I knew about was the web server in my Debian Woody
> distribution with the same name. I was blissfully ignorant to what kind of
> contributions would be welcome in the project and what merit I should gain
> through that.
>
> A year or two later I was a bit closer open source at Apache, I had subscribed
> not only to nutch but also to Lucene mailing lists. A learnt about a conference
> related to the projects close to where I was working back then - namely in
> Amsterdam. I looked at the ticket prices expecting to see something like FOSDEM,
> Chaos Communication Congress, Linuxtage or some such which I happened to know.
> But the prices were just way above and beyond anything I could afford. And my
> research group would only really pay for scientific conferences.
>
> Fast forward another year or two, I was working for a small company in Berlin
> (neofonie, back then doing search consultancy mainly), I
> had co-founded Apache Mahout, as a result I was entitled to get a committer
> discount. I asked my manager for vacation days to go - and got a "let me check
> if we can actually send you there on our cost" as a reply. Back at that Apache
> Con EU 2008 in Amsterdam was when I first understood some of the basic concepts
> of how things are supposed to work around here, what kind of contributions
> should be rewarded.
>
> Only much later did I come across books like Producing Open Source Software, did
> I watch several keynotes on how things work over at Debian and friends, did I
> read about C4 at ZeroMQ, did I see things like the "Poisonous People" video.
>
>
> To cut a really long story short: I may be mistaken, but I think even with the
> incubator community members may not even be aware of a lack of merit. Look at
> the thread over here:
>
> http://mail-archives.apache.org/mod_mbox/community-dev/201610.mbox/browser
>
> isn't that already showing that some questions arise only way after incubation?
>
>
>> > If this is breaking down then its a problem within the PMC not with the process, which has served us well for many years across many projects and should, IMHO, serve us well for many more. Rather than starting to look for a solution to a problem purebred by others perhaps we should look at why they have this perception.
> Just for the record: I don't think we have a process problem per se here. We
> might have an education problem: In terms of what (even if it's little) we
> expect from projects, in terms of what people should do when they see those
> expectations not being met, in terms of what is being done within the ASF to
> keep things in line (or phrased more bluntly: If everything goes wrong, what can
> you expect the board to do to your project? How did we educate projects in the
> past?)
>
>
>> > Here's my thoughts...
>> > 
>> > Open source, in general, has changed. Its gone from mostly individual hackers from small collaborating companies "scratching their own itch" to mostly big business and will funded startups paying individuals who sometimes don't care on a personal level. This has resulted in the emergence of a different flavor of open source. One in which money and metrics count more than community and code. I'm the money and metrics model success means market disruption rather than collaboration on code.
> This matches with some of what I see as well.

Yes, there is no doubt that is has happened.  I used to be really
worried about it and I even went emeritus at one point because I
thought the ASF was just becoming a shell for commercial software
ventures.  I realized later that was over-reaction.  That
over-reaction was the result of a naive view on how OSS was going to
grow and change the software industry and a lack of appreciation for
the remarkable resiliency of the ASF. 

I came to the ASF as a user in 2002.  I was a technology leader at a
large financial services company in the US and we were in process of
moving all of our web technologies and a lot of middleware services
to Java.  We were getting a lot of help from two great technology
companies, whose names are easy to guess, both having long and
interesting relationships with the ASF.  Both also, as Sam so
eloquently put it, had an uneven distribution of cluefulness
(nowhere dense, but definitely positive measure :).  One of the
architects from the __  Java Centers pointed out to me that the
internal MVC framework that we had developed looked a whole lot like
Struts.  That led me to start looking at the Struts code.  I saw
that he was right; but Struts was better.  At first, I just wanted
to see if it was something we could use and if so, figure out how to
support it; but that led to getting on the user list, then the dev
list ... and then that personal connection kind of kicked in.

What I saw at the time was a *huge* change in how software products
come to be and evolve.  Instead of marketing strategies, legacy
products and commercial competition driving the agenda, it was going
to be users - people like me and my teams who actually use software
determining what gets built.  I was coming off a string of painful
experiences with heavy application servers and middleware so I was
very excited that I might one day be able to stop pounding the table
with vendors, saying "Please, stop trying to get me to have the
problems that you have solved.  Solve the ones that I actually
have."  I was excited and did everything I could to encourage my
peers and the clueful ones inside $bigCos to engage in OSS
communities and adopt OSS software.  At the same time, I started
contributing myself.

My attitude toward technology leadership has changed dramatically
during my time at the ASF.  I saw that the core concepts of
transparency, community and meritocracy could be applied inside
organizations of all sizes and functions.  I gave a talk about this
at Apachecon US 2008 [1]. 

All of that is background to what happened in 2011 when I realized
that my wonderful dream of "user-driven software" was colliding with
the reality that commercial software companies were not going away. 
I was shocked to realize how many people were actually paid to work
full time on ASF projects.  That could not be a good thing!  Then I
came to realize who these people were.  Some of the most wonderful
community members among us.  And the companies that were paying them
- well, most appeared to actually accept the idea that investing in
an ASF project carries "control risk" and they were willing to take
that risk.  Sure, some were idiots; but the ASF Board had this big
clue bat they could and would take out now and then to remind the
intransigent.

So yes, the nature of OSS has changed since the early
volunteer-dominated days and yes, this means that some people and
even entire project communities need to be reminded now and then
about what we mean by "separation of hats;" but as you and others
have pointed out, the core processes and structure of the ASF really
are robust against this and other changes in the dynamics of the
software industry.  As long as we keep gently but firmly reminding
people that our communities have to be transparent and inclusive and
that ideas have to win on their publicly earned merit, we will keep
Greg's vision of a healthy ASF 50 years from now [2] alive.

Phil

[1] http://www.slideshare.net/psteitz/open-development-in-the-enterprise
[2] A cool feature of this vision is its evergreen nature - I think
he originally said it about 5 years ago :)


>
>> > I maintain that the Apache Way is still a highly valuable and repeatable process that when applied correctly brings the highest chance of success (where success is valuable open source code). It is a process that is designed to ensure that those who care on a personal level have as much influence as those who are motivated by external need. It is a process that leaves money and metrics at the door but recognizes community and code contributions quickly.
> The "recognizes *community* and code contributions *quickly*" is something I
> would like to see validated across communities. I'm pretty sure even Mahout is
> guilty of having waited long to hand committer status out to people who contribute
> on a non-Java-level.
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org