You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Torsten Curdt <tc...@apache.org> on 2009/04/07 22:10:41 UTC
working together
Hey folks,
I thought back and forth about whether this email should go to the
private list or not. But I have a Cocoon background and in Cocoon land
we found it useful to only use the private list when there is really
no other way. This is about the community so I think it deserves to be
discussed in the open. I hope no one is offended by that approach.
Let me try to explain how I see the history of Cassandra. Please
correct me where I am wrong:
Cassandra was (mainly?) developed by Facebook and is being used in
production over there. At some stage it got open sourced on Google
Code. External contributors did not feel like there is a real
community around it as patches did not get applied. Out of frustration
people started (or at least proposed) to fork the code base. As a fork
means community fragmentation the projects has turned to the Apache
Incubator hoping to turn this fragmented contributors into a healthy
community.
In January the project got accepted. Papers were filed and in March we
saw the code enter the Apache subversion repository. A few weeks later
a first committer was proposed. Unfortunately no one noticed that the
actual authors bringing the code were NOT on the private list where
the vote was held. So we got a new committer without the consent
and/or feedback of the original authors. A big surprise. The people
that brought over the code now feel a bit betrayed by the process.
They have to deal with a committer that does changes all over the
place on a code base they depend on for production. They have the
feeling these changes are untested (at least not tested enough to get
right into production at Facebook) and that this destabilize the code
base. On the other hand there is new blood, new drive to the project.
While Facebook needs it's stability, other contributors needs the
change to meet their goals and deadlines.
So the problems I am seeing are:
1. We elected a committer without real community consensus. The
barrier of entry was unnatural low on this one. On the other hand we
need non-FB committers for the graduation. The more the better. (No
reason for low entry barrier though!)
2. A missing definition of development process:
- What is considered a valid code review?
- How much are changes discussed up-front?
- What is the roadmap? ...for whom? (weighted as a community)
3. Is trunk considered "stable"? Or aren't we missing a stable branch
for the required stability? Once we have the separation between stable
and trunk: Will patches really find it's way from trunk into stable?
Is Facebook OK with that approach. Will everyone cope with the
additional work of merging? Would it be useful ...or overkill to use
merge tracking?
4. Real world testing feedback is not publicly available. So the
feedback on changes will only slowly reach the community. This is not
easy for a project like this. But is there a faster way to provide
testing feedback? (IIRC Yahoo was providing testing feedback for
Hadoop. They even try to auto-apply patches from JIRA)
5. Is there really no code ownership issue. Working on a code base for
1-2 years can get you attached to the code you have written. Can
everyone really let go? Is it OK if someone else really just rewrites
parts of what you wrote? (No, it doesn't mean the original code was
bad! But maybe with the new code it is more readable ...
understandable - especially for someone who hasn't spent the past
years working on that code) Is there room for refactoring?
Anything else I am missing?
This is a tough situation but I hope everyone sees this as an
opportunity. Please let's discuss this openly in civilize manner.
Focusing on how to solve these points rather than looking at the past.
Please talk to each other. Can you/we work this out together?
cheers
--
Torsten
Re: working together
Posted by Matthieu Riou <ma...@gmail.com>.
On Wed, Apr 8, 2009 at 1:17 PM, Torsten Curdt <tc...@apache.org> wrote:
> >> Well, I feel tricked into a vote now. That is NOT right. The vote
> >> should come from the community - not the mentors. Whether you like it
> >> or not.
> >>
> >
> > Tricked into a vote? What are you accusing me of, scheming in your back?
>
> I am not accusing you but instead was just expressing a feeling.
> I don't expect you really planed it *that* way ;-)
>
> > I was just mentioning what I thought to be everybody's opinion at that
> point
> > and I regret you didn't know (that was the "my bad" part). But tell me,
> how
> > could I have known you didn't know? Read your mind?
>
> Well, you said "it was by design". If that's the case I think it's not
> too much to ask to bring it forward like that ...as it's rather me no
> being able to read your mind on this approach. After all we had
> invited the committers to the private mailing list already.
>
> What is more likely to expect: That they signed up or that you started
> the vote on purpose without them?
>
> >> "OK, now childs play together" .. it doesn't work that way.
> >
> > Oh really? So how does it work then, enlighten me.
>
> In every somewhat decent kindergarten parents need to accompany their
> child in the first weeks or even months. That's what they call
> "familiarization". Not sure that is the right english term but...
>
I thought that's why we were here in the first place ;-)
>
> Oh ...and the comparison to a kindergarten is purely accidental!! ;-)
>
> cheers
> --
> Torsten
>
Re: working together
Posted by Torsten Curdt <tc...@apache.org>.
>> Well, I feel tricked into a vote now. That is NOT right. The vote
>> should come from the community - not the mentors. Whether you like it
>> or not.
>>
>
> Tricked into a vote? What are you accusing me of, scheming in your back?
I am not accusing you but instead was just expressing a feeling.
I don't expect you really planed it *that* way ;-)
> I was just mentioning what I thought to be everybody's opinion at that point
> and I regret you didn't know (that was the "my bad" part). But tell me, how
> could I have known you didn't know? Read your mind?
Well, you said "it was by design". If that's the case I think it's not
too much to ask to bring it forward like that ...as it's rather me no
being able to read your mind on this approach. After all we had
invited the committers to the private mailing list already.
What is more likely to expect: That they signed up or that you started
the vote on purpose without them?
>> "OK, now childs play together" .. it doesn't work that way.
>
> Oh really? So how does it work then, enlighten me.
In every somewhat decent kindergarten parents need to accompany their
child in the first weeks or even months. That's what they call
"familiarization". Not sure that is the right english term but...
Oh ...and the comparison to a kindergarten is purely accidental!! ;-)
cheers
--
Torsten
Re: working together
Posted by Matthieu Riou <ma...@gmail.com>.
On Wed, Apr 8, 2009 at 12:38 AM, Torsten Curdt <tc...@apache.org> wrote:
> >> Unfortunately no one noticed that the
> >> actual authors bringing the code were NOT on the private list where
> >> the vote was held. So we got a new committer without the consent
> >> and/or feedback of the original authors. A big surprise.
> >
> >
> > I disagree here, I was fully aware that we hadn't formed the PPMC from
> the
> > committership yet (read PPMC = mentors) and at least on my side it was by
> > design. I should probably have pointed it out at the time just in case
> > though, my bad.
>
> Well, I feel tricked into a vote now. That is NOT right. The vote
> should come from the community - not the mentors. Whether you like it
> or not.
>
Tricked into a vote? What are you accusing me of, scheming in your back? I
was just mentioning what I thought to be everybody's opinion at that point
and I regret you didn't know (that was the "my bad" part). But tell me, how
could I have known you didn't know? Read your mind?
Now I have mentored quite a few podlings so far, some coming from
confronting codebases or others with very heavy political backgrounds.
Sometimes, votes coming from the community just lead to project suicide
whether you like it or not.
>
> > Avinash and Prashant are two very nice fellows but the Cassandra
> community
> > has to be larger than FB employees and actually was already, given the
> > different forks that were developed here and there. Waiting to form the
> PPMC
> > until it's more balanced in this case seems like a reasonable thing to
> do.
>
> "OK, now childs play together" .. it doesn't work that way.
Oh really? So how does it work then, enlighten me.
Matthieu
>
> >> 1. We elected a committer without real community consensus. The
> >> barrier of entry was unnatural low on this one. On the other hand we
> >> need non-FB committers for the graduation. The more the better. (No
> >> reason for low entry barrier though!)
> >>
> >
> > If what you mean is that it was low because there was less direct peer
> > review, I would agree
>
> That is what I meant.
>
>
> >> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> >> for the required stability? Once we have the separation between stable
> >> and trunk: Will patches really find it's way from trunk into stable?
> >> Is Facebook OK with that approach. Will everyone cope with the
> >> additional work of merging? Would it be useful ...or overkill to use
> >> merge tracking?
> >>
> >
> > A very useful tool here is what in Apache lingo is called "lazy
> consensus".
> > Anybody can build a proposition and submit it in the open. If nobody
> objects
> > in a reasonable amount of time (usually 72 hours), then it's considered
> > adopted.
> >
> > So I've read a couple of propositions from Jonathan on these points, why
> not
> > formalizing that?
>
> That's the way forward I think.
>
Re: working together
Posted by Torsten Curdt <tc...@apache.org>.
>> Unfortunately no one noticed that the
>> actual authors bringing the code were NOT on the private list where
>> the vote was held. So we got a new committer without the consent
>> and/or feedback of the original authors. A big surprise.
>
>
> I disagree here, I was fully aware that we hadn't formed the PPMC from the
> committership yet (read PPMC = mentors) and at least on my side it was by
> design. I should probably have pointed it out at the time just in case
> though, my bad.
Well, I feel tricked into a vote now. That is NOT right. The vote
should come from the community - not the mentors. Whether you like it
or not.
> Avinash and Prashant are two very nice fellows but the Cassandra community
> has to be larger than FB employees and actually was already, given the
> different forks that were developed here and there. Waiting to form the PPMC
> until it's more balanced in this case seems like a reasonable thing to do.
"OK, now childs play together" .. it doesn't work that way.
>> 1. We elected a committer without real community consensus. The
>> barrier of entry was unnatural low on this one. On the other hand we
>> need non-FB committers for the graduation. The more the better. (No
>> reason for low entry barrier though!)
>>
>
> If what you mean is that it was low because there was less direct peer
> review, I would agree
That is what I meant.
>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>> for the required stability? Once we have the separation between stable
>> and trunk: Will patches really find it's way from trunk into stable?
>> Is Facebook OK with that approach. Will everyone cope with the
>> additional work of merging? Would it be useful ...or overkill to use
>> merge tracking?
>>
>
> A very useful tool here is what in Apache lingo is called "lazy consensus".
> Anybody can build a proposition and submit it in the open. If nobody objects
> in a reasonable amount of time (usually 72 hours), then it's considered
> adopted.
>
> So I've read a couple of propositions from Jonathan on these points, why not
> formalizing that?
That's the way forward I think.
Re: working together
Posted by Matthieu Riou <ma...@gmail.com>.
On Tue, Apr 7, 2009 at 1:10 PM, Torsten Curdt <tc...@apache.org> wrote:
> Hey folks,
>
> I thought back and forth about whether this email should go to the
> private list or not. But I have a Cocoon background and in Cocoon land
> we found it useful to only use the private list when there is really
> no other way. This is about the community so I think it deserves to be
> discussed in the open. I hope no one is offended by that approach.
>
Thanks for doing it this way.
>
> Let me try to explain how I see the history of Cassandra. Please
> correct me where I am wrong:
>
> Cassandra was (mainly?) developed by Facebook and is being used in
> production over there. At some stage it got open sourced on Google
> Code. External contributors did not feel like there is a real
> community around it as patches did not get applied. Out of frustration
> people started (or at least proposed) to fork the code base. As a fork
> means community fragmentation the projects has turned to the Apache
> Incubator hoping to turn this fragmented contributors into a healthy
> community.
>
> In January the project got accepted. Papers were filed and in March we
> saw the code enter the Apache subversion repository. A few weeks later
> a first committer was proposed. Unfortunately no one noticed that the
> actual authors bringing the code were NOT on the private list where
> the vote was held. So we got a new committer without the consent
> and/or feedback of the original authors. A big surprise.
I disagree here, I was fully aware that we hadn't formed the PPMC from the
committership yet (read PPMC = mentors) and at least on my side it was by
design. I should probably have pointed it out at the time just in case
though, my bad.
Avinash and Prashant are two very nice fellows but the Cassandra community
has to be larger than FB employees and actually was already, given the
different forks that were developed here and there. Waiting to form the PPMC
until it's more balanced in this case seems like a reasonable thing to do.
> The people
> that brought over the code now feel a bit betrayed by the process.
> They have to deal with a committer that does changes all over the
> place on a code base they depend on for production. They have the
> feeling these changes are untested (at least not tested enough to get
> right into production at Facebook) and that this destabilize the code
> base.
I wasn't aware of those feelings but I don't think they're justified, many
different projects are in production in many different environments in the
foundation and changes aren't a problem. That's what stable branches are
for.
Cassandra is now an Apache project, if it's successful it will be developed
by a much larger community that it used to. That's the main goal, not a
byproduct of incubation. What I would expect from committers more
experienced in the codebase is guidance and transmission of expertise, not
code freeze.
> On the other hand there is new blood, new drive to the project.
> While Facebook needs it's stability, other contributors needs the
> change to meet their goals and deadlines.
>
> So the problems I am seeing are:
>
> 1. We elected a committer without real community consensus. The
> barrier of entry was unnatural low on this one. On the other hand we
> need non-FB committers for the graduation. The more the better. (No
> reason for low entry barrier though!)
>
If what you mean is that it was low because there was less direct peer
review, I would agree (although I would consider Ian as a peer). If what you
mean is that Jonathan hasn't enough merit to be a committer, I would
disagree. Just have a look at his giithub branch :)
>
> 2. A missing definition of development process:
> - What is considered a valid code review?
> - How much are changes discussed up-front?
> - What is the roadmap? ...for whom? (weighted as a community)
>
> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> for the required stability? Once we have the separation between stable
> and trunk: Will patches really find it's way from trunk into stable?
> Is Facebook OK with that approach. Will everyone cope with the
> additional work of merging? Would it be useful ...or overkill to use
> merge tracking?
>
A very useful tool here is what in Apache lingo is called "lazy consensus".
Anybody can build a proposition and submit it in the open. If nobody objects
in a reasonable amount of time (usually 72 hours), then it's considered
adopted.
So I've read a couple of propositions from Jonathan on these points, why not
formalizing that?
>
> 4. Real world testing feedback is not publicly available. So the
> feedback on changes will only slowly reach the community. This is not
> easy for a project like this. But is there a faster way to provide
> testing feedback? (IIRC Yahoo was providing testing feedback for
> Hadoop. They even try to auto-apply patches from JIRA)
>
> 5. Is there really no code ownership issue. Working on a code base for
> 1-2 years can get you attached to the code you have written. Can
> everyone really let go? Is it OK if someone else really just rewrites
> parts of what you wrote? (No, it doesn't mean the original code was
> bad! But maybe with the new code it is more readable ...
> understandable - especially for someone who hasn't spent the past
> years working on that code) Is there room for refactoring?
>
> Anything else I am missing?
>
That's a good list :)
Matthieu
>
> This is a tough situation but I hope everyone sees this as an
> opportunity. Please let's discuss this openly in civilize manner.
> Focusing on how to solve these points rather than looking at the past.
> Please talk to each other. Can you/we work this out together?
>
> cheers
> --
> Torsten
>
Re: working together
Posted by Torsten Curdt <tc...@vafer.org>.
Indeed ... but we did not see any vote of them. Which should have
gotten us suspicious.
Water under the bridge.
On Wed, Apr 8, 2009 at 02:39, Ian Holsman <ia...@holsman.net> wrote:
> just on a point here.
> They were invited from Day 1 (Actually 20-Jan-2009) to be on there. It
> wasn't done out of malice.
Re: working together
Posted by Ian Holsman <ia...@holsman.net>.
It *should* be.
- mentors
- people mentioned on the initial proposal as comitters.
I've resent the original email to the original people on the list to
make sure they got it.
On 08/04/2009, at 11:34 AM, Matthieu Riou wrote:
> On Tue, Apr 7, 2009 at 5:39 PM, Ian Holsman <ia...@holsman.net> wrote:
>
>> just on a point here.
>> They were invited from Day 1 (Actually 20-Jan-2009) to be on there.
>> It
>> wasn't done out of malice.
>>
>
> So I'm slightly confused as to what the PPMC composition is now.
>
>
>>
>>
>> On 08/04/2009, at 6:10 AM, Torsten Curdt wrote:
>>
>> Unfortunately no one noticed that the
>>> actual authors bringing the code were NOT on the private list where
>>> the vote was held.
>>>
>>
>> --
>> Ian Holsman
>> Ian@Holsman.net
>>
>>
>>
>>
--
Ian Holsman
Ian@Holsman.net
Re: working together
Posted by Matthieu Riou <ma...@gmail.com>.
On Tue, Apr 7, 2009 at 5:39 PM, Ian Holsman <ia...@holsman.net> wrote:
> just on a point here.
> They were invited from Day 1 (Actually 20-Jan-2009) to be on there. It
> wasn't done out of malice.
>
So I'm slightly confused as to what the PPMC composition is now.
>
>
> On 08/04/2009, at 6:10 AM, Torsten Curdt wrote:
>
> Unfortunately no one noticed that the
>> actual authors bringing the code were NOT on the private list where
>> the vote was held.
>>
>
> --
> Ian Holsman
> Ian@Holsman.net
>
>
>
>
Re: working together
Posted by Ian Holsman <ia...@holsman.net>.
just on a point here.
They were invited from Day 1 (Actually 20-Jan-2009) to be on there. It
wasn't done out of malice.
On 08/04/2009, at 6:10 AM, Torsten Curdt wrote:
> Unfortunately no one noticed that the
> actual authors bringing the code were NOT on the private list where
> the vote was held.
--
Ian Holsman
Ian@Holsman.net
Re: Development process (was: working together)
Posted by Jonathan Ellis <jb...@gmail.com>.
+1
On Thu, Apr 9, 2009 at 10:49 AM, Johan Oskarsson <jo...@oskarsson.nu> wrote:
> Thanks Sandeep.
>
> Would we all be comfortable adopting this "process" going forward,
> hopefully reducing friction, bugs and problems in general?
>
> I assume +1 from me and Sandeep so far.
>
> /Johan
>
> Sandeep Tata wrote:
>> Johan, the wiki pages are great! I think they will help iron out our
>> process for contributing and committing.
>>
>> (I added a pointer to the formatting conventions in HowToContribute ,
>> can't think of anything else to add)
>>
>>> http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
>>> http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
>>> http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
>>>
>>> A short summary and description of why these points make sense:
>>> * "Patch-only" evolution of code, attached to a jira issue
>>> * At least one +1 on each issue before it can be committed, -1 stops the
>>> patch.
>>>
>>> Those two points would make sure that if someone disagrees with a
>>> change, a refactoring etc, they have a chance to voice their opinion and
>>> steer it into the right direction.
>>>
>>>
>>> * Trunk is not considered stable, but must pass unit tests
>>> * Any non trivial change should include unit tests
>>> * When a branch is created to prepare for a release extra effort is put
>>> into QA to make sure the release is as stable as possible. Point
>>> releases would then go out to fix issues found after the release was done.
>>> * Once a release has been out for a while and people are using it in
>>> production without problems it is upgraded to "stable" status.
>>>
>>> The purpose of these points is to encourage a "vibrant codebase", to not
>>> be afraid of for example refactoring if it improves the code readability
>>> or testability. I appreciate that Cassandra is a complex system and that
>>> changes might have unwanted side effects, but hopefully adding tests and
>>> code reviews will reduce those. As a final catch-all the release
>>> candidate and "stable release" process should help end users avoid bugs.
>>>
>>>
>>> Thoughts on the wiki pages? Do they help resolve some of the problems?
>>>
>>> /Johan
>>>
>>> Sandeep Tata wrote:
>>>> Thoughts inline:
>>>>
>>>>> So the problems I am seeing are:
>>>>>
>>>>> 1. We elected a committer without real community consensus. The
>>>>> barrier of entry was unnatural low on this one. On the other hand we
>>>>> need non-FB committers for the graduation. The more the better. (No
>>>>> reason for low entry barrier though!)
>>>> I think everyone (including the FB guys) agree that Jonathan has been
>>>> working hard to help move the codebase forward. He has been quick to
>>>> revert changes that broke the code that the FB guys had in the
>>>> pipeline and have committed since. I think much of the friction comes
>>>> from not having a process, which takes us to Torsten's #2:
>>>>
>>>>> 2. A missing definition of development process:
>>>>> - What is considered a valid code review?
>>>>> - How much are changes discussed up-front?
>>>>> - What is the roadmap? ...for whom? (weighted as a community)
>>>> This is probably where we need most work. Here are some simple suggestions:
>>>>
>>>> a) I'm a fan of a "patch-only" evolution of code. All changes come
>>>> from patches, and no changes come from anywhere else (eg. the
>>>> committers IDE). Even if it is something as simple as cleaning up
>>>> comments or changing a variable name.
>>>> b) A patch gets applied if at least one reviewer +1s it, and no one -1s it.
>>>> c) A patch should pass all unit tests. Any significant patch should
>>>> come with additional unit tests.
>>>>
>>>> Some of this, of course, will mean "more work" for the committers.
>>>> Sure, but such processes are essential if the project is to grow
>>>> beyond a small group of core contributors.
>>>>
>>>>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>>>>> for the required stability? Once we have the separation between stable
>>>>> and trunk: Will patches really find it's way from trunk into stable?
>>>>> Is Facebook OK with that approach. Will everyone cope with the
>>>>> additional work of merging? Would it be useful ...or overkill to use
>>>>> merge tracking?
>>>> I agree with Matt. Trunk should pass build + tests, but should not be
>>>> trusted for production. I think 0.2 was supposed to be a stable
>>>> branch. Avinash, Prashant -- what are your thoughts on this? Are you
>>>> guys comfortable with this approach? Do you foresee any problems?
>>>>
>>>> Basically, use a "release" branch for production. The release branches
>>>> only admit stability patches. New feature and cleanup patches go to
>>>> trunk. Folks running Cassandra in production only need to be nervous
>>>> when moving from one release to next, and not worry too much about
>>>> every single patch breaking their running system.
>>>>
>>>>> 4. Real world testing feedback is not publicly available. So the
>>>>> feedback on changes will only slowly reach the community. This is not
>>>>> easy for a project like this. But is there a faster way to provide
>>>>> testing feedback? (IIRC Yahoo was providing testing feedback for
>>>>> Hadoop. They even try to auto-apply patches from JIRA)
>>>> With time, FB may be able to provide feedback from their "divert some
>>>> traffic to the new version" system. Auto-applying patches from JIRA
>>>> sounds a little ambitious right now :-)
>>>>
>>>>> 5. Is there really no code ownership issue. Working on a code base for
>>>>> 1-2 years can get you attached to the code you have written. Can
>>>>> everyone really let go? Is it OK if someone else really just rewrites
>>>>> parts of what you wrote? (No, it doesn't mean the original code was
>>>>> bad! But maybe with the new code it is more readable ...
>>>>> understandable - especially for someone who hasn't spent the past
>>>>> years working on that code) Is there room for refactoring?
>>>> :-)
>>>>
>>>>
>>>>> This is a tough situation but I hope everyone sees this as an
>>>>> opportunity. Please let's discuss this openly in civilize manner.
>>>>> Focusing on how to solve these points rather than looking at the past.
>>>>> Please talk to each other. Can you/we work this out together?
>>>> I agree -- thanks for initiating this conversation!
>>>
>>>
>
>
Re: Development process (was: working together)
Posted by Eric Evans <ee...@rackspace.com>.
On Thu, 2009-04-09 at 16:49 +0100, Johan Oskarsson wrote:
> Thanks Sandeep.
>
> Would we all be comfortable adopting this "process" going forward,
> hopefully reducing friction, bugs and problems in general?
>
> I assume +1 from me and Sandeep so far.
+1
> Sandeep Tata wrote:
> > Johan, the wiki pages are great! I think they will help iron out our
> > process for contributing and committing.
> >
> > (I added a pointer to the formatting conventions in HowToContribute ,
> > can't think of anything else to add)
> >
> >> http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
> >> http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
> >> http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
--
Eric Evans
eevans@rackspace.com
Re: Development process (was: working together)
Posted by Johan Oskarsson <jo...@oskarsson.nu>.
I have made some minor convenience tweaks to the wiki pages:
When contributing a patch you can now mark the issue with "Patch
available", making it easier for the committers and reviewers to find
tickets that needs attention.
For committers: you can create a filter that gives a review queue or
wait for one to be created here:
https://issues.apache.org/jira/browse/CASSANDRA-73
/Johan
Jun Rao wrote:
> +1 from me too.
>
> Jun
> IBM Almaden Research Center
> K55/B1, 650 Harry Road, San Jose, CA 95120-6099
>
> junrao@almaden.ibm.com
>
> Inactive hide details for Johan Oskarsson <jo...@oskarsson.nu>Johan
> Oskarsson <jo...@oskarsson.nu>
>
>
> *Johan Oskarsson <jo...@oskarsson.nu>*
>
> 04/09/2009 08:49 AM
> Please respond to
> cassandra-dev@incubator.apache.org
>
>
>
> To
>
> cassandra-dev@incubator.apache.org
>
> cc
>
>
> Subject
>
> Development process (was: working together)
>
>
>
>
>
> Thanks Sandeep.
>
> Would we all be comfortable adopting this "process" going forward,
> hopefully reducing friction, bugs and problems in general?
>
> I assume +1 from me and Sandeep so far.
>
> /Johan
>
> Sandeep Tata wrote:
> > Johan, the wiki pages are great! I think they will help iron out our
> > process for contributing and committing.
> >
> > (I added a pointer to the formatting conventions in HowToContribute ,
> > can't think of anything else to add)
> >
> >> http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
> >> http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
> >> http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
> >>
> >> A short summary and description of why these points make sense:
> >> * "Patch-only" evolution of code, attached to a jira issue
> >> * At least one +1 on each issue before it can be committed, -1 stops the
> >> patch.
> >>
> >> Those two points would make sure that if someone disagrees with a
> >> change, a refactoring etc, they have a chance to voice their opinion and
> >> steer it into the right direction.
> >>
> >>
> >> * Trunk is not considered stable, but must pass unit tests
> >> * Any non trivial change should include unit tests
> >> * When a branch is created to prepare for a release extra effort is put
> >> into QA to make sure the release is as stable as possible. Point
> >> releases would then go out to fix issues found after the release was
> done.
> >> * Once a release has been out for a while and people are using it in
> >> production without problems it is upgraded to "stable" status.
> >>
> >> The purpose of these points is to encourage a "vibrant codebase", to not
> >> be afraid of for example refactoring if it improves the code readability
> >> or testability. I appreciate that Cassandra is a complex system and that
> >> changes might have unwanted side effects, but hopefully adding tests and
> >> code reviews will reduce those. As a final catch-all the release
> >> candidate and "stable release" process should help end users avoid bugs.
> >>
> >>
> >> Thoughts on the wiki pages? Do they help resolve some of the problems?
> >>
> >> /Johan
> >>
> >> Sandeep Tata wrote:
> >>> Thoughts inline:
> >>>
> >>>> So the problems I am seeing are:
> >>>>
> >>>> 1. We elected a committer without real community consensus. The
> >>>> barrier of entry was unnatural low on this one. On the other hand we
> >>>> need non-FB committers for the graduation. The more the better. (No
> >>>> reason for low entry barrier though!)
> >>> I think everyone (including the FB guys) agree that Jonathan has been
> >>> working hard to help move the codebase forward. He has been quick to
> >>> revert changes that broke the code that the FB guys had in the
> >>> pipeline and have committed since. I think much of the friction comes
> >>> from not having a process, which takes us to Torsten's #2:
> >>>
> >>>> 2. A missing definition of development process:
> >>>> - What is considered a valid code review?
> >>>> - How much are changes discussed up-front?
> >>>> - What is the roadmap? ...for whom? (weighted as a community)
> >>> This is probably where we need most work. Here are some simple
> suggestions:
> >>>
> >>> a) I'm a fan of a "patch-only" evolution of code. All changes come
> >>> from patches, and no changes come from anywhere else (eg. the
> >>> committers IDE). Even if it is something as simple as cleaning up
> >>> comments or changing a variable name.
> >>> b) A patch gets applied if at least one reviewer +1s it, and no one
> -1s it.
> >>> c) A patch should pass all unit tests. Any significant patch should
> >>> come with additional unit tests.
> >>>
> >>> Some of this, of course, will mean "more work" for the committers.
> >>> Sure, but such processes are essential if the project is to grow
> >>> beyond a small group of core contributors.
> >>>
> >>>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> >>>> for the required stability? Once we have the separation between stable
> >>>> and trunk: Will patches really find it's way from trunk into stable?
> >>>> Is Facebook OK with that approach. Will everyone cope with the
> >>>> additional work of merging? Would it be useful ...or overkill to use
> >>>> merge tracking?
> >>> I agree with Matt. Trunk should pass build + tests, but should not be
> >>> trusted for production. I think 0.2 was supposed to be a stable
> >>> branch. Avinash, Prashant -- what are your thoughts on this? Are you
> >>> guys comfortable with this approach? Do you foresee any problems?
> >>>
> >>> Basically, use a "release" branch for production. The release branches
> >>> only admit stability patches. New feature and cleanup patches go to
> >>> trunk. Folks running Cassandra in production only need to be nervous
> >>> when moving from one release to next, and not worry too much about
> >>> every single patch breaking their running system.
> >>>
> >>>> 4. Real world testing feedback is not publicly available. So the
> >>>> feedback on changes will only slowly reach the community. This is not
> >>>> easy for a project like this. But is there a faster way to provide
> >>>> testing feedback? (IIRC Yahoo was providing testing feedback for
> >>>> Hadoop. They even try to auto-apply patches from JIRA)
> >>> With time, FB may be able to provide feedback from their "divert some
> >>> traffic to the new version" system. Auto-applying patches from JIRA
> >>> sounds a little ambitious right now :-)
> >>>
> >>>> 5. Is there really no code ownership issue. Working on a code base for
> >>>> 1-2 years can get you attached to the code you have written. Can
> >>>> everyone really let go? Is it OK if someone else really just rewrites
> >>>> parts of what you wrote? (No, it doesn't mean the original code was
> >>>> bad! But maybe with the new code it is more readable ...
> >>>> understandable - especially for someone who hasn't spent the past
> >>>> years working on that code) Is there room for refactoring?
> >>> :-)
> >>>
> >>>
> >>>> This is a tough situation but I hope everyone sees this as an
> >>>> opportunity. Please let's discuss this openly in civilize manner.
> >>>> Focusing on how to solve these points rather than looking at the past.
> >>>> Please talk to each other. Can you/we work this out together?
> >>> I agree -- thanks for initiating this conversation!
> >>
> >>
>
>
Re: Development process (was: working together)
Posted by Jun Rao <ju...@almaden.ibm.com>.
+1 from me too.
Jun
IBM Almaden Research Center
K55/B1, 650 Harry Road, San Jose, CA 95120-6099
junrao@almaden.ibm.com
Johan Oskarsson
<johan@oskarsson.
nu> To
cassandra-dev@incubator.apache.org
04/09/2009 08:49 cc
AM
Subject
Development process (was: working
Please respond to together)
cassandra-dev@inc
ubator.apache.org
Thanks Sandeep.
Would we all be comfortable adopting this "process" going forward,
hopefully reducing friction, bugs and problems in general?
I assume +1 from me and Sandeep so far.
/Johan
Sandeep Tata wrote:
> Johan, the wiki pages are great! I think they will help iron out our
> process for contributing and committing.
>
> (I added a pointer to the formatting conventions in HowToContribute ,
> can't think of anything else to add)
>
>> http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
>> http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
>> http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
>>
>> A short summary and description of why these points make sense:
>> * "Patch-only" evolution of code, attached to a jira issue
>> * At least one +1 on each issue before it can be committed, -1 stops the
>> patch.
>>
>> Those two points would make sure that if someone disagrees with a
>> change, a refactoring etc, they have a chance to voice their opinion and
>> steer it into the right direction.
>>
>>
>> * Trunk is not considered stable, but must pass unit tests
>> * Any non trivial change should include unit tests
>> * When a branch is created to prepare for a release extra effort is put
>> into QA to make sure the release is as stable as possible. Point
>> releases would then go out to fix issues found after the release was
done.
>> * Once a release has been out for a while and people are using it in
>> production without problems it is upgraded to "stable" status.
>>
>> The purpose of these points is to encourage a "vibrant codebase", to not
>> be afraid of for example refactoring if it improves the code readability
>> or testability. I appreciate that Cassandra is a complex system and that
>> changes might have unwanted side effects, but hopefully adding tests and
>> code reviews will reduce those. As a final catch-all the release
>> candidate and "stable release" process should help end users avoid bugs.
>>
>>
>> Thoughts on the wiki pages? Do they help resolve some of the problems?
>>
>> /Johan
>>
>> Sandeep Tata wrote:
>>> Thoughts inline:
>>>
>>>> So the problems I am seeing are:
>>>>
>>>> 1. We elected a committer without real community consensus. The
>>>> barrier of entry was unnatural low on this one. On the other hand we
>>>> need non-FB committers for the graduation. The more the better. (No
>>>> reason for low entry barrier though!)
>>> I think everyone (including the FB guys) agree that Jonathan has been
>>> working hard to help move the codebase forward. He has been quick to
>>> revert changes that broke the code that the FB guys had in the
>>> pipeline and have committed since. I think much of the friction comes
>>> from not having a process, which takes us to Torsten's #2:
>>>
>>>> 2. A missing definition of development process:
>>>> - What is considered a valid code review?
>>>> - How much are changes discussed up-front?
>>>> - What is the roadmap? ...for whom? (weighted as a community)
>>> This is probably where we need most work. Here are some simple
suggestions:
>>>
>>> a) I'm a fan of a "patch-only" evolution of code. All changes come
>>> from patches, and no changes come from anywhere else (eg. the
>>> committers IDE). Even if it is something as simple as cleaning up
>>> comments or changing a variable name.
>>> b) A patch gets applied if at least one reviewer +1s it, and no one -1s
it.
>>> c) A patch should pass all unit tests. Any significant patch should
>>> come with additional unit tests.
>>>
>>> Some of this, of course, will mean "more work" for the committers.
>>> Sure, but such processes are essential if the project is to grow
>>> beyond a small group of core contributors.
>>>
>>>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>>>> for the required stability? Once we have the separation between stable
>>>> and trunk: Will patches really find it's way from trunk into stable?
>>>> Is Facebook OK with that approach. Will everyone cope with the
>>>> additional work of merging? Would it be useful ...or overkill to use
>>>> merge tracking?
>>> I agree with Matt. Trunk should pass build + tests, but should not be
>>> trusted for production. I think 0.2 was supposed to be a stable
>>> branch. Avinash, Prashant -- what are your thoughts on this? Are you
>>> guys comfortable with this approach? Do you foresee any problems?
>>>
>>> Basically, use a "release" branch for production. The release branches
>>> only admit stability patches. New feature and cleanup patches go to
>>> trunk. Folks running Cassandra in production only need to be nervous
>>> when moving from one release to next, and not worry too much about
>>> every single patch breaking their running system.
>>>
>>>> 4. Real world testing feedback is not publicly available. So the
>>>> feedback on changes will only slowly reach the community. This is not
>>>> easy for a project like this. But is there a faster way to provide
>>>> testing feedback? (IIRC Yahoo was providing testing feedback for
>>>> Hadoop. They even try to auto-apply patches from JIRA)
>>> With time, FB may be able to provide feedback from their "divert some
>>> traffic to the new version" system. Auto-applying patches from JIRA
>>> sounds a little ambitious right now :-)
>>>
>>>> 5. Is there really no code ownership issue. Working on a code base for
>>>> 1-2 years can get you attached to the code you have written. Can
>>>> everyone really let go? Is it OK if someone else really just rewrites
>>>> parts of what you wrote? (No, it doesn't mean the original code was
>>>> bad! But maybe with the new code it is more readable ...
>>>> understandable - especially for someone who hasn't spent the past
>>>> years working on that code) Is there room for refactoring?
>>> :-)
>>>
>>>
>>>> This is a tough situation but I hope everyone sees this as an
>>>> opportunity. Please let's discuss this openly in civilize manner.
>>>> Focusing on how to solve these points rather than looking at the past.
>>>> Please talk to each other. Can you/we work this out together?
>>> I agree -- thanks for initiating this conversation!
>>
>>
Development process (was: working together)
Posted by Johan Oskarsson <jo...@oskarsson.nu>.
Thanks Sandeep.
Would we all be comfortable adopting this "process" going forward,
hopefully reducing friction, bugs and problems in general?
I assume +1 from me and Sandeep so far.
/Johan
Sandeep Tata wrote:
> Johan, the wiki pages are great! I think they will help iron out our
> process for contributing and committing.
>
> (I added a pointer to the formatting conventions in HowToContribute ,
> can't think of anything else to add)
>
>> http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
>> http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
>> http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
>>
>> A short summary and description of why these points make sense:
>> * "Patch-only" evolution of code, attached to a jira issue
>> * At least one +1 on each issue before it can be committed, -1 stops the
>> patch.
>>
>> Those two points would make sure that if someone disagrees with a
>> change, a refactoring etc, they have a chance to voice their opinion and
>> steer it into the right direction.
>>
>>
>> * Trunk is not considered stable, but must pass unit tests
>> * Any non trivial change should include unit tests
>> * When a branch is created to prepare for a release extra effort is put
>> into QA to make sure the release is as stable as possible. Point
>> releases would then go out to fix issues found after the release was done.
>> * Once a release has been out for a while and people are using it in
>> production without problems it is upgraded to "stable" status.
>>
>> The purpose of these points is to encourage a "vibrant codebase", to not
>> be afraid of for example refactoring if it improves the code readability
>> or testability. I appreciate that Cassandra is a complex system and that
>> changes might have unwanted side effects, but hopefully adding tests and
>> code reviews will reduce those. As a final catch-all the release
>> candidate and "stable release" process should help end users avoid bugs.
>>
>>
>> Thoughts on the wiki pages? Do they help resolve some of the problems?
>>
>> /Johan
>>
>> Sandeep Tata wrote:
>>> Thoughts inline:
>>>
>>>> So the problems I am seeing are:
>>>>
>>>> 1. We elected a committer without real community consensus. The
>>>> barrier of entry was unnatural low on this one. On the other hand we
>>>> need non-FB committers for the graduation. The more the better. (No
>>>> reason for low entry barrier though!)
>>> I think everyone (including the FB guys) agree that Jonathan has been
>>> working hard to help move the codebase forward. He has been quick to
>>> revert changes that broke the code that the FB guys had in the
>>> pipeline and have committed since. I think much of the friction comes
>>> from not having a process, which takes us to Torsten's #2:
>>>
>>>> 2. A missing definition of development process:
>>>> - What is considered a valid code review?
>>>> - How much are changes discussed up-front?
>>>> - What is the roadmap? ...for whom? (weighted as a community)
>>> This is probably where we need most work. Here are some simple suggestions:
>>>
>>> a) I'm a fan of a "patch-only" evolution of code. All changes come
>>> from patches, and no changes come from anywhere else (eg. the
>>> committers IDE). Even if it is something as simple as cleaning up
>>> comments or changing a variable name.
>>> b) A patch gets applied if at least one reviewer +1s it, and no one -1s it.
>>> c) A patch should pass all unit tests. Any significant patch should
>>> come with additional unit tests.
>>>
>>> Some of this, of course, will mean "more work" for the committers.
>>> Sure, but such processes are essential if the project is to grow
>>> beyond a small group of core contributors.
>>>
>>>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>>>> for the required stability? Once we have the separation between stable
>>>> and trunk: Will patches really find it's way from trunk into stable?
>>>> Is Facebook OK with that approach. Will everyone cope with the
>>>> additional work of merging? Would it be useful ...or overkill to use
>>>> merge tracking?
>>> I agree with Matt. Trunk should pass build + tests, but should not be
>>> trusted for production. I think 0.2 was supposed to be a stable
>>> branch. Avinash, Prashant -- what are your thoughts on this? Are you
>>> guys comfortable with this approach? Do you foresee any problems?
>>>
>>> Basically, use a "release" branch for production. The release branches
>>> only admit stability patches. New feature and cleanup patches go to
>>> trunk. Folks running Cassandra in production only need to be nervous
>>> when moving from one release to next, and not worry too much about
>>> every single patch breaking their running system.
>>>
>>>> 4. Real world testing feedback is not publicly available. So the
>>>> feedback on changes will only slowly reach the community. This is not
>>>> easy for a project like this. But is there a faster way to provide
>>>> testing feedback? (IIRC Yahoo was providing testing feedback for
>>>> Hadoop. They even try to auto-apply patches from JIRA)
>>> With time, FB may be able to provide feedback from their "divert some
>>> traffic to the new version" system. Auto-applying patches from JIRA
>>> sounds a little ambitious right now :-)
>>>
>>>> 5. Is there really no code ownership issue. Working on a code base for
>>>> 1-2 years can get you attached to the code you have written. Can
>>>> everyone really let go? Is it OK if someone else really just rewrites
>>>> parts of what you wrote? (No, it doesn't mean the original code was
>>>> bad! But maybe with the new code it is more readable ...
>>>> understandable - especially for someone who hasn't spent the past
>>>> years working on that code) Is there room for refactoring?
>>> :-)
>>>
>>>
>>>> This is a tough situation but I hope everyone sees this as an
>>>> opportunity. Please let's discuss this openly in civilize manner.
>>>> Focusing on how to solve these points rather than looking at the past.
>>>> Please talk to each other. Can you/we work this out together?
>>> I agree -- thanks for initiating this conversation!
>>
>>
Re: working together
Posted by Sandeep Tata <sa...@gmail.com>.
Johan, the wiki pages are great! I think they will help iron out our
process for contributing and committing.
(I added a pointer to the formatting conventions in HowToContribute ,
can't think of anything else to add)
>
> http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
> http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
> http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
>
> A short summary and description of why these points make sense:
> * "Patch-only" evolution of code, attached to a jira issue
> * At least one +1 on each issue before it can be committed, -1 stops the
> patch.
>
> Those two points would make sure that if someone disagrees with a
> change, a refactoring etc, they have a chance to voice their opinion and
> steer it into the right direction.
>
>
> * Trunk is not considered stable, but must pass unit tests
> * Any non trivial change should include unit tests
> * When a branch is created to prepare for a release extra effort is put
> into QA to make sure the release is as stable as possible. Point
> releases would then go out to fix issues found after the release was done.
> * Once a release has been out for a while and people are using it in
> production without problems it is upgraded to "stable" status.
>
> The purpose of these points is to encourage a "vibrant codebase", to not
> be afraid of for example refactoring if it improves the code readability
> or testability. I appreciate that Cassandra is a complex system and that
> changes might have unwanted side effects, but hopefully adding tests and
> code reviews will reduce those. As a final catch-all the release
> candidate and "stable release" process should help end users avoid bugs.
>
>
> Thoughts on the wiki pages? Do they help resolve some of the problems?
>
> /Johan
>
> Sandeep Tata wrote:
>> Thoughts inline:
>>
>>> So the problems I am seeing are:
>>>
>>> 1. We elected a committer without real community consensus. The
>>> barrier of entry was unnatural low on this one. On the other hand we
>>> need non-FB committers for the graduation. The more the better. (No
>>> reason for low entry barrier though!)
>>
>> I think everyone (including the FB guys) agree that Jonathan has been
>> working hard to help move the codebase forward. He has been quick to
>> revert changes that broke the code that the FB guys had in the
>> pipeline and have committed since. I think much of the friction comes
>> from not having a process, which takes us to Torsten's #2:
>>
>>> 2. A missing definition of development process:
>>> - What is considered a valid code review?
>>> - How much are changes discussed up-front?
>>> - What is the roadmap? ...for whom? (weighted as a community)
>>
>> This is probably where we need most work. Here are some simple suggestions:
>>
>> a) I'm a fan of a "patch-only" evolution of code. All changes come
>> from patches, and no changes come from anywhere else (eg. the
>> committers IDE). Even if it is something as simple as cleaning up
>> comments or changing a variable name.
>> b) A patch gets applied if at least one reviewer +1s it, and no one -1s it.
>> c) A patch should pass all unit tests. Any significant patch should
>> come with additional unit tests.
>>
>> Some of this, of course, will mean "more work" for the committers.
>> Sure, but such processes are essential if the project is to grow
>> beyond a small group of core contributors.
>>
>>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>>> for the required stability? Once we have the separation between stable
>>> and trunk: Will patches really find it's way from trunk into stable?
>>> Is Facebook OK with that approach. Will everyone cope with the
>>> additional work of merging? Would it be useful ...or overkill to use
>>> merge tracking?
>>
>> I agree with Matt. Trunk should pass build + tests, but should not be
>> trusted for production. I think 0.2 was supposed to be a stable
>> branch. Avinash, Prashant -- what are your thoughts on this? Are you
>> guys comfortable with this approach? Do you foresee any problems?
>>
>> Basically, use a "release" branch for production. The release branches
>> only admit stability patches. New feature and cleanup patches go to
>> trunk. Folks running Cassandra in production only need to be nervous
>> when moving from one release to next, and not worry too much about
>> every single patch breaking their running system.
>>
>>> 4. Real world testing feedback is not publicly available. So the
>>> feedback on changes will only slowly reach the community. This is not
>>> easy for a project like this. But is there a faster way to provide
>>> testing feedback? (IIRC Yahoo was providing testing feedback for
>>> Hadoop. They even try to auto-apply patches from JIRA)
>>
>> With time, FB may be able to provide feedback from their "divert some
>> traffic to the new version" system. Auto-applying patches from JIRA
>> sounds a little ambitious right now :-)
>>
>>> 5. Is there really no code ownership issue. Working on a code base for
>>> 1-2 years can get you attached to the code you have written. Can
>>> everyone really let go? Is it OK if someone else really just rewrites
>>> parts of what you wrote? (No, it doesn't mean the original code was
>>> bad! But maybe with the new code it is more readable ...
>>> understandable - especially for someone who hasn't spent the past
>>> years working on that code) Is there room for refactoring?
>>
>> :-)
>>
>>
>>> This is a tough situation but I hope everyone sees this as an
>>> opportunity. Please let's discuss this openly in civilize manner.
>>> Focusing on how to solve these points rather than looking at the past.
>>> Please talk to each other. Can you/we work this out together?
>>
>> I agree -- thanks for initiating this conversation!
>
>
>
Re: working together
Posted by Johan Oskarsson <jo...@oskarsson.nu>.
+1 for Sandeeps development process suggestions.
In order to address some of the issues brought forward in this thread I
have adapted the following wiki pages from other projects and from
various emails. They could serve as the basis for an initial process.
http://cwiki.apache.org/confluence/display/CSDR/HowToContribute
http://cwiki.apache.org/confluence/display/CSDR/HowToCommit
http://cwiki.apache.org/confluence/display/CSDR/HowToRelease
A short summary and description of why these points make sense:
* "Patch-only" evolution of code, attached to a jira issue
* At least one +1 on each issue before it can be committed, -1 stops the
patch.
Those two points would make sure that if someone disagrees with a
change, a refactoring etc, they have a chance to voice their opinion and
steer it into the right direction.
* Trunk is not considered stable, but must pass unit tests
* Any non trivial change should include unit tests
* When a branch is created to prepare for a release extra effort is put
into QA to make sure the release is as stable as possible. Point
releases would then go out to fix issues found after the release was done.
* Once a release has been out for a while and people are using it in
production without problems it is upgraded to "stable" status.
The purpose of these points is to encourage a "vibrant codebase", to not
be afraid of for example refactoring if it improves the code readability
or testability. I appreciate that Cassandra is a complex system and that
changes might have unwanted side effects, but hopefully adding tests and
code reviews will reduce those. As a final catch-all the release
candidate and "stable release" process should help end users avoid bugs.
Thoughts on the wiki pages? Do they help resolve some of the problems?
/Johan
Sandeep Tata wrote:
> Thoughts inline:
>
>> So the problems I am seeing are:
>>
>> 1. We elected a committer without real community consensus. The
>> barrier of entry was unnatural low on this one. On the other hand we
>> need non-FB committers for the graduation. The more the better. (No
>> reason for low entry barrier though!)
>
> I think everyone (including the FB guys) agree that Jonathan has been
> working hard to help move the codebase forward. He has been quick to
> revert changes that broke the code that the FB guys had in the
> pipeline and have committed since. I think much of the friction comes
> from not having a process, which takes us to Torsten's #2:
>
>> 2. A missing definition of development process:
>> - What is considered a valid code review?
>> - How much are changes discussed up-front?
>> - What is the roadmap? ...for whom? (weighted as a community)
>
> This is probably where we need most work. Here are some simple suggestions:
>
> a) I'm a fan of a "patch-only" evolution of code. All changes come
> from patches, and no changes come from anywhere else (eg. the
> committers IDE). Even if it is something as simple as cleaning up
> comments or changing a variable name.
> b) A patch gets applied if at least one reviewer +1s it, and no one -1s it.
> c) A patch should pass all unit tests. Any significant patch should
> come with additional unit tests.
>
> Some of this, of course, will mean "more work" for the committers.
> Sure, but such processes are essential if the project is to grow
> beyond a small group of core contributors.
>
>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>> for the required stability? Once we have the separation between stable
>> and trunk: Will patches really find it's way from trunk into stable?
>> Is Facebook OK with that approach. Will everyone cope with the
>> additional work of merging? Would it be useful ...or overkill to use
>> merge tracking?
>
> I agree with Matt. Trunk should pass build + tests, but should not be
> trusted for production. I think 0.2 was supposed to be a stable
> branch. Avinash, Prashant -- what are your thoughts on this? Are you
> guys comfortable with this approach? Do you foresee any problems?
>
> Basically, use a "release" branch for production. The release branches
> only admit stability patches. New feature and cleanup patches go to
> trunk. Folks running Cassandra in production only need to be nervous
> when moving from one release to next, and not worry too much about
> every single patch breaking their running system.
>
>> 4. Real world testing feedback is not publicly available. So the
>> feedback on changes will only slowly reach the community. This is not
>> easy for a project like this. But is there a faster way to provide
>> testing feedback? (IIRC Yahoo was providing testing feedback for
>> Hadoop. They even try to auto-apply patches from JIRA)
>
> With time, FB may be able to provide feedback from their "divert some
> traffic to the new version" system. Auto-applying patches from JIRA
> sounds a little ambitious right now :-)
>
>> 5. Is there really no code ownership issue. Working on a code base for
>> 1-2 years can get you attached to the code you have written. Can
>> everyone really let go? Is it OK if someone else really just rewrites
>> parts of what you wrote? (No, it doesn't mean the original code was
>> bad! But maybe with the new code it is more readable ...
>> understandable - especially for someone who hasn't spent the past
>> years working on that code) Is there room for refactoring?
>
> :-)
>
>
>> This is a tough situation but I hope everyone sees this as an
>> opportunity. Please let's discuss this openly in civilize manner.
>> Focusing on how to solve these points rather than looking at the past.
>> Please talk to each other. Can you/we work this out together?
>
> I agree -- thanks for initiating this conversation!
Re: working together
Posted by Sandeep Tata <sa...@gmail.com>.
Thoughts inline:
> So the problems I am seeing are:
>
> 1. We elected a committer without real community consensus. The
> barrier of entry was unnatural low on this one. On the other hand we
> need non-FB committers for the graduation. The more the better. (No
> reason for low entry barrier though!)
I think everyone (including the FB guys) agree that Jonathan has been
working hard to help move the codebase forward. He has been quick to
revert changes that broke the code that the FB guys had in the
pipeline and have committed since. I think much of the friction comes
from not having a process, which takes us to Torsten's #2:
> 2. A missing definition of development process:
> - What is considered a valid code review?
> - How much are changes discussed up-front?
> - What is the roadmap? ...for whom? (weighted as a community)
This is probably where we need most work. Here are some simple suggestions:
a) I'm a fan of a "patch-only" evolution of code. All changes come
from patches, and no changes come from anywhere else (eg. the
committers IDE). Even if it is something as simple as cleaning up
comments or changing a variable name.
b) A patch gets applied if at least one reviewer +1s it, and no one -1s it.
c) A patch should pass all unit tests. Any significant patch should
come with additional unit tests.
Some of this, of course, will mean "more work" for the committers.
Sure, but such processes are essential if the project is to grow
beyond a small group of core contributors.
> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> for the required stability? Once we have the separation between stable
> and trunk: Will patches really find it's way from trunk into stable?
> Is Facebook OK with that approach. Will everyone cope with the
> additional work of merging? Would it be useful ...or overkill to use
> merge tracking?
I agree with Matt. Trunk should pass build + tests, but should not be
trusted for production. I think 0.2 was supposed to be a stable
branch. Avinash, Prashant -- what are your thoughts on this? Are you
guys comfortable with this approach? Do you foresee any problems?
Basically, use a "release" branch for production. The release branches
only admit stability patches. New feature and cleanup patches go to
trunk. Folks running Cassandra in production only need to be nervous
when moving from one release to next, and not worry too much about
every single patch breaking their running system.
> 4. Real world testing feedback is not publicly available. So the
> feedback on changes will only slowly reach the community. This is not
> easy for a project like this. But is there a faster way to provide
> testing feedback? (IIRC Yahoo was providing testing feedback for
> Hadoop. They even try to auto-apply patches from JIRA)
With time, FB may be able to provide feedback from their "divert some
traffic to the new version" system. Auto-applying patches from JIRA
sounds a little ambitious right now :-)
> 5. Is there really no code ownership issue. Working on a code base for
> 1-2 years can get you attached to the code you have written. Can
> everyone really let go? Is it OK if someone else really just rewrites
> parts of what you wrote? (No, it doesn't mean the original code was
> bad! But maybe with the new code it is more readable ...
> understandable - especially for someone who hasn't spent the past
> years working on that code) Is there room for refactoring?
:-)
> This is a tough situation but I hope everyone sees this as an
> opportunity. Please let's discuss this openly in civilize manner.
> Focusing on how to solve these points rather than looking at the past.
> Please talk to each other. Can you/we work this out together?
I agree -- thanks for initiating this conversation!
Re: working together
Posted by Matt Revelle <mr...@gmail.com>.
On Apr 7, 2009, at 5:13 PM, Jonathan Ellis wrote:
> On Tue, Apr 7, 2009 at 3:58 PM, Matt Revelle <mr...@gmail.com>
> wrote:
>> Jonathan Ellis has been working on various Cassandra code bases
>> released by
>> Facebook for at least nine months now,
>
> Sorry, my inner pedant has to say, "four months." :) But thanks.
>
> -Jonathan
Doh, seemed longer. Point still stands.
-Matt
Re: working together
Posted by Jonathan Ellis <jb...@gmail.com>.
On Tue, Apr 7, 2009 at 3:58 PM, Matt Revelle <mr...@gmail.com> wrote:
> Jonathan Ellis has been working on various Cassandra code bases released by
> Facebook for at least nine months now,
Sorry, my inner pedant has to say, "four months." :) But thanks.
-Jonathan
Re: working together
Posted by Matt Revelle <mr...@gmail.com>.
Please take what I say with a grain of salt as I'm not invested in
this project, but I have been watching it since the initial open
release by Facebook.
Comments inline.
On Apr 7, 2009, at 4:10 PM, Torsten Curdt wrote:
> Hey folks,
>
> I thought back and forth about whether this email should go to the
> private list or not. But I have a Cocoon background and in Cocoon land
> we found it useful to only use the private list when there is really
> no other way. This is about the community so I think it deserves to be
> discussed in the open. I hope no one is offended by that approach.
>
> Let me try to explain how I see the history of Cassandra. Please
> correct me where I am wrong:
>
> Cassandra was (mainly?) developed by Facebook and is being used in
> production over there. At some stage it got open sourced on Google
> Code. External contributors did not feel like there is a real
> community around it as patches did not get applied. Out of frustration
> people started (or at least proposed) to fork the code base. As a fork
> means community fragmentation the projects has turned to the Apache
> Incubator hoping to turn this fragmented contributors into a healthy
> community.
>
> In January the project got accepted. Papers were filed and in March we
> saw the code enter the Apache subversion repository. A few weeks later
> a first committer was proposed. Unfortunately no one noticed that the
> actual authors bringing the code were NOT on the private list where
> the vote was held. So we got a new committer without the consent
> and/or feedback of the original authors. A big surprise. The people
> that brought over the code now feel a bit betrayed by the process.
> They have to deal with a committer that does changes all over the
> place on a code base they depend on for production. They have the
> feeling these changes are untested (at least not tested enough to get
> right into production at Facebook) and that this destabilize the code
> base. On the other hand there is new blood, new drive to the project.
> While Facebook needs it's stability, other contributors needs the
> change to meet their goals and deadlines.
>
> So the problems I am seeing are:
>
> 1. We elected a committer without real community consensus. The
> barrier of entry was unnatural low on this one. On the other hand we
> need non-FB committers for the graduation. The more the better. (No
> reason for low entry barrier though!)
Jonathan Ellis has been working on various Cassandra code bases
released by Facebook for at least nine months now,
his Cassandra repository on github had become the community standard.
He's one of three people in the world that are qualified to be a
committer.
The recent issues have to do with Facebook developers having
expectations that differ from open source standard practices.
>
>
> 2. A missing definition of development process:
> - What is considered a valid code review?
> - How much are changes discussed up-front?
> - What is the roadmap? ...for whom? (weighted as a community)
All important items, due to a lack of leadership this hasn't been
addressed.
>
>
> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> for the required stability? Once we have the separation between stable
> and trunk: Will patches really find it's way from trunk into stable?
> Is Facebook OK with that approach. Will everyone cope with the
> additional work of merging? Would it be useful ...or overkill to use
> merge tracking?
No, the trunk should build and pass tests, but recent changes to a
codebase
should never be considered "stable". The definition of "stable" even
indicates that.
>
>
> 4. Real world testing feedback is not publicly available. So the
> feedback on changes will only slowly reach the community. This is not
> easy for a project like this. But is there a faster way to provide
> testing feedback? (IIRC Yahoo was providing testing feedback for
> Hadoop. They even try to auto-apply patches from JIRA)
>
> 5. Is there really no code ownership issue. Working on a code base for
> 1-2 years can get you attached to the code you have written. Can
> everyone really let go? Is it OK if someone else really just rewrites
> parts of what you wrote? (No, it doesn't mean the original code was
> bad! But maybe with the new code it is more readable ...
> understandable - especially for someone who hasn't spent the past
> years working on that code) Is there room for refactoring?
>
> Anything else I am missing?
>
> This is a tough situation but I hope everyone sees this as an
> opportunity. Please let's discuss this openly in civilize manner.
> Focusing on how to solve these points rather than looking at the past.
> Please talk to each other. Can you/we work this out together?
Lack of leadership is the most obvious difference between this project
and healthy open source projects.
I hope that changes.
-Matt
>
>
> cheers
> --
> Torsten
Re: working together
Posted by Jun Rao <ju...@almaden.ibm.com>.
Jonathan Ellis <jb...@gmail.com> wrote on 04/07/2009 09:02:22 PM:
>
> On Tue, Apr 7, 2009 at 10:11 PM, Avinash Lakshman
> <av...@gmail.com> wrote:
> > The part that is very disconcerting are the following:
> > (1) If one becomes a committer one is not expected to blitz through the
code
> > base and start refactoring everything.
>
> There are two reasons I refactor.
>
> One is, I always try to leave the code better than I found it. This
> helps fight technical debt. In a project the size of cassandra you
> have to do this or you end up with fragile code that you cannot change
> without introducing regressions. Usually this is not because the
> original code was bad, but because the requirements changed as the
> project grew and now there is a better way. Or sometimes there is an
> idiom better suited to a situation that the original author was not
> familiar with. That's okay too; nobody's perfect.
>
> An example of this is r762440, replace String.indexOf != -1 with
> String.contains (review by johano).
>
> Two is, if I am going to introduce a new feature I will try to
> refactor first without changing behavior in such away that the feature
> becomes easier to add. This breaks the changes up into smaller pieces
> which are easier to review and easier to validate against regressions.
> (The more things you change at once, the harder it is to find which
> caused a problem.)
>
> A specific class of this kind of change is merging copy/pasted code
> into method calls. Duplicate code makes it hard to add features
> because you have to know about the duplicates and remember change all
> when you change one. There is a lot of this in cassandra. Examples
> include r762440, r762440, r762440. (This specific area of the code
> did bite me while implementing remove; that's what the "os x error"
> thread on google groups was about.)
>
> So there is a method to my madness. :)
>
In my opinion, the kind of refactoring that removes duplicate code is
necessary. This reduces the chance of introducing bugs and makes it easier
for adding new features. This kind of refactoring is probably better done
sooner than later. Of course, it would be great if the original developers
can guide this process.
Jun
IBM Almaden Research Center
K55/B1, 650 Harry Road, San Jose, CA 95120-6099
junrao@almaden.ibm.com
Re: working together
Posted by Jonathan Ellis <jb...@gmail.com>.
On Tue, Apr 7, 2009 at 10:11 PM, Avinash Lakshman
<av...@gmail.com> wrote:
> The part that is very disconcerting are the following:
> (1) If one becomes a committer one is not expected to blitz through the code
> base and start refactoring everything.
There are two reasons I refactor.
One is, I always try to leave the code better than I found it. This
helps fight technical debt. In a project the size of cassandra you
have to do this or you end up with fragile code that you cannot change
without introducing regressions. Usually this is not because the
original code was bad, but because the requirements changed as the
project grew and now there is a better way. Or sometimes there is an
idiom better suited to a situation that the original author was not
familiar with. That's okay too; nobody's perfect.
An example of this is r762440, replace String.indexOf != -1 with
String.contains (review by johano).
Two is, if I am going to introduce a new feature I will try to
refactor first without changing behavior in such away that the feature
becomes easier to add. This breaks the changes up into smaller pieces
which are easier to review and easier to validate against regressions.
(The more things you change at once, the harder it is to find which
caused a problem.)
A specific class of this kind of change is merging copy/pasted code
into method calls. Duplicate code makes it hard to add features
because you have to know about the duplicates and remember change all
when you change one. There is a lot of this in cassandra. Examples
include r762440, r762440, r762440. (This specific area of the code
did bite me while implementing remove; that's what the "os x error"
thread on google groups was about.)
So there is a method to my madness. :)
> In any organization one doesn't just go about ripping out everyone
> else's code for no rhyme or reason. That will offend anybody. I personally
> would not go about ripping someone else's code apart if I had become
> committer. It is just that respect ought to be there.
I think there may be cultural differences here. It is not a sign of
disrespect to think that I can improve your code or anyone's. :)
> (2) This is something that I have said many times over. Certain things are
> the way they are for a reason.
And others are not. For instance, there's a lot of classes that still
implement Serializable, but that's misleading because you roll your
own serialization now.
Or for something more meaningful, the code-by-copy-paste examples.
It's just not reasonable to say "don't touch anything, it's all done
for a reason" when there are clear counterexamples right there in the
code.
I don't see any alternative to your becoming involved in the review
process so you can tell us when something really is important, such as
> For example when I say ConcurrentHashMap is a
> memory hog I say it because we have seen this in practice. How does it
> manifest itself? I obviously do not recall since all this was over a year
> ago.
So I replaced those with NonBlockingHashMap. The system is working. :)
> No one can claim to have run tests the way we have in the last year and
> a half. One cannot just run some simple test and say well I do not see the
> problem. I am not dumb.
Again, maybe there are cultural differences here... Todd was asking
out of curiosity and a desire for more information. Often knowing how
you arrived at a conclusion is as useful or more so than the
conclusion itself. (Otherwise you run the risk of encouraging cargo
cult programming.)
> My understanding was that new committers come in and start with some feature
> implement that and then slowly start looking into what more they could do
> going forward.
That's what happened... in December. I've had a lot of time to look. :)
> It is NOT come in and refactor the hell out of the system
> because you like something to be in a specific way.
I think I addressed this above, but I'm happy to discuss specific
commits in more detail.
-Jonathan
Re: working together
Posted by Torsten Curdt <tc...@vafer.org>.
On Wed, Apr 8, 2009 at 05:11, Avinash Lakshman
<av...@gmail.com> wrote:
> Point #1 I would love to have committers from outside but the way this
> happened took all of us by surprise. Granted we were not on the list but if
> I were one of the committers I would have definitely pinged one of the other
> committters and asked them as to whether they knew what the hell was going
> on. Anyway this is water under bridge now.
Yes ... not much we can do about this anymore. But as said. I am more
than unhappy with that too.
> I hate bitter confrontation since
> it doesn't take anyone forward but only leaves a bitter taste in everyone's
> mouth. I have had many personal conversations with Jonathan via chat and I
> have nothing personal against anyone, perhaps not everyone but definitely
> nothing against Jonathan.
That good
> The part that is very disconcerting are the following:
> (1) If one becomes a committer one is not expected to blitz through the code
> base and start refactoring everything. There is a way this needs to be
> handled. In any organization one doesn't just go about ripping out everyone
> else's code for no rhyme or reason. That will offend anybody. I personally
> would not go about ripping someone else's code apart if I had become
> committer. It is just that respect ought to be there.
Well, this is where I might see a code ownership problem. I think
every committer can change everything he wants. Not that he should.
And you are right about respectful changes. But in an established
project I would not get offended. Unfortunately Cassandra is not an
established project in the notion of community yet.
> There is a way to get
> this done. Changes to code because person X likes something to be in some
> particular form and going and just changing that in person Y's code is just
> plain wrong. It borders on arrogance which is not the way things should be
> done. If I become a committer on Hadoop can I just go and start ripping
> apart every class and start making changes just because I don't like the
> coding style.
Please don't see this as (remotely as) arrogance. What is your stand
on refactoring? Maybe even just to understand code better. Is that
something out of question for you?
> This is a premature project on Apache and I think we need to
> keep the original developers in the loop till everyone has some degree of
> confidence on the changes made by new committers.
+1
> (2) This is something that I have said many times over. Certain things are
> the way they are for a reason. For example when I say ConcurrentHashMap is a
> memory hog I say it because we have seen this in practice. How does it
> manifest itself? I obviously do not recall since all this was over a year
> ago. No one can claim to have run tests the way we have in the last year and
> a half. One cannot just run some simple test and say well I do not see the
> problem. I am not dumb. Anyone having gone through the exercise of having
> built a system like this in an organization will realize that the tests are
> very intermingled with the organization's infrastructure. I have no time to
> rip that all apart and put together a test suite at this point.
As long as it is a goal I think it would be good. Or as I said: maybe
just forward the testing results to the dev community somehow.
> This is just
> an example. There are many such instances - after all - we are the ones who
> have the operational experience with this and I do not think anyone can
> claim to understand the behavior this system in production workloads better
> than we do.
Probably right at this point or is anyone else using the project in a
real world scenario here yet?
> My understanding was that new committers come in and start with some feature
> implement that and then slowly start looking into what more they could do
> going forward. It is NOT come in and refactor the hell out of the system
> because you like something to be in a specific way. I do not beleive this
> will fly in any community. It is something like we now going through the
> entire code base and changing all the stuff just because I like it in a
> specific way. This seems ludicrous.
Well, I cannot (and frankly also don't want to) judge the
contributions over at github. But it appears there is code knowledge.
You can never expect someone to catch up on the 2 years you guys have
been working on the project. So you will have to deal with someone
that is new to the code base. For the project to fly you must be
willing to do some mentoring as well.
> We may have no experience in open source
> but we understand etiquette very well. This just doesn't seem the way things
> work in other Apache projects which are successful. We work very closely
> with two committers from the Hadoop project who were flabbergasted with the
> refactor changes that were going in. That is my gripe with the whole thing.
The incubation of Hadoop was very different pre conditions than
Cassandra. Which makes it very hard to compare.
Re: working together
Posted by Matthieu Riou <ma...@gmail.com>.
On Tue, Apr 7, 2009 at 8:26 PM, Avinash Lakshman <avinash.lakshman@gmail.com
> wrote:
> Hit Send a bit too early. Thanks Torsten for bringing this up. I really
> appreciate it. No one apart from committers I think should be voting for
> other people becoming committer. I am assuming here only the committers are
> involved with the project from a code perspective. With regards to that,
> Matt I respectfully disagree with your assessment about Jonathan becoming a
> committer. I strongly believe that it has to come from the committers
> themselves. In short, I mean absolutely no disrespect to Jonathan or anyone
> else, but Matt's assessment needs to come from the guys involved with this
> on a day-day basis from a code perspective. My aim was to put forth our
> frustration and not meant to put down anyone.
Let me explain the reasoning behind what I said, I hope it will help
clarify. The motto that Apache often put forth is "Community over Code".
Community for code might actually be more accurate but it's less catchy :)
We build communities around open source projects and try to build them as
strongly as possible so they can survive many, many years. And lots of folks
around here actually have quite a bit of experience doing that, so we all
benefit from that experience.
As an example (which, to be clear, doesn't apply here but illustrates the
point) the mythological genius hacker, who's able to write half of a perfect
garbage collector in a night but never communicates to anybody is a total
nightmare for an open source project. It's actually a very good way to stop
attracting any new committer. Another dangerous tendency, which applies a
bit more here, is to "freeze" the codebase, saying it's good enough, without
proposing a clear roadmap or any possible enhancement. How can anybody
volunteer in helping under such condition? I'd rather an unstable code with
a lot to do anyday.
So getting back to your "it has to come from the committers", I would agree
in general. But not for Cassandra. Cassandra is not Hadoop and at this point
is very, very far from it.
I'd like this project to survive. I have absolutely no personal interest in
it but I've pumped in enough time to not see it wasted. And accepting new
committers at this point is vital, otherwise the project will just go to a
slow and painful death, believe me, I've learned to see the spiral early. I
understand you don't like it and it probably comes as an unpleasant jolt but
I also feel like acceptable alternatives have been proposed (working on
branches, discussing patches, actually providing feedback, ...).
So what are your suggestions going forward?
Matthieu
> Cheers
> Avinash
>
> On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <
> avinash.lakshman@gmail.com
> > wrote:
>
> > Point #1 I would love to have committers from outside but the way this
> > happened took all of us by surprise. Granted we were not on the list but
> if
> > I were one of the committers I would have definitely pinged one of the
> other
> > committters and asked them as to whether they knew what the hell was
> going
> > on. Anyway this is water under bridge now. I hate bitter confrontation
> since
> > it doesn't take anyone forward but only leaves a bitter taste in
> everyone's
> > mouth. I have had many personal conversations with Jonathan via chat and
> I
> > have nothing personal against anyone, perhaps not everyone but definitely
> > nothing against Jonathan.
> > The part that is very disconcerting are the following:
> > (1) If one becomes a committer one is not expected to blitz through the
> > code base and start refactoring everything. There is a way this needs to
> be
> > handled. In any organization one doesn't just go about ripping out
> everyone
> > else's code for no rhyme or reason. That will offend anybody. I
> personally
> > would not go about ripping someone else's code apart if I had become
> > committer. It is just that respect ought to be there. There is a way to
> get
> > this done. Changes to code because person X likes something to be in some
> > particular form and going and just changing that in person Y's code is
> just
> > plain wrong. It borders on arrogance which is not the way things should
> be
> > done. If I become a committer on Hadoop can I just go and start ripping
> > apart every class and start making changes just because I don't like the
> > coding style. This is a premature project on Apache and I think we need
> to
> > keep the original developers in the loop till everyone has some degree of
> > confidence on the changes made by new committers.
> >
> > (2) This is something that I have said many times over. Certain things
> are
> > the way they are for a reason. For example when I say ConcurrentHashMap
> is a
> > memory hog I say it because we have seen this in practice. How does it
> > manifest itself? I obviously do not recall since all this was over a year
> > ago. No one can claim to have run tests the way we have in the last year
> and
> > a half. One cannot just run some simple test and say well I do not see
> the
> > problem. I am not dumb. Anyone having gone through the exercise of having
> > built a system like this in an organization will realize that the tests
> are
> > very intermingled with the organization's infrastructure. I have no time
> to
> > rip that all apart and put together a test suite at this point. This is
> just
> > an example. There are many such instances - after all - we are the ones
> who
> > have the operational experience with this and I do not think anyone can
> > claim to understand the behavior this system in production workloads
> better
> > than we do.
> >
> > My understanding was that new committers come in and start with some
> > feature implement that and then slowly start looking into what more they
> > could do going forward. It is NOT come in and refactor the hell out of
> the
> > system because you like something to be in a specific way. I do not
> beleive
> > this will fly in any community. It is something like we now going through
> > the entire code base and changing all the stuff just because I like it in
> a
> > specific way. This seems ludicrous. We may have no experience in open
> source
> > but we understand etiquette very well. This just doesn't seem the way
> things
> > work in other Apache projects which are successful. We work very closely
> > with two committers from the Hadoop project who were flabbergasted with
> the
> > refactor changes that were going in. That is my gripe with the whole
> thing.
> >
> > Cheers
> > Avinash
> >
> >
> >
> > On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> >
> >> On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org>
> wrote:
> >> > So the problems I am seeing are:
> >> >
> >> > 1. We elected a committer without real community consensus. The
> >> > barrier of entry was unnatural low on this one. On the other hand we
> >> > need non-FB committers for the graduation. The more the better. (No
> >> > reason for low entry barrier though!)
> >>
> >> It's unfortunate that Avinash and Prashant weren't part of the
> >> process. Still, when I talked to Avinash on March 1, he told me [and
> >> this is a direct quote] "If I had known you earlier I would have added
> >> you as a committer." So when I asked one of the mentors how to become
> >> a committer and it worked out from there it did not occur to me that
> >> anything was wrong.
> >>
> >> >
> >> > 2. A missing definition of development process:
> >> > - What is considered a valid code review?
> >> > - How much are changes discussed up-front?
> >>
> >> I think we have a handle on this now. All changes are put on Jira for
> >> review and are not committed until there is at least one +1 from a
> >> reviewer. (I personally prefer post-commit review because manually
> >> attaching and applying patches is tedious but we don't have enough
> >> people following the commit log for that to work right now.)
> >>
> >> > - What is the roadmap? ...for whom? (weighted as a community)
> >>
> >> That's worth a separate thread. Such as this one. :)
> >>
> >>
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
> >>
> >> > 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> >> > for the required stability? Once we have the separation between stable
> >> > and trunk: Will patches really find it's way from trunk into stable?
> >> > Is Facebook OK with that approach. Will everyone cope with the
> >> > additional work of merging? Would it be useful ...or overkill to use
> >> > merge tracking?
> >>
> >> I'm happy to assist with merging code to or from stable branches in
> >> this scenario.
> >>
> >> > This is a tough situation but I hope everyone sees this as an
> >> > opportunity. Please let's discuss this openly in civilize manner.
> >> > Focusing on how to solve these points rather than looking at the past.
> >> > Please talk to each other. Can you/we work this out together?
> >>
> >> This can still be a win/win for everyone. I think that historically
> >> facebook has felt like the community hasn't contributed much of value,
> >> but we're starting to change that. The build and test process is
> >> dramatically better than it was before thanks to community
> >> contributions. We have a real daemon mode. (Well, not in the purest
> >> sense, but it runs in the background nicely w/o nohup or screen. :)
> >> We've also found and fixed several concurrency bugs, and we're well on
> >> the way to having remove and range queries implemented.
> >>
> >> Our IRC population has more than doubled. (#cassandra on freenode:
> >>
> >>
> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
> >> for a web client) We have a chance to make this more than a niche
> >> project.
> >>
> >> -Jonathan
> >>
> >
> >
>
Re: working together
Posted by Avinash Lakshman <av...@gmail.com>.
I don't think I ever spoke about anyone's skills ever over here.
Avinash
On Tue, Apr 7, 2009 at 9:12 PM, Ian Holsman <ia...@holsman.net> wrote:
> Matt.
> please don't.
> your comments are just as valuable as anyone else's.
>
> just be aware that if you talk about someone's skills on this list, or
> their possible promotion to a committer it preempts other conversations
> happening, and can discourage that person from participating IF they are not
> made a committer then and there.
>
> many times (on other projects) people's names are put forth as possible
> committers (or members) and the feedback is 'not yet'. This can't be done on
> a public list. and why I am so discouraging of it on here.
>
>
>
>
>
> On 08/04/2009, at 2:06 PM, Matt Revelle wrote:
>
>
>> On Apr 7, 2009, at 11:33 PM, Avinash Lakshman wrote:
>>
>> I think someone else opened that door here.
>>> Avinash
>>>
>>
>> I completely agree with Avinash that current committers need to be
>> involved in the approval of new committers. There appears to have been
>> a miscommunication that resulted in some committers not being subscribed
>> to the private list.
>>
>> Since I'm not a contributing member of this project, I'll refrain from
>> further comment.
>>
>
> --
> Ian Holsman
> Ian@Holsman.net
>
>
>
>
Re: working together
Posted by Torsten Curdt <tc...@apache.org>.
On Wed, Apr 8, 2009 at 06:12, Ian Holsman <ia...@holsman.net> wrote:
> Matt.
> please don't.
> your comments are just as valuable as anyone else's.
Yes, keep it comming!
..
> many times (on other projects) people's names are put forth as possible
> committers (or members) and the feedback is 'not yet'. This can't be done on
> a public list. and why I am so discouraging of it on here.
I agree that is for the private list. But that is not what this
conversation is about I guess.
Re: working together
Posted by Ian Holsman <ia...@holsman.net>.
Matt.
please don't.
your comments are just as valuable as anyone else's.
just be aware that if you talk about someone's skills on this list, or
their possible promotion to a committer it preempts other
conversations happening, and can discourage that person from
participating IF they are not made a committer then and there.
many times (on other projects) people's names are put forth as
possible committers (or members) and the feedback is 'not yet'. This
can't be done on a public list. and why I am so discouraging of it on
here.
On 08/04/2009, at 2:06 PM, Matt Revelle wrote:
>
> On Apr 7, 2009, at 11:33 PM, Avinash Lakshman wrote:
>
>> I think someone else opened that door here.
>> Avinash
>
> I completely agree with Avinash that current committers need to be
> involved in the approval of new committers. There appears to have
> been
> a miscommunication that resulted in some committers not being
> subscribed to the private list.
>
> Since I'm not a contributing member of this project, I'll refrain
> from further comment.
--
Ian Holsman
Ian@Holsman.net
Re: working together
Posted by Matt Revelle <mr...@gmail.com>.
On Apr 7, 2009, at 11:33 PM, Avinash Lakshman wrote:
> I think someone else opened that door here.
> Avinash
I completely agree with Avinash that current committers need to be
involved in the approval of new committers. There appears to have been
a miscommunication that resulted in some committers not being
subscribed to the private list.
Since I'm not a contributing member of this project, I'll refrain from
further comment.
Re: working together
Posted by Avinash Lakshman <av...@gmail.com>.
I think someone else opened that door here.
Avinash
On Tue, Apr 7, 2009 at 8:30 PM, Ian Holsman <ia...@holsman.net> wrote:
> guys.
> we have a private list to discuss the pro's and con's of people being a
> comitter.
> keep these personal discussions off the development list. It doesn't help
> anyone.
>
> as was mentioned several times. we assumed you subscribed when you were
> asked to on the 20th of January.
>
>
> On 08/04/2009, at 1:26 PM, Avinash Lakshman wrote:
>
> Hit Send a bit too early. Thanks Torsten for bringing this up. I really
>> appreciate it. No one apart from committers I think should be voting for
>> other people becoming committer. I am assuming here only the committers
>> are
>> involved with the project from a code perspective. With regards to that,
>> Matt I respectfully disagree with your assessment about Jonathan becoming
>> a
>> committer. I strongly believe that it has to come from the committers
>> themselves. In short, I mean absolutely no disrespect to Jonathan or
>> anyone
>> else, but Matt's assessment needs to come from the guys involved with this
>> on a day-day basis from a code perspective. My aim was to put forth our
>> frustration and not meant to put down anyone.
>> Cheers
>> Avinash
>>
>> On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <
>> avinash.lakshman@gmail.com
>>
>>> wrote:
>>>
>>
>> Point #1 I would love to have committers from outside but the way this
>>> happened took all of us by surprise. Granted we were not on the list but
>>> if
>>> I were one of the committers I would have definitely pinged one of the
>>> other
>>> committters and asked them as to whether they knew what the hell was
>>> going
>>> on. Anyway this is water under bridge now. I hate bitter confrontation
>>> since
>>> it doesn't take anyone forward but only leaves a bitter taste in
>>> everyone's
>>> mouth. I have had many personal conversations with Jonathan via chat and
>>> I
>>> have nothing personal against anyone, perhaps not everyone but definitely
>>> nothing against Jonathan.
>>> The part that is very disconcerting are the following:
>>> (1) If one becomes a committer one is not expected to blitz through the
>>> code base and start refactoring everything. There is a way this needs to
>>> be
>>> handled. In any organization one doesn't just go about ripping out
>>> everyone
>>> else's code for no rhyme or reason. That will offend anybody. I
>>> personally
>>> would not go about ripping someone else's code apart if I had become
>>> committer. It is just that respect ought to be there. There is a way to
>>> get
>>> this done. Changes to code because person X likes something to be in some
>>> particular form and going and just changing that in person Y's code is
>>> just
>>> plain wrong. It borders on arrogance which is not the way things should
>>> be
>>> done. If I become a committer on Hadoop can I just go and start ripping
>>> apart every class and start making changes just because I don't like the
>>> coding style. This is a premature project on Apache and I think we need
>>> to
>>> keep the original developers in the loop till everyone has some degree of
>>> confidence on the changes made by new committers.
>>>
>>> (2) This is something that I have said many times over. Certain things
>>> are
>>> the way they are for a reason. For example when I say ConcurrentHashMap
>>> is a
>>> memory hog I say it because we have seen this in practice. How does it
>>> manifest itself? I obviously do not recall since all this was over a year
>>> ago. No one can claim to have run tests the way we have in the last year
>>> and
>>> a half. One cannot just run some simple test and say well I do not see
>>> the
>>> problem. I am not dumb. Anyone having gone through the exercise of having
>>> built a system like this in an organization will realize that the tests
>>> are
>>> very intermingled with the organization's infrastructure. I have no time
>>> to
>>> rip that all apart and put together a test suite at this point. This is
>>> just
>>> an example. There are many such instances - after all - we are the ones
>>> who
>>> have the operational experience with this and I do not think anyone can
>>> claim to understand the behavior this system in production workloads
>>> better
>>> than we do.
>>>
>>> My understanding was that new committers come in and start with some
>>> feature implement that and then slowly start looking into what more they
>>> could do going forward. It is NOT come in and refactor the hell out of
>>> the
>>> system because you like something to be in a specific way. I do not
>>> beleive
>>> this will fly in any community. It is something like we now going through
>>> the entire code base and changing all the stuff just because I like it in
>>> a
>>> specific way. This seems ludicrous. We may have no experience in open
>>> source
>>> but we understand etiquette very well. This just doesn't seem the way
>>> things
>>> work in other Apache projects which are successful. We work very closely
>>> with two committers from the Hadoop project who were flabbergasted with
>>> the
>>> refactor changes that were going in. That is my gripe with the whole
>>> thing.
>>>
>>> Cheers
>>> Avinash
>>>
>>>
>>>
>>> On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com>
>>> wrote:
>>>
>>> On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org>
>>>> wrote:
>>>>
>>>>> So the problems I am seeing are:
>>>>>
>>>>> 1. We elected a committer without real community consensus. The
>>>>> barrier of entry was unnatural low on this one. On the other hand we
>>>>> need non-FB committers for the graduation. The more the better. (No
>>>>> reason for low entry barrier though!)
>>>>>
>>>>
>>>> It's unfortunate that Avinash and Prashant weren't part of the
>>>> process. Still, when I talked to Avinash on March 1, he told me [and
>>>> this is a direct quote] "If I had known you earlier I would have added
>>>> you as a committer." So when I asked one of the mentors how to become
>>>> a committer and it worked out from there it did not occur to me that
>>>> anything was wrong.
>>>>
>>>>
>>>>> 2. A missing definition of development process:
>>>>> - What is considered a valid code review?
>>>>> - How much are changes discussed up-front?
>>>>>
>>>>
>>>> I think we have a handle on this now. All changes are put on Jira for
>>>> review and are not committed until there is at least one +1 from a
>>>> reviewer. (I personally prefer post-commit review because manually
>>>> attaching and applying patches is tedious but we don't have enough
>>>> people following the commit log for that to work right now.)
>>>>
>>>> - What is the roadmap? ...for whom? (weighted as a community)
>>>>>
>>>>
>>>> That's worth a separate thread. Such as this one. :)
>>>>
>>>>
>>>> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
>>>>
>>>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>>>>> for the required stability? Once we have the separation between stable
>>>>> and trunk: Will patches really find it's way from trunk into stable?
>>>>> Is Facebook OK with that approach. Will everyone cope with the
>>>>> additional work of merging? Would it be useful ...or overkill to use
>>>>> merge tracking?
>>>>>
>>>>
>>>> I'm happy to assist with merging code to or from stable branches in
>>>> this scenario.
>>>>
>>>> This is a tough situation but I hope everyone sees this as an
>>>>> opportunity. Please let's discuss this openly in civilize manner.
>>>>> Focusing on how to solve these points rather than looking at the past.
>>>>> Please talk to each other. Can you/we work this out together?
>>>>>
>>>>
>>>> This can still be a win/win for everyone. I think that historically
>>>> facebook has felt like the community hasn't contributed much of value,
>>>> but we're starting to change that. The build and test process is
>>>> dramatically better than it was before thanks to community
>>>> contributions. We have a real daemon mode. (Well, not in the purest
>>>> sense, but it runs in the background nicely w/o nohup or screen. :)
>>>> We've also found and fixed several concurrency bugs, and we're well on
>>>> the way to having remove and range queries implemented.
>>>>
>>>> Our IRC population has more than doubled. (#cassandra on freenode:
>>>>
>>>>
>>>> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
>>>> for a web client) We have a chance to make this more than a niche
>>>> project.
>>>>
>>>> -Jonathan
>>>>
>>>>
>>>
>>>
> --
> Ian Holsman
> Ian@Holsman.net
>
>
>
>
Re: working together
Posted by Johann Schleier-Smith <js...@gmail.com>.
I joined the Cassandra-dev list earlier today so I post with hesitation,
knowing that others here have much more knowledge of the project. However,
I hope that my thoughts can provide useful and objective "outsider
perspective."
The Facebook team has created a really groundbreaking database that has the
promise of widespread applicability. The architecture, and code make it
clear that this team has exceptional talent and experience.
Please correct me if I'm wrong, but I doubt that anyone else has anywhere
near the amount of production learnings about Cassandra as Facebook has. If
there's any hesitation about stability from their perspective this should be
taken very seriously. It's a lot easier to advance platform code like this
if it never strays very far from production-readiness.
Community contributions during the past months are clearly making an impact.
The code base looked very much like an in-house project before, but there
is clear progress toward a clean distribution that is easy for people to
work with. Daemon mode and unit tests are evidence of this. I'm sure we'll
see useful features, more tests, examples, and documentation rounding out
the contributions before too long.
Refactoring is also a valuable contribution because it makes the code easier
to work with. I have gripes of my own, but keeping the code easy for
Facebook to work with remains of paramount importance. My experience is
with a large 24/7 environment that handles valuable customer data. In this
context you really can't afford to have breakages, and in the tension
between refactoring and stability, the latter usually wins unless you have
an excellent suite of unit tests and an really good stress test platform.
Hopefully Cassandra will have both of these before too long.
It might be a little unusual for an incubator project to put emphasis on
stability as opposed to feature development. Then again it's a little
unusual for an incubator project to have a large production deployment.
I imagine that the roadmap might first go after adding packaging and
features critical to early-adopters outside of Facebook. This will allow
others to build similar production experience and to contribute at a higher
level. Creating a stable branch at that time would allow trunk to progress
with creative enhancements in a somewhat less encumbered way.
On Wed, Apr 8, 2009 at 1:23 AM, Neophytos Demetriou <ne...@gmail.com>wrote:
> Torsten Curdt wrote:
>
>> On Wed, Apr 8, 2009 at 05:30, Ian Holsman <ia...@holsman.net> wrote:
>>
>>> guys.
>>> we have a private list to discuss the pro's and con's of people being a
>>> comitter.
>>> keep these personal discussions off the development list. It doesn't help
>>> anyone.
>>>
>>
>> Not sure I agree here. I did not see the thread talk about the
>> pros/cons. And I think it is about the community so it's fair to have
>> that in the open. (Sorry, that's how I was raised at Cocoon)
>> But I rather leave that up the person in question to decide.
>>
>
> +1
>
Re: working together
Posted by Neophytos Demetriou <ne...@gmail.com>.
Torsten Curdt wrote:
> On Wed, Apr 8, 2009 at 05:30, Ian Holsman <ia...@holsman.net> wrote:
>> guys.
>> we have a private list to discuss the pro's and con's of people being a
>> comitter.
>> keep these personal discussions off the development list. It doesn't help
>> anyone.
>
> Not sure I agree here. I did not see the thread talk about the
> pros/cons. And I think it is about the community so it's fair to have
> that in the open. (Sorry, that's how I was raised at Cocoon)
> But I rather leave that up the person in question to decide.
+1
Re: working together
Posted by Torsten Curdt <tc...@vafer.org>.
On Wed, Apr 8, 2009 at 05:30, Ian Holsman <ia...@holsman.net> wrote:
> guys.
> we have a private list to discuss the pro's and con's of people being a
> comitter.
> keep these personal discussions off the development list. It doesn't help
> anyone.
Not sure I agree here. I did not see the thread talk about the
pros/cons. And I think it is about the community so it's fair to have
that in the open. (Sorry, that's how I was raised at Cocoon)
But I rather leave that up the person in question to decide.
Re: working together
Posted by Ian Holsman <ia...@holsman.net>.
guys.
we have a private list to discuss the pro's and con's of people being
a comitter.
keep these personal discussions off the development list. It doesn't
help anyone.
as was mentioned several times. we assumed you subscribed when you
were asked to on the 20th of January.
On 08/04/2009, at 1:26 PM, Avinash Lakshman wrote:
> Hit Send a bit too early. Thanks Torsten for bringing this up. I
> really
> appreciate it. No one apart from committers I think should be voting
> for
> other people becoming committer. I am assuming here only the
> committers are
> involved with the project from a code perspective. With regards to
> that,
> Matt I respectfully disagree with your assessment about Jonathan
> becoming a
> committer. I strongly believe that it has to come from the committers
> themselves. In short, I mean absolutely no disrespect to Jonathan or
> anyone
> else, but Matt's assessment needs to come from the guys involved
> with this
> on a day-day basis from a code perspective. My aim was to put forth
> our
> frustration and not meant to put down anyone.
> Cheers
> Avinash
>
> On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <avinash.lakshman@gmail.com
>> wrote:
>
>> Point #1 I would love to have committers from outside but the way
>> this
>> happened took all of us by surprise. Granted we were not on the
>> list but if
>> I were one of the committers I would have definitely pinged one of
>> the other
>> committters and asked them as to whether they knew what the hell
>> was going
>> on. Anyway this is water under bridge now. I hate bitter
>> confrontation since
>> it doesn't take anyone forward but only leaves a bitter taste in
>> everyone's
>> mouth. I have had many personal conversations with Jonathan via
>> chat and I
>> have nothing personal against anyone, perhaps not everyone but
>> definitely
>> nothing against Jonathan.
>> The part that is very disconcerting are the following:
>> (1) If one becomes a committer one is not expected to blitz through
>> the
>> code base and start refactoring everything. There is a way this
>> needs to be
>> handled. In any organization one doesn't just go about ripping out
>> everyone
>> else's code for no rhyme or reason. That will offend anybody. I
>> personally
>> would not go about ripping someone else's code apart if I had become
>> committer. It is just that respect ought to be there. There is a
>> way to get
>> this done. Changes to code because person X likes something to be
>> in some
>> particular form and going and just changing that in person Y's code
>> is just
>> plain wrong. It borders on arrogance which is not the way things
>> should be
>> done. If I become a committer on Hadoop can I just go and start
>> ripping
>> apart every class and start making changes just because I don't
>> like the
>> coding style. This is a premature project on Apache and I think we
>> need to
>> keep the original developers in the loop till everyone has some
>> degree of
>> confidence on the changes made by new committers.
>>
>> (2) This is something that I have said many times over. Certain
>> things are
>> the way they are for a reason. For example when I say
>> ConcurrentHashMap is a
>> memory hog I say it because we have seen this in practice. How does
>> it
>> manifest itself? I obviously do not recall since all this was over
>> a year
>> ago. No one can claim to have run tests the way we have in the last
>> year and
>> a half. One cannot just run some simple test and say well I do not
>> see the
>> problem. I am not dumb. Anyone having gone through the exercise of
>> having
>> built a system like this in an organization will realize that the
>> tests are
>> very intermingled with the organization's infrastructure. I have no
>> time to
>> rip that all apart and put together a test suite at this point.
>> This is just
>> an example. There are many such instances - after all - we are the
>> ones who
>> have the operational experience with this and I do not think anyone
>> can
>> claim to understand the behavior this system in production
>> workloads better
>> than we do.
>>
>> My understanding was that new committers come in and start with some
>> feature implement that and then slowly start looking into what more
>> they
>> could do going forward. It is NOT come in and refactor the hell out
>> of the
>> system because you like something to be in a specific way. I do not
>> beleive
>> this will fly in any community. It is something like we now going
>> through
>> the entire code base and changing all the stuff just because I like
>> it in a
>> specific way. This seems ludicrous. We may have no experience in
>> open source
>> but we understand etiquette very well. This just doesn't seem the
>> way things
>> work in other Apache projects which are successful. We work very
>> closely
>> with two committers from the Hadoop project who were flabbergasted
>> with the
>> refactor changes that were going in. That is my gripe with the
>> whole thing.
>>
>> Cheers
>> Avinash
>>
>>
>>
>> On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com>
>> wrote:
>>
>>> On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org>
>>> wrote:
>>>> So the problems I am seeing are:
>>>>
>>>> 1. We elected a committer without real community consensus. The
>>>> barrier of entry was unnatural low on this one. On the other hand
>>>> we
>>>> need non-FB committers for the graduation. The more the better. (No
>>>> reason for low entry barrier though!)
>>>
>>> It's unfortunate that Avinash and Prashant weren't part of the
>>> process. Still, when I talked to Avinash on March 1, he told me
>>> [and
>>> this is a direct quote] "If I had known you earlier I would have
>>> added
>>> you as a committer." So when I asked one of the mentors how to
>>> become
>>> a committer and it worked out from there it did not occur to me that
>>> anything was wrong.
>>>
>>>>
>>>> 2. A missing definition of development process:
>>>> - What is considered a valid code review?
>>>> - How much are changes discussed up-front?
>>>
>>> I think we have a handle on this now. All changes are put on Jira
>>> for
>>> review and are not committed until there is at least one +1 from a
>>> reviewer. (I personally prefer post-commit review because manually
>>> attaching and applying patches is tedious but we don't have enough
>>> people following the commit log for that to work right now.)
>>>
>>>> - What is the roadmap? ...for whom? (weighted as a community)
>>>
>>> That's worth a separate thread. Such as this one. :)
>>>
>>> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
>>>
>>>> 3. Is trunk considered "stable"? Or aren't we missing a stable
>>>> branch
>>>> for the required stability? Once we have the separation between
>>>> stable
>>>> and trunk: Will patches really find it's way from trunk into
>>>> stable?
>>>> Is Facebook OK with that approach. Will everyone cope with the
>>>> additional work of merging? Would it be useful ...or overkill to
>>>> use
>>>> merge tracking?
>>>
>>> I'm happy to assist with merging code to or from stable branches in
>>> this scenario.
>>>
>>>> This is a tough situation but I hope everyone sees this as an
>>>> opportunity. Please let's discuss this openly in civilize manner.
>>>> Focusing on how to solve these points rather than looking at the
>>>> past.
>>>> Please talk to each other. Can you/we work this out together?
>>>
>>> This can still be a win/win for everyone. I think that historically
>>> facebook has felt like the community hasn't contributed much of
>>> value,
>>> but we're starting to change that. The build and test process is
>>> dramatically better than it was before thanks to community
>>> contributions. We have a real daemon mode. (Well, not in the
>>> purest
>>> sense, but it runs in the background nicely w/o nohup or screen. :)
>>> We've also found and fixed several concurrency bugs, and we're
>>> well on
>>> the way to having remove and range queries implemented.
>>>
>>> Our IRC population has more than doubled. (#cassandra on freenode:
>>>
>>> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
>>> for a web client) We have a chance to make this more than a niche
>>> project.
>>>
>>> -Jonathan
>>>
>>
>>
--
Ian Holsman
Ian@Holsman.net
Re: working together
Posted by Avinash Lakshman <av...@gmail.com>.
Hit Send a bit too early. Thanks Torsten for bringing this up. I really
appreciate it. No one apart from committers I think should be voting for
other people becoming committer. I am assuming here only the committers are
involved with the project from a code perspective. With regards to that,
Matt I respectfully disagree with your assessment about Jonathan becoming a
committer. I strongly believe that it has to come from the committers
themselves. In short, I mean absolutely no disrespect to Jonathan or anyone
else, but Matt's assessment needs to come from the guys involved with this
on a day-day basis from a code perspective. My aim was to put forth our
frustration and not meant to put down anyone.
Cheers
Avinash
On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <avinash.lakshman@gmail.com
> wrote:
> Point #1 I would love to have committers from outside but the way this
> happened took all of us by surprise. Granted we were not on the list but if
> I were one of the committers I would have definitely pinged one of the other
> committters and asked them as to whether they knew what the hell was going
> on. Anyway this is water under bridge now. I hate bitter confrontation since
> it doesn't take anyone forward but only leaves a bitter taste in everyone's
> mouth. I have had many personal conversations with Jonathan via chat and I
> have nothing personal against anyone, perhaps not everyone but definitely
> nothing against Jonathan.
> The part that is very disconcerting are the following:
> (1) If one becomes a committer one is not expected to blitz through the
> code base and start refactoring everything. There is a way this needs to be
> handled. In any organization one doesn't just go about ripping out everyone
> else's code for no rhyme or reason. That will offend anybody. I personally
> would not go about ripping someone else's code apart if I had become
> committer. It is just that respect ought to be there. There is a way to get
> this done. Changes to code because person X likes something to be in some
> particular form and going and just changing that in person Y's code is just
> plain wrong. It borders on arrogance which is not the way things should be
> done. If I become a committer on Hadoop can I just go and start ripping
> apart every class and start making changes just because I don't like the
> coding style. This is a premature project on Apache and I think we need to
> keep the original developers in the loop till everyone has some degree of
> confidence on the changes made by new committers.
>
> (2) This is something that I have said many times over. Certain things are
> the way they are for a reason. For example when I say ConcurrentHashMap is a
> memory hog I say it because we have seen this in practice. How does it
> manifest itself? I obviously do not recall since all this was over a year
> ago. No one can claim to have run tests the way we have in the last year and
> a half. One cannot just run some simple test and say well I do not see the
> problem. I am not dumb. Anyone having gone through the exercise of having
> built a system like this in an organization will realize that the tests are
> very intermingled with the organization's infrastructure. I have no time to
> rip that all apart and put together a test suite at this point. This is just
> an example. There are many such instances - after all - we are the ones who
> have the operational experience with this and I do not think anyone can
> claim to understand the behavior this system in production workloads better
> than we do.
>
> My understanding was that new committers come in and start with some
> feature implement that and then slowly start looking into what more they
> could do going forward. It is NOT come in and refactor the hell out of the
> system because you like something to be in a specific way. I do not beleive
> this will fly in any community. It is something like we now going through
> the entire code base and changing all the stuff just because I like it in a
> specific way. This seems ludicrous. We may have no experience in open source
> but we understand etiquette very well. This just doesn't seem the way things
> work in other Apache projects which are successful. We work very closely
> with two committers from the Hadoop project who were flabbergasted with the
> refactor changes that were going in. That is my gripe with the whole thing.
>
> Cheers
> Avinash
>
>
>
> On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org> wrote:
>> > So the problems I am seeing are:
>> >
>> > 1. We elected a committer without real community consensus. The
>> > barrier of entry was unnatural low on this one. On the other hand we
>> > need non-FB committers for the graduation. The more the better. (No
>> > reason for low entry barrier though!)
>>
>> It's unfortunate that Avinash and Prashant weren't part of the
>> process. Still, when I talked to Avinash on March 1, he told me [and
>> this is a direct quote] "If I had known you earlier I would have added
>> you as a committer." So when I asked one of the mentors how to become
>> a committer and it worked out from there it did not occur to me that
>> anything was wrong.
>>
>> >
>> > 2. A missing definition of development process:
>> > - What is considered a valid code review?
>> > - How much are changes discussed up-front?
>>
>> I think we have a handle on this now. All changes are put on Jira for
>> review and are not committed until there is at least one +1 from a
>> reviewer. (I personally prefer post-commit review because manually
>> attaching and applying patches is tedious but we don't have enough
>> people following the commit log for that to work right now.)
>>
>> > - What is the roadmap? ...for whom? (weighted as a community)
>>
>> That's worth a separate thread. Such as this one. :)
>>
>> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
>>
>> > 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>> > for the required stability? Once we have the separation between stable
>> > and trunk: Will patches really find it's way from trunk into stable?
>> > Is Facebook OK with that approach. Will everyone cope with the
>> > additional work of merging? Would it be useful ...or overkill to use
>> > merge tracking?
>>
>> I'm happy to assist with merging code to or from stable branches in
>> this scenario.
>>
>> > This is a tough situation but I hope everyone sees this as an
>> > opportunity. Please let's discuss this openly in civilize manner.
>> > Focusing on how to solve these points rather than looking at the past.
>> > Please talk to each other. Can you/we work this out together?
>>
>> This can still be a win/win for everyone. I think that historically
>> facebook has felt like the community hasn't contributed much of value,
>> but we're starting to change that. The build and test process is
>> dramatically better than it was before thanks to community
>> contributions. We have a real daemon mode. (Well, not in the purest
>> sense, but it runs in the background nicely w/o nohup or screen. :)
>> We've also found and fixed several concurrency bugs, and we're well on
>> the way to having remove and range queries implemented.
>>
>> Our IRC population has more than doubled. (#cassandra on freenode:
>>
>> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
>> for a web client) We have a chance to make this more than a niche
>> project.
>>
>> -Jonathan
>>
>
>
Re: working together
Posted by Jonathan Ellis <jb...@gmail.com>.
I know it's tempting to skip to the end of a long thread without
reading what's already been said, but we've been over this.
The right solution is to have a production branch for stability and
trunk for development of new features.
-Jonathan
On Thu, Apr 9, 2009 at 1:29 AM, Zhu Han <sc...@gmail.com> wrote:
> [so many mistakes without spelling check, sorry for it]
>
> Jonathan,
>
> I can understand why you refactored the code a lot in the last few month.
> And I saw you were working hard to improve it in the last few months.
>
> However, the talents from Facebook have done a lot of work to bring
> Cassandra to the world. And they have deployed it to the production system
> for some time. Any type of patch could bring instability to the system,
> which is very critical for the production environment. Why not freeze
> refactoring the code before the community can bring a whole bunch of unit
> test case of integrate test case and committed it into the code base? The
> Facebook team doesn't have enough time to do it. That's what the community
> can contribute to the project and it should be the top item on the TODO
> list.
>
> If all the test cases are ready, the regression bug could be detected
> easily. Whatever, even the most trivial patch could bring an unexpected bug
> to the system. Most reviewers would not review the patch seriously, just
> because it's so trivial, or the patch cannot show the context clearly, and
> finally, we may not understand the code thoroughly. Let's depend on the test
> case to fight against this type of bug.
>
> Avinash,
>
> The overall architecture and implementation of Cassandra is very clear and
> impressive. But the refactoring is still necessary because it would bring
> the code quality to a higher layer. But we should take it more seriously and
> more cautious, should we?
>
> best regards,
> hanzhu
>
>
> ---------- Forwarded message ----------
> From: Zhu Han <sc...@gmail.com>
> Date: Thu, Apr 9, 2009 at 2:24 PM
> Subject: Re: working together
> To: cassandra-dev@incubator.apache.org
>
>
> Jonathan,
>
> I can understand why you have refactored the code a lot in the last few
> month. And I saw you were working hard to improve it in the last few
> months.
>
> However, the talents from Facebook have done a lot of work to bring
> Cassandra to the world. And they have deployed it to the production system
> for some time. Any type of patch could bring unstability to the system,
> which is very critical for the production environment. Why not freeze
> refactoring the code before the community can bring a whole bunch of unit
> test case of integrate test case and commited it into the code base? The
> facebook team doesn't have enough time to do it. That's what the community
> can contribute to the project and it should be the top item on the TODO
> list.
>
> If all the test cases are done, the regression bug could be detected easily.
> Whatever, even the most trivial patch could bring an unexpected bug to the
> system. Most reviewers would not review the patch seriously, just because
> it's so trivial, or the patch cannot show the context clearly, and finally,
> we may not understand the code thoroughly. Let's depend on the test case to
> fight against this type of bug.
>
> Avinash,
>
> The overall architecture and implementation of Cassandra is very clear and
> impressive. But the refactoring is still necessary because it would bring
> the code quality to a higher layer. But we should take it more seriously and
> more cautious, should we?
>
>
>
> best regards,
> hanzhu
>
>
>
> On Thu, Apr 9, 2009 at 9:38 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> On Wed, Apr 8, 2009 at 6:26 PM, Sandeep Tata <sa...@gmail.com>
>> wrote:
>> > I think it is reasonable that a codebase that has evolved for over two
>> > years has significant opportunity for refactoring when it is opened to
>> > a host of new developers. That said, large scale refactoring *at this
>> > stage* hurts us in two ways:
>> >
>> > 1. We don't have a rich suite of unit tests.
>>
>> True. The solution is to add tests a little at a time, preferably
>> before the refactor so it can help catch regressions.
>>
>> > Even automatic
>> > refactoring without unit tests makes me uncomfortable.
>>
>> I don't know how else to say this: that's an unwarranted overreaction.
>>
>> > 2. Big refactoring makes it difficult for the original developers
>> > (A&P) to review patches quickly.
>>
>> That is why I break the refactoring parts into separate patches. It
>> is not hard to review when changes are split up this way.
>>
>> > perhaps we should hold off on accepting big refactorings
>>
>> Unless we have vastly different ideas of what "big" means (which is
>> possible), I emphatically disagree.
>>
>> The important question is not, "Is refactoring more dangerous than
>> doing nothing?" Of course it is. Writing code is always more
>> dangerous than not writing code.
>>
>> The real question is, "Is adding new features while refactoring more
>> dangerous than adding new features without refactoring?" And the
>> answer is no. Refactoring improves the fit of the code to its current
>> purpose rather than grafting layers of new features on top of a
>> foundation that was not meant to support them. Refactoring also helps
>> prevent bugs -- for example my remove patch introduced a bug fixed in
>> r761093 -- because I only modified one of three copies of a piece of
>> code. It can also expose bugs in existing code. See CASSANDRA-58 for
>> an example of a bug that Jun Rao noticed because of a refactoring
>> patch.
>>
>> -Jonathan
>>
>
Re: working together
Posted by Zhu Han <sc...@gmail.com>.
[so many mistakes without spelling check, sorry for it]
Jonathan,
I can understand why you refactored the code a lot in the last few month.
And I saw you were working hard to improve it in the last few months.
However, the talents from Facebook have done a lot of work to bring
Cassandra to the world. And they have deployed it to the production system
for some time. Any type of patch could bring instability to the system,
which is very critical for the production environment. Why not freeze
refactoring the code before the community can bring a whole bunch of unit
test case of integrate test case and committed it into the code base? The
Facebook team doesn't have enough time to do it. That's what the community
can contribute to the project and it should be the top item on the TODO
list.
If all the test cases are ready, the regression bug could be detected
easily. Whatever, even the most trivial patch could bring an unexpected bug
to the system. Most reviewers would not review the patch seriously, just
because it's so trivial, or the patch cannot show the context clearly, and
finally, we may not understand the code thoroughly. Let's depend on the test
case to fight against this type of bug.
Avinash,
The overall architecture and implementation of Cassandra is very clear and
impressive. But the refactoring is still necessary because it would bring
the code quality to a higher layer. But we should take it more seriously and
more cautious, should we?
best regards,
hanzhu
---------- Forwarded message ----------
From: Zhu Han <sc...@gmail.com>
Date: Thu, Apr 9, 2009 at 2:24 PM
Subject: Re: working together
To: cassandra-dev@incubator.apache.org
Jonathan,
I can understand why you have refactored the code a lot in the last few
month. And I saw you were working hard to improve it in the last few
months.
However, the talents from Facebook have done a lot of work to bring
Cassandra to the world. And they have deployed it to the production system
for some time. Any type of patch could bring unstability to the system,
which is very critical for the production environment. Why not freeze
refactoring the code before the community can bring a whole bunch of unit
test case of integrate test case and commited it into the code base? The
facebook team doesn't have enough time to do it. That's what the community
can contribute to the project and it should be the top item on the TODO
list.
If all the test cases are done, the regression bug could be detected easily.
Whatever, even the most trivial patch could bring an unexpected bug to the
system. Most reviewers would not review the patch seriously, just because
it's so trivial, or the patch cannot show the context clearly, and finally,
we may not understand the code thoroughly. Let's depend on the test case to
fight against this type of bug.
Avinash,
The overall architecture and implementation of Cassandra is very clear and
impressive. But the refactoring is still necessary because it would bring
the code quality to a higher layer. But we should take it more seriously and
more cautious, should we?
best regards,
hanzhu
On Thu, Apr 9, 2009 at 9:38 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> On Wed, Apr 8, 2009 at 6:26 PM, Sandeep Tata <sa...@gmail.com>
> wrote:
> > I think it is reasonable that a codebase that has evolved for over two
> > years has significant opportunity for refactoring when it is opened to
> > a host of new developers. That said, large scale refactoring *at this
> > stage* hurts us in two ways:
> >
> > 1. We don't have a rich suite of unit tests.
>
> True. The solution is to add tests a little at a time, preferably
> before the refactor so it can help catch regressions.
>
> > Even automatic
> > refactoring without unit tests makes me uncomfortable.
>
> I don't know how else to say this: that's an unwarranted overreaction.
>
> > 2. Big refactoring makes it difficult for the original developers
> > (A&P) to review patches quickly.
>
> That is why I break the refactoring parts into separate patches. It
> is not hard to review when changes are split up this way.
>
> > perhaps we should hold off on accepting big refactorings
>
> Unless we have vastly different ideas of what "big" means (which is
> possible), I emphatically disagree.
>
> The important question is not, "Is refactoring more dangerous than
> doing nothing?" Of course it is. Writing code is always more
> dangerous than not writing code.
>
> The real question is, "Is adding new features while refactoring more
> dangerous than adding new features without refactoring?" And the
> answer is no. Refactoring improves the fit of the code to its current
> purpose rather than grafting layers of new features on top of a
> foundation that was not meant to support them. Refactoring also helps
> prevent bugs -- for example my remove patch introduced a bug fixed in
> r761093 -- because I only modified one of three copies of a piece of
> code. It can also expose bugs in existing code. See CASSANDRA-58 for
> an example of a bug that Jun Rao noticed because of a refactoring
> patch.
>
> -Jonathan
>
Re: working together
Posted by Zhu Han <sc...@gmail.com>.
Jonathan,
I can understand why you refactored the code a lot in the last few month.
And I saw you were working hard to improve it in the last few months.
However, the talents from Facebook has done a lot of work to bring Cassandra
to the world. And they have deployed it to the production system for some
time. Any type of patch could bring unstability to the system, which is very
critical for the production environment. Why not freeze refactoring the
code before the community can bring a whole bunch of unit test case of
integrate test case and commited it into the code base? The facebook team
doesn't have enough time to do it. That's what the community can contribute
to the project and it should be the top item on the TODO list.
If all the test cases are done, the regression bug could be detected easily.
Whatever, even the most trival patch could bring an unexpected bug to the
system. Most reviewers would not review the patch seriously, just because
it's so trival, or the patch cannot show the context clearly, and finally,
we may not understand the code thoroughly. Let's depend on the test case to
fight against this type of bug.
Avinash,
The overall architecture and implementation of Cassandra is very clear and
impressive. But the refactoring is still necessary because it would bring
the code quality to a higher layer. But we should take it more seriously and
more cautious, should we?
best regards,
hanzhu
On Thu, Apr 9, 2009 at 9:38 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> On Wed, Apr 8, 2009 at 6:26 PM, Sandeep Tata <sa...@gmail.com>
> wrote:
> > I think it is reasonable that a codebase that has evolved for over two
> > years has significant opportunity for refactoring when it is opened to
> > a host of new developers. That said, large scale refactoring *at this
> > stage* hurts us in two ways:
> >
> > 1. We don't have a rich suite of unit tests.
>
> True. The solution is to add tests a little at a time, preferably
> before the refactor so it can help catch regressions.
>
> > Even automatic
> > refactoring without unit tests makes me uncomfortable.
>
> I don't know how else to say this: that's an unwarranted overreaction.
>
> > 2. Big refactoring makes it difficult for the original developers
> > (A&P) to review patches quickly.
>
> That is why I break the refactoring parts into separate patches. It
> is not hard to review when changes are split up this way.
>
> > perhaps we should hold off on accepting big refactorings
>
> Unless we have vastly different ideas of what "big" means (which is
> possible), I emphatically disagree.
>
> The important question is not, "Is refactoring more dangerous than
> doing nothing?" Of course it is. Writing code is always more
> dangerous than not writing code.
>
> The real question is, "Is adding new features while refactoring more
> dangerous than adding new features without refactoring?" And the
> answer is no. Refactoring improves the fit of the code to its current
> purpose rather than grafting layers of new features on top of a
> foundation that was not meant to support them. Refactoring also helps
> prevent bugs -- for example my remove patch introduced a bug fixed in
> r761093 -- because I only modified one of three copies of a piece of
> code. It can also expose bugs in existing code. See CASSANDRA-58 for
> an example of a bug that Jun Rao noticed because of a refactoring
> patch.
>
> -Jonathan
>
Re: working together
Posted by Sandeep Tata <sa...@gmail.com>.
>> Even automatic
>> refactoring without unit tests makes me uncomfortable.
>
> I don't know how else to say this: that's an unwarranted overreaction.
Probably :-)
I think automatic refactoring tools have been tested enough for java
now that we'd probably be fine using them without unit tests.
However, much of the really useful refactoring of the code we want is
beyond simple automatic ones.
>> 2. Big refactoring makes it difficult for the original developers
>> (A&P) to review patches quickly.
>
> That is why I break the refactoring parts into separate patches. It
> is not hard to review when changes are split up this way.
Agreed. I'm a big fan of refactoring for simplicity. I'm making the
argument to slightly delay big ones because:
1. I think the FB team has had very limited bandwidth to devote here.
(Not a criticism, just an observation)
2. They are relatively new to Apache processes and are learning to
balance with with their FB commitments.
I'm just suggesting we continue to make a few concessions until we
have a first release. We're still figuring out a process for
committing changes! Like it or not, this is still early days!
>> perhaps we should hold off on accepting big refactorings
>
> Unless we have vastly different ideas of what "big" means (which is
> possible), I emphatically disagree.
Again, I think many of refactoring patches we're discussing on JIRA
are perfectly acceptable, and in my opinion, much needed. For
instance, CASSANDRA-52 and CASSANDRA-58. OTOH, refactoring the entire
set of messaging classes (which is something we should get to
eventually -- I'd hate to see more bugs like CASSANDRA-21) would
qualify as a bigger refactoring. I'm not suggesting we put these off
for a long time, but just until we have a basic first release!
> The important question is not, "Is refactoring more dangerous than
> doing nothing?" Of course it is. Writing code is always more
> dangerous than not writing code.
:-)
> The real question is, "Is adding new features while refactoring more
> dangerous than adding new features without refactoring?" And the
> answer is no. Refactoring improves the fit of the code to its current
> purpose rather than grafting layers of new features on top of a
> foundation that was not meant to support them. Refactoring also helps
> prevent bugs -- for example my remove patch introduced a bug fixed in
> r761093 -- because I only modified one of three copies of a piece of
> code. It can also expose bugs in existing code. See CASSANDRA-58 for
> an example of a bug that Jun Rao noticed because of a refactoring
> patch.
I understand the purpose and merits of refactoring code :-) There are
many parts of the code I'd like to clean up too...
My argument is simply an attempt at solving some of the birthing
trouble our community is having. It is not a technical argument
for/against refactoring...just a suggestion for altering their timing
them so that we can move forward more easily.
Re: working together
Posted by Jonathan Ellis <jb...@gmail.com>.
On Wed, Apr 8, 2009 at 6:26 PM, Sandeep Tata <sa...@gmail.com> wrote:
> I think it is reasonable that a codebase that has evolved for over two
> years has significant opportunity for refactoring when it is opened to
> a host of new developers. That said, large scale refactoring *at this
> stage* hurts us in two ways:
>
> 1. We don't have a rich suite of unit tests.
True. The solution is to add tests a little at a time, preferably
before the refactor so it can help catch regressions.
> Even automatic
> refactoring without unit tests makes me uncomfortable.
I don't know how else to say this: that's an unwarranted overreaction.
> 2. Big refactoring makes it difficult for the original developers
> (A&P) to review patches quickly.
That is why I break the refactoring parts into separate patches. It
is not hard to review when changes are split up this way.
> perhaps we should hold off on accepting big refactorings
Unless we have vastly different ideas of what "big" means (which is
possible), I emphatically disagree.
The important question is not, "Is refactoring more dangerous than
doing nothing?" Of course it is. Writing code is always more
dangerous than not writing code.
The real question is, "Is adding new features while refactoring more
dangerous than adding new features without refactoring?" And the
answer is no. Refactoring improves the fit of the code to its current
purpose rather than grafting layers of new features on top of a
foundation that was not meant to support them. Refactoring also helps
prevent bugs -- for example my remove patch introduced a bug fixed in
r761093 -- because I only modified one of three copies of a piece of
code. It can also expose bugs in existing code. See CASSANDRA-58 for
an example of a bug that Jun Rao noticed because of a refactoring
patch.
-Jonathan
Re: working together
Posted by Sandeep Tata <sa...@gmail.com>.
The refactoring question seems to be a bit of thorn:
> My understanding was that new committers come in and start with some feature
> implement that and then slowly start looking into what more they could do
> going forward. It is NOT come in and refactor the hell out of the system
> because you like something to be in a specific way. I do not beleive this
> will fly in any community. It is something like we now going through the
> entire code base and changing all the stuff just because I like it in a
> specific way. This seems ludicrous.
I think it is reasonable that a codebase that has evolved for over two
years has significant opportunity for refactoring when it is opened to
a host of new developers. That said, large scale refactoring *at this
stage* hurts us in two ways:
1. We don't have a rich suite of unit tests. Even automatic
refactoring without unit tests makes me uncomfortable.
2. Big refactoring makes it difficult for the original developers
(A&P) to review patches quickly.
I can understand Avinash's resistance to big refactoring, and to some
extent, I agree.
While I think we may need significant refactoring as the codebase
moves forward (to simplify, keep it healthy and make contributions
easier), perhaps we should hold off on accepting big refactorings
until:
a) We have a richer suite of unit tests.
b) We've done an initial stable release
That seems like a reasonable restriction on the refactoring story, yes ?
Avinash, Prashant, Jonathan, others -- does this seem like a good
strategy? Alternative ideas?
Re: working together
Posted by Neophytos Demetriou <ne...@gmail.com>.
> Look, what you're saying here is basically "we know better and you're
> stupid, so don't touch our code and don't ask questions, we can't provide
> answers anyway". I'm hoping that's not the way you meant it (emails do that)
> but that's the essence of what came across. You just can't run an open
> source project by saying this on its development list.
My understanding is that they do know better. That does not necessarily
imply the rest of us are stupid but the settings in which Cassandra runs
are so complicated the design of such a system can be an extremely
difficult task. Obviously, it is important that it runs correctly and
efficiently.
In all aspects, there needs to be an approach that strives for a balance
between openness towards new participants and contributions, and the
need for control, with the acknowledgment that this balance might shift
over time.
> There are two reasons I refactor. One is, I always try to leave the
> code better than I found it. [...] Two is, if I am going to introduce
> a new feature I will try to refactor first without changing behavior
> in such away that the feature becomes easier to add. [...] So there
> is a method to my madness.
At this point, this is risky. It is risky because code changes that
introduce incompatibilities are very expensive. This may lead to a
coexisting trunk inside FB which also means we might remain unaware of
the existence of a major problem beyond the stage at which it can be
contained and corrected. Like it or not, using the same or a
close-enough version of the code as FB, was appealing to most of us.
I understand that some of us may feel that the original authors are not
interacting with the community the same way grassroot hackers do but,
these are the same people who brought over the code. I kind of agree
with Avinash in relation to etiquette. First, you go along and then you
come along.
> The recent issues have to do with Facebook developers having
> expectations that differ from open source standard practices.
The recent issues have more to do with meritocracy than other open
source practices. Facebook developers would be happy to have new people
coming in and help, and like other projects filter the people they
believe committed enough for the task and match the human attitudes
required to work well with others, especially in disagreement.
Now, I'm not too familiar with the ASF processes but the following
http://www.apache.org/foundation/how-it-works.html
suggests that the PMC is comprised from developers or commiters that
were elected due to merit for the evolution of the project and
demonstration of commitment. Having said that, it is not difficult to
see why Facebook developers could be frustrated by recent events.
I do not mean to be disrespectful to Jonathan. I probably know him
longer than most people in this group and I think he deserves to be a
commiter in the same way that I think Avinash or Prashant should be the
project leaders and in the same way that they deserve more credit and
respect for their contributions (and releasing Cassandra as Open Source
in the first place).
- Neophytos
Re: working together
Posted by Avinash Lakshman <av...@gmail.com>.
The fact of the matter is it is very hard to defend every decision by going
down memory lane. That's all. I apologize if it came across as condescending
but that definitely was NOT the intent.
Avinash
On Tue, Apr 7, 2009 at 9:51 PM, Avinash Lakshman <avinash.lakshman@gmail.com
> wrote:
> That is NOT what I meant. I thought I made that very clear. I said "It is
> very hard to recall all experiences and that there is a reason for
> everything". That is all. I am very careful about what I state and I always
> put myself in the shoes of the reader.
> Avinash
>
>
> On Tue, Apr 7, 2009 at 9:47 PM, Matthieu Riou <ma...@gmail.com>wrote:
>
>> On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <
>> avinash.lakshman@gmail.com
>> > wrote:
>>
>> > <snipped>
>> > (2) This is something that I have said many times over. Certain things
>> are
>> > the way they are for a reason. For example when I say ConcurrentHashMap
>> is
>> > a
>> > memory hog I say it because we have seen this in practice. How does it
>> > manifest itself? I obviously do not recall since all this was over a
>> year
>> > ago. No one can claim to have run tests the way we have in the last year
>> > and
>> > a half. One cannot just run some simple test and say well I do not see
>> the
>> > problem. I am not dumb. Anyone having gone through the exercise of
>> having
>> > built a system like this in an organization will realize that the tests
>> are
>> > very intermingled with the organization's infrastructure. I have no time
>> to
>> > rip that all apart and put together a test suite at this point. This is
>> > just
>> > an example. There are many such instances - after all - we are the ones
>> who
>> > have the operational experience with this and I do not think anyone can
>> > claim to understand the behavior this system in production workloads
>> better
>> > than we do.
>> >
>>
>> Look, what you're saying here is basically "we know better and you're
>> stupid, so don't touch our code and don't ask questions, we can't provide
>> answers anyway". I'm hoping that's not the way you meant it (emails do
>> that)
>> but that's the essence of what came across. You just can't run an open
>> source project by saying this on its development list.
>>
>> Matthieu
>>
>>
>> >
>> > My understanding was that new committers come in and start with some
>> > feature
>> > implement that and then slowly start looking into what more they could
>> do
>> > going forward. It is NOT come in and refactor the hell out of the system
>> > because you like something to be in a specific way. I do not beleive
>> this
>> > will fly in any community. It is something like we now going through the
>> > entire code base and changing all the stuff just because I like it in a
>> > specific way. This seems ludicrous. We may have no experience in open
>> > source
>> > but we understand etiquette very well. This just doesn't seem the way
>> > things
>> > work in other Apache projects which are successful. We work very closely
>> > with two committers from the Hadoop project who were flabbergasted with
>> the
>> > refactor changes that were going in. That is my gripe with the whole
>> thing.
>> >
>> > Cheers
>> > Avinash
>> >
>> >
>> >
>> > On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com>
>> wrote:
>> >
>> > > On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org>
>> wrote:
>> > > > So the problems I am seeing are:
>> > > >
>> > > > 1. We elected a committer without real community consensus. The
>> > > > barrier of entry was unnatural low on this one. On the other hand we
>> > > > need non-FB committers for the graduation. The more the better. (No
>> > > > reason for low entry barrier though!)
>> > >
>> > > It's unfortunate that Avinash and Prashant weren't part of the
>> > > process. Still, when I talked to Avinash on March 1, he told me [and
>> > > this is a direct quote] "If I had known you earlier I would have added
>> > > you as a committer." So when I asked one of the mentors how to become
>> > > a committer and it worked out from there it did not occur to me that
>> > > anything was wrong.
>> > >
>> > > >
>> > > > 2. A missing definition of development process:
>> > > > - What is considered a valid code review?
>> > > > - How much are changes discussed up-front?
>> > >
>> > > I think we have a handle on this now. All changes are put on Jira for
>> > > review and are not committed until there is at least one +1 from a
>> > > reviewer. (I personally prefer post-commit review because manually
>> > > attaching and applying patches is tedious but we don't have enough
>> > > people following the commit log for that to work right now.)
>> > >
>> > > > - What is the roadmap? ...for whom? (weighted as a community)
>> > >
>> > > That's worth a separate thread. Such as this one. :)
>> > >
>> > >
>> >
>> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
>> > >
>> > > > 3. Is trunk considered "stable"? Or aren't we missing a stable
>> branch
>> > > > for the required stability? Once we have the separation between
>> stable
>> > > > and trunk: Will patches really find it's way from trunk into stable?
>> > > > Is Facebook OK with that approach. Will everyone cope with the
>> > > > additional work of merging? Would it be useful ...or overkill to use
>> > > > merge tracking?
>> > >
>> > > I'm happy to assist with merging code to or from stable branches in
>> > > this scenario.
>> > >
>> > > > This is a tough situation but I hope everyone sees this as an
>> > > > opportunity. Please let's discuss this openly in civilize manner.
>> > > > Focusing on how to solve these points rather than looking at the
>> past.
>> > > > Please talk to each other. Can you/we work this out together?
>> > >
>> > > This can still be a win/win for everyone. I think that historically
>> > > facebook has felt like the community hasn't contributed much of value,
>> > > but we're starting to change that. The build and test process is
>> > > dramatically better than it was before thanks to community
>> > > contributions. We have a real daemon mode. (Well, not in the purest
>> > > sense, but it runs in the background nicely w/o nohup or screen. :)
>> > > We've also found and fixed several concurrency bugs, and we're well on
>> > > the way to having remove and range queries implemented.
>> > >
>> > > Our IRC population has more than doubled. (#cassandra on freenode:
>> > >
>> > >
>> >
>> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
>> > > for a web client) We have a chance to make this more than a niche
>> > > project.
>> > >
>> > > -Jonathan
>> > >
>> >
>>
>
>
Re: working together
Posted by Avinash Lakshman <av...@gmail.com>.
That is NOT what I meant. I thought I made that very clear. I said "It is
very hard to recall all experiences and that there is a reason for
everything". That is all. I am very careful about what I state and I always
put myself in the shoes of the reader.
Avinash
On Tue, Apr 7, 2009 at 9:47 PM, Matthieu Riou <ma...@gmail.com>wrote:
> On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <
> avinash.lakshman@gmail.com
> > wrote:
>
> > <snipped>
> > (2) This is something that I have said many times over. Certain things
> are
> > the way they are for a reason. For example when I say ConcurrentHashMap
> is
> > a
> > memory hog I say it because we have seen this in practice. How does it
> > manifest itself? I obviously do not recall since all this was over a year
> > ago. No one can claim to have run tests the way we have in the last year
> > and
> > a half. One cannot just run some simple test and say well I do not see
> the
> > problem. I am not dumb. Anyone having gone through the exercise of having
> > built a system like this in an organization will realize that the tests
> are
> > very intermingled with the organization's infrastructure. I have no time
> to
> > rip that all apart and put together a test suite at this point. This is
> > just
> > an example. There are many such instances - after all - we are the ones
> who
> > have the operational experience with this and I do not think anyone can
> > claim to understand the behavior this system in production workloads
> better
> > than we do.
> >
>
> Look, what you're saying here is basically "we know better and you're
> stupid, so don't touch our code and don't ask questions, we can't provide
> answers anyway". I'm hoping that's not the way you meant it (emails do
> that)
> but that's the essence of what came across. You just can't run an open
> source project by saying this on its development list.
>
> Matthieu
>
>
> >
> > My understanding was that new committers come in and start with some
> > feature
> > implement that and then slowly start looking into what more they could do
> > going forward. It is NOT come in and refactor the hell out of the system
> > because you like something to be in a specific way. I do not beleive this
> > will fly in any community. It is something like we now going through the
> > entire code base and changing all the stuff just because I like it in a
> > specific way. This seems ludicrous. We may have no experience in open
> > source
> > but we understand etiquette very well. This just doesn't seem the way
> > things
> > work in other Apache projects which are successful. We work very closely
> > with two committers from the Hadoop project who were flabbergasted with
> the
> > refactor changes that were going in. That is my gripe with the whole
> thing.
> >
> > Cheers
> > Avinash
> >
> >
> >
> > On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> >
> > > On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org>
> wrote:
> > > > So the problems I am seeing are:
> > > >
> > > > 1. We elected a committer without real community consensus. The
> > > > barrier of entry was unnatural low on this one. On the other hand we
> > > > need non-FB committers for the graduation. The more the better. (No
> > > > reason for low entry barrier though!)
> > >
> > > It's unfortunate that Avinash and Prashant weren't part of the
> > > process. Still, when I talked to Avinash on March 1, he told me [and
> > > this is a direct quote] "If I had known you earlier I would have added
> > > you as a committer." So when I asked one of the mentors how to become
> > > a committer and it worked out from there it did not occur to me that
> > > anything was wrong.
> > >
> > > >
> > > > 2. A missing definition of development process:
> > > > - What is considered a valid code review?
> > > > - How much are changes discussed up-front?
> > >
> > > I think we have a handle on this now. All changes are put on Jira for
> > > review and are not committed until there is at least one +1 from a
> > > reviewer. (I personally prefer post-commit review because manually
> > > attaching and applying patches is tedious but we don't have enough
> > > people following the commit log for that to work right now.)
> > >
> > > > - What is the roadmap? ...for whom? (weighted as a community)
> > >
> > > That's worth a separate thread. Such as this one. :)
> > >
> > >
> >
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
> > >
> > > > 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> > > > for the required stability? Once we have the separation between
> stable
> > > > and trunk: Will patches really find it's way from trunk into stable?
> > > > Is Facebook OK with that approach. Will everyone cope with the
> > > > additional work of merging? Would it be useful ...or overkill to use
> > > > merge tracking?
> > >
> > > I'm happy to assist with merging code to or from stable branches in
> > > this scenario.
> > >
> > > > This is a tough situation but I hope everyone sees this as an
> > > > opportunity. Please let's discuss this openly in civilize manner.
> > > > Focusing on how to solve these points rather than looking at the
> past.
> > > > Please talk to each other. Can you/we work this out together?
> > >
> > > This can still be a win/win for everyone. I think that historically
> > > facebook has felt like the community hasn't contributed much of value,
> > > but we're starting to change that. The build and test process is
> > > dramatically better than it was before thanks to community
> > > contributions. We have a real daemon mode. (Well, not in the purest
> > > sense, but it runs in the background nicely w/o nohup or screen. :)
> > > We've also found and fixed several concurrency bugs, and we're well on
> > > the way to having remove and range queries implemented.
> > >
> > > Our IRC population has more than doubled. (#cassandra on freenode:
> > >
> > >
> >
> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
> > > for a web client) We have a chance to make this more than a niche
> > > project.
> > >
> > > -Jonathan
> > >
> >
>
Re: working together
Posted by Matthieu Riou <ma...@gmail.com>.
On Tue, Apr 7, 2009 at 8:11 PM, Avinash Lakshman <avinash.lakshman@gmail.com
> wrote:
> <snipped>
> (2) This is something that I have said many times over. Certain things are
> the way they are for a reason. For example when I say ConcurrentHashMap is
> a
> memory hog I say it because we have seen this in practice. How does it
> manifest itself? I obviously do not recall since all this was over a year
> ago. No one can claim to have run tests the way we have in the last year
> and
> a half. One cannot just run some simple test and say well I do not see the
> problem. I am not dumb. Anyone having gone through the exercise of having
> built a system like this in an organization will realize that the tests are
> very intermingled with the organization's infrastructure. I have no time to
> rip that all apart and put together a test suite at this point. This is
> just
> an example. There are many such instances - after all - we are the ones who
> have the operational experience with this and I do not think anyone can
> claim to understand the behavior this system in production workloads better
> than we do.
>
Look, what you're saying here is basically "we know better and you're
stupid, so don't touch our code and don't ask questions, we can't provide
answers anyway". I'm hoping that's not the way you meant it (emails do that)
but that's the essence of what came across. You just can't run an open
source project by saying this on its development list.
Matthieu
>
> My understanding was that new committers come in and start with some
> feature
> implement that and then slowly start looking into what more they could do
> going forward. It is NOT come in and refactor the hell out of the system
> because you like something to be in a specific way. I do not beleive this
> will fly in any community. It is something like we now going through the
> entire code base and changing all the stuff just because I like it in a
> specific way. This seems ludicrous. We may have no experience in open
> source
> but we understand etiquette very well. This just doesn't seem the way
> things
> work in other Apache projects which are successful. We work very closely
> with two committers from the Hadoop project who were flabbergasted with the
> refactor changes that were going in. That is my gripe with the whole thing.
>
> Cheers
> Avinash
>
>
>
> On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>
> > On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org> wrote:
> > > So the problems I am seeing are:
> > >
> > > 1. We elected a committer without real community consensus. The
> > > barrier of entry was unnatural low on this one. On the other hand we
> > > need non-FB committers for the graduation. The more the better. (No
> > > reason for low entry barrier though!)
> >
> > It's unfortunate that Avinash and Prashant weren't part of the
> > process. Still, when I talked to Avinash on March 1, he told me [and
> > this is a direct quote] "If I had known you earlier I would have added
> > you as a committer." So when I asked one of the mentors how to become
> > a committer and it worked out from there it did not occur to me that
> > anything was wrong.
> >
> > >
> > > 2. A missing definition of development process:
> > > - What is considered a valid code review?
> > > - How much are changes discussed up-front?
> >
> > I think we have a handle on this now. All changes are put on Jira for
> > review and are not committed until there is at least one +1 from a
> > reviewer. (I personally prefer post-commit review because manually
> > attaching and applying patches is tedious but we don't have enough
> > people following the commit log for that to work right now.)
> >
> > > - What is the roadmap? ...for whom? (weighted as a community)
> >
> > That's worth a separate thread. Such as this one. :)
> >
> >
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
> >
> > > 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> > > for the required stability? Once we have the separation between stable
> > > and trunk: Will patches really find it's way from trunk into stable?
> > > Is Facebook OK with that approach. Will everyone cope with the
> > > additional work of merging? Would it be useful ...or overkill to use
> > > merge tracking?
> >
> > I'm happy to assist with merging code to or from stable branches in
> > this scenario.
> >
> > > This is a tough situation but I hope everyone sees this as an
> > > opportunity. Please let's discuss this openly in civilize manner.
> > > Focusing on how to solve these points rather than looking at the past.
> > > Please talk to each other. Can you/we work this out together?
> >
> > This can still be a win/win for everyone. I think that historically
> > facebook has felt like the community hasn't contributed much of value,
> > but we're starting to change that. The build and test process is
> > dramatically better than it was before thanks to community
> > contributions. We have a real daemon mode. (Well, not in the purest
> > sense, but it runs in the background nicely w/o nohup or screen. :)
> > We've also found and fixed several concurrency bugs, and we're well on
> > the way to having remove and range queries implemented.
> >
> > Our IRC population has more than doubled. (#cassandra on freenode:
> >
> >
> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
> > for a web client) We have a chance to make this more than a niche
> > project.
> >
> > -Jonathan
> >
>
Re: working together
Posted by Roland Dreier <ro...@digitalvampire.org>.
> (2) This is something that I have said many times over. Certain things are
> the way they are for a reason. For example when I say ConcurrentHashMap is a
> memory hog I say it because we have seen this in practice. How does it
> manifest itself? I obviously do not recall since all this was over a year
> ago.
Not to sidetrack the conversation too much -- but this is a perfect
example of why I suggested more informative changelog entries would be
valuable to the project going forward. If you record this sort of
information along with changes to a file, then someone wondering why
code is written in a certain way at least has a hope of using "svn
blame" or the like and actually finding an answer.
- R.
Re: working together
Posted by Avinash Lakshman <av...@gmail.com>.
Point #1 I would love to have committers from outside but the way this
happened took all of us by surprise. Granted we were not on the list but if
I were one of the committers I would have definitely pinged one of the other
committters and asked them as to whether they knew what the hell was going
on. Anyway this is water under bridge now. I hate bitter confrontation since
it doesn't take anyone forward but only leaves a bitter taste in everyone's
mouth. I have had many personal conversations with Jonathan via chat and I
have nothing personal against anyone, perhaps not everyone but definitely
nothing against Jonathan.
The part that is very disconcerting are the following:
(1) If one becomes a committer one is not expected to blitz through the code
base and start refactoring everything. There is a way this needs to be
handled. In any organization one doesn't just go about ripping out everyone
else's code for no rhyme or reason. That will offend anybody. I personally
would not go about ripping someone else's code apart if I had become
committer. It is just that respect ought to be there. There is a way to get
this done. Changes to code because person X likes something to be in some
particular form and going and just changing that in person Y's code is just
plain wrong. It borders on arrogance which is not the way things should be
done. If I become a committer on Hadoop can I just go and start ripping
apart every class and start making changes just because I don't like the
coding style. This is a premature project on Apache and I think we need to
keep the original developers in the loop till everyone has some degree of
confidence on the changes made by new committers.
(2) This is something that I have said many times over. Certain things are
the way they are for a reason. For example when I say ConcurrentHashMap is a
memory hog I say it because we have seen this in practice. How does it
manifest itself? I obviously do not recall since all this was over a year
ago. No one can claim to have run tests the way we have in the last year and
a half. One cannot just run some simple test and say well I do not see the
problem. I am not dumb. Anyone having gone through the exercise of having
built a system like this in an organization will realize that the tests are
very intermingled with the organization's infrastructure. I have no time to
rip that all apart and put together a test suite at this point. This is just
an example. There are many such instances - after all - we are the ones who
have the operational experience with this and I do not think anyone can
claim to understand the behavior this system in production workloads better
than we do.
My understanding was that new committers come in and start with some feature
implement that and then slowly start looking into what more they could do
going forward. It is NOT come in and refactor the hell out of the system
because you like something to be in a specific way. I do not beleive this
will fly in any community. It is something like we now going through the
entire code base and changing all the stuff just because I like it in a
specific way. This seems ludicrous. We may have no experience in open source
but we understand etiquette very well. This just doesn't seem the way things
work in other Apache projects which are successful. We work very closely
with two committers from the Hadoop project who were flabbergasted with the
refactor changes that were going in. That is my gripe with the whole thing.
Cheers
Avinash
On Tue, Apr 7, 2009 at 7:30 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org> wrote:
> > So the problems I am seeing are:
> >
> > 1. We elected a committer without real community consensus. The
> > barrier of entry was unnatural low on this one. On the other hand we
> > need non-FB committers for the graduation. The more the better. (No
> > reason for low entry barrier though!)
>
> It's unfortunate that Avinash and Prashant weren't part of the
> process. Still, when I talked to Avinash on March 1, he told me [and
> this is a direct quote] "If I had known you earlier I would have added
> you as a committer." So when I asked one of the mentors how to become
> a committer and it worked out from there it did not occur to me that
> anything was wrong.
>
> >
> > 2. A missing definition of development process:
> > - What is considered a valid code review?
> > - How much are changes discussed up-front?
>
> I think we have a handle on this now. All changes are put on Jira for
> review and are not committed until there is at least one +1 from a
> reviewer. (I personally prefer post-commit review because manually
> attaching and applying patches is tedious but we don't have enough
> people following the commit log for that to work right now.)
>
> > - What is the roadmap? ...for whom? (weighted as a community)
>
> That's worth a separate thread. Such as this one. :)
>
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
>
> > 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> > for the required stability? Once we have the separation between stable
> > and trunk: Will patches really find it's way from trunk into stable?
> > Is Facebook OK with that approach. Will everyone cope with the
> > additional work of merging? Would it be useful ...or overkill to use
> > merge tracking?
>
> I'm happy to assist with merging code to or from stable branches in
> this scenario.
>
> > This is a tough situation but I hope everyone sees this as an
> > opportunity. Please let's discuss this openly in civilize manner.
> > Focusing on how to solve these points rather than looking at the past.
> > Please talk to each other. Can you/we work this out together?
>
> This can still be a win/win for everyone. I think that historically
> facebook has felt like the community hasn't contributed much of value,
> but we're starting to change that. The build and test process is
> dramatically better than it was before thanks to community
> contributions. We have a real daemon mode. (Well, not in the purest
> sense, but it runs in the background nicely w/o nohup or screen. :)
> We've also found and fixed several concurrency bugs, and we're well on
> the way to having remove and range queries implemented.
>
> Our IRC population has more than doubled. (#cassandra on freenode:
>
> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
> for a web client) We have a chance to make this more than a niche
> project.
>
> -Jonathan
>
Re: working together
Posted by Torsten Curdt <tc...@apache.org>.
Post-commit review works for *many* Apache projects ... but I agree -
it might not work Cassandra at this stage yet. The DVCS approach we
already had. While I am a big fan usually that doesn't sound like the
right path here.
cheers
--
Torsten
Re: working together
Posted by Roland Dreier <ro...@digitalvampire.org>.
> I think we have a handle on this now. All changes are put on Jira for
> review and are not committed until there is at least one +1 from a
> reviewer. (I personally prefer post-commit review because manually
> attaching and applying patches is tedious but we don't have enough
> people following the commit log for that to work right now.)
I don't see how post-commit review could work. The fundamental issue
is that with post-commit review, lack of response means the patch
stays committed. For very obvious or trivial changes, that makes
sense, but for anything with real meat, the bandwidth of the people
qualified to review may be very limited, and a lack of response should
not be read as acceptance. And then if a reviewer finds issues after
other patches have been committed on top, reverting the original patch
may be a big hassle.
Pre-commit review on the other hand makes the social aspects of review
work to the benefit of the project. If someone wants to get a change
onto the trunk, then the process gives them an incentive to make the
change readable, work with reviewers to make sure the change is
understood, build consensus, etc.
With that said it would be nice to have a low-barrier way for people
(including non-committers) to publish branches so that their work is
easily obtainable and testable, even pre-merge. Unfortunately the
incubator svn doesn't make that very easy but perhaps something can be
done short of switching to a DVCS.
- R.
Re: working together
Posted by Torsten Curdt <tc...@apache.org>.
>> 2. A missing definition of development process:
>> - What is considered a valid code review?
>> - How much are changes discussed up-front?
>
> I think we have a handle on this now. All changes are put on Jira for
> review and are not committed until there is at least one +1 from a
> reviewer. (I personally prefer post-commit review because manually
> attaching and applying patches is tedious but we don't have enough
> people following the commit log for that to work right now.)
I think it is important we come to a point where we discuss on dev and
there is enough freedom and trust for post-commit. Maybe in the
beginning going through JIRA is OK.
...but I fear that this is cumbersome and won't scale.
>> - What is the roadmap? ...for whom? (weighted as a community)
>
> That's worth a separate thread. Such as this one. :)
> http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
Thanks for that. Indeed.
>> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
>> for the required stability? Once we have the separation between stable
>> and trunk: Will patches really find it's way from trunk into stable?
>> Is Facebook OK with that approach. Will everyone cope with the
>> additional work of merging? Would it be useful ...or overkill to use
>> merge tracking?
>
> I'm happy to assist with merging code to or from stable branches in
> this scenario.
Thanks!
>> This is a tough situation but I hope everyone sees this as an
>> opportunity. Please let's discuss this openly in civilize manner.
>> Focusing on how to solve these points rather than looking at the past.
>> Please talk to each other. Can you/we work this out together?
>
> This can still be a win/win for everyone. I think that historically
> facebook has felt like the community hasn't contributed much of value,
> but we're starting to change that. The build and test process is
> dramatically better than it was before thanks to community
> contributions. We have a real daemon mode. (Well, not in the purest
> sense, but it runs in the background nicely w/o nohup or screen. :)
> We've also found and fixed several concurrency bugs, and we're well on
> the way to having remove and range queries implemented.
>
> Our IRC population has more than doubled. (#cassandra on freenode:
> http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
> for a web client) We have a chance to make this more than a niche
> project.
+1 to that
Re: working together
Posted by Jonathan Ellis <jb...@gmail.com>.
On Tue, Apr 7, 2009 at 3:10 PM, Torsten Curdt <tc...@apache.org> wrote:
> So the problems I am seeing are:
>
> 1. We elected a committer without real community consensus. The
> barrier of entry was unnatural low on this one. On the other hand we
> need non-FB committers for the graduation. The more the better. (No
> reason for low entry barrier though!)
It's unfortunate that Avinash and Prashant weren't part of the
process. Still, when I talked to Avinash on March 1, he told me [and
this is a direct quote] "If I had known you earlier I would have added
you as a committer." So when I asked one of the mentors how to become
a committer and it worked out from there it did not occur to me that
anything was wrong.
>
> 2. A missing definition of development process:
> - What is considered a valid code review?
> - How much are changes discussed up-front?
I think we have a handle on this now. All changes are put on Jira for
review and are not committed until there is at least one +1 from a
reviewer. (I personally prefer post-commit review because manually
attaching and applying patches is tedious but we don't have enough
people following the commit log for that to work right now.)
> - What is the roadmap? ...for whom? (weighted as a community)
That's worth a separate thread. Such as this one. :)
http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg00160.html
> 3. Is trunk considered "stable"? Or aren't we missing a stable branch
> for the required stability? Once we have the separation between stable
> and trunk: Will patches really find it's way from trunk into stable?
> Is Facebook OK with that approach. Will everyone cope with the
> additional work of merging? Would it be useful ...or overkill to use
> merge tracking?
I'm happy to assist with merging code to or from stable branches in
this scenario.
> This is a tough situation but I hope everyone sees this as an
> opportunity. Please let's discuss this openly in civilize manner.
> Focusing on how to solve these points rather than looking at the past.
> Please talk to each other. Can you/we work this out together?
This can still be a win/win for everyone. I think that historically
facebook has felt like the community hasn't contributed much of value,
but we're starting to change that. The build and test process is
dramatically better than it was before thanks to community
contributions. We have a real daemon mode. (Well, not in the purest
sense, but it runs in the background nicely w/o nohup or screen. :)
We've also found and fixed several concurrency bugs, and we're well on
the way to having remove and range queries implemented.
Our IRC population has more than doubled. (#cassandra on freenode:
http://www.mibbit.com/?server=irc.freenode.net&channel=%23cassandra&nick=mibbit
for a web client) We have a chance to make this more than a niche
project.
-Jonathan