Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Gordon Smith <go...@adobe.com>]

Why all the focus on branching strategies? It makes more sense to focus on a testing strategy. We need an 'ant tests' and a requirement that it pass for all changes to trunk.

Didn't the mustella tests get submitted? What more do we need other than an easy way to run them?

- Gordon

Sent from my iPad

On Aug 8, 2012, at 7:21 PM, "Om" <bi...@gmail.com> wrote:

> On Wed, Aug 8, 2012 at 6:40 PM, Justin Mclean <ju...@classsoftware.com>wrote:
> 
>> Hi,
>> 
>>> I would use the name "alt-trunk" instead of "unstable" for the other
>>> branch.  Every rule that applies to trunk would apply to the alt-trunk
>>> branch.
>> The issue I see is having one single branch not the name. You would be
>> fine with the multi step step check in process (with unresolved issues) as
>> describe below every time you wanted to make a change?
>> 
>> 
> If we cannot guarantee that the branch is stable everytime anyone wants to
> make a change, how can we do it in trunk?  What special properties that
> trunk have that makes it easier there?
> 
> If I check in something that breaks the build or causes numerous issues
> into alt-trunk, I will have to either pull it out or fix it in alt-trunk
> itself.  Trunk does not even come into picture here.
> 
> Only when we are closer to a release, we will merge alt-trunk into trunk.
> It would be a much smoother process then because we would have brought
> alt-trunk to stability before we attempt the merge into trunk.
> 
> 
>>> Remember that if we do all the development in trunk, we would have
>>> to do the same thing except that it would be a much bigger problem
>> because
>>> we dont have any place to go when we screw things up.
>> How about svn revert?
>> 
> 
> That is the command we would use, yes.  But what if we dont find out the
> issues in time?  What if we find out multiple huge issues when it is time
> to cut a release.  The complexity that would ensue is something we are not
> set up for now.
> Maybe after a couple of releases, yes.
> 
>> When are happy with alt-trunk's state, we merge alt-trunk into trunk and
>> cut a release off of trunk.
> 
>> So we would have to have a code freeze while we sort out merge issues and
>> the like?
> 
> 
> Yes.
> 
> 
>> How would you not include the changes you did not want to go into trunk?
>> 
>> How would we do it if everyone checks into trunk directly?
> 
> 
> 
>> Just want to make it clear what people are agreeing to.
>> 
>> 
> Let's use alt-trunk as a training ground.  Lets get it right there and we
> can move to working directly off of trunk.
> 
> Om

Re: Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Omar Gonzalez <om...@gmail.com>]

On Wed, Aug 8, 2012 at 11:04 PM, Gordon Smith <go...@adobe.com> wrote:

> The solution for that is test-driven development. Apache doesn't have a QA
> team like Adobe. If developers write new stuff and don't write any tests,
> how are you ever going to trust anything new?
>
> Sent from my iPad
>
>
I would not be opposed to requiring that unit tests/mustella tests are
written for patches before they're considered for integration if we put out
the guidelines to developers on what is required of those tests. It would
make every committer's life easier if we had test suites that came w/
patches not only to ensure the quality of the code but to help to
understand what is going on in the mind of the developer that wrote the
patch.

-omar

Re: Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Dave Fisher <da...@comcast.net>]

On Aug 9, 2012, at 8:10 AM, Greg Reddin wrote:

> On Thu, Aug 9, 2012 at 1:16 AM, Alex Harui <ah...@adobe.com> wrote:
>> Absolutely correct, but I don't want to hold off another release until we
>> have a mustella test for every code path and boundary condition, so I am
>> proposing using an "unstable" branch as the proving ground until we prove we
>> don't need it.
> 
> Similar to my previous post, but maybe we avoid argument by simply
> flipping the model.
> 
> Instead of stable trunk and unstable branch what about this?
> 
> svn cp trunk branches/stable
> 
> Trunk is now unstable and "branches/stable" is ready for a quick
> release. The only complication there is how to "promote" code from
> trunk to stable, but maybe we can do that on a case by case basis.

IMO the Release Vote is the main promotion process, but there might be other cases.

FYI - Some projects like Tomcat and HTTPD maintain multiple branches that are stable (or not so stable) releases.

Regards,
Dave


> 
> Greg

Re: Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Greg Reddin <gr...@gmail.com>]

On Thu, Aug 9, 2012 at 1:16 AM, Alex Harui <ah...@adobe.com> wrote:
> Absolutely correct, but I don't want to hold off another release until we
> have a mustella test for every code path and boundary condition, so I am
> proposing using an "unstable" branch as the proving ground until we prove we
> don't need it.

Similar to my previous post, but maybe we avoid argument by simply
flipping the model.

Instead of stable trunk and unstable branch what about this?

svn cp trunk branches/stable

Trunk is now unstable and "branches/stable" is ready for a quick
release. The only complication there is how to "promote" code from
trunk to stable, but maybe we can do that on a case by case basis.

Greg

Re: Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Alex Harui <ah...@adobe.com>]

On 8/8/12 11:04 PM, "Gordon Smith" <go...@adobe.com> wrote:

> The solution for that is test-driven development. Apache doesn't have a QA
> team like Adobe. If developers write new stuff and don't write any tests, how
> are you ever going to trust anything new?
> 
Absolutely correct, but I don't want to hold off another release until we
have a mustella test for every code path and boundary condition, so I am
proposing using an "unstable" branch as the proving ground until we prove we
don't need it. 

-- 
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui

Re: Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Gordon Smith <go...@adobe.com>]

The solution for that is test-driven development. Apache doesn't have a QA team like Adobe. If developers write new stuff and don't write any tests, how are you ever going to trust anything new?

Sent from my iPad

On Aug 8, 2012, at 10:11 PM, "Alex Harui" <ah...@adobe.com> wrote:

> 
> 
> 
> On 8/8/12 10:00 PM, "Gordon Smith" <go...@adobe.com> wrote:
> 
>> Why all the focus on branching strategies? It makes more sense to focus on a
>> testing strategy. We need an 'ant tests' and a requirement that it pass for
>> all changes to trunk.
>> 
>> Didn't the mustella tests get submitted? What more do we need other than an
>> easy way to run them?
>> 
> Because I don't have faith that Mustella catches everything, and I'd rather
> merge stuff we feel good about rather than try to remove stuff we don't feel
> good about when were trying to get a release out.
> 
> You know as well as I do what the bug curve looked like for most things
> after they landed.
> 
> -- 
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
> 

Re: Testing strategy (was: How to Merge Unstable Branch (was: [POLL] Use Unstable Branch))

[Alex Harui <ah...@adobe.com>]

On 8/8/12 10:00 PM, "Gordon Smith" <go...@adobe.com> wrote:

> Why all the focus on branching strategies? It makes more sense to focus on a
> testing strategy. We need an 'ant tests' and a requirement that it pass for
> all changes to trunk.
> 
> Didn't the mustella tests get submitted? What more do we need other than an
> easy way to run them?
> 
Because I don't have faith that Mustella catches everything, and I'd rather
merge stuff we feel good about rather than try to remove stuff we don't feel
good about when were trying to get a release out.

You know as well as I do what the bug curve looked like for most things
after they landed.

-- 
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui