You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Animesh Chaturvedi <an...@citrix.com> on 2013/04/22 23:19:58 UTC
[DISCUSS] ACS Release 4 month v/s 6 month
Folks
We started discussing 4 month v/s 6 month release cycle in a another thread [1]. Since the subject of that thread was different, community may not have participated in this important discussion fully. I am are bringing this discussion to its own thread. Here is the summary so far please refer to [1] for more details.
Summary of discussion:
- Animesh pointed out the technical debt that we have accumulated so far needs extra time to resolve
- David, Chip favor shorter release cycle of 4 month and keeping master always stable and in good quality and enhancing automation as a solution to reduce QA manual effort. A focused defect fixing activity may be needed to reduce technical debt
- Will brought up several points in the discussion: He called out heavy dependence on manual QA for a release and pointed out that manual QA may not be always available to match up ACS release schedule. Release overhead for 4 month release is still high and suggest that moving to 6 month will save on release overhead and that time can be used for strengthening automation.
- Joe agrees partly in release overhead being significant for major release
If I missed out any important point please feel free to bring into the thread.
There were some other discussion in [1] on release planning conference and chip's clarification on time based v/s feature based releases but we will not discuss those in this thread. Community has agreed to time-based release already.
[1] http://markmail.org/thread/6suq2fhltdvgvcxd
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Joe Brockmeier <jz...@zonker.net>.
On Mon, Apr 22, 2013, at 04:19 PM, Animesh Chaturvedi wrote:
> - Joe agrees partly in release overhead being significant for major
> release
This is only a partial summary of what I've said before. Yes, I think
there's overhead in a release that's really non-flexible.
However, we've really not given a four-month release a proper shot yet.
As already mentioned - our first release cycle was all about getting the
infra up in place. The second one was punctuated by graduation and other
things that kept it from being as "smooth" as it could have been.
The 4.2 cycle would be the first one we really have a shot at a "normal"
release cycle.
Am I 100% convinced in a four-month cycle? Not yet, but I am convinced
it's a goal we should be aiming for, and I don't think we get there by
going to a six-month cycle "until we're ready." We get there by aiming
for four-month cycles and improving until we reach the ability to do a
four-month cycle.
+1 to a four month cycle.
Best,
jzb
--
Joe Brockmeier
jzb@zonker.net
Twitter: @jzb
http://www.dissociatedpress.net/
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Sudha Ponnaganti <su...@citrix.com>.
+1 for 6 month cycle till Automation is in place and current backlog of defects are reduced. Once we have automation we can switch to shorter cycles
May be having release goals set to meet this criteria ( % of backlog to be cleared + %age of automation to be achieved) might help to reduce the cycles in 4.2.
1. Once IP Clearance is done, this may get to a little better to have community run automated tests on their own for check-ins. Helps to reduce the general QA timeline from current 2+ months to much shorter cycle with better quality code reaching QA.
2. Hope community can pick up a few more features for 4.2 compared to 4.1 so things can move a little faster.
3.Current 72 hour waiting period for code to get in to master is killing some time. Is there a way to reduce this timeline??
4. Reviews got a lot better recently. This is helping definitely.
Thanks
/sudha
-----Original Message-----
From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
Sent: Monday, April 22, 2013 2:20 PM
To: cloudstack-dev@incubator.apache.org
Subject: [DISCUSS] ACS Release 4 month v/s 6 month
Folks
We started discussing 4 month v/s 6 month release cycle in a another thread [1]. Since the subject of that thread was different, community may not have participated in this important discussion fully. I am are bringing this discussion to its own thread. Here is the summary so far please refer to [1] for more details.
Summary of discussion:
- Animesh pointed out the technical debt that we have accumulated so far needs extra time to resolve
- David, Chip favor shorter release cycle of 4 month and keeping master always stable and in good quality and enhancing automation as a solution to reduce QA manual effort. A focused defect fixing activity may be needed to reduce technical debt
- Will brought up several points in the discussion: He called out heavy dependence on manual QA for a release and pointed out that manual QA may not be always available to match up ACS release schedule. Release overhead for 4 month release is still high and suggest that moving to 6 month will save on release overhead and that time can be used for strengthening automation.
- Joe agrees partly in release overhead being significant for major release
If I missed out any important point please feel free to bring into the thread.
There were some other discussion in [1] on release planning conference and chip's clarification on time based v/s feature based releases but we will not discuss those in this thread. Community has agreed to time-based release already.
[1] http://markmail.org/thread/6suq2fhltdvgvcxd
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Will Chan <wi...@citrix.com>.
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Tuesday, April 23, 2013 11:19 AM
> To: dev@cloudstack.apache.org
> Cc: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
>
> How do you see the release schedule laying out with multiple releases over
> the course of time? What overlaps exist?
>
> We *really should not* plan on blocking new features from coming into
> master as they are completed. That's just an inhibitor to progress.
> Given that assumption, I fail to see how the we would be doing anything
> *but* increasing the overall change size by increasing the time between
> feature releases. That leads to increased "cost of release".
>
I'm not saying we should block new features. I'm simply saying that if a feature is not of certain quality before we are ready to release, we should consider not including it in the release notes. For example, the bare metal feature was something written by Frank just before he personally went on a month long vacation. I'm sure the community may have been able to help him out but without his help in such a short release cycle, the community IMO did the right thing and did not deem this feature GA for 4.1. With a longer test/fix cycle, either Frank could have come back in time to help OR the community may have *learned* enough of the feature to fix those issues. In any case, this is simply one example and even I wouldn't simply use this case as the only example to why we should increase the QA/Fix time.
Also, as I said previously, it takes more than 3 months from the time it takes to code freeze on a CCP major release to the time where we are comfortable to GA it. Now, of course ACS does not require the quality of CCP but on the other hand, our process here allows me to effectively ensure that certain bugs and QA tasks are done according to a tight schedule. ACS does not have this luxury.
Also, I do see more patch/maintenance releases coming out in between major releases. We will quickly find out that there the cadence would be too fast.
Will
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Sudha Ponnaganti <su...@citrix.com>.
For bigger features (eg IPV6) and refactoring work, it is beneficial to test on the feature branches. If a feature which impacts several other areas gets checked in, we do not want the testing done so far invalid. This approach is taken to avoid risk in the last minute merges.
- Quality is better as defects are isolated and gets fixed before check-in
- Helps to keep master in stable condition without any interruption throughout development cycles
However this would not help to keep the cycles shorter. If it does, it would be marginal.
- QA need to validate feature both on feature branch and on Master once check in happens. This takes more or less same amount of time.
-----Original Message-----
From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
Sent: Tuesday, April 23, 2013 3:33 PM
To: dev@cloudstack.apache.org
Cc: cloudstack-dev@incubator.apache.org
Subject: RE: [DISCUSS] ACS Release 4 month v/s 6 month
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Tuesday, April 23, 2013 11:19 AM
> To: dev@cloudstack.apache.org
> Cc: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
> On Tue, Apr 23, 2013 at 10:55:50AM -0700, Will Chan wrote:
> > Again, I am not disputing that more features will make it in given
> > Dave's
> argument. In fact, I'm not really that keen on using the fixed cost of release
> mgmt. for the reason of the move as well. Heck, I'm not even sure a 6
> month cycle would fix any of the issues that have already been
> outlined but it'd be nice to try. However, I do know a couple of things:
> >
> > 1. We all want CS releases to be of certain quality. It needs to
> > work and
> upgrades need to work.
>
> +1 - Absolutely
>
> > 2. ACS still relies too heavily on manual testing and most of it
> > unfortunately
> comes from 1 company. We cannot be dependent on another company's
> schedule to ensure ACS has gone through enough proper testing to
> achieve (1).
>
> Also agreed, but this only relates to release schedule for regression
> and upgrade testing really. Feature testing is something that should
> / can be handled prior to a merge in my mind.
>
[Animesh>] If feature testing could mostly be completed in feature branch that would be ideal. However given our limited automation, manual QA testing on feature branch is logistically challenging when a QA person is testing multiple features. Sudha/ folks doing primarily QA function can you provide your feedback?
> > 3. Most importantly, the extra 2 months will give QA more time, give
> features more time to settle in, and more automated tests to be
> written for existing features as well. I agree with Dave that more feature will come in.
> So what? If they are not of good enough quality, we don't release it
> as part of the ACS release. However, that extra 2 month will give
> earlier written features more time to be potentially tested and used
> by people so that we fix the most egregious bugs before we ship it.
> It will also better accommodate people's schedule so they can fix bugs for their features.
> >
>
> How do you see the release schedule laying out with multiple releases
> over the course of time? What overlaps exist?
>
> We *really should not* plan on blocking new features from coming into
> master as they are completed. That's just an inhibitor to progress.
> Given that assumption, I fail to see how the we would be doing
> anything
> *but* increasing the overall change size by increasing the time
> between feature releases. That leads to increased "cost of release".
>
> > Personally, so far, I feel 4 months seems a bit rushed.
> >
> > Will
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Animesh Chaturvedi <an...@citrix.com>.
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Tuesday, April 23, 2013 11:19 AM
> To: dev@cloudstack.apache.org
> Cc: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
> On Tue, Apr 23, 2013 at 10:55:50AM -0700, Will Chan wrote:
> > Again, I am not disputing that more features will make it in given Dave's
> argument. In fact, I'm not really that keen on using the fixed cost of release
> mgmt. for the reason of the move as well. Heck, I'm not even sure a 6
> month cycle would fix any of the issues that have already been outlined but
> it'd be nice to try. However, I do know a couple of things:
> >
> > 1. We all want CS releases to be of certain quality. It needs to work and
> upgrades need to work.
>
> +1 - Absolutely
>
> > 2. ACS still relies too heavily on manual testing and most of it unfortunately
> comes from 1 company. We cannot be dependent on another company's
> schedule to ensure ACS has gone through enough proper testing to achieve
> (1).
>
> Also agreed, but this only relates to release schedule for regression and
> upgrade testing really. Feature testing is something that should / can be
> handled prior to a merge in my mind.
>
[Animesh>] If feature testing could mostly be completed in feature branch that would be ideal. However given our limited automation, manual QA testing on feature branch is logistically challenging when a QA person is testing multiple features. Sudha/ folks doing primarily QA function can you provide your feedback?
> > 3. Most importantly, the extra 2 months will give QA more time, give
> features more time to settle in, and more automated tests to be written for
> existing features as well. I agree with Dave that more feature will come in.
> So what? If they are not of good enough quality, we don't release it as part
> of the ACS release. However, that extra 2 month will give earlier written
> features more time to be potentially tested and used by people so that we
> fix the most egregious bugs before we ship it. It will also better
> accommodate people's schedule so they can fix bugs for their features.
> >
>
> How do you see the release schedule laying out with multiple releases over
> the course of time? What overlaps exist?
>
> We *really should not* plan on blocking new features from coming into
> master as they are completed. That's just an inhibitor to progress.
> Given that assumption, I fail to see how the we would be doing anything
> *but* increasing the overall change size by increasing the time between
> feature releases. That leads to increased "cost of release".
>
> > Personally, so far, I feel 4 months seems a bit rushed.
> >
> > Will
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Chip Childers <ch...@sungard.com>.
On Tue, Apr 23, 2013 at 10:55:50AM -0700, Will Chan wrote:
> Again, I am not disputing that more features will make it in given Dave's argument. In fact, I'm not really that keen on using the fixed cost of release mgmt. for the reason of the move as well. Heck, I'm not even sure a 6 month cycle would fix any of the issues that have already been outlined but it'd be nice to try. However, I do know a couple of things:
>
> 1. We all want CS releases to be of certain quality. It needs to work and upgrades need to work.
+1 - Absolutely
> 2. ACS still relies too heavily on manual testing and most of it unfortunately comes from 1 company. We cannot be dependent on another company's schedule to ensure ACS has gone through enough proper testing to achieve (1).
Also agreed, but this only relates to release schedule for regression
and upgrade testing really. Feature testing is something that should /
can be handled prior to a merge in my mind.
> 3. Most importantly, the extra 2 months will give QA more time, give features more time to settle in, and more automated tests to be written for existing features as well. I agree with Dave that more feature will come in. So what? If they are not of good enough quality, we don't release it as part of the ACS release. However, that extra 2 month will give earlier written features more time to be potentially tested and used by people so that we fix the most egregious bugs before we ship it. It will also better accommodate people's schedule so they can fix bugs for their features.
>
How do you see the release schedule laying out with multiple releases
over the course of time? What overlaps exist?
We *really should not* plan on blocking new features from coming into
master as they are completed. That's just an inhibitor to progress.
Given that assumption, I fail to see how the we would be doing anything
*but* increasing the overall change size by increasing the time between
feature releases. That leads to increased "cost of release".
> Personally, so far, I feel 4 months seems a bit rushed.
>
> Will
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Will Chan <wi...@citrix.com>.
Again, I am not disputing that more features will make it in given Dave's argument. In fact, I'm not really that keen on using the fixed cost of release mgmt. for the reason of the move as well. Heck, I'm not even sure a 6 month cycle would fix any of the issues that have already been outlined but it'd be nice to try. However, I do know a couple of things:
1. We all want CS releases to be of certain quality. It needs to work and upgrades need to work.
2. ACS still relies too heavily on manual testing and most of it unfortunately comes from 1 company. We cannot be dependent on another company's schedule to ensure ACS has gone through enough proper testing to achieve (1).
3. Most importantly, the extra 2 months will give QA more time, give features more time to settle in, and more automated tests to be written for existing features as well. I agree with Dave that more feature will come in. So what? If they are not of good enough quality, we don't release it as part of the ACS release. However, that extra 2 month will give earlier written features more time to be potentially tested and used by people so that we fix the most egregious bugs before we ship it. It will also better accommodate people's schedule so they can fix bugs for their features.
Personally, so far, I feel 4 months seems a bit rushed.
Will
> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Monday, April 22, 2013 7:52 PM
> To: dev@cloudstack.apache.org
> Cc: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
> On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
> <an...@citrix.com> wrote:
> > Folks
> >
> > We started discussing 4 month v/s 6 month release cycle in a another
> thread [1]. Since the subject of that thread was different, community may
> not have participated in this important discussion fully. I am are bringing
> this discussion to its own thread. Here is the summary so far please refer to
> [1] for more details.
> >
> > Summary of discussion:
> > - Animesh pointed out the technical debt that we have accumulated so
> > far needs extra time to resolve
> > - David, Chip favor shorter release cycle of 4 month and keeping
> > master always stable and in good quality and enhancing automation as a
> > solution to reduce QA manual effort. A focused defect fixing activity
> > may be needed to reduce technical debt
> > - Will brought up several points in the discussion: He called out heavy
> dependence on manual QA for a release and pointed out that manual QA
> may not be always available to match up ACS release schedule. Release
> overhead for 4 month release is still high and suggest that moving to 6
> month will save on release overhead and that time can be used for
> strengthening automation.
> > - Joe agrees partly in release overhead being significant for major
> > release
> >
> > If I missed out any important point please feel free to bring into the
> thread.
> >
> > There were some other discussion in [1] on release planning conference
> and chip's clarification on time based v/s feature based releases but we will
> not discuss those in this thread. Community has agreed to time-based
> release already.
> >
> > [1] http://markmail.org/thread/6suq2fhltdvgvcxd
> >
>
>
> Hi Animesh:
>
> I thought I would add a few comments. To the folks reading - I apologize for
> the length. If you haven't started yet, you may want to get some coffee/tea.
> You've been warned. :)
>
> I think there are two concerns people think about when talking about
> changing the length of the release cycle.
>
> The first concern is workload. Generating a release has a certain fixed
> amount of overhead. Writing release notes, running votes, and a key part of
> the discussion currently is the QA cycle. Currently a large portion of our QA
> cycle is manual testing, which means we need lots of fingers on keyboards
> to get it done.
>
> The other concern is quality. We always want to try and put our best foot
> forward, and minimize bugs, yet the very act of developing software, and
> especially developing new features, introduces bugs.
>
> Before I start telling you my reasoning, I want to lay out something that
> only hit me tonight. We've talked about how extending the release cycle
> only extends the length of development. We've talked about keeping the
> length of the development period (e.g. pre-code freeze) the same, and
> essentially only extending the amount of time for QA.
> That works for the first release after a move from 4 to 6 months. It falls
> apart there after though. The problem is that at code freeze we branch.
> Master immediately becomes open for future feature development, and
> you've just extended it by an additional 2 months, because your cycle is
> that much longer.
>
> I personally don't think that either concern is addressed well by lengthening
> the cycle. Let me explain why
>
> On the quality front - the immediate threat to quality is instability inserted
> by additional development/new features/improvements. By increasing the
> length of time for that disruption to occur, we get more of it, as people will
> continue adding things that might have been deferred to a later release. We
> have code merged that, realistically, we won't know if it works properly
> until QA has gone in and verified it, perhaps months later. The more paths
> changed in a given release, the greater the propensity for failure.
>
> On the overhead front - the overhead of writing release notes and
> generating releases goes down. In addition the 'basic' testing that QA does
> now is additional overhead in each cycle as well. What isn't overhead
> though is testing for new features. Because effectively longer cycles mean
> greater amount of codebase change, more features, etc. The actual testing
> in a given cycle must also increase.
>
> If we were really focused on absolute best quality we could deliver with our
> current tooling, I believe the easiest would be to make our release cycle the
> length of a single feature development period + the QA cycle (e.g.
> shortening the release cycle.)
>
> Is there a right or wrong answer? There shouldn't be. Our software should
> be deliverable at any point in the development cycle we choose.
> 4 months or 6 months would not be a quality, or QA workload issue. We
> would have automated test suites written for 80+% of our testing and we
> would run comprehensive tests against propose merges to guarantee that
> an incoming features both works as intended, and doesn't harm the rest of
> the platform in the process. We'd make release cycle decisions based upon
> the same kind of preferences that make us choose a monkey mascot over a
> polar bear mascot.
>
> That sadly isn't where we are today, but I do think it illuminates a clear path
> of where we need to go. (more automated testing)
>
> So where do I fall on 4 months versus 6 months?
>
> I think we should stay the 4 month course. Heres why:
> * We set the 4-month release cycle expectation already. In theory we've
> been through 2 releases, but really only 4.1 - 4.0 was dominated by getting
> into the incubator, getting releases okayed legally, etc.
> It took us 7 months to release. I am loathe to change something we only
> recently set, and have only been through once. I am also worried about the
> signal that such rapid contrary decision changes would send to our users.
>
> * I am worried about the impact to quality of our releases. Greater
> introduction of features means an increased threat to stability. It may not
> impact us in the 4.2 cycle, but without massive automated QA it will impact
> us going forward in the 4.3 and later time frames.
>
>
> If you've made it thus far, I hope your tea/coffee was enjoyable.
> Thanks for reading. :)
>
> --David
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Prasanna Santhanam <ts...@apache.org>.
The problem (I am beginning to realize) with extending the release is
the temptation to cram in more features. May be use the collab
conference to improve release management aspects. What we did right
in 4.1, what went wrong, how to better plan 5.0. And that can happen
with a 4 or 6 month cycle. I understand the exclusive nature of this
but I do think that everything must/should be brought back to the
lists judiciously.
To me making our release cadence easier to follow is an important step
towards bringing in more contributions. If that happens soon the
better - offline or online. Either way the effect is going to be felt
on the list since release management affects everyone.
IMO, even with a time-based release we need to think of things like:
- Is this proposal targeted for the next release going to be
disruptive to others
- Are we changing architecture aspects of the system - that work must
happen well in advance.
- What features are possibly stalled by architecture work/refactors
etc.
- When to pull the plug on proposals to make it to the next release.
Or do we just consume everything until code freeze
- What gets manual tested, what gets automated. Are we reviewing our
test plans?
It seems like anything that's had a discussion and a wiki page under
4.2 release is already targeted for that release now. Just go look at
it. It's massive! :)
On Mon, Apr 22, 2013 at 08:13:55PM -0700, Edison Su wrote:
> +1, on the automation test. When will we have good automation test?
After a long-winded struggle from our startup days - automated tests
are _finally_ becoming a priority. That in itself is a great leap
forward for us as a project. Until now I'd say testing hasn't received
the same love as the love for adding new features. But it'll take us a
release or two to build up those test suites strong enough to ensure
that our releases are not delayed. Time is nigh to think along the
lines of testability of features being merged, proposed and churned in
our topic branches.
I'd like to see the day when our "threads on release management" grow
shorter than those done as reviews for patches and topic branches. We
need to seriously start paying attention to:
- The work happening around in those silos (topic branches).
- And to the bugs users have been reporting.
- Reviews of test plans is going to have to be considered seriously
- During those reviews think about automating the QA tests. If
framework support doesn't exist, extend the test framework to
support that.
It's great that we at least begin to add tests when we need our
feature merged now. Where we need to get to is start thinking about
testability from the beginning of feature planning. All of this if we
do even with a reasonable efficiency I think 4 months should be
doable.
--
Prasanna.,
------------------------
Powered by BigRock.com
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Edison Su <Ed...@citrix.com>.
+1, on the automation test. When will we have good automation test?
> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Monday, April 22, 2013 7:52 PM
> To: dev@cloudstack.apache.org
> Cc: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
> On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
> <an...@citrix.com> wrote:
> > Folks
> >
> > We started discussing 4 month v/s 6 month release cycle in a another
> thread [1]. Since the subject of that thread was different, community may
> not have participated in this important discussion fully. I am are bringing this
> discussion to its own thread. Here is the summary so far please refer to [1]
> for more details.
> >
> > Summary of discussion:
> > - Animesh pointed out the technical debt that we have accumulated so
> > far needs extra time to resolve
> > - David, Chip favor shorter release cycle of 4 month and keeping
> > master always stable and in good quality and enhancing automation as a
> > solution to reduce QA manual effort. A focused defect fixing activity
> > may be needed to reduce technical debt
> > - Will brought up several points in the discussion: He called out heavy
> dependence on manual QA for a release and pointed out that manual QA
> may not be always available to match up ACS release schedule. Release
> overhead for 4 month release is still high and suggest that moving to 6 month
> will save on release overhead and that time can be used for strengthening
> automation.
> > - Joe agrees partly in release overhead being significant for major
> > release
> >
> > If I missed out any important point please feel free to bring into the thread.
> >
> > There were some other discussion in [1] on release planning conference
> and chip's clarification on time based v/s feature based releases but we will
> not discuss those in this thread. Community has agreed to time-based
> release already.
> >
> > [1] http://markmail.org/thread/6suq2fhltdvgvcxd
> >
>
>
> Hi Animesh:
>
> I thought I would add a few comments. To the folks reading - I apologize for
> the length. If you haven't started yet, you may want to get some coffee/tea.
> You've been warned. :)
>
> I think there are two concerns people think about when talking about
> changing the length of the release cycle.
>
> The first concern is workload. Generating a release has a certain fixed
> amount of overhead. Writing release notes, running votes, and a key part of
> the discussion currently is the QA cycle. Currently a large portion of our QA
> cycle is manual testing, which means we need lots of fingers on keyboards to
> get it done.
>
> The other concern is quality. We always want to try and put our best foot
> forward, and minimize bugs, yet the very act of developing software, and
> especially developing new features, introduces bugs.
>
> Before I start telling you my reasoning, I want to lay out something that only
> hit me tonight. We've talked about how extending the release cycle only
> extends the length of development. We've talked about keeping the length
> of the development period (e.g. pre-code freeze) the same, and essentially
> only extending the amount of time for QA.
> That works for the first release after a move from 4 to 6 months. It falls apart
> there after though. The problem is that at code freeze we branch. Master
> immediately becomes open for future feature development, and you've just
> extended it by an additional 2 months, because your cycle is that much
> longer.
>
> I personally don't think that either concern is addressed well by lengthening
> the cycle. Let me explain why
>
> On the quality front - the immediate threat to quality is instability inserted by
> additional development/new features/improvements. By increasing the
> length of time for that disruption to occur, we get more of it, as people will
> continue adding things that might have been deferred to a later release. We
> have code merged that, realistically, we won't know if it works properly until
> QA has gone in and verified it, perhaps months later. The more paths
> changed in a given release, the greater the propensity for failure.
>
> On the overhead front - the overhead of writing release notes and
> generating releases goes down. In addition the 'basic' testing that QA does
> now is additional overhead in each cycle as well. What isn't overhead though
> is testing for new features. Because effectively longer cycles mean greater
> amount of codebase change, more features, etc. The actual testing in a given
> cycle must also increase.
>
> If we were really focused on absolute best quality we could deliver with our
> current tooling, I believe the easiest would be to make our release cycle the
> length of a single feature development period + the QA cycle (e.g.
> shortening the release cycle.)
>
> Is there a right or wrong answer? There shouldn't be. Our software should be
> deliverable at any point in the development cycle we choose.
> 4 months or 6 months would not be a quality, or QA workload issue. We
> would have automated test suites written for 80+% of our testing and we
> would run comprehensive tests against propose merges to guarantee that an
> incoming features both works as intended, and doesn't harm the rest of the
> platform in the process. We'd make release cycle decisions based upon the
> same kind of preferences that make us choose a monkey mascot over a polar
> bear mascot.
>
> That sadly isn't where we are today, but I do think it illuminates a clear path
> of where we need to go. (more automated testing)
>
> So where do I fall on 4 months versus 6 months?
>
> I think we should stay the 4 month course. Heres why:
> * We set the 4-month release cycle expectation already. In theory we've
> been through 2 releases, but really only 4.1 - 4.0 was dominated by getting
> into the incubator, getting releases okayed legally, etc.
> It took us 7 months to release. I am loathe to change something we only
> recently set, and have only been through once. I am also worried about the
> signal that such rapid contrary decision changes would send to our users.
>
> * I am worried about the impact to quality of our releases. Greater
> introduction of features means an increased threat to stability. It may not
> impact us in the 4.2 cycle, but without massive automated QA it will impact us
> going forward in the 4.3 and later time frames.
>
>
> If you've made it thus far, I hope your tea/coffee was enjoyable.
> Thanks for reading. :)
>
> --David
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Rohit Yadav <bh...@apache.org>.
On Tue, Apr 23, 2013 at 5:05 PM, Hugo Trippaers <
HTrippaers@schubergphilis.com> wrote:
> I would favor sticking to the current 4 month release cycle. We have only
> been doing it once and I think we need to have some more experience before
> changing this.
>
+1 Stick with early release if not >4 months release cycle.
I would prefer less than 4 months release cycle, and propose that we "ship
early and ship often". Say 4 months dev overlapping with QA (where say, 2
months dev, 1 months dev+qa feedback/overlap, 1 month bugfixing/qa/feedback
and release).
If we have longer release cycles:
- new stuff would take time to show up
- we would end up breaking a lot of things
- longer feedback and fixing from the real world deployments/prod.
environment
Early releases and a good frequency (i.e. major+minor ones, thanks to jzb
for example we have the 4.0.x hotfix/bugfix/minors) would ensure;
- a good feedback loop and support.
- easier to upgrade, support, have a migration path
- we would be cautious with backward compatibility
- have frequent hotfixes/maintenance releases
- would have new features with better upgrade/paths so when we end up
changing fundamentals stuff such as the architecture, design, structure,
deployment designs it will be less costly and penalizing for both developer
and the user (See 4.1 and 4.2 changelog, woah! the community thought :)
Cheers.
>
> For me another important factor is the amount of features that are coming
> into the system at this moment. With a four month release cycle I would
> have to wait a maximum of four months on the new feature, with 6 months its
> even longer. If we want to get our features out there I think we need to
> release as often as possible so everybody can enjoy the features. I'd
> rather leave out a few features to ease the burden on QA than delay the
> release.
>
> The same feature thing also has another issue, as David pointed out. The
> longer the development cycle, the more features with hit master. That means
> more testing complexity and an even greater risk when you want to upgrade
> to a new version. And I'm not only talking about our tests, but any user
> will want to test a release before it hits his/her production systems. Make
> the releases to complex and they might be skipped because of testing
> complexity. From a user perspective I'd rather upgrade 3 times a year with
> a smaller change set than two time a year with a bigger changeset. Actually
> I'd rather upgrade 12 times a year with a very small changeset, but that's
> a dream for the future.
>
> Last but not least, if we start delaying releases because we don't have
> the automated test capacity, we will never get it. I believe a bit of
> pressure goes a long way to ensuring we will have automated tests. If
> everybody that is working on features spends just a little bit of time on
> testing, it's done in no time at all. If you look at what happened during
> this release cycle 4.0 to 4.1, I'm really impressed by the level of
> automation that is already there. And much of the groundwork has been
> completed as well so making future test is going to be even easier.
>
> Cheers,
>
> Hugo
>
> > -----Original Message-----
> > From: Sebastien Goasguen [mailto:runseb@gmail.com]
> > Sent: Tuesday, April 23, 2013 9:31 AM
> > To: dev@cloudstack.apache.org
> > Cc: cloudstack-dev@incubator.apache.org; Chip Childers
> > Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
> >
> >
> > On Apr 23, 2013, at 1:32 AM, Nitin Mehta <ni...@citrix.com> wrote:
> >
> > > David - You have some compelling logic :) Given the debate about 4
> > > month to 6 month, lets make an educated decision
> > >
> > > First - Lets hear from the "release managers" as to what they have to
> > > say about the last 2 releases. What went well and what didn't. Chip - ?
> > > Second - If we plan to do a 6 month release what would be the
> > > timelines in place ? Like time allocated for what features going in
> > > the release, feature discussion and coding, bug fixing, testing,
> > > documentation etc. If I get a rough idea on that for the 6 month
> > > release it would help me understand the differences.
> > >
> > > At this point I would still +1 a 6 month release because it gives me
> > > good time to discuss, code, unit test and bake my feature.
> >
> > We agreed on a time based release cycle, not a feature based one.
> >
> > Moving from 4 to 6 just means that a feature will get delayed 2 months,
> > giving even more time to finish it and test it properly.
> >
> > IMHO, improving quality and reducing QA time comes from increasing
> > testing/review at commit time and making sure feature branches do not
> > diverge from master (too much).
> >
> > I'd rather aim for having releases that have incremental changes and an
> easy
> > upgrade path than a longer release cycle in order to give more time for
> "big"
> > changes that will disrupt upgrade and operations.
> >
> > I also agree with David that we have only gone through the 4 months cycle
> > once (4.1) and this release has some major refactoring. We at least need
> to
> > do it once or twice more to see if there is a real issue.
> >
> > -sebastien
> >
> >
> > > From my experience
> > > at this point this definitely makes more sense.
> > > We can revisit the time line after a couple releases as we all become
> > > more accustomed with the processes in place.
> > >
> > > Thanks,
> > > -Nitin
> > >
> > > On 23/04/13 8:21 AM, "David Nalley" <da...@gnsa.us> wrote:
> > >
> > >> On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
> > >> <an...@citrix.com> wrote:
> > >>> Folks
> > >>>
> > >>> We started discussing 4 month v/s 6 month release cycle in a another
> > >>> thread [1]. Since the subject of that thread was different,
> > >>> community may not have participated in this important discussion
> > >>> fully. I am are bringing this discussion to its own thread. Here is
> > >>> the summary so far please refer to [1] for more details.
> > >>>
> > >>> Summary of discussion:
> > >>> - Animesh pointed out the technical debt that we have accumulated so
> > >>> far needs extra time to resolve
> > >>> - David, Chip favor shorter release cycle of 4 month and keeping
> > >>> master always stable and in good quality and enhancing automation as
> > >>> a solution to reduce QA manual effort. A focused defect fixing
> > >>> activity may be needed to reduce technical debt
> > >>> - Will brought up several points in the discussion: He called out
> > >>> heavy dependence on manual QA for a release and pointed out that
> > >>> manual QA may not be always available to match up ACS release
> > >>> schedule. Release overhead for 4 month release is still high and
> > >>> suggest that moving to 6 month will save on release overhead and
> > >>> that time can be used for strengthening automation.
> > >>> - Joe agrees partly in release overhead being significant for major
> > >>> release
> > >>>
> > >>> If I missed out any important point please feel free to bring into
> > >>> the thread.
> > >>>
> > >>> There were some other discussion in [1] on release planning
> > >>> conference and chip's clarification on time based v/s feature based
> > >>> releases but we will not discuss those in this thread. Community has
> > >>> agreed to time-based release already.
> > >>>
> > >>> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
> > >>>
> > >>
> > >>
> > >> Hi Animesh:
> > >>
> > >> I thought I would add a few comments. To the folks reading - I
> > >> apologize for the length. If you haven't started yet, you may want to
> > >> get some coffee/tea. You've been warned. :)
> > >>
> > >> I think there are two concerns people think about when talking about
> > >> changing the length of the release cycle.
> > >>
> > >> The first concern is workload. Generating a release has a certain
> > >> fixed amount of overhead. Writing release notes, running votes, and a
> > >> key part of the discussion currently is the QA cycle. Currently a
> > >> large portion of our QA cycle is manual testing, which means we need
> > >> lots of fingers on keyboards to get it done.
> > >>
> > >> The other concern is quality. We always want to try and put our best
> > >> foot forward, and minimize bugs, yet the very act of developing
> > >> software, and especially developing new features, introduces bugs.
> > >>
> > >> Before I start telling you my reasoning, I want to lay out something
> > >> that only hit me tonight. We've talked about how extending the
> > >> release cycle only extends the length of development. We've talked
> > >> about keeping the length of the development period (e.g. pre-code
> > >> freeze) the same, and essentially only extending the amount of time
> for
> > QA.
> > >> That works for the first release after a move from 4 to 6 months. It
> > >> falls apart there after though. The problem is that at code freeze we
> > >> branch. Master immediately becomes open for future feature
> > >> development, and you've just extended it by an additional 2 months,
> > >> because your cycle is that much longer.
> > >>
> > >> I personally don't think that either concern is addressed well by
> > >> lengthening the cycle. Let me explain why
> > >>
> > >> On the quality front - the immediate threat to quality is instability
> > >> inserted by additional development/new features/improvements. By
> > >> increasing the length of time for that disruption to occur, we get
> > >> more of it, as people will continue adding things that might have
> > >> been deferred to a later release. We have code merged that,
> > >> realistically, we won't know if it works properly until QA has gone
> > >> in and verified it, perhaps months later. The more paths changed in a
> > >> given release, the greater the propensity for failure.
> > >
> > > Agreed, but merely changing the time duration doesn't really solve the
> > > problem. More automation, having a staging branch would still do a
> > > better job.
> > > But with the current context what you say makes sense.
> > >
> > >>
> > >> On the overhead front - the overhead of writing release notes and
> > >> generating releases goes down. In addition the 'basic' testing that
> > >> QA does now is additional overhead in each cycle as well. What isn't
> > >> overhead though is testing for new features. Because effectively
> > >> longer cycles mean greater amount of codebase change, more features,
> > >> etc. The actual testing in a given cycle must also increase.
> > >>
> > >> If we were really focused on absolute best quality we could deliver
> > >> with our current tooling, I believe the easiest would be to make our
> > >> release cycle the length of a single feature development period + the
> > >> QA cycle (e.g. shortening the release cycle.)
> > >>
> > >> Is there a right or wrong answer? There shouldn't be. Our software
> > >> should be deliverable at any point in the development cycle we choose.
> > >> 4 months or 6 months would not be a quality, or QA workload issue. We
> > >> would have automated test suites written for 80+% of our testing and
> > >> we would run comprehensive tests against propose merges to guarantee
> > >> that an incoming features both works as intended, and doesn't harm
> > >> the rest of the platform in the process. We'd make release cycle
> > >> decisions based upon the same kind of preferences that make us choose
> > >> a monkey mascot over a polar bear mascot.
> > >>
> > >> That sadly isn't where we are today, but I do think it illuminates a
> > >> clear path of where we need to go. (more automated testing)
> > >>
> > >> So where do I fall on 4 months versus 6 months?
> > >>
> > >> I think we should stay the 4 month course. Heres why:
> > >> * We set the 4-month release cycle expectation already. In theory
> > >> we've been through 2 releases, but really only 4.1 - 4.0 was
> > >> dominated by getting into the incubator, getting releases okayed
> legally,
> > etc.
> > >> It took us 7 months to release. I am loathe to change something we
> > >> only recently set, and have only been through once. I am also worried
> > >> about the signal that such rapid contrary decision changes would send
> > >> to our users.
> > >>
> > >> * I am worried about the impact to quality of our releases. Greater
> > >> introduction of features means an increased threat to stability. It
> > >> may not impact us in the 4.2 cycle, but without massive automated QA
> > >> it will impact us going forward in the 4.3 and later time frames.
> > >>
> > >
> > > Great point and I agree this can definitely be an issue.
> > >
> > >>
> > >> If you've made it thus far, I hope your tea/coffee was enjoyable.
> > >> Thanks for reading. :)
> > >
> > > Had to take 2 :).
> > >
> > >>
> > >> --David
> > >
>
>
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Hugo Trippaers <HT...@schubergphilis.com>.
I would favor sticking to the current 4 month release cycle. We have only been doing it once and I think we need to have some more experience before changing this.
For me another important factor is the amount of features that are coming into the system at this moment. With a four month release cycle I would have to wait a maximum of four months on the new feature, with 6 months its even longer. If we want to get our features out there I think we need to release as often as possible so everybody can enjoy the features. I'd rather leave out a few features to ease the burden on QA than delay the release.
The same feature thing also has another issue, as David pointed out. The longer the development cycle, the more features with hit master. That means more testing complexity and an even greater risk when you want to upgrade to a new version. And I'm not only talking about our tests, but any user will want to test a release before it hits his/her production systems. Make the releases to complex and they might be skipped because of testing complexity. From a user perspective I'd rather upgrade 3 times a year with a smaller change set than two time a year with a bigger changeset. Actually I'd rather upgrade 12 times a year with a very small changeset, but that's a dream for the future.
Last but not least, if we start delaying releases because we don't have the automated test capacity, we will never get it. I believe a bit of pressure goes a long way to ensuring we will have automated tests. If everybody that is working on features spends just a little bit of time on testing, it's done in no time at all. If you look at what happened during this release cycle 4.0 to 4.1, I'm really impressed by the level of automation that is already there. And much of the groundwork has been completed as well so making future test is going to be even easier.
Cheers,
Hugo
> -----Original Message-----
> From: Sebastien Goasguen [mailto:runseb@gmail.com]
> Sent: Tuesday, April 23, 2013 9:31 AM
> To: dev@cloudstack.apache.org
> Cc: cloudstack-dev@incubator.apache.org; Chip Childers
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
>
> On Apr 23, 2013, at 1:32 AM, Nitin Mehta <ni...@citrix.com> wrote:
>
> > David - You have some compelling logic :) Given the debate about 4
> > month to 6 month, lets make an educated decision
> >
> > First - Lets hear from the "release managers" as to what they have to
> > say about the last 2 releases. What went well and what didn't. Chip - ?
> > Second - If we plan to do a 6 month release what would be the
> > timelines in place ? Like time allocated for what features going in
> > the release, feature discussion and coding, bug fixing, testing,
> > documentation etc. If I get a rough idea on that for the 6 month
> > release it would help me understand the differences.
> >
> > At this point I would still +1 a 6 month release because it gives me
> > good time to discuss, code, unit test and bake my feature.
>
> We agreed on a time based release cycle, not a feature based one.
>
> Moving from 4 to 6 just means that a feature will get delayed 2 months,
> giving even more time to finish it and test it properly.
>
> IMHO, improving quality and reducing QA time comes from increasing
> testing/review at commit time and making sure feature branches do not
> diverge from master (too much).
>
> I'd rather aim for having releases that have incremental changes and an easy
> upgrade path than a longer release cycle in order to give more time for "big"
> changes that will disrupt upgrade and operations.
>
> I also agree with David that we have only gone through the 4 months cycle
> once (4.1) and this release has some major refactoring. We at least need to
> do it once or twice more to see if there is a real issue.
>
> -sebastien
>
>
> > From my experience
> > at this point this definitely makes more sense.
> > We can revisit the time line after a couple releases as we all become
> > more accustomed with the processes in place.
> >
> > Thanks,
> > -Nitin
> >
> > On 23/04/13 8:21 AM, "David Nalley" <da...@gnsa.us> wrote:
> >
> >> On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
> >> <an...@citrix.com> wrote:
> >>> Folks
> >>>
> >>> We started discussing 4 month v/s 6 month release cycle in a another
> >>> thread [1]. Since the subject of that thread was different,
> >>> community may not have participated in this important discussion
> >>> fully. I am are bringing this discussion to its own thread. Here is
> >>> the summary so far please refer to [1] for more details.
> >>>
> >>> Summary of discussion:
> >>> - Animesh pointed out the technical debt that we have accumulated so
> >>> far needs extra time to resolve
> >>> - David, Chip favor shorter release cycle of 4 month and keeping
> >>> master always stable and in good quality and enhancing automation as
> >>> a solution to reduce QA manual effort. A focused defect fixing
> >>> activity may be needed to reduce technical debt
> >>> - Will brought up several points in the discussion: He called out
> >>> heavy dependence on manual QA for a release and pointed out that
> >>> manual QA may not be always available to match up ACS release
> >>> schedule. Release overhead for 4 month release is still high and
> >>> suggest that moving to 6 month will save on release overhead and
> >>> that time can be used for strengthening automation.
> >>> - Joe agrees partly in release overhead being significant for major
> >>> release
> >>>
> >>> If I missed out any important point please feel free to bring into
> >>> the thread.
> >>>
> >>> There were some other discussion in [1] on release planning
> >>> conference and chip's clarification on time based v/s feature based
> >>> releases but we will not discuss those in this thread. Community has
> >>> agreed to time-based release already.
> >>>
> >>> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
> >>>
> >>
> >>
> >> Hi Animesh:
> >>
> >> I thought I would add a few comments. To the folks reading - I
> >> apologize for the length. If you haven't started yet, you may want to
> >> get some coffee/tea. You've been warned. :)
> >>
> >> I think there are two concerns people think about when talking about
> >> changing the length of the release cycle.
> >>
> >> The first concern is workload. Generating a release has a certain
> >> fixed amount of overhead. Writing release notes, running votes, and a
> >> key part of the discussion currently is the QA cycle. Currently a
> >> large portion of our QA cycle is manual testing, which means we need
> >> lots of fingers on keyboards to get it done.
> >>
> >> The other concern is quality. We always want to try and put our best
> >> foot forward, and minimize bugs, yet the very act of developing
> >> software, and especially developing new features, introduces bugs.
> >>
> >> Before I start telling you my reasoning, I want to lay out something
> >> that only hit me tonight. We've talked about how extending the
> >> release cycle only extends the length of development. We've talked
> >> about keeping the length of the development period (e.g. pre-code
> >> freeze) the same, and essentially only extending the amount of time for
> QA.
> >> That works for the first release after a move from 4 to 6 months. It
> >> falls apart there after though. The problem is that at code freeze we
> >> branch. Master immediately becomes open for future feature
> >> development, and you've just extended it by an additional 2 months,
> >> because your cycle is that much longer.
> >>
> >> I personally don't think that either concern is addressed well by
> >> lengthening the cycle. Let me explain why
> >>
> >> On the quality front - the immediate threat to quality is instability
> >> inserted by additional development/new features/improvements. By
> >> increasing the length of time for that disruption to occur, we get
> >> more of it, as people will continue adding things that might have
> >> been deferred to a later release. We have code merged that,
> >> realistically, we won't know if it works properly until QA has gone
> >> in and verified it, perhaps months later. The more paths changed in a
> >> given release, the greater the propensity for failure.
> >
> > Agreed, but merely changing the time duration doesn't really solve the
> > problem. More automation, having a staging branch would still do a
> > better job.
> > But with the current context what you say makes sense.
> >
> >>
> >> On the overhead front - the overhead of writing release notes and
> >> generating releases goes down. In addition the 'basic' testing that
> >> QA does now is additional overhead in each cycle as well. What isn't
> >> overhead though is testing for new features. Because effectively
> >> longer cycles mean greater amount of codebase change, more features,
> >> etc. The actual testing in a given cycle must also increase.
> >>
> >> If we were really focused on absolute best quality we could deliver
> >> with our current tooling, I believe the easiest would be to make our
> >> release cycle the length of a single feature development period + the
> >> QA cycle (e.g. shortening the release cycle.)
> >>
> >> Is there a right or wrong answer? There shouldn't be. Our software
> >> should be deliverable at any point in the development cycle we choose.
> >> 4 months or 6 months would not be a quality, or QA workload issue. We
> >> would have automated test suites written for 80+% of our testing and
> >> we would run comprehensive tests against propose merges to guarantee
> >> that an incoming features both works as intended, and doesn't harm
> >> the rest of the platform in the process. We'd make release cycle
> >> decisions based upon the same kind of preferences that make us choose
> >> a monkey mascot over a polar bear mascot.
> >>
> >> That sadly isn't where we are today, but I do think it illuminates a
> >> clear path of where we need to go. (more automated testing)
> >>
> >> So where do I fall on 4 months versus 6 months?
> >>
> >> I think we should stay the 4 month course. Heres why:
> >> * We set the 4-month release cycle expectation already. In theory
> >> we've been through 2 releases, but really only 4.1 - 4.0 was
> >> dominated by getting into the incubator, getting releases okayed legally,
> etc.
> >> It took us 7 months to release. I am loathe to change something we
> >> only recently set, and have only been through once. I am also worried
> >> about the signal that such rapid contrary decision changes would send
> >> to our users.
> >>
> >> * I am worried about the impact to quality of our releases. Greater
> >> introduction of features means an increased threat to stability. It
> >> may not impact us in the 4.2 cycle, but without massive automated QA
> >> it will impact us going forward in the 4.3 and later time frames.
> >>
> >
> > Great point and I agree this can definitely be an issue.
> >
> >>
> >> If you've made it thus far, I hope your tea/coffee was enjoyable.
> >> Thanks for reading. :)
> >
> > Had to take 2 :).
> >
> >>
> >> --David
> >
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Apr 23, 2013, at 1:32 AM, Nitin Mehta <ni...@citrix.com> wrote:
> David - You have some compelling logic :)
> Given the debate about 4 month to 6 month, lets make an educated decision
>
> First - Lets hear from the "release managers" as to what they have to say
> about the last 2 releases. What went well and what didn't. Chip - ?
> Second - If we plan to do a 6 month release what would be the timelines in
> place ? Like time allocated for what features going in the release,
> feature discussion and coding, bug fixing, testing, documentation etc. If
> I get a rough idea on that for the 6 month release it would help me
> understand the differences.
>
> At this point I would still +1 a 6 month release because it gives me good
> time to discuss, code, unit test and bake my feature.
We agreed on a time based release cycle, not a feature based one.
Moving from 4 to 6 just means that a feature will get delayed 2 months, giving even more time to finish it and test it properly.
IMHO, improving quality and reducing QA time comes from increasing testing/review at commit time and making sure feature branches do not diverge from master (too much).
I'd rather aim for having releases that have incremental changes and an easy upgrade path than a longer release cycle in order to give more time for "big" changes that will disrupt upgrade and operations.
I also agree with David that we have only gone through the 4 months cycle once (4.1) and this release has some major refactoring. We at least need to do it once or twice more to see if there is a real issue.
-sebastien
> From my experience
> at this point this definitely makes more sense.
> We can revisit the time line after a couple releases as we all become more
> accustomed with the processes in place.
>
> Thanks,
> -Nitin
>
> On 23/04/13 8:21 AM, "David Nalley" <da...@gnsa.us> wrote:
>
>> On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
>> <an...@citrix.com> wrote:
>>> Folks
>>>
>>> We started discussing 4 month v/s 6 month release cycle in a another
>>> thread [1]. Since the subject of that thread was different, community
>>> may not have participated in this important discussion fully. I am are
>>> bringing this discussion to its own thread. Here is the summary so far
>>> please refer to [1] for more details.
>>>
>>> Summary of discussion:
>>> - Animesh pointed out the technical debt that we have accumulated so
>>> far needs extra time to resolve
>>> - David, Chip favor shorter release cycle of 4 month and keeping master
>>> always stable and in good quality and enhancing automation as a solution
>>> to reduce QA manual effort. A focused defect fixing activity may be
>>> needed to reduce technical debt
>>> - Will brought up several points in the discussion: He called out heavy
>>> dependence on manual QA for a release and pointed out that manual QA may
>>> not be always available to match up ACS release schedule. Release
>>> overhead for 4 month release is still high and suggest that moving to 6
>>> month will save on release overhead and that time can be used for
>>> strengthening automation.
>>> - Joe agrees partly in release overhead being significant for major
>>> release
>>>
>>> If I missed out any important point please feel free to bring into the
>>> thread.
>>>
>>> There were some other discussion in [1] on release planning conference
>>> and chip's clarification on time based v/s feature based releases but we
>>> will not discuss those in this thread. Community has agreed to
>>> time-based release already.
>>>
>>> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
>>>
>>
>>
>> Hi Animesh:
>>
>> I thought I would add a few comments. To the folks reading - I
>> apologize for the length. If you haven't started yet, you may want to
>> get some coffee/tea. You've been warned. :)
>>
>> I think there are two concerns people think about when talking about
>> changing the length of the release cycle.
>>
>> The first concern is workload. Generating a release has a certain
>> fixed amount of overhead. Writing release notes, running votes, and a
>> key part of the discussion currently is the QA cycle. Currently a
>> large portion of our QA cycle is manual testing, which means we need
>> lots of fingers on keyboards to get it done.
>>
>> The other concern is quality. We always want to try and put our best
>> foot forward, and minimize bugs, yet the very act of developing
>> software, and especially developing new features, introduces bugs.
>>
>> Before I start telling you my reasoning, I want to lay out something
>> that only hit me tonight. We've talked about how extending the release
>> cycle only extends the length of development. We've talked about
>> keeping the length of the development period (e.g. pre-code freeze)
>> the same, and essentially only extending the amount of time for QA.
>> That works for the first release after a move from 4 to 6 months. It
>> falls apart there after though. The problem is that at code freeze we
>> branch. Master immediately becomes open for future feature
>> development, and you've just extended it by an additional 2 months,
>> because your cycle is that much longer.
>>
>> I personally don't think that either concern is addressed well by
>> lengthening the cycle. Let me explain why
>>
>> On the quality front - the immediate threat to quality is instability
>> inserted by additional development/new features/improvements. By
>> increasing the length of time for that disruption to occur, we get
>> more of it, as people will continue adding things that might have been
>> deferred to a later release. We have code merged that, realistically,
>> we won't know if it works properly until QA has gone in and verified
>> it, perhaps months later. The more paths changed in a given release,
>> the greater the propensity for failure.
>
> Agreed, but merely changing the time duration doesn't really solve the
> problem. More automation, having a staging branch would still do a better
> job.
> But with the current context what you say makes sense.
>
>>
>> On the overhead front - the overhead of writing release notes and
>> generating releases goes down. In addition the 'basic' testing that QA
>> does now is additional overhead in each cycle as well. What isn't
>> overhead though is testing for new features. Because effectively
>> longer cycles mean greater amount of codebase change, more features,
>> etc. The actual testing in a given cycle must also increase.
>>
>> If we were really focused on absolute best quality we could deliver
>> with our current tooling, I believe the easiest would be to make our
>> release cycle the length of a single feature development period + the
>> QA cycle (e.g. shortening the release cycle.)
>>
>> Is there a right or wrong answer? There shouldn't be. Our software
>> should be deliverable at any point in the development cycle we choose.
>> 4 months or 6 months would not be a quality, or QA workload issue. We
>> would have automated test suites written for 80+% of our testing and
>> we would run comprehensive tests against propose merges to guarantee
>> that an incoming features both works as intended, and doesn't harm the
>> rest of the platform in the process. We'd make release cycle decisions
>> based upon the same kind of preferences that make us choose a monkey
>> mascot over a polar bear mascot.
>>
>> That sadly isn't where we are today, but I do think it illuminates a
>> clear path of where we need to go. (more automated testing)
>>
>> So where do I fall on 4 months versus 6 months?
>>
>> I think we should stay the 4 month course. Heres why:
>> * We set the 4-month release cycle expectation already. In theory
>> we've been through 2 releases, but really only 4.1 - 4.0 was dominated
>> by getting into the incubator, getting releases okayed legally, etc.
>> It took us 7 months to release. I am loathe to change something we
>> only recently set, and have only been through once. I am also worried
>> about the signal that such rapid contrary decision changes would send
>> to our users.
>>
>> * I am worried about the impact to quality of our releases. Greater
>> introduction of features means an increased threat to stability. It
>> may not impact us in the 4.2 cycle, but without massive automated QA
>> it will impact us going forward in the 4.3 and later time frames.
>>
>
> Great point and I agree this can definitely be an issue.
>
>>
>> If you've made it thus far, I hope your tea/coffee was enjoyable.
>> Thanks for reading. :)
>
> Had to take 2 :).
>
>>
>> --David
>
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Nitin Mehta <Ni...@citrix.com>.
David - You have some compelling logic :)
Given the debate about 4 month to 6 month, lets make an educated decision
First - Lets hear from the "release managers" as to what they have to say
about the last 2 releases. What went well and what didn't. Chip - ?
Second - If we plan to do a 6 month release what would be the timelines in
place ? Like time allocated for what features going in the release,
feature discussion and coding, bug fixing, testing, documentation etc. If
I get a rough idea on that for the 6 month release it would help me
understand the differences.
At this point I would still +1 a 6 month release because it gives me good
time to discuss, code, unit test and bake my feature. From my experience
at this point this definitely makes more sense.
We can revisit the time line after a couple releases as we all become more
accustomed with the processes in place.
Thanks,
-Nitin
On 23/04/13 8:21 AM, "David Nalley" <da...@gnsa.us> wrote:
>On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
><an...@citrix.com> wrote:
>> Folks
>>
>> We started discussing 4 month v/s 6 month release cycle in a another
>>thread [1]. Since the subject of that thread was different, community
>>may not have participated in this important discussion fully. I am are
>>bringing this discussion to its own thread. Here is the summary so far
>>please refer to [1] for more details.
>>
>> Summary of discussion:
>> - Animesh pointed out the technical debt that we have accumulated so
>>far needs extra time to resolve
>> - David, Chip favor shorter release cycle of 4 month and keeping master
>>always stable and in good quality and enhancing automation as a solution
>>to reduce QA manual effort. A focused defect fixing activity may be
>>needed to reduce technical debt
>> - Will brought up several points in the discussion: He called out heavy
>>dependence on manual QA for a release and pointed out that manual QA may
>>not be always available to match up ACS release schedule. Release
>>overhead for 4 month release is still high and suggest that moving to 6
>>month will save on release overhead and that time can be used for
>>strengthening automation.
>> - Joe agrees partly in release overhead being significant for major
>>release
>>
>> If I missed out any important point please feel free to bring into the
>>thread.
>>
>> There were some other discussion in [1] on release planning conference
>>and chip's clarification on time based v/s feature based releases but we
>>will not discuss those in this thread. Community has agreed to
>>time-based release already.
>>
>> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
>>
>
>
>Hi Animesh:
>
>I thought I would add a few comments. To the folks reading - I
>apologize for the length. If you haven't started yet, you may want to
>get some coffee/tea. You've been warned. :)
>
>I think there are two concerns people think about when talking about
>changing the length of the release cycle.
>
>The first concern is workload. Generating a release has a certain
>fixed amount of overhead. Writing release notes, running votes, and a
>key part of the discussion currently is the QA cycle. Currently a
>large portion of our QA cycle is manual testing, which means we need
>lots of fingers on keyboards to get it done.
>
>The other concern is quality. We always want to try and put our best
>foot forward, and minimize bugs, yet the very act of developing
>software, and especially developing new features, introduces bugs.
>
>Before I start telling you my reasoning, I want to lay out something
>that only hit me tonight. We've talked about how extending the release
>cycle only extends the length of development. We've talked about
>keeping the length of the development period (e.g. pre-code freeze)
>the same, and essentially only extending the amount of time for QA.
>That works for the first release after a move from 4 to 6 months. It
>falls apart there after though. The problem is that at code freeze we
>branch. Master immediately becomes open for future feature
>development, and you've just extended it by an additional 2 months,
>because your cycle is that much longer.
>
>I personally don't think that either concern is addressed well by
>lengthening the cycle. Let me explain why
>
>On the quality front - the immediate threat to quality is instability
>inserted by additional development/new features/improvements. By
>increasing the length of time for that disruption to occur, we get
>more of it, as people will continue adding things that might have been
>deferred to a later release. We have code merged that, realistically,
>we won't know if it works properly until QA has gone in and verified
>it, perhaps months later. The more paths changed in a given release,
>the greater the propensity for failure.
Agreed, but merely changing the time duration doesn't really solve the
problem. More automation, having a staging branch would still do a better
job.
But with the current context what you say makes sense.
>
>On the overhead front - the overhead of writing release notes and
>generating releases goes down. In addition the 'basic' testing that QA
>does now is additional overhead in each cycle as well. What isn't
>overhead though is testing for new features. Because effectively
>longer cycles mean greater amount of codebase change, more features,
>etc. The actual testing in a given cycle must also increase.
>
>If we were really focused on absolute best quality we could deliver
>with our current tooling, I believe the easiest would be to make our
>release cycle the length of a single feature development period + the
>QA cycle (e.g. shortening the release cycle.)
>
>Is there a right or wrong answer? There shouldn't be. Our software
>should be deliverable at any point in the development cycle we choose.
>4 months or 6 months would not be a quality, or QA workload issue. We
>would have automated test suites written for 80+% of our testing and
>we would run comprehensive tests against propose merges to guarantee
>that an incoming features both works as intended, and doesn't harm the
>rest of the platform in the process. We'd make release cycle decisions
>based upon the same kind of preferences that make us choose a monkey
>mascot over a polar bear mascot.
>
>That sadly isn't where we are today, but I do think it illuminates a
>clear path of where we need to go. (more automated testing)
>
>So where do I fall on 4 months versus 6 months?
>
>I think we should stay the 4 month course. Heres why:
>* We set the 4-month release cycle expectation already. In theory
>we've been through 2 releases, but really only 4.1 - 4.0 was dominated
>by getting into the incubator, getting releases okayed legally, etc.
>It took us 7 months to release. I am loathe to change something we
>only recently set, and have only been through once. I am also worried
>about the signal that such rapid contrary decision changes would send
>to our users.
>
>* I am worried about the impact to quality of our releases. Greater
>introduction of features means an increased threat to stability. It
>may not impact us in the 4.2 cycle, but without massive automated QA
>it will impact us going forward in the 4.3 and later time frames.
>
Great point and I agree this can definitely be an issue.
>
>If you've made it thus far, I hope your tea/coffee was enjoyable.
>Thanks for reading. :)
Had to take 2 :).
>
>--David
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by David Nalley <da...@gnsa.us>.
On Mon, Apr 22, 2013 at 5:19 PM, Animesh Chaturvedi
<an...@citrix.com> wrote:
> Folks
>
> We started discussing 4 month v/s 6 month release cycle in a another thread [1]. Since the subject of that thread was different, community may not have participated in this important discussion fully. I am are bringing this discussion to its own thread. Here is the summary so far please refer to [1] for more details.
>
> Summary of discussion:
> - Animesh pointed out the technical debt that we have accumulated so far needs extra time to resolve
> - David, Chip favor shorter release cycle of 4 month and keeping master always stable and in good quality and enhancing automation as a solution to reduce QA manual effort. A focused defect fixing activity may be needed to reduce technical debt
> - Will brought up several points in the discussion: He called out heavy dependence on manual QA for a release and pointed out that manual QA may not be always available to match up ACS release schedule. Release overhead for 4 month release is still high and suggest that moving to 6 month will save on release overhead and that time can be used for strengthening automation.
> - Joe agrees partly in release overhead being significant for major release
>
> If I missed out any important point please feel free to bring into the thread.
>
> There were some other discussion in [1] on release planning conference and chip's clarification on time based v/s feature based releases but we will not discuss those in this thread. Community has agreed to time-based release already.
>
> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
>
Hi Animesh:
I thought I would add a few comments. To the folks reading - I
apologize for the length. If you haven't started yet, you may want to
get some coffee/tea. You've been warned. :)
I think there are two concerns people think about when talking about
changing the length of the release cycle.
The first concern is workload. Generating a release has a certain
fixed amount of overhead. Writing release notes, running votes, and a
key part of the discussion currently is the QA cycle. Currently a
large portion of our QA cycle is manual testing, which means we need
lots of fingers on keyboards to get it done.
The other concern is quality. We always want to try and put our best
foot forward, and minimize bugs, yet the very act of developing
software, and especially developing new features, introduces bugs.
Before I start telling you my reasoning, I want to lay out something
that only hit me tonight. We've talked about how extending the release
cycle only extends the length of development. We've talked about
keeping the length of the development period (e.g. pre-code freeze)
the same, and essentially only extending the amount of time for QA.
That works for the first release after a move from 4 to 6 months. It
falls apart there after though. The problem is that at code freeze we
branch. Master immediately becomes open for future feature
development, and you've just extended it by an additional 2 months,
because your cycle is that much longer.
I personally don't think that either concern is addressed well by
lengthening the cycle. Let me explain why
On the quality front - the immediate threat to quality is instability
inserted by additional development/new features/improvements. By
increasing the length of time for that disruption to occur, we get
more of it, as people will continue adding things that might have been
deferred to a later release. We have code merged that, realistically,
we won't know if it works properly until QA has gone in and verified
it, perhaps months later. The more paths changed in a given release,
the greater the propensity for failure.
On the overhead front - the overhead of writing release notes and
generating releases goes down. In addition the 'basic' testing that QA
does now is additional overhead in each cycle as well. What isn't
overhead though is testing for new features. Because effectively
longer cycles mean greater amount of codebase change, more features,
etc. The actual testing in a given cycle must also increase.
If we were really focused on absolute best quality we could deliver
with our current tooling, I believe the easiest would be to make our
release cycle the length of a single feature development period + the
QA cycle (e.g. shortening the release cycle.)
Is there a right or wrong answer? There shouldn't be. Our software
should be deliverable at any point in the development cycle we choose.
4 months or 6 months would not be a quality, or QA workload issue. We
would have automated test suites written for 80+% of our testing and
we would run comprehensive tests against propose merges to guarantee
that an incoming features both works as intended, and doesn't harm the
rest of the platform in the process. We'd make release cycle decisions
based upon the same kind of preferences that make us choose a monkey
mascot over a polar bear mascot.
That sadly isn't where we are today, but I do think it illuminates a
clear path of where we need to go. (more automated testing)
So where do I fall on 4 months versus 6 months?
I think we should stay the 4 month course. Heres why:
* We set the 4-month release cycle expectation already. In theory
we've been through 2 releases, but really only 4.1 - 4.0 was dominated
by getting into the incubator, getting releases okayed legally, etc.
It took us 7 months to release. I am loathe to change something we
only recently set, and have only been through once. I am also worried
about the signal that such rapid contrary decision changes would send
to our users.
* I am worried about the impact to quality of our releases. Greater
introduction of features means an increased threat to stability. It
may not impact us in the 4.2 cycle, but without massive automated QA
it will impact us going forward in the 4.3 and later time frames.
If you've made it thus far, I hope your tea/coffee was enjoyable.
Thanks for reading. :)
--David
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Noah Slater <ns...@apache.org>.
Thanks for the great summary Animesh. Agree with Chip. Great thread. :)
On 26 April 2013 01:11, Chip Childers <ch...@sungard.com> wrote:
> On Thu, Apr 25, 2013 at 05:02:05PM -0700, Animesh Chaturvedi wrote:
> > Let me attempt to summarize this thread, if I missed any glaring points
> feel free to bring them up
> >
> > 4 months:
> > Proponents (9): Chip, Alex, David, Noah, Hugo, Joe, Sebastian,
> Prasanna, Rohit
> > Reasoning:
> > * We have not given proper shot to 4 month cycle, this was just the
> first time. Level of automation has increased between 4.0 to 4.1 which lays
> groundwork for better automation
> > * Longer feature cycle will mean more features and bigger and more
> complex release
> > * Faster feedback loop to respond and address problems and shorter
> wait time for feature delivery
> >
> >
> > 6 months:
> > Proponents (12): Will, Animesh, Edison, Frank, Min, Ilya, Kelven,
> Edison, Sudha, Radhika, Nitin, Mice
> > Reasoning:
> > * ACS currently has heavy reliance on manual testing and majority
> of QA comes from 1 company. Shorter release cycle puts more dependence on
> timely availability of QA to keep up to quality goals
> > * ACS release is expected to be of good quality and support
> upgrades. Longer QA cycle will mean more soak time and better quality.
> > * Less overhead on release fixed cost work (release notes,
> generating release artifacts)
> > * Longer cycles also provides more flexibility in schedule for
> individuals in defect fixing
> >
> >
> > I still see there is difference of opinion and not a clear consensus
> with 12 out of 21 ( approx. 60%) preferring 6 months. But going by the
> argument of not having given proper shot to 4 month cycle I will say we can
> keep 4.2 as a 4 month cycle and pull in all effort to make it successful.
> If it turns out that we can work with 4 month schedule that's well and
> good otherwise we can bring this topic again based on the results of
> running 4 month cycle.
> >
> > If there is no objection I will proceed with creating 4.2 release page,
> dashboards etc. on Monday
> >
> > Thanks
> > Animesh
>
> Well summarized, and the right way forward when there is no consensus to
> change is to "stay the course". I'm quite happy that this didn't
> degenerate into a "holy war" [1] of sorts actually. Well debated folks.
>
> Yes, let's revisit after 4.2, and even possibly again after that.
>
> -chip
>
> [1] http://producingoss.com/en/common-pitfalls.html#holy-wars
>
--
NS
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Animesh Chaturvedi <an...@citrix.com>.
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Thursday, April 25, 2013 5:11 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
> On Thu, Apr 25, 2013 at 05:02:05PM -0700, Animesh Chaturvedi wrote:
> > Let me attempt to summarize this thread, if I missed any glaring
> > points feel free to bring them up
> >
> > 4 months:
> > Proponents (9): Chip, Alex, David, Noah, Hugo, Joe, Sebastian,
> > Prasanna, Rohit
> > Reasoning:
> > * We have not given proper shot to 4 month cycle, this was just the
> first time. Level of automation has increased between 4.0 to 4.1 which lays
> groundwork for better automation
> > * Longer feature cycle will mean more features and bigger and more
> complex release
> > * Faster feedback loop to respond and address problems and shorter
> wait time for feature delivery
> >
> >
> > 6 months:
> > Proponents (12): Will, Animesh, Edison, Frank, Min, Ilya, Kelven,
> > Edison, Sudha, Radhika, Nitin, Mice
> > Reasoning:
> > * ACS currently has heavy reliance on manual testing and majority of
> QA comes from 1 company. Shorter release cycle puts more dependence on
> timely availability of QA to keep up to quality goals
> > * ACS release is expected to be of good quality and support upgrades.
> Longer QA cycle will mean more soak time and better quality.
> > * Less overhead on release fixed cost work (release notes, generating
> release artifacts)
> > * Longer cycles also provides more flexibility in schedule for individuals
> in defect fixing
> >
> >
> > I still see there is difference of opinion and not a clear consensus with 12
> out of 21 ( approx. 60%) preferring 6 months. But going by the argument of
> not having given proper shot to 4 month cycle I will say we can keep 4.2 as a 4
> month cycle and pull in all effort to make it successful. If it turns out that we
> can work with 4 month schedule that's well and good otherwise we can bring
> this topic again based on the results of running 4 month cycle.
> >
> > If there is no objection I will proceed with creating 4.2 release
> > page, dashboards etc. on Monday
> >
> > Thanks
> > Animesh
>
> Well summarized, and the right way forward when there is no consensus to
> change is to "stay the course". I'm quite happy that this didn't degenerate
> into a "holy war" [1] of sorts actually. Well debated folks.
>
> Yes, let's revisit after 4.2, and even possibly again after that.
>
> -chip
[Animesh>] Ross is really the bible :)
>
> [1] http://producingoss.com/en/common-pitfalls.html#holy-wars
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Chip Childers <ch...@sungard.com>.
On Thu, Apr 25, 2013 at 05:02:05PM -0700, Animesh Chaturvedi wrote:
> Let me attempt to summarize this thread, if I missed any glaring points feel free to bring them up
>
> 4 months:
> Proponents (9): Chip, Alex, David, Noah, Hugo, Joe, Sebastian, Prasanna, Rohit
> Reasoning:
> * We have not given proper shot to 4 month cycle, this was just the first time. Level of automation has increased between 4.0 to 4.1 which lays groundwork for better automation
> * Longer feature cycle will mean more features and bigger and more complex release
> * Faster feedback loop to respond and address problems and shorter wait time for feature delivery
>
>
> 6 months:
> Proponents (12): Will, Animesh, Edison, Frank, Min, Ilya, Kelven, Edison, Sudha, Radhika, Nitin, Mice
> Reasoning:
> * ACS currently has heavy reliance on manual testing and majority of QA comes from 1 company. Shorter release cycle puts more dependence on timely availability of QA to keep up to quality goals
> * ACS release is expected to be of good quality and support upgrades. Longer QA cycle will mean more soak time and better quality.
> * Less overhead on release fixed cost work (release notes, generating release artifacts)
> * Longer cycles also provides more flexibility in schedule for individuals in defect fixing
>
>
> I still see there is difference of opinion and not a clear consensus with 12 out of 21 ( approx. 60%) preferring 6 months. But going by the argument of not having given proper shot to 4 month cycle I will say we can keep 4.2 as a 4 month cycle and pull in all effort to make it successful. If it turns out that we can work with 4 month schedule that's well and good otherwise we can bring this topic again based on the results of running 4 month cycle.
>
> If there is no objection I will proceed with creating 4.2 release page, dashboards etc. on Monday
>
> Thanks
> Animesh
Well summarized, and the right way forward when there is no consensus to
change is to "stay the course". I'm quite happy that this didn't
degenerate into a "holy war" [1] of sorts actually. Well debated folks.
Yes, let's revisit after 4.2, and even possibly again after that.
-chip
[1] http://producingoss.com/en/common-pitfalls.html#holy-wars
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by prasanna <ts...@apache.org>.
On 26 April 2013 05:32, Animesh Chaturvedi
<an...@citrix.com> wrote:
> Let me attempt to summarize this thread, if I missed any glaring points feel free to bring them up
>
> 4 months:
> Proponents (9): Chip, Alex, David, Noah, Hugo, Joe, Sebastian, Prasanna, Rohit
> Reasoning:
> * We have not given proper shot to 4 month cycle, this was just the first time. Level of automation has increased between 4.0 to 4.1 which lays groundwork for better automation
> * Longer feature cycle will mean more features and bigger and more complex release
> * Faster feedback loop to respond and address problems and shorter wait time for feature delivery
>
>
> 6 months:
> Proponents (12): Will, Animesh, Edison, Frank, Min, Ilya, Kelven, Edison, Sudha, Radhika, Nitin, Mice
I see Edison's stuffed two votes in there :)
Good Summary! Thanks,
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Animesh Chaturvedi <an...@citrix.com>.
Let me attempt to summarize this thread, if I missed any glaring points feel free to bring them up
4 months:
Proponents (9): Chip, Alex, David, Noah, Hugo, Joe, Sebastian, Prasanna, Rohit
Reasoning:
* We have not given proper shot to 4 month cycle, this was just the first time. Level of automation has increased between 4.0 to 4.1 which lays groundwork for better automation
* Longer feature cycle will mean more features and bigger and more complex release
* Faster feedback loop to respond and address problems and shorter wait time for feature delivery
6 months:
Proponents (12): Will, Animesh, Edison, Frank, Min, Ilya, Kelven, Edison, Sudha, Radhika, Nitin, Mice
Reasoning:
* ACS currently has heavy reliance on manual testing and majority of QA comes from 1 company. Shorter release cycle puts more dependence on timely availability of QA to keep up to quality goals
* ACS release is expected to be of good quality and support upgrades. Longer QA cycle will mean more soak time and better quality.
* Less overhead on release fixed cost work (release notes, generating release artifacts)
* Longer cycles also provides more flexibility in schedule for individuals in defect fixing
I still see there is difference of opinion and not a clear consensus with 12 out of 21 ( approx. 60%) preferring 6 months. But going by the argument of not having given proper shot to 4 month cycle I will say we can keep 4.2 as a 4 month cycle and pull in all effort to make it successful. If it turns out that we can work with 4 month schedule that's well and good otherwise we can bring this topic again based on the results of running 4 month cycle.
If there is no objection I will proceed with creating 4.2 release page, dashboards etc. on Monday
Thanks
Animesh
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Thursday, April 25, 2013 9:24 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] ACS Release 4 month v/s 6 month
>
> On Wed, Apr 24, 2013 at 12:22:58PM +0100, Noah Slater wrote:
> > I see where David is coming from.
> >
> > The longer you leave a release branch, the harder it becomes to QA,
> > the harder it comes to test, and the harder it becomes to release. As
> > has been mentioned already, you can think of this as a "release cost".
> > More regular releases keep complexity down, and reduce anxiety over
> > "will my feature make the next release?" (Only applicable in a
> > time-based system, like we have it.)
>
> Indeed. And frankly the longer the "QA" cycle, the less interest the
> community will have (seems to have) in resolving bugs from the pending
> feature release. People move on, naturally, to the next feature they want to
> work on.
>
> Frankly this is the reason that I feel like we are still waiting to ship 4.1.0.
>
> -chip
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Chip Childers <ch...@sungard.com>.
On Wed, Apr 24, 2013 at 12:22:58PM +0100, Noah Slater wrote:
> I see where David is coming from.
>
> The longer you leave a release branch, the harder it becomes to QA, the
> harder it comes to test, and the harder it becomes to release. As has been
> mentioned already, you can think of this as a "release cost". More regular
> releases keep complexity down, and reduce anxiety over "will my feature
> make the next release?" (Only applicable in a time-based system, like we
> have it.)
Indeed. And frankly the longer the "QA" cycle, the less interest the
community will have (seems to have) in resolving bugs from the pending
feature release. People move on, naturally, to the next feature they
want to work on.
Frankly this is the reason that I feel like we are still waiting to ship
4.1.0.
-chip
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Noah Slater <ns...@apache.org>.
I see where David is coming from.
The longer you leave a release branch, the harder it becomes to QA, the
harder it comes to test, and the harder it becomes to release. As has been
mentioned already, you can think of this as a "release cost". More regular
releases keep complexity down, and reduce anxiety over "will my feature
make the next release?" (Only applicable in a time-based system, like we
have it.)
There's also something else to consider. Releases are like the heartbeat of
a project. They stimulate activity, and they are pillars around which to
market the project and spread the word. Hence, they bring new contributors
to the project. We should be aiming to do as many of them as possible.
In fact, frequency of releases is a reasonable proxy indicator for project
health. Not that that there's much difference between 4 months and 6
months. (Now, 2 years might be a different matter...) But I'm just
illustrating a point here. :)
I think Joe hits the nail on the head when he points out that we haven't
really given it enough time to know whether 4 months is working for us. I
would suggest we give it give it a while longer, and then re-evaluate.
On 24 April 2013 06:30, Alex Huang <Al...@citrix.com> wrote:
> I side with Dave and Chip on this one. The 4 month dev cycle has forced
> CloudStack to own up to its problems in planning, testing, and automation.
> Until that has been settled, going to a longer release cycle is actually
> not beneficial to this community.
>
> --Alex
>
> > -----Original Message-----
> > From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> > Sent: Tuesday, April 23, 2013 2:50 AM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: [DISCUSS] ACS Release 4 month v/s 6 month
> >
> > Folks
> >
> > We started discussing 4 month v/s 6 month release cycle in a another
> thread
> > [1]. Since the subject of that thread was different, community may not
> have
> > participated in this important discussion fully. I am are bringing this
> > discussion to its own thread. Here is the summary so far please refer to
> [1]
> > for more details.
> >
> > Summary of discussion:
> > - Animesh pointed out the technical debt that we have accumulated so far
> > needs extra time to resolve
> > - David, Chip favor shorter release cycle of 4 month and keeping master
> > always stable and in good quality and enhancing automation as a solution
> to
> > reduce QA manual effort. A focused defect fixing activity may be needed
> to
> > reduce technical debt
> > - Will brought up several points in the discussion: He called out heavy
> > dependence on manual QA for a release and pointed out that manual QA
> > may not be always available to match up ACS release schedule. Release
> > overhead for 4 month release is still high and suggest that moving to 6
> month
> > will save on release overhead and that time can be used for
> strengthening
> > automation.
> > - Joe agrees partly in release overhead being significant for major
> release
> >
> > If I missed out any important point please feel free to bring into the
> thread.
> >
> > There were some other discussion in [1] on release planning conference
> and
> > chip's clarification on time based v/s feature based releases but we
> will not
> > discuss those in this thread. Community has agreed to time-based release
> > already.
> >
> > [1] http://markmail.org/thread/6suq2fhltdvgvcxd
>
>
--
NS
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Alex Huang <Al...@citrix.com>.
I side with Dave and Chip on this one. The 4 month dev cycle has forced CloudStack to own up to its problems in planning, testing, and automation. Until that has been settled, going to a longer release cycle is actually not beneficial to this community.
--Alex
> -----Original Message-----
> From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> Sent: Tuesday, April 23, 2013 2:50 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] ACS Release 4 month v/s 6 month
>
> Folks
>
> We started discussing 4 month v/s 6 month release cycle in a another thread
> [1]. Since the subject of that thread was different, community may not have
> participated in this important discussion fully. I am are bringing this
> discussion to its own thread. Here is the summary so far please refer to [1]
> for more details.
>
> Summary of discussion:
> - Animesh pointed out the technical debt that we have accumulated so far
> needs extra time to resolve
> - David, Chip favor shorter release cycle of 4 month and keeping master
> always stable and in good quality and enhancing automation as a solution to
> reduce QA manual effort. A focused defect fixing activity may be needed to
> reduce technical debt
> - Will brought up several points in the discussion: He called out heavy
> dependence on manual QA for a release and pointed out that manual QA
> may not be always available to match up ACS release schedule. Release
> overhead for 4 month release is still high and suggest that moving to 6 month
> will save on release overhead and that time can be used for strengthening
> automation.
> - Joe agrees partly in release overhead being significant for major release
>
> If I missed out any important point please feel free to bring into the thread.
>
> There were some other discussion in [1] on release planning conference and
> chip's clarification on time based v/s feature based releases but we will not
> discuss those in this thread. Community has agreed to time-based release
> already.
>
> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Mice Xia <mi...@tcloudcomputing.com>.
+1 for 6 months, leave buffer for testing and bug fixing
-Mice
-----Original Message-----
From: Radhika Puthiyetath [mailto:radhika.puthiyetath@citrix.com]
Sent: Tuesday, April 23, 2013 12:36 PM
To: dev@cloudstack.apache.org; cloudstack-dev@incubator.apache.org
Subject: RE: [DISCUSS] ACS Release 4 month v/s 6 month
+ 1 for 6 months
From documentation perspective, it gives the doc contributors ample time to play around with the features; accommodate all the review cycles (unit tests, technical, and editorial if that applies); fix more number of defects.
-----Original Message-----
From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
Sent: Tuesday, April 23, 2013 4:38 AM
To: dev@cloudstack.apache.org; cloudstack-dev@incubator.apache.org
Subject: RE: [DISCUSS] ACS Release 4 month v/s 6 month
> -----Original Message-----
> From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> Sent: Monday, April 22, 2013 2:20 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] ACS Release 4 month v/s 6 month
>
> Folks
>
> We started discussing 4 month v/s 6 month release cycle in a another
> thread [1]. Since the subject of that thread was different, community
> may not have participated in this important discussion fully. I am
> are bringing this discussion to its own thread. Here is the summary so
> far please refer to [1] for more details.
>
> Summary of discussion:
> - Animesh pointed out the technical debt that we have accumulated so
> far needs extra time to resolve
> - David, Chip favor shorter release cycle of 4 month and keeping
> master always stable and in good quality and enhancing automation as a
> solution to reduce QA manual effort. A focused defect fixing activity
> may be needed to reduce technical debt
> - Will brought up several points in the discussion: He called out
> heavy dependence on manual QA for a release and pointed out that
> manual QA may not be always available to match up ACS release
> schedule. Release overhead for 4 month release is still high and
> suggest that moving to 6 month will save on release overhead and that time can be used for strengthening automation.
> - Joe agrees partly in release overhead being significant for major
> release
>
> If I missed out any important point please feel free to bring into the thread.
>
> There were some other discussion in [1] on release planning conference
> and chip's clarification on time based v/s feature based releases but
> we will not discuss those in this thread. Community has agreed to
> time-based release already.
>
> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
[Animesh>] Please provide your input.
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Radhika Puthiyetath <ra...@citrix.com>.
+ 1 for 6 months
>From documentation perspective, it gives the doc contributors ample time to play around with the features; accommodate all the review cycles (unit tests, technical, and editorial if that applies); fix more number of defects.
-----Original Message-----
From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
Sent: Tuesday, April 23, 2013 4:38 AM
To: dev@cloudstack.apache.org; cloudstack-dev@incubator.apache.org
Subject: RE: [DISCUSS] ACS Release 4 month v/s 6 month
> -----Original Message-----
> From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> Sent: Monday, April 22, 2013 2:20 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] ACS Release 4 month v/s 6 month
>
> Folks
>
> We started discussing 4 month v/s 6 month release cycle in a another
> thread [1]. Since the subject of that thread was different, community
> may not have participated in this important discussion fully. I am
> are bringing this discussion to its own thread. Here is the summary so
> far please refer to [1] for more details.
>
> Summary of discussion:
> - Animesh pointed out the technical debt that we have accumulated so
> far needs extra time to resolve
> - David, Chip favor shorter release cycle of 4 month and keeping
> master always stable and in good quality and enhancing automation as a
> solution to reduce QA manual effort. A focused defect fixing activity
> may be needed to reduce technical debt
> - Will brought up several points in the discussion: He called out
> heavy dependence on manual QA for a release and pointed out that
> manual QA may not be always available to match up ACS release
> schedule. Release overhead for 4 month release is still high and
> suggest that moving to 6 month will save on release overhead and that time can be used for strengthening automation.
> - Joe agrees partly in release overhead being significant for major
> release
>
> If I missed out any important point please feel free to bring into the thread.
>
> There were some other discussion in [1] on release planning conference
> and chip's clarification on time based v/s feature based releases but
> we will not discuss those in this thread. Community has agreed to
> time-based release already.
>
> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
[Animesh>] Please provide your input.
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Min Chen <mi...@citrix.com>.
Same feelings here.
-min
On 4/22/13 4:18 PM, "Frank Zhang" <Fr...@citrix.com> wrote:
>Before we have most of tests automated, 4 months release cycle seems too
>tight to me
>
>> -----Original Message-----
>> From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
>> Sent: Monday, April 22, 2013 4:08 PM
>> To: dev@cloudstack.apache.org; cloudstack-dev@incubator.apache.org
>> Subject: RE: [DISCUSS] ACS Release 4 month v/s 6 month
>>
>>
>>
>> > -----Original Message-----
>> > From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
>> > Sent: Monday, April 22, 2013 2:20 PM
>> > To: cloudstack-dev@incubator.apache.org
>> > Subject: [DISCUSS] ACS Release 4 month v/s 6 month
>> >
>> > Folks
>> >
>> > We started discussing 4 month v/s 6 month release cycle in a another
>> > thread [1]. Since the subject of that thread was different, community
>> > may not have participated in this important discussion fully. I am
>> > are bringing this discussion to its own thread. Here is the summary so
>> > far please refer to [1] for more details.
>> >
>> > Summary of discussion:
>> > - Animesh pointed out the technical debt that we have accumulated so
>> > far needs extra time to resolve
>> > - David, Chip favor shorter release cycle of 4 month and keeping
>> > master always stable and in good quality and enhancing automation as a
>> > solution to reduce QA manual effort. A focused defect fixing activity
>> > may be needed to reduce technical debt
>> > - Will brought up several points in the discussion: He called out
>> > heavy dependence on manual QA for a release and pointed out that
>> > manual QA may not be always available to match up ACS release
>> > schedule. Release overhead for 4 month release is still high and
>> > suggest that moving to 6 month will save on release overhead and that
>> time can be used for strengthening automation.
>> > - Joe agrees partly in release overhead being significant for major
>> > release
>> >
>> > If I missed out any important point please feel free to bring into
>>the thread.
>> >
>> > There were some other discussion in [1] on release planning conference
>> > and chip's clarification on time based v/s feature based releases but
>> > we will not discuss those in this thread. Community has agreed to
>> > time-based release already.
>> >
>> > [1] http://markmail.org/thread/6suq2fhltdvgvcxd
>>
>> [Animesh>] Please provide your input.
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Frank Zhang <Fr...@citrix.com>.
Before we have most of tests automated, 4 months release cycle seems too tight to me
> -----Original Message-----
> From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> Sent: Monday, April 22, 2013 4:08 PM
> To: dev@cloudstack.apache.org; cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] ACS Release 4 month v/s 6 month
>
>
>
> > -----Original Message-----
> > From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> > Sent: Monday, April 22, 2013 2:20 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: [DISCUSS] ACS Release 4 month v/s 6 month
> >
> > Folks
> >
> > We started discussing 4 month v/s 6 month release cycle in a another
> > thread [1]. Since the subject of that thread was different, community
> > may not have participated in this important discussion fully. I am
> > are bringing this discussion to its own thread. Here is the summary so
> > far please refer to [1] for more details.
> >
> > Summary of discussion:
> > - Animesh pointed out the technical debt that we have accumulated so
> > far needs extra time to resolve
> > - David, Chip favor shorter release cycle of 4 month and keeping
> > master always stable and in good quality and enhancing automation as a
> > solution to reduce QA manual effort. A focused defect fixing activity
> > may be needed to reduce technical debt
> > - Will brought up several points in the discussion: He called out
> > heavy dependence on manual QA for a release and pointed out that
> > manual QA may not be always available to match up ACS release
> > schedule. Release overhead for 4 month release is still high and
> > suggest that moving to 6 month will save on release overhead and that
> time can be used for strengthening automation.
> > - Joe agrees partly in release overhead being significant for major
> > release
> >
> > If I missed out any important point please feel free to bring into the thread.
> >
> > There were some other discussion in [1] on release planning conference
> > and chip's clarification on time based v/s feature based releases but
> > we will not discuss those in this thread. Community has agreed to
> > time-based release already.
> >
> > [1] http://markmail.org/thread/6suq2fhltdvgvcxd
>
> [Animesh>] Please provide your input.
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Animesh Chaturvedi <an...@citrix.com>.
> -----Original Message-----
> From: Animesh Chaturvedi [mailto:animesh.chaturvedi@citrix.com]
> Sent: Monday, April 22, 2013 2:20 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] ACS Release 4 month v/s 6 month
>
> Folks
>
> We started discussing 4 month v/s 6 month release cycle in a another thread
> [1]. Since the subject of that thread was different, community may not have
> participated in this important discussion fully. I am are bringing this discussion
> to its own thread. Here is the summary so far please refer to [1] for more
> details.
>
> Summary of discussion:
> - Animesh pointed out the technical debt that we have accumulated so far
> needs extra time to resolve
> - David, Chip favor shorter release cycle of 4 month and keeping master always
> stable and in good quality and enhancing automation as a solution to reduce
> QA manual effort. A focused defect fixing activity may be needed to reduce
> technical debt
> - Will brought up several points in the discussion: He called out heavy
> dependence on manual QA for a release and pointed out that manual QA may
> not be always available to match up ACS release schedule. Release overhead
> for 4 month release is still high and suggest that moving to 6 month will save
> on release overhead and that time can be used for strengthening automation.
> - Joe agrees partly in release overhead being significant for major release
>
> If I missed out any important point please feel free to bring into the thread.
>
> There were some other discussion in [1] on release planning conference and
> chip's clarification on time based v/s feature based releases but we will not
> discuss those in this thread. Community has agreed to time-based release
> already.
>
> [1] http://markmail.org/thread/6suq2fhltdvgvcxd
[Animesh>] Please provide your input.
Re: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by Kelven Yang <ke...@citrix.com>.
Given the remaining work to reshape CloudStack into a better architecture
to response other "Stacks" (i.e., OpenStack) at architecture level in
addition to feature development. I prefer a 6 month release cycle.
Kelven
On 4/22/13 4:47 PM, "Musayev, Ilya" <im...@webmd.net> wrote:
>Animesh
>
>I personally believe 4 months cycle is too rapid and we need more time to
>Qa and fix all issues.
>
>I'm finding issues constantly that automated tests missed.
>
>My opinion, we should release a stable product when it's ready,
>hopefully following a 6 months cycle.
>
>Regards
>Ilya
>
>
>
>-------- Original message --------
>From: Animesh Chaturvedi <an...@citrix.com>
>Date:
>To: cloudstack-dev@incubator.apache.org
>Subject: [DISCUSS] ACS Release 4 month v/s 6 month
>
>
>Folks
>
>We started discussing 4 month v/s 6 month release cycle in a another
>thread [1]. Since the subject of that thread was different, community may
>not have participated in this important discussion fully. I am are
>bringing this discussion to its own thread. Here is the summary so far
>please refer to [1] for more details.
>
>Summary of discussion:
>- Animesh pointed out the technical debt that we have accumulated so far
>needs extra time to resolve
>- David, Chip favor shorter release cycle of 4 month and keeping master
>always stable and in good quality and enhancing automation as a solution
>to reduce QA manual effort. A focused defect fixing activity may be
>needed to reduce technical debt
>- Will brought up several points in the discussion: He called out heavy
>dependence on manual QA for a release and pointed out that manual QA may
>not be always available to match up ACS release schedule. Release
>overhead for 4 month release is still high and suggest that moving to 6
>month will save on release overhead and that time can be used for
>strengthening automation.
> - Joe agrees partly in release overhead being significant for major
>release
>
>If I missed out any important point please feel free to bring into the
>thread.
>
>There were some other discussion in [1] on release planning conference
>and chip's clarification on time based v/s feature based releases but we
>will not discuss those in this thread. Community has agreed to time-based
>release already.
>
>[1] http://markmail.org/thread/6suq2fhltdvgvcxd
>
>
RE: [DISCUSS] ACS Release 4 month v/s 6 month
Posted by "Musayev, Ilya" <im...@webmd.net>.
Animesh
I personally believe 4 months cycle is too rapid and we need more time to Qa and fix all issues.
I'm finding issues constantly that automated tests missed.
My opinion, we should release a stable product when it's ready, hopefully following a 6 months cycle.
Regards
Ilya
-------- Original message --------
From: Animesh Chaturvedi <an...@citrix.com>
Date:
To: cloudstack-dev@incubator.apache.org
Subject: [DISCUSS] ACS Release 4 month v/s 6 month
Folks
We started discussing 4 month v/s 6 month release cycle in a another thread [1]. Since the subject of that thread was different, community may not have participated in this important discussion fully. I am are bringing this discussion to its own thread. Here is the summary so far please refer to [1] for more details.
Summary of discussion:
- Animesh pointed out the technical debt that we have accumulated so far needs extra time to resolve
- David, Chip favor shorter release cycle of 4 month and keeping master always stable and in good quality and enhancing automation as a solution to reduce QA manual effort. A focused defect fixing activity may be needed to reduce technical debt
- Will brought up several points in the discussion: He called out heavy dependence on manual QA for a release and pointed out that manual QA may not be always available to match up ACS release schedule. Release overhead for 4 month release is still high and suggest that moving to 6 month will save on release overhead and that time can be used for strengthening automation.
- Joe agrees partly in release overhead being significant for major release
If I missed out any important point please feel free to bring into the thread.
There were some other discussion in [1] on release planning conference and chip's clarification on time based v/s feature based releases but we will not discuss those in this thread. Community has agreed to time-based release already.
[1] http://markmail.org/thread/6suq2fhltdvgvcxd