Re: In-place upgrades of Bigtop releases

[Konstantin Boudnik <co...@apache.org>]

Yes, Bigtop doesn't have any magic hooks to make the component upgrades
possible: if a component doesn't support the upgrade it is on them, not on us.

That said, there's nothing preventing us from having minor Bigtop releases
including just some of the component updates.

Cos
 
On Wed, Jul 27, 2016 at 01:17PM, Konstantinos Tsakalozos wrote:
> Hi everyone,
> 
> For sure, offering an in-place upgrade might not always be possible given
> that some Apache projects do not offer such an upgrade option. A
> side-by-side deployment with a traffic manager in front could handle the
> upgrade in some cases. However, a side-by-side deployment of components
> storing data on persistence storage is rather impractical, for instance you
> cannot have two HBase deployments. We need to make sure (and assist) the
> users follow the proper upgrade path as described by each project's
> documentation especially whenever storage is affected.
> 
> To a more practical point. I think we should clearly state the version of
> the components shipped with each Bigtop release. IMO the component version
> should be stated in each package form (deb, rpm, charm etc) and in the
> Bigtop release notes (I liked the table of the 1.0.0 release notes [0]). In
> this way the user would know what to expect if he upgrades to a new Bigtop
> version. In the case of charms we should make sure the user is aware of any
> upgrade operation that would happen due to a charm upgrade*. I wonder if it
> would be useful to keep the charms of each Bigtop release under their own
> namespace (eg cs:~bigtop-1.1.0/hbase) and promulgate the latest Bigtop has
> to offer. In this way we can continue to support any customers that do not
> want to (or cannot) upgrade. I guess we could do that for snapshots as well
> (eg cs:~bigtop-1.2.0-SNAPSHOT/hbase).
> 
> 
> Cheers,
> Konstantinos
> 
> * Our READMEs should also point to the right documentation version. For
> example the Kafka charm README build from BT 1.1.0 should point to the
> Kafka 8.1.1 documentation.
> 
> [0] https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.0.0+Release
> 
> 
> On Wed, Jul 20, 2016 at 3:10 AM, Antonio Rosales <
> antonio.rosales@canonical.com> wrote:
> 
> > On Tue, Jul 19, 2016 at 4:16 PM, Roman Shaposhnik <ro...@shaposhnik.org>
> > wrote:
> > > On Tue, Jul 19, 2016 at 2:58 PM, Cory Johns <co...@canonical.com>
> > wrote:
> > >> In the Juju charms we are adding to Apache Bigtop, one of the scenarios
> > we
> > >> want to handle is upgrading from one Bigtop release to another.  I
> > tried to
> > >> find documentation on how to upgrade a Bigtop release, but my google-fu
> > >> failed me.
> > >
> > > That's because I don't think it exists ;-)
> > >
> > > The biggest hurdle is the fact that you can really only upgrade at
> > stack-level,
> > > not individual component's level (the dependencies seep into the
> > > implementation).
> >
> > Is it correct to stay one can install bigtop-1.0.0 and then upgrade to
> > 1.1.0. If so, how is service specific upgrades between Bigtop versions
> > handled?  I think that is the root of the question, and the answer may
> > very well be it is service specific.  We just wanted to check if there
> > were any specific "hooks" we could call to invoke an in-place upgrade,
> > or migrate service/jobs/data to a newer version of the application.
> >
> > >
> > >> What is the recommended way to upgrade a deployment of Apache Bigtop to
> > a
> > >> new Bigtop release?  Does Bigtop support in-place upgrades, or is it
> > >> designed to do a side-by-side deployment of the new version and
> > transition
> > >> to that?
> > >
> > > Actually, that's a great question for you, Canonical guys. Isn't part
> > > of the reason
> > > you came up with the new packaging format (snap) the fact that
> > traditional DEBs
> > > make side-by-side installs of different versions of the same stack
> > > next to impossible?
> >
> > Snaps are intended to help encapsulate application dependencies and
> > provide a transactional method for updates. Both snaps and debs
> > provide a method for the user to install a specific version of the
> > app.  Where snaps excel is the transactional layer of dependencies a
> > given version of the application requires. Snaps also provide the
> > ability to just change the bits in the application needed for the
> > upgrade. In debs we can accomplish the same but it may require extra
> > work on an apt cache or modifying the package to require specific
> > version. These are just frameworks to model how an application should
> > be upgraded.  Debs or snaps can certainly be upgraded side-by-side or
> > in-place, but I think that is very dependent on how the application is
> > designed to be upgraded.
> >
> > In this context we would certainly like all upstreams to use snaps,
> > but today that isn't the case. Thus, we wanted to query folks here on
> > the best practices to upgrade applications provided by Bigtop. For
> > example, is there Bigtop best practice to upgrade Sqoop, Spark, or
> > Hadoop?  Or should we query the Apache specific project for best
> > practices on upgrading?  We wanted to ensure if there was a Bigtop
> > best practice for upgrades we were using that, and if not we wanted to
> > open a discussion on the best way to tackle that issue.
> >
> > Thoughts?
> >
> > -thanks,
> > Antonio
> >
> >
> > >
> > > So let me flip that question on you: what's Ubuntu's guidance on that?
> > With RPM
> > > you can at least try to do relocatable RPMs and install them under the
> > stack
> > > trees to facilitate rolling upgrades, with DEB, I don't think it is
> > possible.
> > >
> > > Thanks,
> > > Roman.
> >