You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Hugo Trippaers <HT...@schubergphilis.com> on 2013/02/03 08:24:44 UTC
[DISCUSS] Packaging in 4.1
Heya all,
As Devid already mentioned we met up at Build-A-Cloud-Day in Ghent with Wido, Noa and a few other folks. During this day (with lots of nice talks) we had a chance to sit down and discuss packaging. Something that Noa, Wido and myself were planning to do for a long time. Some other people joined in the discussion and with the help of the available black board (the event was held in a school ;-) ) we managed to sync our ideas on packaging the 4.1 release.
With our ideas synced we thought it time to bring our ideas to the list and ask for feedback from the community. We are also using our current time at FOSDEM to sync up with some of the people from the distros to see what they think of the new ideas, it would still be nice to have CloudStack packaged up and shipped with some of the distributions out there.
So the main goals of redoing packaging are getting rid of ant and waf completely. A secondary goal is to create a reference set of packages which in itself should be enough to get anyone going with CloudStack, but will hardly depend on the underlying distro. Real distro specific stuff should be handled by packagers from those distros. We just aim to provide a reference implementation.
Our goal is to have a reference implementation that will install on the following list of operating systems: CentOS 6.3, Ubuntu 12.04 and Fedora 18. This means that it will probably install and run on a lot more, but this is the set that we will test against (i'm using a jenkins system at the office to automatically build and install and these images will be used for the tests).
Next we will remove as much system dependencies as possible, so we will use maven to gather the dependencies and make sure that they are packaged and shipped with the RPMs. This makes for slightly bigger packages, but reduced the overhead of having to check each operating system and run the risk of version mismatched with versions of jar files present on the distro.
We also intend to change the name of the packages to cloudstack to make it perfectly clear what somebody is installing, this will also affect the location of several files like configuration files and scripts, but we plan to include symlinks for backwards compatibility. The feasibility of this will obviously be tested in the packaging street my collegues are building for me.
The planned packages for now are cloudstack-management, cloudstack-agent, cloudstack-common, cloudstack-cli, cloudstack-docs, cloudstack-awsapi and cloudstack-usage. You might already have seen these names in some of the checkins. I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name} and the main jar will be located there and any dependencies will be located in the lib directory beneath that location. With the exception of management which will be created as an exploded webapplication archive in the same directory. Scripts will be located in /usr/share/cloudstack-common/scripts and symlinks will be made to the previous locations for backwards compatibility.
I think these are the highlights of what we intend to do for release 4.1. We have a lot of plans for subsequenst release and on how to get us into distros. But for now we thought it prudent to focus on getting packages for the 4.1 release as son as possible and focus on other improvement later.
@Wido, @Noa, @David did i miss anything important?
Cheers,
Hugo
Re: [DISCUSS] Packaging in 4.1
Posted by Chip Childers <ch...@sungard.com>.
Thanks for bringing this back to the list Hugo! Hope you guys had a
good time at FOSDEM.
On Sun, Feb 3, 2013 at 2:24 AM, Hugo Trippaers
<HT...@schubergphilis.com> wrote:
> Heya all,
>
> As Devid already mentioned we met up at Build-A-Cloud-Day in Ghent with Wido, Noa and a few other folks. During this day (with lots of nice talks) we had a chance to sit down and discuss packaging. Something that Noa, Wido and myself were planning to do for a long time. Some other people joined in the discussion and with the help of the available black board (the event was held in a school ;-) ) we managed to sync our ideas on packaging the 4.1 release.
>
> With our ideas synced we thought it time to bring our ideas to the list and ask for feedback from the community. We are also using our current time at FOSDEM to sync up with some of the people from the distros to see what they think of the new ideas, it would still be nice to have CloudStack packaged up and shipped with some of the distributions out there.
>
> So the main goals of redoing packaging are getting rid of ant and waf completely. A secondary goal is to create a reference set of packages which in itself should be enough to get anyone going with CloudStack, but will hardly depend on the underlying distro. Real distro specific stuff should be handled by packagers from those distros. We just aim to provide a reference implementation.
>
> Our goal is to have a reference implementation that will install on the following list of operating systems: CentOS 6.3, Ubuntu 12.04 and Fedora 18. This means that it will probably install and run on a lot more, but this is the set that we will test against (i'm using a jenkins system at the office to automatically build and install and these images will be used for the tests).
>
> Next we will remove as much system dependencies as possible, so we will use maven to gather the dependencies and make sure that they are packaged and shipped with the RPMs. This makes for slightly bigger packages, but reduced the overhead of having to check each operating system and run the risk of version mismatched with versions of jar files present on the distro.
>
> We also intend to change the name of the packages to cloudstack to make it perfectly clear what somebody is installing, this will also affect the location of several files like configuration files and scripts, but we plan to include symlinks for backwards compatibility. The feasibility of this will obviously be tested in the packaging street my collegues are building for me.
>
> The planned packages for now are cloudstack-management, cloudstack-agent, cloudstack-common, cloudstack-cli, cloudstack-docs, cloudstack-awsapi and cloudstack-usage. You might already have seen these names in some of the checkins. I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name} and the main jar will be located there and any dependencies will be located in the lib directory beneath that location. With the exception of management which will be created as an exploded webapplication archive in the same directory. Scripts will be located in /usr/share/cloudstack-common/scripts and symlinks will be made to the previous locations for backwards compatibility.
>
> I think these are the highlights of what we intend to do for release 4.1. We have a lot of plans for subsequenst release and on how to get us into distros. But for now we thought it prudent to focus on getting packages for the 4.1 release as son as possible and focus on other improvement later.
>
+1 for the approaches mentioned. I love the idea that our packages
are the "reference" packages (which we test and have faith in), but
that we concurrently work with the distros to get the code into their
packaging schemes.
> @Wido, @Noa, @David did i miss anything important?
>
> Cheers,
>
> Hugo
>
>
RE: [DISCUSS] Packaging in 4.1
Posted by Hugo Trippaers <HT...@schubergphilis.com>.
Hey Sudha,
I'm working on this as much as my $dayjob allows, but I can't give an estimate yet.
This old systems was already broken and did not create any useful packages, so I removed even the last remnants. For the moment I think the team will have to work from the maven builds until we have this done and integrated in the 4.1 branch.
Cheers,
Hugo
> -----Original Message-----
> From: Sudha Ponnaganti [mailto:sudha.ponnaganti@citrix.com]
> Sent: Monday, February 04, 2013 7:13 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] Packaging in 4.1
>
> Wanted to check when would this be implemented?? Several QA folks
> depend on the packages and need this working.
> We have been still building using waf but today that is also not working as
> some references are removed.
>
> Is it possible to accelerate this process or leave old way of packaging in place
> till you are done with the new changes
>
> Thanks
> /sudha
>
> -----Original Message-----
> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com] On Behalf
> Of Rohit Yadav
> Sent: Sunday, February 03, 2013 5:14 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
> > On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
> >> ...
> >>>
> >>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
> >>> worth than clint (clint is in EPEL, but no new version of pygments
> >>> in
> >>> EPEL/CentOS-Extras/CentOS-Plus)
> >>
> >> I want people to use pip to install the cli because it's the easiest
> >> and because rpm/deb packages may have dependency issues like you
> >> mentioned => may not work on all distros, what we can do is when
> >> people install cloudstack-cli rpm or deb, it runs a script that
> >> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
> >> python, so the rpm/deb can also ship bundling src tarballs of
> >> cloudmonkey and its dependencies and install from it. Advise best way
> >> of doing this?
> >
> > I guess we won't be installing the CLI via RPMs at least for EL6.
> >
> > You are assuming that they would have internet access when installing
> > - which is not a valid assumption.
> >
> > Honestly, the above idea makes me blanch. A package that reports as
> > installed, and may or may not have installed - may have installed a
> > compromised package (see rubygems.org compromise recently,
> kernel.org,
> > and a number of other site compromises.), or might have installed
> > packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
> > know you are installing some of the dependencies, there is no record
> > of those packages in the package manager, and there might potentially
> > be conflicts with system packages, a security vulnerability in one of
> > those dependencies wouldn't be caught on audit, etc etc.
>
> /facepalm\, it's just a problem of packaging. The package can include cli or
> any other artifact's dependencies, so in case of cli, you bundle both
> pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure python
> so easily installable. The cool people can use pip to install.
>
> >
> > And I really don't intend for this to sound like a rant, but the one
> > of the important benefits behind using packages and a package manager
> > is that a sysadmin needs (and often is required to have by government
> > regulations) a single source of truth about the software installed on
> > a machine.
>
> No, it's not a rant, I understand.
>
> > Developers love things like Maven central, pypi, CPAN, and rubygems,
> > and for good reason, they are fast, flexible, and make their life
> > easy. To a sysadmin managing machines in production, they are
> > anathema; they make system state difficult or impossible to determine,
> > they make audits painful.
>
> I just assumed the sysadmin who would install CloudStack, cli and whatnot
> won't be stupid, at the same time I want his life to be less miserable.
>
> > In addition they make troubleshooting
> > incredibly difficult. Do I have $foo installed - which version? Are
> > there multiple copies of $foo installed on the system? Which one is
> > actually being called/loaded?
>
> Alright, but I'm talking only about the cli, since most users, admins included,
> would want it to run on their machines, the installation should be easiest,
> that's why I said they can use pip, so it works on their windows, osx, linux,
> bsd boxes and that's why it's pure python (written that way).
>
> Regards.
> >
> > --David
Re: [DISCUSS] Packaging in 4.1
Posted by Henri Gomez <he...@gmail.com>.
Well, some distributions are very stricts with binary contents in
packages, ie Debian, but not all.
BTW, there is no need to have RPM or DEB in distributions, as they
could be provided by Cloudstack as companion/extras packages
(yum/zypper, ppa).
For RPM distros, openSUSE Build System (https://build.opensuse.org/),
provide required build infrastructure.
This build/package factory may be also set elsewhere (may be even ASF)
and is not too complex to set (Jenkins, slaves for distros).
I'll be happy to help in such area if needed.
Cheers
2013/2/4 David Nalley <da...@gnsa.us>:
> On Mon, Feb 4, 2013 at 3:22 PM, Henri Gomez <he...@gmail.com> wrote:
>> Nice to see native packages for Cloudstack :
>>
>> https://github.com/noaresare/incubator-cloudstack/tree/master/packaging
>>
>> CentOS 6.3 and Debian.
>>
>>
>> Did there is plan to add Fedora, openSUSE, SLES or will it be done by
>> these distributions packagers ?
>>
>
> So we have some openSuSE/SLES packagers on the list, and some Fedora
> packagers as well (me at least) While we want them to be packaged, I
> think the primary focus atm, is getting things working again.
> Essentially the packages that are being created would be unacceptable
> for inclusion in most distributions because we bundle jars. That
> doesn't mean we can't publish repos that are consumable by those
> distributions, but is another point to consider.
>
> --David
Re: [DISCUSS] Packaging in 4.1
Posted by Henri Gomez <he...@gmail.com>.
Did bigtop infra provide a large choice of Linux distros ?
Better build packages for distro X under distro X.
2013/2/5 Andrew Bayer <an...@gmail.com>:
> I should probably pipe up with my suggestion of getting CloudStack into
> Apache Bigtop here. =) http://bigtop.apache.org/ - it's mainly focused on
> the Hadoop ecosystem currently, but at the very least it'd provide an
> infrastructure, some consistent guidelines on builds/layout/etc.
>
> A.
>
> On Mon, Feb 4, 2013 at 12:29 PM, David Nalley <da...@gnsa.us> wrote:
>
>> On Mon, Feb 4, 2013 at 3:22 PM, Henri Gomez <he...@gmail.com> wrote:
>> > Nice to see native packages for Cloudstack :
>> >
>> > https://github.com/noaresare/incubator-cloudstack/tree/master/packaging
>> >
>> > CentOS 6.3 and Debian.
>> >
>> >
>> > Did there is plan to add Fedora, openSUSE, SLES or will it be done by
>> > these distributions packagers ?
>> >
>>
>> So we have some openSuSE/SLES packagers on the list, and some Fedora
>> packagers as well (me at least) While we want them to be packaged, I
>> think the primary focus atm, is getting things working again.
>> Essentially the packages that are being created would be unacceptable
>> for inclusion in most distributions because we bundle jars. That
>> doesn't mean we can't publish repos that are consumable by those
>> distributions, but is another point to consider.
>>
>> --David
>>
Re: [DISCUSS] Packaging in 4.1
Posted by Andrew Bayer <an...@gmail.com>.
I should probably pipe up with my suggestion of getting CloudStack into
Apache Bigtop here. =) http://bigtop.apache.org/ - it's mainly focused on
the Hadoop ecosystem currently, but at the very least it'd provide an
infrastructure, some consistent guidelines on builds/layout/etc.
A.
On Mon, Feb 4, 2013 at 12:29 PM, David Nalley <da...@gnsa.us> wrote:
> On Mon, Feb 4, 2013 at 3:22 PM, Henri Gomez <he...@gmail.com> wrote:
> > Nice to see native packages for Cloudstack :
> >
> > https://github.com/noaresare/incubator-cloudstack/tree/master/packaging
> >
> > CentOS 6.3 and Debian.
> >
> >
> > Did there is plan to add Fedora, openSUSE, SLES or will it be done by
> > these distributions packagers ?
> >
>
> So we have some openSuSE/SLES packagers on the list, and some Fedora
> packagers as well (me at least) While we want them to be packaged, I
> think the primary focus atm, is getting things working again.
> Essentially the packages that are being created would be unacceptable
> for inclusion in most distributions because we bundle jars. That
> doesn't mean we can't publish repos that are consumable by those
> distributions, but is another point to consider.
>
> --David
>
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Mon, Feb 4, 2013 at 3:22 PM, Henri Gomez <he...@gmail.com> wrote:
> Nice to see native packages for Cloudstack :
>
> https://github.com/noaresare/incubator-cloudstack/tree/master/packaging
>
> CentOS 6.3 and Debian.
>
>
> Did there is plan to add Fedora, openSUSE, SLES or will it be done by
> these distributions packagers ?
>
So we have some openSuSE/SLES packagers on the list, and some Fedora
packagers as well (me at least) While we want them to be packaged, I
think the primary focus atm, is getting things working again.
Essentially the packages that are being created would be unacceptable
for inclusion in most distributions because we bundle jars. That
doesn't mean we can't publish repos that are consumable by those
distributions, but is another point to consider.
--David
Re: [DISCUSS] Packaging in 4.1
Posted by Henri Gomez <he...@gmail.com>.
Nice to see native packages for Cloudstack :
https://github.com/noaresare/incubator-cloudstack/tree/master/packaging
CentOS 6.3 and Debian.
Did there is plan to add Fedora, openSUSE, SLES or will it be done by
these distributions packagers ?
2013/2/4 Wido den Hollander <wi...@widodh.nl>:
> Hey,
>
> I just pushed everything to the Github repo[0]:
>
> dpkg-genchanges -b >../cloudstack_4.1.0-0.0.snapshot+2_amd64.changes
> dpkg-genchanges: binary-only upload - not including any source code
> dpkg-source --after-build cloudstack
> dpkg-buildpackage: binary only upload (no source included)
>
> wido@wido-desktop:~/repos/cloudstack$ ls -l ../*.deb
> -rw-r--r-- 1 wido wido 176298 Feb 4 21:14
> ../cloudstack-agent_4.1.0-0.0.snapshot+2_all.deb
> -rw-r--r-- 1 wido wido 2382 Feb 4 21:14
> ../cloudstack-awsapi_4.1.0-0.0.snapshot+2_all.deb
> -rw-r--r-- 1 wido wido 2302 Feb 4 21:14
> ../cloudstack-cli_4.1.0-0.0.snapshot+2_all.deb
> -rw-r--r-- 1 wido wido 24629096 Feb 4 21:14
> ../cloudstack-common_4.1.0-0.0.snapshot+2_all.deb
> -rw-r--r-- 1 wido wido 2280 Feb 4 21:14
> ../cloudstack-docs_4.1.0-0.0.snapshot+2_all.deb
> -rw-r--r-- 1 wido wido 85635152 Feb 4 21:14
> ../cloudstack-management_4.1.0-0.0.snapshot+2_all.deb
> -rw-r--r-- 1 wido wido 91854 Feb 4 21:14
> ../cloudstack-usage_4.1.0-0.0.snapshot+2_all.deb
> wido@wido-desktop:~/repos/cloudstack$
>
> I can't guarantee that they work for 100%, but it should be a start!
>
> Wido
>
> [0]: https://github.com/noaresare/incubator-cloudstack/commits/packaging
>
>
> On 02/04/2013 06:04 PM, Wido den Hollander wrote:
>>
>>
>>
>> On 02/04/2013 06:03 PM, Hugo Trippaers wrote:
>>>
>>> Hey guys,
>>>
>>> What do we do with the database scripts? They are needed by
>>> cloud(stack)-setup-management. I'm thinking about adding them to the
>>> management server for now together with the scripts. There are some
>>> python modules (cloudutils and cloud_utils) which are needed by both
>>> setup tools in management and agent to I thought to put those in common.
>>>
>>
>> Ran into the same! Had the idea to put them indeed in the management
>> server package.
>>
>> The python libs should indeed go into common.
>>
>> Wido
>>
>>> Cheers,
>>>
>>> Hugo
>>>
>>>> -----Original Message-----
>>>> From: Wido den Hollander [mailto:wido@widodh.nl]
>>>> Sent: Monday, February 04, 2013 2:48 PM
>>>> To: cloudstack-dev@incubator.apache.org
>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>
>>>> On 02/04/2013 02:39 PM, Noa Resare wrote:
>>>>>
>>>>> I'm on the last leg of my trip home from FOSDEM right now, I hope to
>>>>> be able to put in some work on deb packages in the next few days.
>>>>>
>>>>
>>>> I still have some pending changes on my laptop which I forgot to push
>>>> to the
>>>> Github repo[0].
>>>>
>>>> I'll push them in a couple of hours, they contain a lot of what we
>>>> discussed
>>>> regarding the DEB packaging.
>>>>
>>>> Wido
>>>>
>>>> [0]: https://github.com/noaresare/incubator-cloudstack
>>>>
>>>>> /n
>>>>>
>>>>>
>>>>> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl>
>>>>
>>>> wrote:
>>>>>
>>>>>
>>>>>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>>>>>
>>>>>>> Wanted to check when would this be implemented?? Several QA folks
>>>>>>> depend on the packages and need this working.
>>>>>>> We have been still building using waf but today that is also not
>>>>>>> working as some references are removed.
>>>>>>>
>>>>>>> Is it possible to accelerate this process or leave old way of
>>>>>>> packaging in place till you are done with the new changes
>>>>>>>
>>>>>>>
>>>>>> I fully understand. As I told David, I think the DEB work is about
>>>>>> one day of work, but then again, there is something like $dayjob.
>>>>>>
>>>>>> What might be even tougher is to get the RPM and DEB packages fully
>>>>>> synced. cloudstack-common for example should contain exactly the same
>>>>>> files in the RPM and DEB version, so Hugo and I will have to keep
>>>>>> in touch.
>>>>>>
>>>>>> I really want to have the DEB packaging working this week, period.
>>>>>>
>>>>>> Wido
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>>>
>>>>>>> /sudha
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
>>>>>>> Behalf Of Rohit Yadav
>>>>>>> Sent: Sunday, February 03, 2013 5:14 PM
>>>>>>> To:
>>>>>>> cloudstack-dev@incubator.**apache.org<cloudstack-
>>>>
>>>> dev@incubator.apach
>>>>>>>
>>>>>>> e.org>
>>>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>>>
>>>>>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>>>>>
>>>>>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
>>>>
>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>>>>>> worth than clint (clint is in EPEL, but no new version of
>>>>>>>>>> pygments in
>>>>>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I want people to use pip to install the cli because it's the
>>>>>>>>> easiest and because rpm/deb packages may have dependency issues
>>>>>>>>> like you mentioned => may not work on all distros, what we can do
>>>>>>>>> is when people install cloudstack-cli rpm or deb, it runs a script
>>>>>>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
>>>>>>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
>>>>>>>>> cloudmonkey and its dependencies and install from it. Advise best
>>>>>>>>> way of doing this?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>>>>>
>>>>>>>> You are assuming that they would have internet access when
>>>>>>>> installing
>>>>>>>> - which is not a valid assumption.
>>>>>>>>
>>>>>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>>>>>> installed, and may or may not have installed - may have installed a
>>>>>>>> compromised package (see rubygems.org compromise recently,
>>>>>>>> kernel.org, and a number of other site compromises.), or might have
>>>>>>>> installed packages I didn't know about is a Bad Idea (tm) The
>>>>>>>> sysadmin doesn't know you are installing some of the dependencies,
>>>>>>>> there is no record of those packages in the package manager, and
>>>>>>>> there might potentially be conflicts with system packages, a
>>>>>>>> security vulnerability in one of those dependencies wouldn't be
>>>>>>>> caught
>>>>
>>>> on audit, etc etc.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> /facepalm\, it's just a problem of packaging. The package can
>>>>>>> include cli or any other artifact's dependencies, so in case of cli,
>>>>>>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
>>>>>>> of them are pure python so easily installable. The cool people can
>>>>>>> use
>>>>
>>>> pip to install.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> And I really don't intend for this to sound like a rant, but the
>>>>>>>> one of the important benefits behind using packages and a package
>>>>>>>> manager is that a sysadmin needs (and often is required to have by
>>>>>>>> government
>>>>>>>> regulations) a single source of truth about the software installed
>>>>>>>> on a machine.
>>>>>>>>
>>>>>>>
>>>>>>> No, it's not a rant, I understand.
>>>>>>>
>>>>>>> Developers love things like Maven central, pypi, CPAN, and
>>>>>>> rubygems,
>>>>>>>>
>>>>>>>> and for good reason, they are fast, flexible, and make their life
>>>>>>>> easy. To a sysadmin managing machines in production, they are
>>>>>>>> anathema; they make system state difficult or impossible to
>>>>>>>> determine, they make audits painful.
>>>>>>>>
>>>>>>>
>>>>>>> I just assumed the sysadmin who would install CloudStack, cli and
>>>>>>> whatnot won't be stupid, at the same time I want his life to be less
>>>>
>>>> miserable.
>>>>>>>
>>>>>>>
>>>>>>> In addition they make troubleshooting
>>>>>>>>
>>>>>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>>>>>> there multiple copies of $foo installed on the system? Which one is
>>>>>>>> actually being called/loaded?
>>>>>>>>
>>>>>>>
>>>>>>> Alright, but I'm talking only about the cli, since most users,
>>>>>>> admins included, would want it to run on their machines, the
>>>>>>> installation should be easiest, that's why I said they can use pip,
>>>>>>> so it works on their windows, osx, linux, bsd boxes and that's why
>>>>>>> it's pure python (written that way).
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>>>
>>>>>>>> --David
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
Hey,
I just pushed everything to the Github repo[0]:
dpkg-genchanges -b >../cloudstack_4.1.0-0.0.snapshot+2_amd64.changes
dpkg-genchanges: binary-only upload - not including any source code
dpkg-source --after-build cloudstack
dpkg-buildpackage: binary only upload (no source included)
wido@wido-desktop:~/repos/cloudstack$ ls -l ../*.deb
-rw-r--r-- 1 wido wido 176298 Feb 4 21:14
../cloudstack-agent_4.1.0-0.0.snapshot+2_all.deb
-rw-r--r-- 1 wido wido 2382 Feb 4 21:14
../cloudstack-awsapi_4.1.0-0.0.snapshot+2_all.deb
-rw-r--r-- 1 wido wido 2302 Feb 4 21:14
../cloudstack-cli_4.1.0-0.0.snapshot+2_all.deb
-rw-r--r-- 1 wido wido 24629096 Feb 4 21:14
../cloudstack-common_4.1.0-0.0.snapshot+2_all.deb
-rw-r--r-- 1 wido wido 2280 Feb 4 21:14
../cloudstack-docs_4.1.0-0.0.snapshot+2_all.deb
-rw-r--r-- 1 wido wido 85635152 Feb 4 21:14
../cloudstack-management_4.1.0-0.0.snapshot+2_all.deb
-rw-r--r-- 1 wido wido 91854 Feb 4 21:14
../cloudstack-usage_4.1.0-0.0.snapshot+2_all.deb
wido@wido-desktop:~/repos/cloudstack$
I can't guarantee that they work for 100%, but it should be a start!
Wido
[0]: https://github.com/noaresare/incubator-cloudstack/commits/packaging
On 02/04/2013 06:04 PM, Wido den Hollander wrote:
>
>
> On 02/04/2013 06:03 PM, Hugo Trippaers wrote:
>> Hey guys,
>>
>> What do we do with the database scripts? They are needed by
>> cloud(stack)-setup-management. I'm thinking about adding them to the
>> management server for now together with the scripts. There are some
>> python modules (cloudutils and cloud_utils) which are needed by both
>> setup tools in management and agent to I thought to put those in common.
>>
>
> Ran into the same! Had the idea to put them indeed in the management
> server package.
>
> The python libs should indeed go into common.
>
> Wido
>
>> Cheers,
>>
>> Hugo
>>
>>> -----Original Message-----
>>> From: Wido den Hollander [mailto:wido@widodh.nl]
>>> Sent: Monday, February 04, 2013 2:48 PM
>>> To: cloudstack-dev@incubator.apache.org
>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>
>>> On 02/04/2013 02:39 PM, Noa Resare wrote:
>>>> I'm on the last leg of my trip home from FOSDEM right now, I hope to
>>>> be able to put in some work on deb packages in the next few days.
>>>>
>>>
>>> I still have some pending changes on my laptop which I forgot to push
>>> to the
>>> Github repo[0].
>>>
>>> I'll push them in a couple of hours, they contain a lot of what we
>>> discussed
>>> regarding the DEB packaging.
>>>
>>> Wido
>>>
>>> [0]: https://github.com/noaresare/incubator-cloudstack
>>>
>>>> /n
>>>>
>>>>
>>>> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl>
>>> wrote:
>>>>
>>>>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>>>>
>>>>>> Wanted to check when would this be implemented?? Several QA folks
>>>>>> depend on the packages and need this working.
>>>>>> We have been still building using waf but today that is also not
>>>>>> working as some references are removed.
>>>>>>
>>>>>> Is it possible to accelerate this process or leave old way of
>>>>>> packaging in place till you are done with the new changes
>>>>>>
>>>>>>
>>>>> I fully understand. As I told David, I think the DEB work is about
>>>>> one day of work, but then again, there is something like $dayjob.
>>>>>
>>>>> What might be even tougher is to get the RPM and DEB packages fully
>>>>> synced. cloudstack-common for example should contain exactly the same
>>>>> files in the RPM and DEB version, so Hugo and I will have to keep
>>>>> in touch.
>>>>>
>>>>> I really want to have the DEB packaging working this week, period.
>>>>>
>>>>> Wido
>>>>>
>>>>>
>>>>> Thanks
>>>>>> /sudha
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
>>>>>> Behalf Of Rohit Yadav
>>>>>> Sent: Sunday, February 03, 2013 5:14 PM
>>>>>> To:
>>>>>> cloudstack-dev@incubator.**apache.org<cloudstack-
>>> dev@incubator.apach
>>>>>> e.org>
>>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>>
>>>>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>>>>
>>>>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
>>> wrote:
>>>>>>>
>>>>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>>>>> ...
>>>>>>>>
>>>>>>>>>
>>>>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>>>>> worth than clint (clint is in EPEL, but no new version of
>>>>>>>>> pygments in
>>>>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>>>>
>>>>>>>>
>>>>>>>> I want people to use pip to install the cli because it's the
>>>>>>>> easiest and because rpm/deb packages may have dependency issues
>>>>>>>> like you mentioned => may not work on all distros, what we can do
>>>>>>>> is when people install cloudstack-cli rpm or deb, it runs a script
>>>>>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
>>>>>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
>>>>>>>> cloudmonkey and its dependencies and install from it. Advise best
>>>>>>>> way of doing this?
>>>>>>>>
>>>>>>>
>>>>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>>>>
>>>>>>> You are assuming that they would have internet access when
>>>>>>> installing
>>>>>>> - which is not a valid assumption.
>>>>>>>
>>>>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>>>>> installed, and may or may not have installed - may have installed a
>>>>>>> compromised package (see rubygems.org compromise recently,
>>>>>>> kernel.org, and a number of other site compromises.), or might have
>>>>>>> installed packages I didn't know about is a Bad Idea (tm) The
>>>>>>> sysadmin doesn't know you are installing some of the dependencies,
>>>>>>> there is no record of those packages in the package manager, and
>>>>>>> there might potentially be conflicts with system packages, a
>>>>>>> security vulnerability in one of those dependencies wouldn't be
>>>>>>> caught
>>> on audit, etc etc.
>>>>>>>
>>>>>>
>>>>>> /facepalm\, it's just a problem of packaging. The package can
>>>>>> include cli or any other artifact's dependencies, so in case of cli,
>>>>>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
>>>>>> of them are pure python so easily installable. The cool people can
>>>>>> use
>>> pip to install.
>>>>>>
>>>>>>
>>>>>>> And I really don't intend for this to sound like a rant, but the
>>>>>>> one of the important benefits behind using packages and a package
>>>>>>> manager is that a sysadmin needs (and often is required to have by
>>>>>>> government
>>>>>>> regulations) a single source of truth about the software installed
>>>>>>> on a machine.
>>>>>>>
>>>>>>
>>>>>> No, it's not a rant, I understand.
>>>>>>
>>>>>> Developers love things like Maven central, pypi, CPAN, and
>>>>>> rubygems,
>>>>>>> and for good reason, they are fast, flexible, and make their life
>>>>>>> easy. To a sysadmin managing machines in production, they are
>>>>>>> anathema; they make system state difficult or impossible to
>>>>>>> determine, they make audits painful.
>>>>>>>
>>>>>>
>>>>>> I just assumed the sysadmin who would install CloudStack, cli and
>>>>>> whatnot won't be stupid, at the same time I want his life to be less
>>> miserable.
>>>>>>
>>>>>> In addition they make troubleshooting
>>>>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>>>>> there multiple copies of $foo installed on the system? Which one is
>>>>>>> actually being called/loaded?
>>>>>>>
>>>>>>
>>>>>> Alright, but I'm talking only about the cli, since most users,
>>>>>> admins included, would want it to run on their machines, the
>>>>>> installation should be easiest, that's why I said they can use pip,
>>>>>> so it works on their windows, osx, linux, bsd boxes and that's why
>>>>>> it's pure python (written that way).
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>>>
>>>>>>> --David
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/04/2013 06:03 PM, Hugo Trippaers wrote:
> Hey guys,
>
> What do we do with the database scripts? They are needed by cloud(stack)-setup-management. I'm thinking about adding them to the management server for now together with the scripts. There are some python modules (cloudutils and cloud_utils) which are needed by both setup tools in management and agent to I thought to put those in common.
>
Ran into the same! Had the idea to put them indeed in the management
server package.
The python libs should indeed go into common.
Wido
> Cheers,
>
> Hugo
>
>> -----Original Message-----
>> From: Wido den Hollander [mailto:wido@widodh.nl]
>> Sent: Monday, February 04, 2013 2:48 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: [DISCUSS] Packaging in 4.1
>>
>> On 02/04/2013 02:39 PM, Noa Resare wrote:
>>> I'm on the last leg of my trip home from FOSDEM right now, I hope to
>>> be able to put in some work on deb packages in the next few days.
>>>
>>
>> I still have some pending changes on my laptop which I forgot to push to the
>> Github repo[0].
>>
>> I'll push them in a couple of hours, they contain a lot of what we discussed
>> regarding the DEB packaging.
>>
>> Wido
>>
>> [0]: https://github.com/noaresare/incubator-cloudstack
>>
>>> /n
>>>
>>>
>>> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl>
>> wrote:
>>>
>>>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>>>
>>>>> Wanted to check when would this be implemented?? Several QA folks
>>>>> depend on the packages and need this working.
>>>>> We have been still building using waf but today that is also not
>>>>> working as some references are removed.
>>>>>
>>>>> Is it possible to accelerate this process or leave old way of
>>>>> packaging in place till you are done with the new changes
>>>>>
>>>>>
>>>> I fully understand. As I told David, I think the DEB work is about
>>>> one day of work, but then again, there is something like $dayjob.
>>>>
>>>> What might be even tougher is to get the RPM and DEB packages fully
>>>> synced. cloudstack-common for example should contain exactly the same
>>>> files in the RPM and DEB version, so Hugo and I will have to keep in touch.
>>>>
>>>> I really want to have the DEB packaging working this week, period.
>>>>
>>>> Wido
>>>>
>>>>
>>>> Thanks
>>>>> /sudha
>>>>>
>>>>> -----Original Message-----
>>>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
>>>>> Behalf Of Rohit Yadav
>>>>> Sent: Sunday, February 03, 2013 5:14 PM
>>>>> To:
>>>>> cloudstack-dev@incubator.**apache.org<cloudstack-
>> dev@incubator.apach
>>>>> e.org>
>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>
>>>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>>>
>>>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
>> wrote:
>>>>>>
>>>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>>>> ...
>>>>>>>
>>>>>>>>
>>>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>>>> worth than clint (clint is in EPEL, but no new version of
>>>>>>>> pygments in
>>>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>>>
>>>>>>>
>>>>>>> I want people to use pip to install the cli because it's the
>>>>>>> easiest and because rpm/deb packages may have dependency issues
>>>>>>> like you mentioned => may not work on all distros, what we can do
>>>>>>> is when people install cloudstack-cli rpm or deb, it runs a script
>>>>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
>>>>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
>>>>>>> cloudmonkey and its dependencies and install from it. Advise best
>>>>>>> way of doing this?
>>>>>>>
>>>>>>
>>>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>>>
>>>>>> You are assuming that they would have internet access when
>>>>>> installing
>>>>>> - which is not a valid assumption.
>>>>>>
>>>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>>>> installed, and may or may not have installed - may have installed a
>>>>>> compromised package (see rubygems.org compromise recently,
>>>>>> kernel.org, and a number of other site compromises.), or might have
>>>>>> installed packages I didn't know about is a Bad Idea (tm) The
>>>>>> sysadmin doesn't know you are installing some of the dependencies,
>>>>>> there is no record of those packages in the package manager, and
>>>>>> there might potentially be conflicts with system packages, a
>>>>>> security vulnerability in one of those dependencies wouldn't be caught
>> on audit, etc etc.
>>>>>>
>>>>>
>>>>> /facepalm\, it's just a problem of packaging. The package can
>>>>> include cli or any other artifact's dependencies, so in case of cli,
>>>>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
>>>>> of them are pure python so easily installable. The cool people can use
>> pip to install.
>>>>>
>>>>>
>>>>>> And I really don't intend for this to sound like a rant, but the
>>>>>> one of the important benefits behind using packages and a package
>>>>>> manager is that a sysadmin needs (and often is required to have by
>>>>>> government
>>>>>> regulations) a single source of truth about the software installed
>>>>>> on a machine.
>>>>>>
>>>>>
>>>>> No, it's not a rant, I understand.
>>>>>
>>>>> Developers love things like Maven central, pypi, CPAN, and
>>>>> rubygems,
>>>>>> and for good reason, they are fast, flexible, and make their life
>>>>>> easy. To a sysadmin managing machines in production, they are
>>>>>> anathema; they make system state difficult or impossible to
>>>>>> determine, they make audits painful.
>>>>>>
>>>>>
>>>>> I just assumed the sysadmin who would install CloudStack, cli and
>>>>> whatnot won't be stupid, at the same time I want his life to be less
>> miserable.
>>>>>
>>>>> In addition they make troubleshooting
>>>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>>>> there multiple copies of $foo installed on the system? Which one is
>>>>>> actually being called/loaded?
>>>>>>
>>>>>
>>>>> Alright, but I'm talking only about the cli, since most users,
>>>>> admins included, would want it to run on their machines, the
>>>>> installation should be easiest, that's why I said they can use pip,
>>>>> so it works on their windows, osx, linux, bsd boxes and that's why
>>>>> it's pure python (written that way).
>>>>>
>>>>> Regards.
>>>>>
>>>>>>
>>>>>> --David
>>>>>>
>>>>>
>>>>
>>>
>>>
>
RE: [DISCUSS] Packaging in 4.1
Posted by Hugo Trippaers <HT...@schubergphilis.com>.
Hey guys,
What do we do with the database scripts? They are needed by cloud(stack)-setup-management. I'm thinking about adding them to the management server for now together with the scripts. There are some python modules (cloudutils and cloud_utils) which are needed by both setup tools in management and agent to I thought to put those in common.
Cheers,
Hugo
> -----Original Message-----
> From: Wido den Hollander [mailto:wido@widodh.nl]
> Sent: Monday, February 04, 2013 2:48 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
> On 02/04/2013 02:39 PM, Noa Resare wrote:
> > I'm on the last leg of my trip home from FOSDEM right now, I hope to
> > be able to put in some work on deb packages in the next few days.
> >
>
> I still have some pending changes on my laptop which I forgot to push to the
> Github repo[0].
>
> I'll push them in a couple of hours, they contain a lot of what we discussed
> regarding the DEB packaging.
>
> Wido
>
> [0]: https://github.com/noaresare/incubator-cloudstack
>
> > /n
> >
> >
> > On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >
> >> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
> >>
> >>> Wanted to check when would this be implemented?? Several QA folks
> >>> depend on the packages and need this working.
> >>> We have been still building using waf but today that is also not
> >>> working as some references are removed.
> >>>
> >>> Is it possible to accelerate this process or leave old way of
> >>> packaging in place till you are done with the new changes
> >>>
> >>>
> >> I fully understand. As I told David, I think the DEB work is about
> >> one day of work, but then again, there is something like $dayjob.
> >>
> >> What might be even tougher is to get the RPM and DEB packages fully
> >> synced. cloudstack-common for example should contain exactly the same
> >> files in the RPM and DEB version, so Hugo and I will have to keep in touch.
> >>
> >> I really want to have the DEB packaging working this week, period.
> >>
> >> Wido
> >>
> >>
> >> Thanks
> >>> /sudha
> >>>
> >>> -----Original Message-----
> >>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
> >>> Behalf Of Rohit Yadav
> >>> Sent: Sunday, February 03, 2013 5:14 PM
> >>> To:
> >>> cloudstack-dev@incubator.**apache.org<cloudstack-
> dev@incubator.apach
> >>> e.org>
> >>> Subject: Re: [DISCUSS] Packaging in 4.1
> >>>
> >>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
> >>>
> >>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >>>>
> >>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
> >>>>> ...
> >>>>>
> >>>>>>
> >>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
> >>>>>> worth than clint (clint is in EPEL, but no new version of
> >>>>>> pygments in
> >>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
> >>>>>>
> >>>>>
> >>>>> I want people to use pip to install the cli because it's the
> >>>>> easiest and because rpm/deb packages may have dependency issues
> >>>>> like you mentioned => may not work on all distros, what we can do
> >>>>> is when people install cloudstack-cli rpm or deb, it runs a script
> >>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
> >>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
> >>>>> cloudmonkey and its dependencies and install from it. Advise best
> >>>>> way of doing this?
> >>>>>
> >>>>
> >>>> I guess we won't be installing the CLI via RPMs at least for EL6.
> >>>>
> >>>> You are assuming that they would have internet access when
> >>>> installing
> >>>> - which is not a valid assumption.
> >>>>
> >>>> Honestly, the above idea makes me blanch. A package that reports as
> >>>> installed, and may or may not have installed - may have installed a
> >>>> compromised package (see rubygems.org compromise recently,
> >>>> kernel.org, and a number of other site compromises.), or might have
> >>>> installed packages I didn't know about is a Bad Idea (tm) The
> >>>> sysadmin doesn't know you are installing some of the dependencies,
> >>>> there is no record of those packages in the package manager, and
> >>>> there might potentially be conflicts with system packages, a
> >>>> security vulnerability in one of those dependencies wouldn't be caught
> on audit, etc etc.
> >>>>
> >>>
> >>> /facepalm\, it's just a problem of packaging. The package can
> >>> include cli or any other artifact's dependencies, so in case of cli,
> >>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
> >>> of them are pure python so easily installable. The cool people can use
> pip to install.
> >>>
> >>>
> >>>> And I really don't intend for this to sound like a rant, but the
> >>>> one of the important benefits behind using packages and a package
> >>>> manager is that a sysadmin needs (and often is required to have by
> >>>> government
> >>>> regulations) a single source of truth about the software installed
> >>>> on a machine.
> >>>>
> >>>
> >>> No, it's not a rant, I understand.
> >>>
> >>> Developers love things like Maven central, pypi, CPAN, and
> >>> rubygems,
> >>>> and for good reason, they are fast, flexible, and make their life
> >>>> easy. To a sysadmin managing machines in production, they are
> >>>> anathema; they make system state difficult or impossible to
> >>>> determine, they make audits painful.
> >>>>
> >>>
> >>> I just assumed the sysadmin who would install CloudStack, cli and
> >>> whatnot won't be stupid, at the same time I want his life to be less
> miserable.
> >>>
> >>> In addition they make troubleshooting
> >>>> incredibly difficult. Do I have $foo installed - which version? Are
> >>>> there multiple copies of $foo installed on the system? Which one is
> >>>> actually being called/loaded?
> >>>>
> >>>
> >>> Alright, but I'm talking only about the cli, since most users,
> >>> admins included, would want it to run on their machines, the
> >>> installation should be easiest, that's why I said they can use pip,
> >>> so it works on their windows, osx, linux, bsd boxes and that's why
> >>> it's pure python (written that way).
> >>>
> >>> Regards.
> >>>
> >>>>
> >>>> --David
> >>>>
> >>>
> >>
> >
> >
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/04/2013 02:39 PM, Noa Resare wrote:
> I'm on the last leg of my trip home from FOSDEM right now, I hope to be
> able to put in some work on deb packages in the next few days.
>
I still have some pending changes on my laptop which I forgot to push to
the Github repo[0].
I'll push them in a couple of hours, they contain a lot of what we
discussed regarding the DEB packaging.
Wido
[0]: https://github.com/noaresare/incubator-cloudstack
> /n
>
>
> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>
>>> Wanted to check when would this be implemented?? Several QA folks depend
>>> on the packages and need this working.
>>> We have been still building using waf but today that is also not working
>>> as some references are removed.
>>>
>>> Is it possible to accelerate this process or leave old way of packaging
>>> in place till you are done with the new changes
>>>
>>>
>> I fully understand. As I told David, I think the DEB work is about one day
>> of work, but then again, there is something like $dayjob.
>>
>> What might be even tougher is to get the RPM and DEB packages fully
>> synced. cloudstack-common for example should contain exactly the same files
>> in the RPM and DEB version, so Hugo and I will have to keep in touch.
>>
>> I really want to have the DEB packaging working this week, period.
>>
>> Wido
>>
>>
>> Thanks
>>> /sudha
>>>
>>> -----Original Message-----
>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On Behalf
>>> Of Rohit Yadav
>>> Sent: Sunday, February 03, 2013 5:14 PM
>>> To: cloudstack-dev@incubator.**apache.org<cl...@incubator.apache.org>
>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>
>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>
>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>>>>
>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>> ...
>>>>>
>>>>>>
>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>> worth than clint (clint is in EPEL, but no new version of pygments
>>>>>> in
>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>
>>>>>
>>>>> I want people to use pip to install the cli because it's the easiest
>>>>> and because rpm/deb packages may have dependency issues like you
>>>>> mentioned => may not work on all distros, what we can do is when
>>>>> people install cloudstack-cli rpm or deb, it runs a script that
>>>>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>>>>> python, so the rpm/deb can also ship bundling src tarballs of
>>>>> cloudmonkey and its dependencies and install from it. Advise best way
>>>>> of doing this?
>>>>>
>>>>
>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>
>>>> You are assuming that they would have internet access when installing
>>>> - which is not a valid assumption.
>>>>
>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>> installed, and may or may not have installed - may have installed a
>>>> compromised package (see rubygems.org compromise recently, kernel.org,
>>>> and a number of other site compromises.), or might have installed
>>>> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
>>>> know you are installing some of the dependencies, there is no record
>>>> of those packages in the package manager, and there might potentially
>>>> be conflicts with system packages, a security vulnerability in one of
>>>> those dependencies wouldn't be caught on audit, etc etc.
>>>>
>>>
>>> /facepalm\, it's just a problem of packaging. The package can include cli
>>> or any other artifact's dependencies, so in case of cli, you bundle both
>>> pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure
>>> python so easily installable. The cool people can use pip to install.
>>>
>>>
>>>> And I really don't intend for this to sound like a rant, but the one
>>>> of the important benefits behind using packages and a package manager
>>>> is that a sysadmin needs (and often is required to have by government
>>>> regulations) a single source of truth about the software installed on
>>>> a machine.
>>>>
>>>
>>> No, it's not a rant, I understand.
>>>
>>> Developers love things like Maven central, pypi, CPAN, and rubygems,
>>>> and for good reason, they are fast, flexible, and make their life
>>>> easy. To a sysadmin managing machines in production, they are
>>>> anathema; they make system state difficult or impossible to determine,
>>>> they make audits painful.
>>>>
>>>
>>> I just assumed the sysadmin who would install CloudStack, cli and whatnot
>>> won't be stupid, at the same time I want his life to be less miserable.
>>>
>>> In addition they make troubleshooting
>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>> there multiple copies of $foo installed on the system? Which one is
>>>> actually being called/loaded?
>>>>
>>>
>>> Alright, but I'm talking only about the cli, since most users, admins
>>> included, would want it to run on their machines, the installation should
>>> be easiest, that's why I said they can use pip, so it works on their
>>> windows, osx, linux, bsd boxes and that's why it's pure python (written
>>> that way).
>>>
>>> Regards.
>>>
>>>>
>>>> --David
>>>>
>>>
>>
>
>
Re: [DISCUSS] Packaging in 4.1
Posted by Noa Resare <no...@spotify.com>.
I'm on the last leg of my trip home from FOSDEM right now, I hope to be
able to put in some work on deb packages in the next few days.
/n
On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>
>> Wanted to check when would this be implemented?? Several QA folks depend
>> on the packages and need this working.
>> We have been still building using waf but today that is also not working
>> as some references are removed.
>>
>> Is it possible to accelerate this process or leave old way of packaging
>> in place till you are done with the new changes
>>
>>
> I fully understand. As I told David, I think the DEB work is about one day
> of work, but then again, there is something like $dayjob.
>
> What might be even tougher is to get the RPM and DEB packages fully
> synced. cloudstack-common for example should contain exactly the same files
> in the RPM and DEB version, so Hugo and I will have to keep in touch.
>
> I really want to have the DEB packaging working this week, period.
>
> Wido
>
>
> Thanks
>> /sudha
>>
>> -----Original Message-----
>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On Behalf
>> Of Rohit Yadav
>> Sent: Sunday, February 03, 2013 5:14 PM
>> To: cloudstack-dev@incubator.**apache.org<cl...@incubator.apache.org>
>> Subject: Re: [DISCUSS] Packaging in 4.1
>>
>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>
>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>>>
>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>> ...
>>>>
>>>>>
>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>> worth than clint (clint is in EPEL, but no new version of pygments
>>>>> in
>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>
>>>>
>>>> I want people to use pip to install the cli because it's the easiest
>>>> and because rpm/deb packages may have dependency issues like you
>>>> mentioned => may not work on all distros, what we can do is when
>>>> people install cloudstack-cli rpm or deb, it runs a script that
>>>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>>>> python, so the rpm/deb can also ship bundling src tarballs of
>>>> cloudmonkey and its dependencies and install from it. Advise best way
>>>> of doing this?
>>>>
>>>
>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>
>>> You are assuming that they would have internet access when installing
>>> - which is not a valid assumption.
>>>
>>> Honestly, the above idea makes me blanch. A package that reports as
>>> installed, and may or may not have installed - may have installed a
>>> compromised package (see rubygems.org compromise recently, kernel.org,
>>> and a number of other site compromises.), or might have installed
>>> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
>>> know you are installing some of the dependencies, there is no record
>>> of those packages in the package manager, and there might potentially
>>> be conflicts with system packages, a security vulnerability in one of
>>> those dependencies wouldn't be caught on audit, etc etc.
>>>
>>
>> /facepalm\, it's just a problem of packaging. The package can include cli
>> or any other artifact's dependencies, so in case of cli, you bundle both
>> pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure
>> python so easily installable. The cool people can use pip to install.
>>
>>
>>> And I really don't intend for this to sound like a rant, but the one
>>> of the important benefits behind using packages and a package manager
>>> is that a sysadmin needs (and often is required to have by government
>>> regulations) a single source of truth about the software installed on
>>> a machine.
>>>
>>
>> No, it's not a rant, I understand.
>>
>> Developers love things like Maven central, pypi, CPAN, and rubygems,
>>> and for good reason, they are fast, flexible, and make their life
>>> easy. To a sysadmin managing machines in production, they are
>>> anathema; they make system state difficult or impossible to determine,
>>> they make audits painful.
>>>
>>
>> I just assumed the sysadmin who would install CloudStack, cli and whatnot
>> won't be stupid, at the same time I want his life to be less miserable.
>>
>> In addition they make troubleshooting
>>> incredibly difficult. Do I have $foo installed - which version? Are
>>> there multiple copies of $foo installed on the system? Which one is
>>> actually being called/loaded?
>>>
>>
>> Alright, but I'm talking only about the cli, since most users, admins
>> included, would want it to run on their machines, the installation should
>> be easiest, that's why I said they can use pip, so it works on their
>> windows, osx, linux, bsd boxes and that's why it's pure python (written
>> that way).
>>
>> Regards.
>>
>>>
>>> --David
>>>
>>
>
--
Engineering Experience, Infrastructure tribe, Spotify
Re: [DISCUSS] Packaging in 4.1
Posted by Prasanna Santhanam <ts...@apache.org>.
On Fri, Feb 08, 2013 at 12:32:02PM -0500, Hugo Trippaers wrote:
> Hey guys,
>
> Just a quick note before the weekend with a status update on RPM.
>
> The management server package is pretty much done and installation
> on a clean system works like a charm. This is actually tested every
> few hours with a Jenkins setup a colleague and I built. We take the
> sources, compile and package. The packages are added to a repo and
> chef is used to deploy two new clean CentOS 6.3 boxes. One is
> configured as database server and another one as CloudStack
> management server by chef. After installation an ApiKey is created
> for the admin user. This proves that the package can be installed on
> a clean system and that the management server starts.
Just tested the rpms myself and they are working well. Well done! I'll
be setting up a public repository to host packages so such test jobs
can use them downstream. Wido's repo can hold the released binaries
while these can be unstable.
> Next week:
> * we will continue with the setup and add some real tests to create zones and add hypervisors.
> *I will also start testing with the agent and usage package, they are created at the moment but not tested for functionality.
> * Deploy fedora 18 image and extend the test to that
> * Deploy Ubuntu 12.04 and add packaging scripts for that (check with wido/noa)
This is really cool Hugo. I've connected my test infrastructure to run
your packaging job as a test as well [1]. The system I'm using is similar to
yours except I'm using puppet in place of chef for my configuration.
Would you guys be able to setup your jobs via jenkins.cs.o as well?
I'd ideally love to see multi-site environments running tests.
Also we should perhaps standardize on some marvin configurations for
the zones. I'm going to start with advanced zone with kvm and
xenserver.
I've got hypervisors refreshing using cobbler kickstarts and added
some dashboards to cobbler and puppet so it's not all command line now :)
On my TODO list this week are:
0) ubuntu packaging tests
1) configuring zones for kvm and xen
2) jobs for marvin smoke tests
3) jobs for regression tests per component
4) upstreaming debug logs to jenkins for troubleshooting
[1] http://jenkins.cloudstack.org/view/cloudstack-qa/job/test-packaging-rhel63/
--
Prasanna.,
RE: [DISCUSS] Packaging in 4.1
Posted by Pradeep Soundararajan <pr...@citrix.com>.
I am able to package rpm successfully now. Thank you very much.
Apart from the packaging directory is there any way could you please get us the list of files/scripts/commits touched for this to update from cloud* to cloudstack*. Example: python/lib/cloudutils/serviceConfig.py, serviceConfigServer.py.
I hope the list is small.
Thanks,
Pradeep S
-----Original Message-----
From: Chip Childers [mailto:chip.childers@sungard.com]
Sent: Thursday, February 07, 2013 7:09 AM
To: David Nalley
Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander; cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Packaging in 4.1
On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com> wrote:
> >> Well, first, Apache CloudStack only releases source code.
> >>
> >> But Wido is kind enough to also host RPM / DEB package repos for
> >> users to take advantage of. Our install guide explains how to
> >> build from source, as well as how to use Wido's repos.
> >>
> >> This was all true for 4.0.0-incubating, and I think it still holds
> >> true for all future releases.
> >>
> > Chip,
> >
> > Can you refresh my memory as to why this is? I look at something like cxf or tomcat, they all have binary downloads available.
> >
> > http://cxf.apache.org/download.html
> >
>
> Because providing 'binaries' isn't necessarily problematic, but making
> yum and apt repos work in the ASF mirror system seems a bit more of an
> issue. Plus, Wido stepped up to do the work, no one else has offered
> any other alternatives.
>
> --David
>
Yup - exactly what David said. We had discussed trying to get ASF Infra to help us host package repos somewhere, but I don't think that went anywhere. And since Wido's doing it, it avoided all sorts of questions from the infra team around mirrors, archiving, etc...
-chip
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
Debian still has to change.
In the postinst scripts we have to change the homedir of an existing
user "cloud" to the new directoy.
We haven't synced the Debian and CentOS packages yet, Noa is working on
that.
Wido
On 02/13/2013 08:41 PM, Marcus Sorensen wrote:
> CentOS is currently experiencing bugs around changing user cloud's
> home to /var/cloudstack, from /var/lib/cloud. Is Debian changing
> /var/lib/cloud to /var/cloudstack as well? I see references to it in:
>
> debian/cloud-client.postinst
> debian/cloud-client.install
> debian/cloud-usage.postinst
> packaging/debian/replace.properties
>
>
> On Wed, Feb 13, 2013 at 10:52 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>> Yes, this line here in package.sh:
>>
>> VERSION=`(cd ../../; mvn
>> org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate
>> -Dexpression=project.version) | grep -v '^\['`
>>
>> if you don't have all of the maven stuff downloaded, instead of spitting out:
>>
>> 4.1.0-SNAPSHOT
>>
>> It spits out:
>>
>> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
>> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
>> (2 KB at 1.1 KB/sec)
>> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
>> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
>> (135 KB at 1094.3 KB/sec)
>> 4.1.0-SNAPSHOT
>>
>>
>> On Wed, Feb 13, 2013 at 10:45 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>> It works for me subsequent to the first run.
>>>
>>> I think it's because of the output of downloading maven stuff, the
>>> script doesn't seem to like that.
>>>
>>> On Wed, Feb 13, 2013 at 10:42 AM, Pradeep Soundararajan
>>> <pr...@citrix.com> wrote:
>>>> I am following these steps:
>>>>
>>>> chmod 755 ./packaging/centos63/package.sh
>>>> cd packaging/centos63
>>>> ./package.sh
>>>>
>>>> cd $WORKSPACE
>>>> tempdir=`mktemp -d`
>>>> mkdir -p "$tempdir"
>>>> cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
>>>> createrepo $tempdir/
>>>>
>>>> The above is working well for me.
>>>>
>>>> Thanks,
>>>> Pradeep S
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
>>>> Sent: Wednesday, February 13, 2013 10:31 PM
>>>> To: 'Marcus Sorensen'; Wido den Hollander
>>>> Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
>>>> Subject: RE: [DISCUSS] Packaging in 4.1
>>>>
>>>> Hey Marcus,
>>>>
>>>> I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
>>>>
>>>> <from toplevel project dir>
>>>> rm -rf dist
>>>> mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
>>>>
>>>> tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
>>>>
>>>> rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
>>>>
>>>>
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Hugo
>>>>> -----Original Message-----
>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>> Sent: Wednesday, February 13, 2013 3:08 PM
>>>>> To: Wido den Hollander
>>>>> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
>>>>> Soundararajan; cloudstack-dev@incubator.apache.org
>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>
>>>>> Hm, this package.sh is still doing weird things for me. If I pull a
>>>>> fresh incubator-cloudstack and run this:
>>>>>
>>>>> cd packaging/centos63
>>>>> chmod +x package.sh
>>>>> ./package.sh
>>>>>
>>>>> I get this output (truncated, but you get the idea)
>>>>>
>>>>> tar: \rDownloaded\:: Cannot stat: No such file or directory
>>>>> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
>>>>> metadata.xml:
>>>>> Cannot stat: No such file or directory
>>>>> tar: (22: Cannot stat: No such file or directory
>>>>> tar: KB: Cannot stat: No such file or directory
>>>>> tar: at: Cannot stat: No such file or directory
>>>>> tar: 96.8: Cannot stat: No such file or directory
>>>>> tar: KB/sec): Cannot stat: No such file or directory
>>>>>
>>>>> [root@devcloud-kvm centos63]# ls
>>>>> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
>>>>> ?54 ?72 92
>>>>> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
>>>>> ?55 728.8 ?92
>>>>> (10 121.3 14.5 173 204 238 ?271 326.8 (402
>>>>> 551 73 921.8
>>>>> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
>>>>> (551 ?73 93
>>>>> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
>>>>> 55.7 739.4 ?93
>>>>> ?100 122.1 148 176 ?205 ?239 275.6 ?33
>>>>> 4.1.0-SNAPSHOT 557.5 74 93.0
>>>>> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
>>>>> 56 ?74 93.3
>>>>> ?101 123 149 177 207 (24 278.3 331 ?42
>>>>> ?56 75 94
>>>>> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
>>>>> 56.0 75.5 ?94
>>>>> 102 124 15 ?178 ?208 ?240 28 335 (424
>>>>> ?57 755.1 94.2
>>>>> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
>>>>> 58 ?76 95
>>>>> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
>>>>> ?58 762.9 954.6
>>>>> 104 126 150 (18 209.4 ?243 280.9 339 437.4
>>>>> 580.7 77 96
>>>>> ?104 ?126 (150 ?18 210 243.6 282 34 44
>>>>> 58.2 ?77 ?96
>>>>> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
>>>>> 582.0 78 96.1
>>>>> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
>>>>> 582.9 ?78 96.8
>>>>> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
>>>>> (59 79 ?97
>>>>> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
>>>>> ?59 8 97.5
>>>>> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
>>>>> 598.6 (8 98
>>>>> 108 ?13 153 184.0 214 25 ?289 35 455.7
>>>>> 6 ?8 ?98
>>>>> ?108 130 ?153 185 215 (25 (29 ?35 46
>>>>> (6 ?80 99
>>>>> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
>>>>> ?6 ?81 998
>>>>> (11 ?131 153.9 1873.3 (216 250 290 355 468
>>>>> 60 818.7 (998
>>>>> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
>>>>> (60 82 99.8
>>>>> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
>>>>> ?60 ?82 at
>>>>> ?110 ?132 156 1883.0 218 251.6 294 36 47
>>>>> ?61 83 available
>>>>> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
>>>>> 618.0 83.3 B
>>>>> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
>>>>> 62 84 cloud-agent.rc
>>>>> 111 ?134 ?158 19 (22 254 29.7 363 471
>>>>> ?62 ?84 cloud-ipallocator.rc
>>>>> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
>>>>> 63 8.4 cloud-management.rc
>>>>> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
>>>>> (63 ?85 cloud-management.sysconfig
>>>>> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
>>>>> ?63 85.7 cloud.spec
>>>>> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
>>>>> 636.5 859.6 cloud-usage.rc
>>>>> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
>>>>> 64 86 dependency
>>>>> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
>>>>> ?64 ?86 ?Downloaded:
>>>>> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
>>>>> 66 86.8 Downloading:
>>>>> 115 138 161 ?197 226 ?258 ?30 374 488.4
>>>>> ?66 87 for
>>>>> 11.5 ?138 16.1 198 227 258.0 302 378 (49
>>>>> 67 ?87 http:
>>>>> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
>>>>> ?67 87.7 information
>>>>> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
>>>>> 68 88 is
>>>>> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
>>>>> ?68 (88 KB
>>>>> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
>>>>> 68.2 ?88 missing,
>>>>> (117 ?14 166 20 (23 (260 (31 ?39 (5
>>>>> 684.9 89 no
>>>>> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
>>>>> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
>>>>> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
>>>>> ?69 ?89 package.sh
>>>>> 118 141 168 20.0 230 262 311 398 ?50
>>>>> (7 89.0 POM
>>>>> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
>>>>> ?7 (9 replace.properties
>>>>> (119 142 169 201 ?231 ?263 31.8 399.5 (51
>>>>> 70 ?9 The
>>>>> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
>>>>> ?70 90 ?[WARNING]
>>>>> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
>>>>> 70.2 ?90
>>>>> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
>>>>> 71 901.6
>>>>> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
>>>>> ?71 91
>>>>> 120 144.8 172 203 ?235 (27 ?32 ?40 54
>>>>> 72 91.4
>>>>>
>>>>> Then I 'git clean -fxd', rerun package.sh, and everything works. I
>>>>> wonder if it's setting an env variable that allows it to work the second time.
>>>>>
>>>>> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
>>>>> <sh...@gmail.com> wrote:
>>>>>> The packaging/centos63/package.sh makes some assumptions about how
>>>>>> it's being run that end up with some ugly results if it's not done
>>>>>> exactly right. For example, I tried from the incubator-cloudstack
>>>>>> directory: "sh ./packaging/centos63/package.sh", which seemed to
>>>>>> copy /proc into my current directory and attemped to tar it up. Then
>>>>>> I did "cd packaging/centos63; sh ./package.sh", which ended up with
>>>>>> roughly the same result, although it died trying to run "Downloading:
>>>>>> http://repo.maven..." as a bash command.
>>>>>>
>>>>>> Seems it didn't like being run as an 'sh' either way, even though
>>>>>> it's not in the code as executable. After doing a chmod to make it
>>>>>> executable, it seems to work but only if your cwd is
>>>>>> incubator-cloudstack/packaging/centos63.
>>>>>>
>>>>>> Maybe we could change it to fail gracefully if your current path
>>>>>> doesn't end in "packaging/centos63", and make it executable in git?
>>>>>>
>>>>>> On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>>>>>>>
>>>>>>>> Hey guys,
>>>>>>>>
>>>>>>>> Just a quick note before the weekend with a status update on RPM.
>>>>>>>>
>>>>>>>> The management server package is pretty much done and installation
>>>>>>>> on a clean system works like a charm. This is actually tested
>>>>>>>> every few hours with a Jenkins setup a colleague and I built. We
>>>>>>>> take the sources, compile and package. The packages are added to a
>>>>>>>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
>>>>>>>> One is configured as database server and another one as CloudStack
>>>>>>>> management server by chef. After installation an ApiKey is created
>>>>>>>> for the admin user. This proves that the package can be installed
>>>>>>>> on a
>>>>> clean system and that the management server starts.
>>>>>>>>
>>>>>>>> With this testing we have found several issues of which a few
>>>>>>>> haven't been resolved yet (hopefully this weekend):
>>>>>>>>
>>>>>>>> * 4.1-new-db-schema.sql is not loaded by
>>>>>>>> cloudstack-setup-databases
>>>>>>>> * userid is null in reponse to a login call with the admin user,
>>>>>>>> expected
>>>>>>>> 2
>>>>>>>> * Excryption initialization is now done in Transaction, this
>>>>>>>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail is
>>>>>>>> db.properties is not in the classpath
>>>>>>>>
>>>>>>>> Next week:
>>>>>>>> * we will continue with the setup and add some real tests to
>>>>>>>> create zones and add hypervisors.
>>>>>>>> *I will also start testing with the agent and usage package,
>>>>>>>> they are created at the moment but not tested for functionality.
>>>>>>>> * Deploy fedora 18 image and extend the test to that
>>>>>>>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
>>>>>>>> with
>>>>>>>> wido/noa)
>>>>>>>>
>>>>>>>
>>>>>>> We'll sync next week! A lot of the .deb work is already done, but
>>>>>>> we just have to make sure the RPM and DEB packages contain the same files.
>>>>>>>
>>>>>>> Then it will just be tuning and some work in the pre and postinst
>>>>>>> files, but that could be a pain, but we'll just see when we go along.
>>>>>>>
>>>>>>> Wido
>>>>>>>
>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Hugo
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>>>>>>>> Sent: Thursday, February 07, 2013 2:39 AM
>>>>>>>>> To: David Nalley
>>>>>>>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
>>>>>>>>> cloudstack- dev@incubator.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>>>>>
>>>>>>>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
>>>>> <Al...@citrix.com>
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Well, first, Apache CloudStack only releases source code.
>>>>>>>>>>>>
>>>>>>>>>>>> But Wido is kind enough to also host RPM / DEB package repos
>>>>>>>>>>>> for users to take advantage of. Our install guide explains
>>>>>>>>>>>> how to build from source, as well as how to use Wido's repos.
>>>>>>>>>>>>
>>>>>>>>>>>> This was all true for 4.0.0-incubating, and I think it still
>>>>>>>>>>>> holds true for all future releases.
>>>>>>>>>>>>
>>>>>>>>>>> Chip,
>>>>>>>>>>>
>>>>>>>>>>> Can you refresh my memory as to why this is? I look at
>>>>>>>>>>> something like cxf
>>>>>>>>>
>>>>>>>>> or tomcat, they all have binary downloads available.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://cxf.apache.org/download.html
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Because providing 'binaries' isn't necessarily problematic, but
>>>>>>>>>> making yum and apt repos work in the ASF mirror system seems a
>>>>>>>>>> bit more of an issue. Plus, Wido stepped up to do the work, no
>>>>>>>>>> one else has offered any other alternatives.
>>>>>>>>>>
>>>>>>>>>> --David
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yup - exactly what David said. We had discussed trying to get
>>>>>>>>> ASF Infra to help us host package repos somewhere, but I don't
>>>>>>>>> think that went anywhere. And since Wido's doing it, it avoided
>>>>>>>>> all sorts of questions from the infra team around mirrors,
>>>>>>>>> archiving, etc...
>>>>>>>>>
>>>>>>>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Marcus Sorensen <sh...@gmail.com>.
FYI, I cleaned up all remaining references to /var/lib/cloud except
for the Debian ones. Just want the packager to do that in case they're
being accounted for somewhere else and they're being left behind for
compatibility, maybe symlinking or something
On Wed, Feb 13, 2013 at 12:41 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> CentOS is currently experiencing bugs around changing user cloud's
> home to /var/cloudstack, from /var/lib/cloud. Is Debian changing
> /var/lib/cloud to /var/cloudstack as well? I see references to it in:
>
> debian/cloud-client.postinst
> debian/cloud-client.install
> debian/cloud-usage.postinst
> packaging/debian/replace.properties
>
>
> On Wed, Feb 13, 2013 at 10:52 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>> Yes, this line here in package.sh:
>>
>> VERSION=`(cd ../../; mvn
>> org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate
>> -Dexpression=project.version) | grep -v '^\['`
>>
>> if you don't have all of the maven stuff downloaded, instead of spitting out:
>>
>> 4.1.0-SNAPSHOT
>>
>> It spits out:
>>
>> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
>> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
>> (2 KB at 1.1 KB/sec)
>> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
>> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
>> (135 KB at 1094.3 KB/sec)
>> 4.1.0-SNAPSHOT
>>
>>
>> On Wed, Feb 13, 2013 at 10:45 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>> It works for me subsequent to the first run.
>>>
>>> I think it's because of the output of downloading maven stuff, the
>>> script doesn't seem to like that.
>>>
>>> On Wed, Feb 13, 2013 at 10:42 AM, Pradeep Soundararajan
>>> <pr...@citrix.com> wrote:
>>>> I am following these steps:
>>>>
>>>> chmod 755 ./packaging/centos63/package.sh
>>>> cd packaging/centos63
>>>> ./package.sh
>>>>
>>>> cd $WORKSPACE
>>>> tempdir=`mktemp -d`
>>>> mkdir -p "$tempdir"
>>>> cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
>>>> createrepo $tempdir/
>>>>
>>>> The above is working well for me.
>>>>
>>>> Thanks,
>>>> Pradeep S
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
>>>> Sent: Wednesday, February 13, 2013 10:31 PM
>>>> To: 'Marcus Sorensen'; Wido den Hollander
>>>> Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
>>>> Subject: RE: [DISCUSS] Packaging in 4.1
>>>>
>>>> Hey Marcus,
>>>>
>>>> I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
>>>>
>>>> <from toplevel project dir>
>>>> rm -rf dist
>>>> mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
>>>>
>>>> tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
>>>>
>>>> rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
>>>>
>>>>
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Hugo
>>>>> -----Original Message-----
>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>> Sent: Wednesday, February 13, 2013 3:08 PM
>>>>> To: Wido den Hollander
>>>>> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
>>>>> Soundararajan; cloudstack-dev@incubator.apache.org
>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>
>>>>> Hm, this package.sh is still doing weird things for me. If I pull a
>>>>> fresh incubator-cloudstack and run this:
>>>>>
>>>>> cd packaging/centos63
>>>>> chmod +x package.sh
>>>>> ./package.sh
>>>>>
>>>>> I get this output (truncated, but you get the idea)
>>>>>
>>>>> tar: \rDownloaded\:: Cannot stat: No such file or directory
>>>>> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
>>>>> metadata.xml:
>>>>> Cannot stat: No such file or directory
>>>>> tar: (22: Cannot stat: No such file or directory
>>>>> tar: KB: Cannot stat: No such file or directory
>>>>> tar: at: Cannot stat: No such file or directory
>>>>> tar: 96.8: Cannot stat: No such file or directory
>>>>> tar: KB/sec): Cannot stat: No such file or directory
>>>>>
>>>>> [root@devcloud-kvm centos63]# ls
>>>>> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
>>>>> ?54 ?72 92
>>>>> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
>>>>> ?55 728.8 ?92
>>>>> (10 121.3 14.5 173 204 238 ?271 326.8 (402
>>>>> 551 73 921.8
>>>>> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
>>>>> (551 ?73 93
>>>>> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
>>>>> 55.7 739.4 ?93
>>>>> ?100 122.1 148 176 ?205 ?239 275.6 ?33
>>>>> 4.1.0-SNAPSHOT 557.5 74 93.0
>>>>> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
>>>>> 56 ?74 93.3
>>>>> ?101 123 149 177 207 (24 278.3 331 ?42
>>>>> ?56 75 94
>>>>> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
>>>>> 56.0 75.5 ?94
>>>>> 102 124 15 ?178 ?208 ?240 28 335 (424
>>>>> ?57 755.1 94.2
>>>>> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
>>>>> 58 ?76 95
>>>>> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
>>>>> ?58 762.9 954.6
>>>>> 104 126 150 (18 209.4 ?243 280.9 339 437.4
>>>>> 580.7 77 96
>>>>> ?104 ?126 (150 ?18 210 243.6 282 34 44
>>>>> 58.2 ?77 ?96
>>>>> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
>>>>> 582.0 78 96.1
>>>>> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
>>>>> 582.9 ?78 96.8
>>>>> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
>>>>> (59 79 ?97
>>>>> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
>>>>> ?59 8 97.5
>>>>> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
>>>>> 598.6 (8 98
>>>>> 108 ?13 153 184.0 214 25 ?289 35 455.7
>>>>> 6 ?8 ?98
>>>>> ?108 130 ?153 185 215 (25 (29 ?35 46
>>>>> (6 ?80 99
>>>>> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
>>>>> ?6 ?81 998
>>>>> (11 ?131 153.9 1873.3 (216 250 290 355 468
>>>>> 60 818.7 (998
>>>>> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
>>>>> (60 82 99.8
>>>>> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
>>>>> ?60 ?82 at
>>>>> ?110 ?132 156 1883.0 218 251.6 294 36 47
>>>>> ?61 83 available
>>>>> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
>>>>> 618.0 83.3 B
>>>>> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
>>>>> 62 84 cloud-agent.rc
>>>>> 111 ?134 ?158 19 (22 254 29.7 363 471
>>>>> ?62 ?84 cloud-ipallocator.rc
>>>>> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
>>>>> 63 8.4 cloud-management.rc
>>>>> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
>>>>> (63 ?85 cloud-management.sysconfig
>>>>> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
>>>>> ?63 85.7 cloud.spec
>>>>> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
>>>>> 636.5 859.6 cloud-usage.rc
>>>>> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
>>>>> 64 86 dependency
>>>>> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
>>>>> ?64 ?86 ?Downloaded:
>>>>> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
>>>>> 66 86.8 Downloading:
>>>>> 115 138 161 ?197 226 ?258 ?30 374 488.4
>>>>> ?66 87 for
>>>>> 11.5 ?138 16.1 198 227 258.0 302 378 (49
>>>>> 67 ?87 http:
>>>>> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
>>>>> ?67 87.7 information
>>>>> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
>>>>> 68 88 is
>>>>> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
>>>>> ?68 (88 KB
>>>>> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
>>>>> 68.2 ?88 missing,
>>>>> (117 ?14 166 20 (23 (260 (31 ?39 (5
>>>>> 684.9 89 no
>>>>> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
>>>>> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
>>>>> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
>>>>> ?69 ?89 package.sh
>>>>> 118 141 168 20.0 230 262 311 398 ?50
>>>>> (7 89.0 POM
>>>>> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
>>>>> ?7 (9 replace.properties
>>>>> (119 142 169 201 ?231 ?263 31.8 399.5 (51
>>>>> 70 ?9 The
>>>>> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
>>>>> ?70 90 ?[WARNING]
>>>>> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
>>>>> 70.2 ?90
>>>>> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
>>>>> 71 901.6
>>>>> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
>>>>> ?71 91
>>>>> 120 144.8 172 203 ?235 (27 ?32 ?40 54
>>>>> 72 91.4
>>>>>
>>>>> Then I 'git clean -fxd', rerun package.sh, and everything works. I
>>>>> wonder if it's setting an env variable that allows it to work the second time.
>>>>>
>>>>> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
>>>>> <sh...@gmail.com> wrote:
>>>>> > The packaging/centos63/package.sh makes some assumptions about how
>>>>> > it's being run that end up with some ugly results if it's not done
>>>>> > exactly right. For example, I tried from the incubator-cloudstack
>>>>> > directory: "sh ./packaging/centos63/package.sh", which seemed to
>>>>> > copy /proc into my current directory and attemped to tar it up. Then
>>>>> > I did "cd packaging/centos63; sh ./package.sh", which ended up with
>>>>> > roughly the same result, although it died trying to run "Downloading:
>>>>> > http://repo.maven..." as a bash command.
>>>>> >
>>>>> > Seems it didn't like being run as an 'sh' either way, even though
>>>>> > it's not in the code as executable. After doing a chmod to make it
>>>>> > executable, it seems to work but only if your cwd is
>>>>> > incubator-cloudstack/packaging/centos63.
>>>>> >
>>>>> > Maybe we could change it to fail gracefully if your current path
>>>>> > doesn't end in "packaging/centos63", and make it executable in git?
>>>>> >
>>>>> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
>>>>> wrote:
>>>>> >>
>>>>> >>
>>>>> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>>>> >>>
>>>>> >>> Hey guys,
>>>>> >>>
>>>>> >>> Just a quick note before the weekend with a status update on RPM.
>>>>> >>>
>>>>> >>> The management server package is pretty much done and installation
>>>>> >>> on a clean system works like a charm. This is actually tested
>>>>> >>> every few hours with a Jenkins setup a colleague and I built. We
>>>>> >>> take the sources, compile and package. The packages are added to a
>>>>> >>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
>>>>> >>> One is configured as database server and another one as CloudStack
>>>>> >>> management server by chef. After installation an ApiKey is created
>>>>> >>> for the admin user. This proves that the package can be installed
>>>>> >>> on a
>>>>> clean system and that the management server starts.
>>>>> >>>
>>>>> >>> With this testing we have found several issues of which a few
>>>>> >>> haven't been resolved yet (hopefully this weekend):
>>>>> >>>
>>>>> >>> * 4.1-new-db-schema.sql is not loaded by
>>>>> >>> cloudstack-setup-databases
>>>>> >>> * userid is null in reponse to a login call with the admin user,
>>>>> >>> expected
>>>>> >>> 2
>>>>> >>> * Excryption initialization is now done in Transaction, this
>>>>> >>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail is
>>>>> >>> db.properties is not in the classpath
>>>>> >>>
>>>>> >>> Next week:
>>>>> >>> * we will continue with the setup and add some real tests to
>>>>> >>> create zones and add hypervisors.
>>>>> >>> *I will also start testing with the agent and usage package,
>>>>> >>> they are created at the moment but not tested for functionality.
>>>>> >>> * Deploy fedora 18 image and extend the test to that
>>>>> >>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
>>>>> >>> with
>>>>> >>> wido/noa)
>>>>> >>>
>>>>> >>
>>>>> >> We'll sync next week! A lot of the .deb work is already done, but
>>>>> >> we just have to make sure the RPM and DEB packages contain the same files.
>>>>> >>
>>>>> >> Then it will just be tuning and some work in the pre and postinst
>>>>> >> files, but that could be a pain, but we'll just see when we go along.
>>>>> >>
>>>>> >> Wido
>>>>> >>
>>>>> >>
>>>>> >>> Cheers,
>>>>> >>>
>>>>> >>> Hugo
>>>>> >>>
>>>>> >>>> -----Original Message-----
>>>>> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>>>> >>>> Sent: Thursday, February 07, 2013 2:39 AM
>>>>> >>>> To: David Nalley
>>>>> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
>>>>> >>>> cloudstack- dev@incubator.apache.org
>>>>> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>> >>>>
>>>>> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>>>> >>>>>
>>>>> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
>>>>> <Al...@citrix.com>
>>>>> >>>>
>>>>> >>>> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Well, first, Apache CloudStack only releases source code.
>>>>> >>>>>>>
>>>>> >>>>>>> But Wido is kind enough to also host RPM / DEB package repos
>>>>> >>>>>>> for users to take advantage of. Our install guide explains
>>>>> >>>>>>> how to build from source, as well as how to use Wido's repos.
>>>>> >>>>>>>
>>>>> >>>>>>> This was all true for 4.0.0-incubating, and I think it still
>>>>> >>>>>>> holds true for all future releases.
>>>>> >>>>>>>
>>>>> >>>>>> Chip,
>>>>> >>>>>>
>>>>> >>>>>> Can you refresh my memory as to why this is? I look at
>>>>> >>>>>> something like cxf
>>>>> >>>>
>>>>> >>>> or tomcat, they all have binary downloads available.
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> http://cxf.apache.org/download.html
>>>>> >>>>>>
>>>>> >>>>>
>>>>> >>>>> Because providing 'binaries' isn't necessarily problematic, but
>>>>> >>>>> making yum and apt repos work in the ASF mirror system seems a
>>>>> >>>>> bit more of an issue. Plus, Wido stepped up to do the work, no
>>>>> >>>>> one else has offered any other alternatives.
>>>>> >>>>>
>>>>> >>>>> --David
>>>>> >>>>>
>>>>> >>>>
>>>>> >>>> Yup - exactly what David said. We had discussed trying to get
>>>>> >>>> ASF Infra to help us host package repos somewhere, but I don't
>>>>> >>>> think that went anywhere. And since Wido's doing it, it avoided
>>>>> >>>> all sorts of questions from the infra team around mirrors,
>>>>> >>>> archiving, etc...
>>>>> >>>>
>>>>> >>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Marcus Sorensen <sh...@gmail.com>.
CentOS is currently experiencing bugs around changing user cloud's
home to /var/cloudstack, from /var/lib/cloud. Is Debian changing
/var/lib/cloud to /var/cloudstack as well? I see references to it in:
debian/cloud-client.postinst
debian/cloud-client.install
debian/cloud-usage.postinst
packaging/debian/replace.properties
On Wed, Feb 13, 2013 at 10:52 AM, Marcus Sorensen <sh...@gmail.com> wrote:
> Yes, this line here in package.sh:
>
> VERSION=`(cd ../../; mvn
> org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate
> -Dexpression=project.version) | grep -v '^\['`
>
> if you don't have all of the maven stuff downloaded, instead of spitting out:
>
> 4.1.0-SNAPSHOT
>
> It spits out:
>
> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
> (2 KB at 1.1 KB/sec)
> Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
> Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
> (135 KB at 1094.3 KB/sec)
> 4.1.0-SNAPSHOT
>
>
> On Wed, Feb 13, 2013 at 10:45 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>> It works for me subsequent to the first run.
>>
>> I think it's because of the output of downloading maven stuff, the
>> script doesn't seem to like that.
>>
>> On Wed, Feb 13, 2013 at 10:42 AM, Pradeep Soundararajan
>> <pr...@citrix.com> wrote:
>>> I am following these steps:
>>>
>>> chmod 755 ./packaging/centos63/package.sh
>>> cd packaging/centos63
>>> ./package.sh
>>>
>>> cd $WORKSPACE
>>> tempdir=`mktemp -d`
>>> mkdir -p "$tempdir"
>>> cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
>>> createrepo $tempdir/
>>>
>>> The above is working well for me.
>>>
>>> Thanks,
>>> Pradeep S
>>>
>>>
>>> -----Original Message-----
>>> From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
>>> Sent: Wednesday, February 13, 2013 10:31 PM
>>> To: 'Marcus Sorensen'; Wido den Hollander
>>> Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
>>> Subject: RE: [DISCUSS] Packaging in 4.1
>>>
>>> Hey Marcus,
>>>
>>> I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
>>>
>>> <from toplevel project dir>
>>> rm -rf dist
>>> mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
>>>
>>> tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
>>>
>>> rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
>>>
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Hugo
>>>> -----Original Message-----
>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>> Sent: Wednesday, February 13, 2013 3:08 PM
>>>> To: Wido den Hollander
>>>> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
>>>> Soundararajan; cloudstack-dev@incubator.apache.org
>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>
>>>> Hm, this package.sh is still doing weird things for me. If I pull a
>>>> fresh incubator-cloudstack and run this:
>>>>
>>>> cd packaging/centos63
>>>> chmod +x package.sh
>>>> ./package.sh
>>>>
>>>> I get this output (truncated, but you get the idea)
>>>>
>>>> tar: \rDownloaded\:: Cannot stat: No such file or directory
>>>> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
>>>> metadata.xml:
>>>> Cannot stat: No such file or directory
>>>> tar: (22: Cannot stat: No such file or directory
>>>> tar: KB: Cannot stat: No such file or directory
>>>> tar: at: Cannot stat: No such file or directory
>>>> tar: 96.8: Cannot stat: No such file or directory
>>>> tar: KB/sec): Cannot stat: No such file or directory
>>>>
>>>> [root@devcloud-kvm centos63]# ls
>>>> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
>>>> ?54 ?72 92
>>>> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
>>>> ?55 728.8 ?92
>>>> (10 121.3 14.5 173 204 238 ?271 326.8 (402
>>>> 551 73 921.8
>>>> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
>>>> (551 ?73 93
>>>> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
>>>> 55.7 739.4 ?93
>>>> ?100 122.1 148 176 ?205 ?239 275.6 ?33
>>>> 4.1.0-SNAPSHOT 557.5 74 93.0
>>>> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
>>>> 56 ?74 93.3
>>>> ?101 123 149 177 207 (24 278.3 331 ?42
>>>> ?56 75 94
>>>> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
>>>> 56.0 75.5 ?94
>>>> 102 124 15 ?178 ?208 ?240 28 335 (424
>>>> ?57 755.1 94.2
>>>> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
>>>> 58 ?76 95
>>>> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
>>>> ?58 762.9 954.6
>>>> 104 126 150 (18 209.4 ?243 280.9 339 437.4
>>>> 580.7 77 96
>>>> ?104 ?126 (150 ?18 210 243.6 282 34 44
>>>> 58.2 ?77 ?96
>>>> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
>>>> 582.0 78 96.1
>>>> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
>>>> 582.9 ?78 96.8
>>>> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
>>>> (59 79 ?97
>>>> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
>>>> ?59 8 97.5
>>>> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
>>>> 598.6 (8 98
>>>> 108 ?13 153 184.0 214 25 ?289 35 455.7
>>>> 6 ?8 ?98
>>>> ?108 130 ?153 185 215 (25 (29 ?35 46
>>>> (6 ?80 99
>>>> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
>>>> ?6 ?81 998
>>>> (11 ?131 153.9 1873.3 (216 250 290 355 468
>>>> 60 818.7 (998
>>>> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
>>>> (60 82 99.8
>>>> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
>>>> ?60 ?82 at
>>>> ?110 ?132 156 1883.0 218 251.6 294 36 47
>>>> ?61 83 available
>>>> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
>>>> 618.0 83.3 B
>>>> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
>>>> 62 84 cloud-agent.rc
>>>> 111 ?134 ?158 19 (22 254 29.7 363 471
>>>> ?62 ?84 cloud-ipallocator.rc
>>>> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
>>>> 63 8.4 cloud-management.rc
>>>> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
>>>> (63 ?85 cloud-management.sysconfig
>>>> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
>>>> ?63 85.7 cloud.spec
>>>> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
>>>> 636.5 859.6 cloud-usage.rc
>>>> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
>>>> 64 86 dependency
>>>> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
>>>> ?64 ?86 ?Downloaded:
>>>> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
>>>> 66 86.8 Downloading:
>>>> 115 138 161 ?197 226 ?258 ?30 374 488.4
>>>> ?66 87 for
>>>> 11.5 ?138 16.1 198 227 258.0 302 378 (49
>>>> 67 ?87 http:
>>>> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
>>>> ?67 87.7 information
>>>> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
>>>> 68 88 is
>>>> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
>>>> ?68 (88 KB
>>>> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
>>>> 68.2 ?88 missing,
>>>> (117 ?14 166 20 (23 (260 (31 ?39 (5
>>>> 684.9 89 no
>>>> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
>>>> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
>>>> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
>>>> ?69 ?89 package.sh
>>>> 118 141 168 20.0 230 262 311 398 ?50
>>>> (7 89.0 POM
>>>> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
>>>> ?7 (9 replace.properties
>>>> (119 142 169 201 ?231 ?263 31.8 399.5 (51
>>>> 70 ?9 The
>>>> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
>>>> ?70 90 ?[WARNING]
>>>> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
>>>> 70.2 ?90
>>>> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
>>>> 71 901.6
>>>> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
>>>> ?71 91
>>>> 120 144.8 172 203 ?235 (27 ?32 ?40 54
>>>> 72 91.4
>>>>
>>>> Then I 'git clean -fxd', rerun package.sh, and everything works. I
>>>> wonder if it's setting an env variable that allows it to work the second time.
>>>>
>>>> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
>>>> <sh...@gmail.com> wrote:
>>>> > The packaging/centos63/package.sh makes some assumptions about how
>>>> > it's being run that end up with some ugly results if it's not done
>>>> > exactly right. For example, I tried from the incubator-cloudstack
>>>> > directory: "sh ./packaging/centos63/package.sh", which seemed to
>>>> > copy /proc into my current directory and attemped to tar it up. Then
>>>> > I did "cd packaging/centos63; sh ./package.sh", which ended up with
>>>> > roughly the same result, although it died trying to run "Downloading:
>>>> > http://repo.maven..." as a bash command.
>>>> >
>>>> > Seems it didn't like being run as an 'sh' either way, even though
>>>> > it's not in the code as executable. After doing a chmod to make it
>>>> > executable, it seems to work but only if your cwd is
>>>> > incubator-cloudstack/packaging/centos63.
>>>> >
>>>> > Maybe we could change it to fail gracefully if your current path
>>>> > doesn't end in "packaging/centos63", and make it executable in git?
>>>> >
>>>> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
>>>> wrote:
>>>> >>
>>>> >>
>>>> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>>> >>>
>>>> >>> Hey guys,
>>>> >>>
>>>> >>> Just a quick note before the weekend with a status update on RPM.
>>>> >>>
>>>> >>> The management server package is pretty much done and installation
>>>> >>> on a clean system works like a charm. This is actually tested
>>>> >>> every few hours with a Jenkins setup a colleague and I built. We
>>>> >>> take the sources, compile and package. The packages are added to a
>>>> >>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
>>>> >>> One is configured as database server and another one as CloudStack
>>>> >>> management server by chef. After installation an ApiKey is created
>>>> >>> for the admin user. This proves that the package can be installed
>>>> >>> on a
>>>> clean system and that the management server starts.
>>>> >>>
>>>> >>> With this testing we have found several issues of which a few
>>>> >>> haven't been resolved yet (hopefully this weekend):
>>>> >>>
>>>> >>> * 4.1-new-db-schema.sql is not loaded by
>>>> >>> cloudstack-setup-databases
>>>> >>> * userid is null in reponse to a login call with the admin user,
>>>> >>> expected
>>>> >>> 2
>>>> >>> * Excryption initialization is now done in Transaction, this
>>>> >>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail is
>>>> >>> db.properties is not in the classpath
>>>> >>>
>>>> >>> Next week:
>>>> >>> * we will continue with the setup and add some real tests to
>>>> >>> create zones and add hypervisors.
>>>> >>> *I will also start testing with the agent and usage package,
>>>> >>> they are created at the moment but not tested for functionality.
>>>> >>> * Deploy fedora 18 image and extend the test to that
>>>> >>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
>>>> >>> with
>>>> >>> wido/noa)
>>>> >>>
>>>> >>
>>>> >> We'll sync next week! A lot of the .deb work is already done, but
>>>> >> we just have to make sure the RPM and DEB packages contain the same files.
>>>> >>
>>>> >> Then it will just be tuning and some work in the pre and postinst
>>>> >> files, but that could be a pain, but we'll just see when we go along.
>>>> >>
>>>> >> Wido
>>>> >>
>>>> >>
>>>> >>> Cheers,
>>>> >>>
>>>> >>> Hugo
>>>> >>>
>>>> >>>> -----Original Message-----
>>>> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>>> >>>> Sent: Thursday, February 07, 2013 2:39 AM
>>>> >>>> To: David Nalley
>>>> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
>>>> >>>> cloudstack- dev@incubator.apache.org
>>>> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>> >>>>
>>>> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>>> >>>>>
>>>> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
>>>> <Al...@citrix.com>
>>>> >>>>
>>>> >>>> wrote:
>>>> >>>>>>>
>>>> >>>>>>> Well, first, Apache CloudStack only releases source code.
>>>> >>>>>>>
>>>> >>>>>>> But Wido is kind enough to also host RPM / DEB package repos
>>>> >>>>>>> for users to take advantage of. Our install guide explains
>>>> >>>>>>> how to build from source, as well as how to use Wido's repos.
>>>> >>>>>>>
>>>> >>>>>>> This was all true for 4.0.0-incubating, and I think it still
>>>> >>>>>>> holds true for all future releases.
>>>> >>>>>>>
>>>> >>>>>> Chip,
>>>> >>>>>>
>>>> >>>>>> Can you refresh my memory as to why this is? I look at
>>>> >>>>>> something like cxf
>>>> >>>>
>>>> >>>> or tomcat, they all have binary downloads available.
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> http://cxf.apache.org/download.html
>>>> >>>>>>
>>>> >>>>>
>>>> >>>>> Because providing 'binaries' isn't necessarily problematic, but
>>>> >>>>> making yum and apt repos work in the ASF mirror system seems a
>>>> >>>>> bit more of an issue. Plus, Wido stepped up to do the work, no
>>>> >>>>> one else has offered any other alternatives.
>>>> >>>>>
>>>> >>>>> --David
>>>> >>>>>
>>>> >>>>
>>>> >>>> Yup - exactly what David said. We had discussed trying to get
>>>> >>>> ASF Infra to help us host package repos somewhere, but I don't
>>>> >>>> think that went anywhere. And since Wido's doing it, it avoided
>>>> >>>> all sorts of questions from the infra team around mirrors,
>>>> >>>> archiving, etc...
>>>> >>>>
>>>> >>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Marcus Sorensen <sh...@gmail.com>.
Yes, this line here in package.sh:
VERSION=`(cd ../../; mvn
org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate
-Dexpression=project.version) | grep -v '^\['`
if you don't have all of the maven stuff downloaded, instead of spitting out:
4.1.0-SNAPSHOT
It spits out:
Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.pom
(2 KB at 1.1 KB/sec)
Downloading: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
Downloaded: http://repo.maven.apache.org/maven2/jtidy/jtidy/4aug2000r7-dev/jtidy-4aug2000r7-dev.jar
(135 KB at 1094.3 KB/sec)
4.1.0-SNAPSHOT
On Wed, Feb 13, 2013 at 10:45 AM, Marcus Sorensen <sh...@gmail.com> wrote:
> It works for me subsequent to the first run.
>
> I think it's because of the output of downloading maven stuff, the
> script doesn't seem to like that.
>
> On Wed, Feb 13, 2013 at 10:42 AM, Pradeep Soundararajan
> <pr...@citrix.com> wrote:
>> I am following these steps:
>>
>> chmod 755 ./packaging/centos63/package.sh
>> cd packaging/centos63
>> ./package.sh
>>
>> cd $WORKSPACE
>> tempdir=`mktemp -d`
>> mkdir -p "$tempdir"
>> cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
>> createrepo $tempdir/
>>
>> The above is working well for me.
>>
>> Thanks,
>> Pradeep S
>>
>>
>> -----Original Message-----
>> From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
>> Sent: Wednesday, February 13, 2013 10:31 PM
>> To: 'Marcus Sorensen'; Wido den Hollander
>> Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
>> Subject: RE: [DISCUSS] Packaging in 4.1
>>
>> Hey Marcus,
>>
>> I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
>>
>> <from toplevel project dir>
>> rm -rf dist
>> mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
>>
>> tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
>>
>> rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
>>
>>
>>
>>
>> Cheers,
>>
>> Hugo
>>> -----Original Message-----
>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>> Sent: Wednesday, February 13, 2013 3:08 PM
>>> To: Wido den Hollander
>>> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
>>> Soundararajan; cloudstack-dev@incubator.apache.org
>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>
>>> Hm, this package.sh is still doing weird things for me. If I pull a
>>> fresh incubator-cloudstack and run this:
>>>
>>> cd packaging/centos63
>>> chmod +x package.sh
>>> ./package.sh
>>>
>>> I get this output (truncated, but you get the idea)
>>>
>>> tar: \rDownloaded\:: Cannot stat: No such file or directory
>>> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
>>> metadata.xml:
>>> Cannot stat: No such file or directory
>>> tar: (22: Cannot stat: No such file or directory
>>> tar: KB: Cannot stat: No such file or directory
>>> tar: at: Cannot stat: No such file or directory
>>> tar: 96.8: Cannot stat: No such file or directory
>>> tar: KB/sec): Cannot stat: No such file or directory
>>>
>>> [root@devcloud-kvm centos63]# ls
>>> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
>>> ?54 ?72 92
>>> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
>>> ?55 728.8 ?92
>>> (10 121.3 14.5 173 204 238 ?271 326.8 (402
>>> 551 73 921.8
>>> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
>>> (551 ?73 93
>>> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
>>> 55.7 739.4 ?93
>>> ?100 122.1 148 176 ?205 ?239 275.6 ?33
>>> 4.1.0-SNAPSHOT 557.5 74 93.0
>>> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
>>> 56 ?74 93.3
>>> ?101 123 149 177 207 (24 278.3 331 ?42
>>> ?56 75 94
>>> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
>>> 56.0 75.5 ?94
>>> 102 124 15 ?178 ?208 ?240 28 335 (424
>>> ?57 755.1 94.2
>>> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
>>> 58 ?76 95
>>> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
>>> ?58 762.9 954.6
>>> 104 126 150 (18 209.4 ?243 280.9 339 437.4
>>> 580.7 77 96
>>> ?104 ?126 (150 ?18 210 243.6 282 34 44
>>> 58.2 ?77 ?96
>>> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
>>> 582.0 78 96.1
>>> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
>>> 582.9 ?78 96.8
>>> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
>>> (59 79 ?97
>>> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
>>> ?59 8 97.5
>>> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
>>> 598.6 (8 98
>>> 108 ?13 153 184.0 214 25 ?289 35 455.7
>>> 6 ?8 ?98
>>> ?108 130 ?153 185 215 (25 (29 ?35 46
>>> (6 ?80 99
>>> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
>>> ?6 ?81 998
>>> (11 ?131 153.9 1873.3 (216 250 290 355 468
>>> 60 818.7 (998
>>> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
>>> (60 82 99.8
>>> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
>>> ?60 ?82 at
>>> ?110 ?132 156 1883.0 218 251.6 294 36 47
>>> ?61 83 available
>>> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
>>> 618.0 83.3 B
>>> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
>>> 62 84 cloud-agent.rc
>>> 111 ?134 ?158 19 (22 254 29.7 363 471
>>> ?62 ?84 cloud-ipallocator.rc
>>> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
>>> 63 8.4 cloud-management.rc
>>> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
>>> (63 ?85 cloud-management.sysconfig
>>> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
>>> ?63 85.7 cloud.spec
>>> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
>>> 636.5 859.6 cloud-usage.rc
>>> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
>>> 64 86 dependency
>>> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
>>> ?64 ?86 ?Downloaded:
>>> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
>>> 66 86.8 Downloading:
>>> 115 138 161 ?197 226 ?258 ?30 374 488.4
>>> ?66 87 for
>>> 11.5 ?138 16.1 198 227 258.0 302 378 (49
>>> 67 ?87 http:
>>> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
>>> ?67 87.7 information
>>> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
>>> 68 88 is
>>> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
>>> ?68 (88 KB
>>> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
>>> 68.2 ?88 missing,
>>> (117 ?14 166 20 (23 (260 (31 ?39 (5
>>> 684.9 89 no
>>> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
>>> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
>>> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
>>> ?69 ?89 package.sh
>>> 118 141 168 20.0 230 262 311 398 ?50
>>> (7 89.0 POM
>>> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
>>> ?7 (9 replace.properties
>>> (119 142 169 201 ?231 ?263 31.8 399.5 (51
>>> 70 ?9 The
>>> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
>>> ?70 90 ?[WARNING]
>>> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
>>> 70.2 ?90
>>> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
>>> 71 901.6
>>> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
>>> ?71 91
>>> 120 144.8 172 203 ?235 (27 ?32 ?40 54
>>> 72 91.4
>>>
>>> Then I 'git clean -fxd', rerun package.sh, and everything works. I
>>> wonder if it's setting an env variable that allows it to work the second time.
>>>
>>> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
>>> <sh...@gmail.com> wrote:
>>> > The packaging/centos63/package.sh makes some assumptions about how
>>> > it's being run that end up with some ugly results if it's not done
>>> > exactly right. For example, I tried from the incubator-cloudstack
>>> > directory: "sh ./packaging/centos63/package.sh", which seemed to
>>> > copy /proc into my current directory and attemped to tar it up. Then
>>> > I did "cd packaging/centos63; sh ./package.sh", which ended up with
>>> > roughly the same result, although it died trying to run "Downloading:
>>> > http://repo.maven..." as a bash command.
>>> >
>>> > Seems it didn't like being run as an 'sh' either way, even though
>>> > it's not in the code as executable. After doing a chmod to make it
>>> > executable, it seems to work but only if your cwd is
>>> > incubator-cloudstack/packaging/centos63.
>>> >
>>> > Maybe we could change it to fail gracefully if your current path
>>> > doesn't end in "packaging/centos63", and make it executable in git?
>>> >
>>> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
>>> wrote:
>>> >>
>>> >>
>>> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>> >>>
>>> >>> Hey guys,
>>> >>>
>>> >>> Just a quick note before the weekend with a status update on RPM.
>>> >>>
>>> >>> The management server package is pretty much done and installation
>>> >>> on a clean system works like a charm. This is actually tested
>>> >>> every few hours with a Jenkins setup a colleague and I built. We
>>> >>> take the sources, compile and package. The packages are added to a
>>> >>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
>>> >>> One is configured as database server and another one as CloudStack
>>> >>> management server by chef. After installation an ApiKey is created
>>> >>> for the admin user. This proves that the package can be installed
>>> >>> on a
>>> clean system and that the management server starts.
>>> >>>
>>> >>> With this testing we have found several issues of which a few
>>> >>> haven't been resolved yet (hopefully this weekend):
>>> >>>
>>> >>> * 4.1-new-db-schema.sql is not loaded by
>>> >>> cloudstack-setup-databases
>>> >>> * userid is null in reponse to a login call with the admin user,
>>> >>> expected
>>> >>> 2
>>> >>> * Excryption initialization is now done in Transaction, this
>>> >>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail is
>>> >>> db.properties is not in the classpath
>>> >>>
>>> >>> Next week:
>>> >>> * we will continue with the setup and add some real tests to
>>> >>> create zones and add hypervisors.
>>> >>> *I will also start testing with the agent and usage package,
>>> >>> they are created at the moment but not tested for functionality.
>>> >>> * Deploy fedora 18 image and extend the test to that
>>> >>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
>>> >>> with
>>> >>> wido/noa)
>>> >>>
>>> >>
>>> >> We'll sync next week! A lot of the .deb work is already done, but
>>> >> we just have to make sure the RPM and DEB packages contain the same files.
>>> >>
>>> >> Then it will just be tuning and some work in the pre and postinst
>>> >> files, but that could be a pain, but we'll just see when we go along.
>>> >>
>>> >> Wido
>>> >>
>>> >>
>>> >>> Cheers,
>>> >>>
>>> >>> Hugo
>>> >>>
>>> >>>> -----Original Message-----
>>> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>> >>>> Sent: Thursday, February 07, 2013 2:39 AM
>>> >>>> To: David Nalley
>>> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
>>> >>>> cloudstack- dev@incubator.apache.org
>>> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>> >>>>
>>> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>> >>>>>
>>> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
>>> <Al...@citrix.com>
>>> >>>>
>>> >>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Well, first, Apache CloudStack only releases source code.
>>> >>>>>>>
>>> >>>>>>> But Wido is kind enough to also host RPM / DEB package repos
>>> >>>>>>> for users to take advantage of. Our install guide explains
>>> >>>>>>> how to build from source, as well as how to use Wido's repos.
>>> >>>>>>>
>>> >>>>>>> This was all true for 4.0.0-incubating, and I think it still
>>> >>>>>>> holds true for all future releases.
>>> >>>>>>>
>>> >>>>>> Chip,
>>> >>>>>>
>>> >>>>>> Can you refresh my memory as to why this is? I look at
>>> >>>>>> something like cxf
>>> >>>>
>>> >>>> or tomcat, they all have binary downloads available.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> http://cxf.apache.org/download.html
>>> >>>>>>
>>> >>>>>
>>> >>>>> Because providing 'binaries' isn't necessarily problematic, but
>>> >>>>> making yum and apt repos work in the ASF mirror system seems a
>>> >>>>> bit more of an issue. Plus, Wido stepped up to do the work, no
>>> >>>>> one else has offered any other alternatives.
>>> >>>>>
>>> >>>>> --David
>>> >>>>>
>>> >>>>
>>> >>>> Yup - exactly what David said. We had discussed trying to get
>>> >>>> ASF Infra to help us host package repos somewhere, but I don't
>>> >>>> think that went anywhere. And since Wido's doing it, it avoided
>>> >>>> all sorts of questions from the infra team around mirrors,
>>> >>>> archiving, etc...
>>> >>>>
>>> >>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Marcus Sorensen <sh...@gmail.com>.
It works for me subsequent to the first run.
I think it's because of the output of downloading maven stuff, the
script doesn't seem to like that.
On Wed, Feb 13, 2013 at 10:42 AM, Pradeep Soundararajan
<pr...@citrix.com> wrote:
> I am following these steps:
>
> chmod 755 ./packaging/centos63/package.sh
> cd packaging/centos63
> ./package.sh
>
> cd $WORKSPACE
> tempdir=`mktemp -d`
> mkdir -p "$tempdir"
> cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
> createrepo $tempdir/
>
> The above is working well for me.
>
> Thanks,
> Pradeep S
>
>
> -----Original Message-----
> From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
> Sent: Wednesday, February 13, 2013 10:31 PM
> To: 'Marcus Sorensen'; Wido den Hollander
> Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] Packaging in 4.1
>
> Hey Marcus,
>
> I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
>
> <from toplevel project dir>
> rm -rf dist
> mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
>
> tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
>
> rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
>
>
>
>
> Cheers,
>
> Hugo
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Wednesday, February 13, 2013 3:08 PM
>> To: Wido den Hollander
>> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
>> Soundararajan; cloudstack-dev@incubator.apache.org
>> Subject: Re: [DISCUSS] Packaging in 4.1
>>
>> Hm, this package.sh is still doing weird things for me. If I pull a
>> fresh incubator-cloudstack and run this:
>>
>> cd packaging/centos63
>> chmod +x package.sh
>> ./package.sh
>>
>> I get this output (truncated, but you get the idea)
>>
>> tar: \rDownloaded\:: Cannot stat: No such file or directory
>> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
>> metadata.xml:
>> Cannot stat: No such file or directory
>> tar: (22: Cannot stat: No such file or directory
>> tar: KB: Cannot stat: No such file or directory
>> tar: at: Cannot stat: No such file or directory
>> tar: 96.8: Cannot stat: No such file or directory
>> tar: KB/sec): Cannot stat: No such file or directory
>>
>> [root@devcloud-kvm centos63]# ls
>> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
>> ?54 ?72 92
>> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
>> ?55 728.8 ?92
>> (10 121.3 14.5 173 204 238 ?271 326.8 (402
>> 551 73 921.8
>> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
>> (551 ?73 93
>> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
>> 55.7 739.4 ?93
>> ?100 122.1 148 176 ?205 ?239 275.6 ?33
>> 4.1.0-SNAPSHOT 557.5 74 93.0
>> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
>> 56 ?74 93.3
>> ?101 123 149 177 207 (24 278.3 331 ?42
>> ?56 75 94
>> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
>> 56.0 75.5 ?94
>> 102 124 15 ?178 ?208 ?240 28 335 (424
>> ?57 755.1 94.2
>> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
>> 58 ?76 95
>> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
>> ?58 762.9 954.6
>> 104 126 150 (18 209.4 ?243 280.9 339 437.4
>> 580.7 77 96
>> ?104 ?126 (150 ?18 210 243.6 282 34 44
>> 58.2 ?77 ?96
>> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
>> 582.0 78 96.1
>> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
>> 582.9 ?78 96.8
>> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
>> (59 79 ?97
>> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
>> ?59 8 97.5
>> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
>> 598.6 (8 98
>> 108 ?13 153 184.0 214 25 ?289 35 455.7
>> 6 ?8 ?98
>> ?108 130 ?153 185 215 (25 (29 ?35 46
>> (6 ?80 99
>> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
>> ?6 ?81 998
>> (11 ?131 153.9 1873.3 (216 250 290 355 468
>> 60 818.7 (998
>> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
>> (60 82 99.8
>> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
>> ?60 ?82 at
>> ?110 ?132 156 1883.0 218 251.6 294 36 47
>> ?61 83 available
>> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
>> 618.0 83.3 B
>> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
>> 62 84 cloud-agent.rc
>> 111 ?134 ?158 19 (22 254 29.7 363 471
>> ?62 ?84 cloud-ipallocator.rc
>> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
>> 63 8.4 cloud-management.rc
>> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
>> (63 ?85 cloud-management.sysconfig
>> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
>> ?63 85.7 cloud.spec
>> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
>> 636.5 859.6 cloud-usage.rc
>> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
>> 64 86 dependency
>> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
>> ?64 ?86 ?Downloaded:
>> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
>> 66 86.8 Downloading:
>> 115 138 161 ?197 226 ?258 ?30 374 488.4
>> ?66 87 for
>> 11.5 ?138 16.1 198 227 258.0 302 378 (49
>> 67 ?87 http:
>> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
>> ?67 87.7 information
>> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
>> 68 88 is
>> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
>> ?68 (88 KB
>> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
>> 68.2 ?88 missing,
>> (117 ?14 166 20 (23 (260 (31 ?39 (5
>> 684.9 89 no
>> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
>> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
>> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
>> ?69 ?89 package.sh
>> 118 141 168 20.0 230 262 311 398 ?50
>> (7 89.0 POM
>> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
>> ?7 (9 replace.properties
>> (119 142 169 201 ?231 ?263 31.8 399.5 (51
>> 70 ?9 The
>> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
>> ?70 90 ?[WARNING]
>> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
>> 70.2 ?90
>> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
>> 71 901.6
>> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
>> ?71 91
>> 120 144.8 172 203 ?235 (27 ?32 ?40 54
>> 72 91.4
>>
>> Then I 'git clean -fxd', rerun package.sh, and everything works. I
>> wonder if it's setting an env variable that allows it to work the second time.
>>
>> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
>> <sh...@gmail.com> wrote:
>> > The packaging/centos63/package.sh makes some assumptions about how
>> > it's being run that end up with some ugly results if it's not done
>> > exactly right. For example, I tried from the incubator-cloudstack
>> > directory: "sh ./packaging/centos63/package.sh", which seemed to
>> > copy /proc into my current directory and attemped to tar it up. Then
>> > I did "cd packaging/centos63; sh ./package.sh", which ended up with
>> > roughly the same result, although it died trying to run "Downloading:
>> > http://repo.maven..." as a bash command.
>> >
>> > Seems it didn't like being run as an 'sh' either way, even though
>> > it's not in the code as executable. After doing a chmod to make it
>> > executable, it seems to work but only if your cwd is
>> > incubator-cloudstack/packaging/centos63.
>> >
>> > Maybe we could change it to fail gracefully if your current path
>> > doesn't end in "packaging/centos63", and make it executable in git?
>> >
>> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
>> wrote:
>> >>
>> >>
>> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>> >>>
>> >>> Hey guys,
>> >>>
>> >>> Just a quick note before the weekend with a status update on RPM.
>> >>>
>> >>> The management server package is pretty much done and installation
>> >>> on a clean system works like a charm. This is actually tested
>> >>> every few hours with a Jenkins setup a colleague and I built. We
>> >>> take the sources, compile and package. The packages are added to a
>> >>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
>> >>> One is configured as database server and another one as CloudStack
>> >>> management server by chef. After installation an ApiKey is created
>> >>> for the admin user. This proves that the package can be installed
>> >>> on a
>> clean system and that the management server starts.
>> >>>
>> >>> With this testing we have found several issues of which a few
>> >>> haven't been resolved yet (hopefully this weekend):
>> >>>
>> >>> * 4.1-new-db-schema.sql is not loaded by
>> >>> cloudstack-setup-databases
>> >>> * userid is null in reponse to a login call with the admin user,
>> >>> expected
>> >>> 2
>> >>> * Excryption initialization is now done in Transaction, this
>> >>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail is
>> >>> db.properties is not in the classpath
>> >>>
>> >>> Next week:
>> >>> * we will continue with the setup and add some real tests to
>> >>> create zones and add hypervisors.
>> >>> *I will also start testing with the agent and usage package,
>> >>> they are created at the moment but not tested for functionality.
>> >>> * Deploy fedora 18 image and extend the test to that
>> >>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
>> >>> with
>> >>> wido/noa)
>> >>>
>> >>
>> >> We'll sync next week! A lot of the .deb work is already done, but
>> >> we just have to make sure the RPM and DEB packages contain the same files.
>> >>
>> >> Then it will just be tuning and some work in the pre and postinst
>> >> files, but that could be a pain, but we'll just see when we go along.
>> >>
>> >> Wido
>> >>
>> >>
>> >>> Cheers,
>> >>>
>> >>> Hugo
>> >>>
>> >>>> -----Original Message-----
>> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>> >>>> Sent: Thursday, February 07, 2013 2:39 AM
>> >>>> To: David Nalley
>> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
>> >>>> cloudstack- dev@incubator.apache.org
>> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
>> >>>>
>> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>> >>>>>
>> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
>> <Al...@citrix.com>
>> >>>>
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>> Well, first, Apache CloudStack only releases source code.
>> >>>>>>>
>> >>>>>>> But Wido is kind enough to also host RPM / DEB package repos
>> >>>>>>> for users to take advantage of. Our install guide explains
>> >>>>>>> how to build from source, as well as how to use Wido's repos.
>> >>>>>>>
>> >>>>>>> This was all true for 4.0.0-incubating, and I think it still
>> >>>>>>> holds true for all future releases.
>> >>>>>>>
>> >>>>>> Chip,
>> >>>>>>
>> >>>>>> Can you refresh my memory as to why this is? I look at
>> >>>>>> something like cxf
>> >>>>
>> >>>> or tomcat, they all have binary downloads available.
>> >>>>>>
>> >>>>>>
>> >>>>>> http://cxf.apache.org/download.html
>> >>>>>>
>> >>>>>
>> >>>>> Because providing 'binaries' isn't necessarily problematic, but
>> >>>>> making yum and apt repos work in the ASF mirror system seems a
>> >>>>> bit more of an issue. Plus, Wido stepped up to do the work, no
>> >>>>> one else has offered any other alternatives.
>> >>>>>
>> >>>>> --David
>> >>>>>
>> >>>>
>> >>>> Yup - exactly what David said. We had discussed trying to get
>> >>>> ASF Infra to help us host package repos somewhere, but I don't
>> >>>> think that went anywhere. And since Wido's doing it, it avoided
>> >>>> all sorts of questions from the infra team around mirrors,
>> >>>> archiving, etc...
>> >>>>
>> >>>> -chip
RE: [DISCUSS] Packaging in 4.1
Posted by Pradeep Soundararajan <pr...@citrix.com>.
I am following these steps:
chmod 755 ./packaging/centos63/package.sh
cd packaging/centos63
./package.sh
cd $WORKSPACE
tempdir=`mktemp -d`
mkdir -p "$tempdir"
cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
createrepo $tempdir/
The above is working well for me.
Thanks,
Pradeep S
-----Original Message-----
From: Hugo Trippaers [mailto:HTrippaers@schubergphilis.com]
Sent: Wednesday, February 13, 2013 10:31 PM
To: 'Marcus Sorensen'; Wido den Hollander
Cc: Chip Childers; David Nalley; Alex Huang; Pradeep Soundararajan; cloudstack-dev@incubator.apache.org
Subject: RE: [DISCUSS] Packaging in 4.1
Hey Marcus,
I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
<from toplevel project dir>
rm -rf dist
mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
Cheers,
Hugo
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Wednesday, February 13, 2013 3:08 PM
> To: Wido den Hollander
> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
> Soundararajan; cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
> Hm, this package.sh is still doing weird things for me. If I pull a
> fresh incubator-cloudstack and run this:
>
> cd packaging/centos63
> chmod +x package.sh
> ./package.sh
>
> I get this output (truncated, but you get the idea)
>
> tar: \rDownloaded\:: Cannot stat: No such file or directory
> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
> metadata.xml:
> Cannot stat: No such file or directory
> tar: (22: Cannot stat: No such file or directory
> tar: KB: Cannot stat: No such file or directory
> tar: at: Cannot stat: No such file or directory
> tar: 96.8: Cannot stat: No such file or directory
> tar: KB/sec): Cannot stat: No such file or directory
>
> [root@devcloud-kvm centos63]# ls
> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
> ?54 ?72 92
> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
> ?55 728.8 ?92
> (10 121.3 14.5 173 204 238 ?271 326.8 (402
> 551 73 921.8
> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
> (551 ?73 93
> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
> 55.7 739.4 ?93
> ?100 122.1 148 176 ?205 ?239 275.6 ?33
> 4.1.0-SNAPSHOT 557.5 74 93.0
> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
> 56 ?74 93.3
> ?101 123 149 177 207 (24 278.3 331 ?42
> ?56 75 94
> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
> 56.0 75.5 ?94
> 102 124 15 ?178 ?208 ?240 28 335 (424
> ?57 755.1 94.2
> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
> 58 ?76 95
> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
> ?58 762.9 954.6
> 104 126 150 (18 209.4 ?243 280.9 339 437.4
> 580.7 77 96
> ?104 ?126 (150 ?18 210 243.6 282 34 44
> 58.2 ?77 ?96
> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
> 582.0 78 96.1
> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
> 582.9 ?78 96.8
> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
> (59 79 ?97
> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
> ?59 8 97.5
> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
> 598.6 (8 98
> 108 ?13 153 184.0 214 25 ?289 35 455.7
> 6 ?8 ?98
> ?108 130 ?153 185 215 (25 (29 ?35 46
> (6 ?80 99
> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
> ?6 ?81 998
> (11 ?131 153.9 1873.3 (216 250 290 355 468
> 60 818.7 (998
> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
> (60 82 99.8
> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
> ?60 ?82 at
> ?110 ?132 156 1883.0 218 251.6 294 36 47
> ?61 83 available
> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
> 618.0 83.3 B
> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
> 62 84 cloud-agent.rc
> 111 ?134 ?158 19 (22 254 29.7 363 471
> ?62 ?84 cloud-ipallocator.rc
> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
> 63 8.4 cloud-management.rc
> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
> (63 ?85 cloud-management.sysconfig
> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
> ?63 85.7 cloud.spec
> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
> 636.5 859.6 cloud-usage.rc
> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
> 64 86 dependency
> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
> ?64 ?86 ?Downloaded:
> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
> 66 86.8 Downloading:
> 115 138 161 ?197 226 ?258 ?30 374 488.4
> ?66 87 for
> 11.5 ?138 16.1 198 227 258.0 302 378 (49
> 67 ?87 http:
> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
> ?67 87.7 information
> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
> 68 88 is
> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
> ?68 (88 KB
> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
> 68.2 ?88 missing,
> (117 ?14 166 20 (23 (260 (31 ?39 (5
> 684.9 89 no
> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
> ?69 ?89 package.sh
> 118 141 168 20.0 230 262 311 398 ?50
> (7 89.0 POM
> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
> ?7 (9 replace.properties
> (119 142 169 201 ?231 ?263 31.8 399.5 (51
> 70 ?9 The
> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
> ?70 90 ?[WARNING]
> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
> 70.2 ?90
> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
> 71 901.6
> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
> ?71 91
> 120 144.8 172 203 ?235 (27 ?32 ?40 54
> 72 91.4
>
> Then I 'git clean -fxd', rerun package.sh, and everything works. I
> wonder if it's setting an env variable that allows it to work the second time.
>
> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
> <sh...@gmail.com> wrote:
> > The packaging/centos63/package.sh makes some assumptions about how
> > it's being run that end up with some ugly results if it's not done
> > exactly right. For example, I tried from the incubator-cloudstack
> > directory: "sh ./packaging/centos63/package.sh", which seemed to
> > copy /proc into my current directory and attemped to tar it up. Then
> > I did "cd packaging/centos63; sh ./package.sh", which ended up with
> > roughly the same result, although it died trying to run "Downloading:
> > http://repo.maven..." as a bash command.
> >
> > Seems it didn't like being run as an 'sh' either way, even though
> > it's not in the code as executable. After doing a chmod to make it
> > executable, it seems to work but only if your cwd is
> > incubator-cloudstack/packaging/centos63.
> >
> > Maybe we could change it to fail gracefully if your current path
> > doesn't end in "packaging/centos63", and make it executable in git?
> >
> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >>
> >>
> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
> >>>
> >>> Hey guys,
> >>>
> >>> Just a quick note before the weekend with a status update on RPM.
> >>>
> >>> The management server package is pretty much done and installation
> >>> on a clean system works like a charm. This is actually tested
> >>> every few hours with a Jenkins setup a colleague and I built. We
> >>> take the sources, compile and package. The packages are added to a
> >>> repo and chef is used to deploy two new clean CentOS 6.3 boxes.
> >>> One is configured as database server and another one as CloudStack
> >>> management server by chef. After installation an ApiKey is created
> >>> for the admin user. This proves that the package can be installed
> >>> on a
> clean system and that the management server starts.
> >>>
> >>> With this testing we have found several issues of which a few
> >>> haven't been resolved yet (hopefully this weekend):
> >>>
> >>> * 4.1-new-db-schema.sql is not loaded by
> >>> cloudstack-setup-databases
> >>> * userid is null in reponse to a login call with the admin user,
> >>> expected
> >>> 2
> >>> * Excryption initialization is now done in Transaction, this
> >>> causes the mvn -Pdeveloper -pl developer -D deploydb to fail is
> >>> db.properties is not in the classpath
> >>>
> >>> Next week:
> >>> * we will continue with the setup and add some real tests to
> >>> create zones and add hypervisors.
> >>> *I will also start testing with the agent and usage package,
> >>> they are created at the moment but not tested for functionality.
> >>> * Deploy fedora 18 image and extend the test to that
> >>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
> >>> with
> >>> wido/noa)
> >>>
> >>
> >> We'll sync next week! A lot of the .deb work is already done, but
> >> we just have to make sure the RPM and DEB packages contain the same files.
> >>
> >> Then it will just be tuning and some work in the pre and postinst
> >> files, but that could be a pain, but we'll just see when we go along.
> >>
> >> Wido
> >>
> >>
> >>> Cheers,
> >>>
> >>> Hugo
> >>>
> >>>> -----Original Message-----
> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
> >>>> Sent: Thursday, February 07, 2013 2:39 AM
> >>>> To: David Nalley
> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
> >>>> cloudstack- dev@incubator.apache.org
> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
> >>>>
> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
> >>>>>
> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
> <Al...@citrix.com>
> >>>>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Well, first, Apache CloudStack only releases source code.
> >>>>>>>
> >>>>>>> But Wido is kind enough to also host RPM / DEB package repos
> >>>>>>> for users to take advantage of. Our install guide explains
> >>>>>>> how to build from source, as well as how to use Wido's repos.
> >>>>>>>
> >>>>>>> This was all true for 4.0.0-incubating, and I think it still
> >>>>>>> holds true for all future releases.
> >>>>>>>
> >>>>>> Chip,
> >>>>>>
> >>>>>> Can you refresh my memory as to why this is? I look at
> >>>>>> something like cxf
> >>>>
> >>>> or tomcat, they all have binary downloads available.
> >>>>>>
> >>>>>>
> >>>>>> http://cxf.apache.org/download.html
> >>>>>>
> >>>>>
> >>>>> Because providing 'binaries' isn't necessarily problematic, but
> >>>>> making yum and apt repos work in the ASF mirror system seems a
> >>>>> bit more of an issue. Plus, Wido stepped up to do the work, no
> >>>>> one else has offered any other alternatives.
> >>>>>
> >>>>> --David
> >>>>>
> >>>>
> >>>> Yup - exactly what David said. We had discussed trying to get
> >>>> ASF Infra to help us host package repos somewhere, but I don't
> >>>> think that went anywhere. And since Wido's doing it, it avoided
> >>>> all sorts of questions from the infra team around mirrors,
> >>>> archiving, etc...
> >>>>
> >>>> -chip
RE: [DISCUSS] Packaging in 4.1
Posted by Hugo Trippaers <HT...@schubergphilis.com>.
Hey Marcus,
I haven't updated package.sh in some time as I do most of by build directly with Jenkins. This is procedure that I'm currently using:
<from toplevel project dir>
rm -rf dist
mkdir -p dist/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
tar --transform 's,^\./,cloudstack-4.1.0-SNAPSHOT/,' -c -z -f dist/rpmbuild/SOURCES/cloudstack-4.1.0-SNAPSHOT.tgz --exclude .git --exclude dist .
rpmbuild -D"%_topdir ${WORKSPACE}/dist/rpmbuild" --ba -D"_ver 4.1.0" -D"_prerelease true" -D"_rel SNAPSHOT_${BUILD_NUMBER}" packaging/centos63/cloud.spec
Cheers,
Hugo
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Wednesday, February 13, 2013 3:08 PM
> To: Wido den Hollander
> Cc: Hugo Trippaers; Chip Childers; David Nalley; Alex Huang; Pradeep
> Soundararajan; cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
> Hm, this package.sh is still doing weird things for me. If I pull a fresh
> incubator-cloudstack and run this:
>
> cd packaging/centos63
> chmod +x package.sh
> ./package.sh
>
> I get this output (truncated, but you get the idea)
>
> tar: \rDownloaded\:: Cannot stat: No such file or directory
> tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-
> metadata.xml:
> Cannot stat: No such file or directory
> tar: (22: Cannot stat: No such file or directory
> tar: KB: Cannot stat: No such file or directory
> tar: at: Cannot stat: No such file or directory
> tar: 96.8: Cannot stat: No such file or directory
> tar: KB/sec): Cannot stat: No such file or directory
>
> [root@devcloud-kvm centos63]# ls
> ? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
> ?54 ?72 92
> 10 ?121 ?145 172.4 20.3 ?236 270 323 402
> ?55 728.8 ?92
> (10 121.3 14.5 173 204 238 ?271 326.8 (402
> 551 73 921.8
> ?10 121.8 146 ?174 ?204 238.6 274 327 405.7
> (551 ?73 93
> 100 122 ?146 175.9 20.4 239 ?275 (33 406.3
> 55.7 739.4 ?93
> ?100 122.1 148 176 ?205 ?239 275.6 ?33
> 4.1.0-SNAPSHOT 557.5 74 93.0
> 10.0 1227.3 ?148 ?176 206 24 278 330.2 42
> 56 ?74 93.3
> ?101 123 149 177 207 (24 278.3 331 ?42
> ?56 75 94
> 101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
> 56.0 75.5 ?94
> 102 124 15 ?178 ?208 ?240 28 335 (424
> ?57 755.1 94.2
> ?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
> 58 ?76 95
> 103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
> ?58 762.9 954.6
> 104 126 150 (18 209.4 ?243 280.9 339 437.4
> 580.7 77 96
> ?104 ?126 (150 ?18 210 243.6 282 34 44
> 58.2 ?77 ?96
> ?105 ?127 ?150 180 211 ?244 ?283 (34 (44
> 582.0 78 96.1
> 106 128 151 ?180 21.1 246 283.0 ?34 ?44
> 582.9 ?78 96.8
> ?106 ?128 15.1 181 211.4 247 286 34.1 443.3
> (59 79 ?97
> 107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
> ?59 8 97.5
> 10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
> 598.6 (8 98
> 108 ?13 153 184.0 214 25 ?289 35 455.7
> 6 ?8 ?98
> ?108 130 ?153 185 215 (25 (29 ?35 46
> (6 ?80 99
> ?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
> ?6 ?81 998
> (11 ?131 153.9 1873.3 (216 250 290 355 468
> 60 818.7 (998
> ?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
> (60 82 99.8
> 110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
> ?60 ?82 at
> ?110 ?132 156 1883.0 218 251.6 294 36 47
> ?61 83 available
> 11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
> 618.0 83.3 B
> 110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
> 62 84 cloud-agent.rc
> 111 ?134 ?158 19 (22 254 29.7 363 471
> ?62 ?84 cloud-ipallocator.rc
> 112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
> 63 8.4 cloud-management.rc
> ?112 ?135 16 ?190 ?220 255 298 367.6 47.9
> (63 ?85 cloud-management.sysconfig
> ?113 1359.0 (16 ?192 ?221 ?255 3 368 48
> ?63 85.7 cloud.spec
> 113.0 136 ?16 193 222 256 (3 3683.1 ?48
> 636.5 859.6 cloud-usage.rc
> 114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
> 64 86 dependency
> ?114 137 160 ?196 ?223 ?256 30 ?37 483.2
> ?64 ?86 ?Downloaded:
> 114.6 137.8 ?160 197 ?225 258 (30 370 483.5
> 66 86.8 Downloading:
> 115 138 161 ?197 226 ?258 ?30 374 488.4
> ?66 87 for
> 11.5 ?138 16.1 198 227 258.0 302 378 (49
> 67 ?87 http:
> 116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
> ?67 87.7 information
> ?116 1396.8 164 2 ?228 26 305 ?38 491.0
> 68 88 is
> 116.0 14 ?164 (2 229.4 (26 306.3 382 492
> ?68 (88 KB
> 116.7 (14 16.5 ?2 23 ?26 307 386 (492
> 68.2 ?88 missing,
> (117 ?14 166 20 (23 (260 (31 ?39 (5
> 684.9 89 no
> ?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
> 69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
> 117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
> ?69 ?89 package.sh
> 118 141 168 20.0 230 262 311 398 ?50
> (7 89.0 POM
> 119 ?141 ?168 200.9 231 26.2 315 39.9 51
> ?7 (9 replace.properties
> (119 142 169 201 ?231 ?263 31.8 399.5 (51
> 70 ?9 The
> ?119 ?142 (17 ?201 ?232 266 319 4 ?51
> ?70 90 ?[WARNING]
> 12 142.5 ?17 202 23.3 266.8 31.9 (4 52
> 70.2 ?90
> (12 144 ?170 ?202 234 ?267 32 ?4 ?52
> 71 901.6
> ?12 ?144 170.4 202.9 235 27 (32 40 5.2
> ?71 91
> 120 144.8 172 203 ?235 (27 ?32 ?40 54
> 72 91.4
>
> Then I 'git clean -fxd', rerun package.sh, and everything works. I wonder if it's
> setting an env variable that allows it to work the second time.
>
> On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen
> <sh...@gmail.com> wrote:
> > The packaging/centos63/package.sh makes some assumptions about how
> > it's being run that end up with some ugly results if it's not done
> > exactly right. For example, I tried from the incubator-cloudstack
> > directory: "sh ./packaging/centos63/package.sh", which seemed to copy
> > /proc into my current directory and attemped to tar it up. Then I did
> > "cd packaging/centos63; sh ./package.sh", which ended up with roughly
> > the same result, although it died trying to run "Downloading:
> > http://repo.maven..." as a bash command.
> >
> > Seems it didn't like being run as an 'sh' either way, even though it's
> > not in the code as executable. After doing a chmod to make it
> > executable, it seems to work but only if your cwd is
> > incubator-cloudstack/packaging/centos63.
> >
> > Maybe we could change it to fail gracefully if your current path
> > doesn't end in "packaging/centos63", and make it executable in git?
> >
> > On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >>
> >>
> >> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
> >>>
> >>> Hey guys,
> >>>
> >>> Just a quick note before the weekend with a status update on RPM.
> >>>
> >>> The management server package is pretty much done and installation
> >>> on a clean system works like a charm. This is actually tested every
> >>> few hours with a Jenkins setup a colleague and I built. We take the
> >>> sources, compile and package. The packages are added to a repo and
> >>> chef is used to deploy two new clean CentOS 6.3 boxes. One is
> >>> configured as database server and another one as CloudStack
> >>> management server by chef. After installation an ApiKey is created
> >>> for the admin user. This proves that the package can be installed on a
> clean system and that the management server starts.
> >>>
> >>> With this testing we have found several issues of which a few
> >>> haven't been resolved yet (hopefully this weekend):
> >>>
> >>> * 4.1-new-db-schema.sql is not loaded by
> >>> cloudstack-setup-databases
> >>> * userid is null in reponse to a login call with the admin user,
> >>> expected
> >>> 2
> >>> * Excryption initialization is now done in Transaction, this causes
> >>> the mvn -Pdeveloper -pl developer -D deploydb to fail is
> >>> db.properties is not in the classpath
> >>>
> >>> Next week:
> >>> * we will continue with the setup and add some real tests to
> >>> create zones and add hypervisors.
> >>> *I will also start testing with the agent and usage package, they
> >>> are created at the moment but not tested for functionality.
> >>> * Deploy fedora 18 image and extend the test to that
> >>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check
> >>> with
> >>> wido/noa)
> >>>
> >>
> >> We'll sync next week! A lot of the .deb work is already done, but we
> >> just have to make sure the RPM and DEB packages contain the same files.
> >>
> >> Then it will just be tuning and some work in the pre and postinst
> >> files, but that could be a pain, but we'll just see when we go along.
> >>
> >> Wido
> >>
> >>
> >>> Cheers,
> >>>
> >>> Hugo
> >>>
> >>>> -----Original Message-----
> >>>> From: Chip Childers [mailto:chip.childers@sungard.com]
> >>>> Sent: Thursday, February 07, 2013 2:39 AM
> >>>> To: David Nalley
> >>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander;
> >>>> cloudstack- dev@incubator.apache.org
> >>>> Subject: Re: [DISCUSS] Packaging in 4.1
> >>>>
> >>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
> >>>>>
> >>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang
> <Al...@citrix.com>
> >>>>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Well, first, Apache CloudStack only releases source code.
> >>>>>>>
> >>>>>>> But Wido is kind enough to also host RPM / DEB package repos for
> >>>>>>> users to take advantage of. Our install guide explains how to
> >>>>>>> build from source, as well as how to use Wido's repos.
> >>>>>>>
> >>>>>>> This was all true for 4.0.0-incubating, and I think it still
> >>>>>>> holds true for all future releases.
> >>>>>>>
> >>>>>> Chip,
> >>>>>>
> >>>>>> Can you refresh my memory as to why this is? I look at something
> >>>>>> like cxf
> >>>>
> >>>> or tomcat, they all have binary downloads available.
> >>>>>>
> >>>>>>
> >>>>>> http://cxf.apache.org/download.html
> >>>>>>
> >>>>>
> >>>>> Because providing 'binaries' isn't necessarily problematic, but
> >>>>> making yum and apt repos work in the ASF mirror system seems a bit
> >>>>> more of an issue. Plus, Wido stepped up to do the work, no one
> >>>>> else has offered any other alternatives.
> >>>>>
> >>>>> --David
> >>>>>
> >>>>
> >>>> Yup - exactly what David said. We had discussed trying to get ASF
> >>>> Infra to help us host package repos somewhere, but I don't think
> >>>> that went anywhere. And since Wido's doing it, it avoided all
> >>>> sorts of questions from the infra team around mirrors, archiving,
> >>>> etc...
> >>>>
> >>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Marcus Sorensen <sh...@gmail.com>.
Hm, this package.sh is still doing weird things for me. If I pull a
fresh incubator-cloudstack and run this:
cd packaging/centos63
chmod +x package.sh
./package.sh
I get this output (truncated, but you get the idea)
tar: \rDownloaded\:: Cannot stat: No such file or directory
tar: http\://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml:
Cannot stat: No such file or directory
tar: (22: Cannot stat: No such file or directory
tar: KB: Cannot stat: No such file or directory
tar: at: Cannot stat: No such file or directory
tar: 96.8: Cannot stat: No such file or directory
tar: KB/sec): Cannot stat: No such file or directory
[root@devcloud-kvm centos63]# ls
? ?120 145 ?172 ?203 235.3 ?27 320.7 401.7
?54 ?72 92
10 ?121 ?145 172.4 20.3 ?236 270 323 402
?55 728.8 ?92
(10 121.3 14.5 173 204 238 ?271 326.8 (402
551 73 921.8
?10 121.8 146 ?174 ?204 238.6 274 327 405.7
(551 ?73 93
100 122 ?146 175.9 20.4 239 ?275 (33 406.3
55.7 739.4 ?93
?100 122.1 148 176 ?205 ?239 275.6 ?33
4.1.0-SNAPSHOT 557.5 74 93.0
10.0 1227.3 ?148 ?176 206 24 278 330.2 42
56 ?74 93.3
?101 123 149 177 207 (24 278.3 331 ?42
?56 75 94
101.0 ?123 ?149 ?177 20.7 ?24 ?279 33.4 424
56.0 75.5 ?94
102 124 15 ?178 ?208 ?240 28 335 (424
?57 755.1 94.2
?102 ?124 (15 179.3 ?209 242 (28 33.5 429.8
58 ?76 95
103 ?125 ?15 18 209.0 243 ?28 3352.2 ?43
?58 762.9 954.6
104 126 150 (18 209.4 ?243 280.9 339 437.4
580.7 77 96
?104 ?126 (150 ?18 210 243.6 282 34 44
58.2 ?77 ?96
?105 ?127 ?150 180 211 ?244 ?283 (34 (44
582.0 78 96.1
106 128 151 ?180 21.1 246 283.0 ?34 ?44
582.9 ?78 96.8
?106 ?128 15.1 181 211.4 247 286 34.1 443.3
(59 79 ?97
107 ?129 152 ?182 ?212 ?247 ?287 343 44.4
?59 8 97.5
10.7 (13 ?152 ?184 ?213 ?248 (289 347 44.8
598.6 (8 98
108 ?13 153 184.0 214 25 ?289 35 455.7
6 ?8 ?98
?108 130 ?153 185 215 (25 (29 ?35 46
(6 ?80 99
?109 ?130 153.3 ?186 2157.1 ?25 ?29 351 ?46
?6 ?81 998
(11 ?131 153.9 1873.3 (216 250 290 355 468
60 818.7 (998
?11 131.2 ?154 188 ?216 251 29.1 35.7 (468
(60 82 99.8
110 132 154.4 ?188 ?217 ?251 29.3 359 469.1
?60 ?82 at
?110 ?132 156 1883.0 218 251.6 294 36 47
?61 83 available
11.0 132.5 ?156 189 219 ?252 295.3 ?36 (47
618.0 83.3 B
110.9 134 157 189.0 22 252.0 296.1 36.0 ?47
62 84 cloud-agent.rc
111 ?134 ?158 19 (22 254 29.7 363 471
?62 ?84 cloud-ipallocator.rc
112 (135 158.3 ?19 ?22 254.8 2972.9 367 (471
63 8.4 cloud-management.rc
?112 ?135 16 ?190 ?220 255 298 367.6 47.9
(63 ?85 cloud-management.sysconfig
?113 1359.0 (16 ?192 ?221 ?255 3 368 48
?63 85.7 cloud.spec
113.0 136 ?16 193 222 256 (3 3683.1 ?48
636.5 859.6 cloud-usage.rc
114 ?136 1.6 ?194 223 (256 ?3 (37 48.1
64 86 dependency
?114 137 160 ?196 ?223 ?256 30 ?37 483.2
?64 ?86 ?Downloaded:
114.6 137.8 ?160 197 ?225 258 (30 370 483.5
66 86.8 Downloading:
115 138 161 ?197 226 ?258 ?30 374 488.4
?66 87 for
11.5 ?138 16.1 198 227 258.0 302 378 (49
67 ?87 http:
116 ?139 ?162 ?198 ?227 ?259 30.2 38 ?49
?67 87.7 information
?116 1396.8 164 2 ?228 26 305 ?38 491.0
68 88 is
116.0 14 ?164 (2 229.4 (26 306.3 382 492
?68 (88 KB
116.7 (14 16.5 ?2 23 ?26 307 386 (492
68.2 ?88 missing,
(117 ?14 166 20 (23 (260 (31 ?39 (5
684.9 89 no
?117 140 ?166 ?20 ?23 ?260 ?31 390 ?5
69 (89 org.eclipse.m2e:lifecycle-mapping:jar:1.0.0
117.7 ?140 167.5 ?200 2.3 26.0 310.2 394 50
?69 ?89 package.sh
118 141 168 20.0 230 262 311 398 ?50
(7 89.0 POM
119 ?141 ?168 200.9 231 26.2 315 39.9 51
?7 (9 replace.properties
(119 142 169 201 ?231 ?263 31.8 399.5 (51
70 ?9 The
?119 ?142 (17 ?201 ?232 266 319 4 ?51
?70 90 ?[WARNING]
12 142.5 ?17 202 23.3 266.8 31.9 (4 52
70.2 ?90
(12 144 ?170 ?202 234 ?267 32 ?4 ?52
71 901.6
?12 ?144 170.4 202.9 235 27 (32 40 5.2
?71 91
120 144.8 172 203 ?235 (27 ?32 ?40 54
72 91.4
Then I 'git clean -fxd', rerun package.sh, and everything works. I
wonder if it's setting an env variable that allows it to work the
second time.
On Tue, Feb 12, 2013 at 10:13 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> The packaging/centos63/package.sh makes some assumptions about how
> it's being run that end up with some ugly results if it's not done
> exactly right. For example, I tried from the incubator-cloudstack
> directory: "sh ./packaging/centos63/package.sh", which seemed to copy
> /proc into my current directory and attemped to tar it up. Then I did
> "cd packaging/centos63; sh ./package.sh", which ended up with roughly
> the same result, although it died trying to run "Downloading:
> http://repo.maven..." as a bash command.
>
> Seems it didn't like being run as an 'sh' either way, even though it's
> not in the code as executable. After doing a chmod to make it
> executable, it seems to work but only if your cwd is
> incubator-cloudstack/packaging/centos63.
>
> Maybe we could change it to fail gracefully if your current path
> doesn't end in "packaging/centos63", and make it executable in git?
>
> On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl> wrote:
>>
>>
>> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>>
>>> Hey guys,
>>>
>>> Just a quick note before the weekend with a status update on RPM.
>>>
>>> The management server package is pretty much done and installation on a
>>> clean system works like a charm. This is actually tested every few hours
>>> with a Jenkins setup a colleague and I built. We take the sources, compile
>>> and package. The packages are added to a repo and chef is used to deploy two
>>> new clean CentOS 6.3 boxes. One is configured as database server and another
>>> one as CloudStack management server by chef. After installation an ApiKey is
>>> created for the admin user. This proves that the package can be installed on
>>> a clean system and that the management server starts.
>>>
>>> With this testing we have found several issues of which a few haven't been
>>> resolved yet (hopefully this weekend):
>>>
>>> * 4.1-new-db-schema.sql is not loaded by cloudstack-setup-databases
>>> * userid is null in reponse to a login call with the admin user, expected
>>> 2
>>> * Excryption initialization is now done in Transaction, this causes the
>>> mvn -Pdeveloper -pl developer -D deploydb to fail is db.properties is not in
>>> the classpath
>>>
>>> Next week:
>>> * we will continue with the setup and add some real tests to create
>>> zones and add hypervisors.
>>> *I will also start testing with the agent and usage package, they are
>>> created at the moment but not tested for functionality.
>>> * Deploy fedora 18 image and extend the test to that
>>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check with
>>> wido/noa)
>>>
>>
>> We'll sync next week! A lot of the .deb work is already done, but we just
>> have to make sure the RPM and DEB packages contain the same files.
>>
>> Then it will just be tuning and some work in the pre and postinst files, but
>> that could be a pain, but we'll just see when we go along.
>>
>> Wido
>>
>>
>>> Cheers,
>>>
>>> Hugo
>>>
>>>> -----Original Message-----
>>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>>> Sent: Thursday, February 07, 2013 2:39 AM
>>>> To: David Nalley
>>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander; cloudstack-
>>>> dev@incubator.apache.org
>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>
>>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>>>>
>>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com>
>>>>
>>>> wrote:
>>>>>>>
>>>>>>> Well, first, Apache CloudStack only releases source code.
>>>>>>>
>>>>>>> But Wido is kind enough to also host RPM / DEB package repos for
>>>>>>> users to take advantage of. Our install guide explains how to
>>>>>>> build from source, as well as how to use Wido's repos.
>>>>>>>
>>>>>>> This was all true for 4.0.0-incubating, and I think it still holds
>>>>>>> true for all future releases.
>>>>>>>
>>>>>> Chip,
>>>>>>
>>>>>> Can you refresh my memory as to why this is? I look at something like
>>>>>> cxf
>>>>
>>>> or tomcat, they all have binary downloads available.
>>>>>>
>>>>>>
>>>>>> http://cxf.apache.org/download.html
>>>>>>
>>>>>
>>>>> Because providing 'binaries' isn't necessarily problematic, but making
>>>>> yum and apt repos work in the ASF mirror system seems a bit more of an
>>>>> issue. Plus, Wido stepped up to do the work, no one else has offered
>>>>> any other alternatives.
>>>>>
>>>>> --David
>>>>>
>>>>
>>>> Yup - exactly what David said. We had discussed trying to get ASF Infra
>>>> to
>>>> help us host package repos somewhere, but I don't think that went
>>>> anywhere. And since Wido's doing it, it avoided all sorts of questions
>>>> from
>>>> the infra team around mirrors, archiving, etc...
>>>>
>>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Marcus Sorensen <sh...@gmail.com>.
The packaging/centos63/package.sh makes some assumptions about how
it's being run that end up with some ugly results if it's not done
exactly right. For example, I tried from the incubator-cloudstack
directory: "sh ./packaging/centos63/package.sh", which seemed to copy
/proc into my current directory and attemped to tar it up. Then I did
"cd packaging/centos63; sh ./package.sh", which ended up with roughly
the same result, although it died trying to run "Downloading:
http://repo.maven..." as a bash command.
Seems it didn't like being run as an 'sh' either way, even though it's
not in the code as executable. After doing a chmod to make it
executable, it seems to work but only if your cwd is
incubator-cloudstack/packaging/centos63.
Maybe we could change it to fail gracefully if your current path
doesn't end in "packaging/centos63", and make it executable in git?
On Sat, Feb 9, 2013 at 12:15 PM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>
> On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
>>
>> Hey guys,
>>
>> Just a quick note before the weekend with a status update on RPM.
>>
>> The management server package is pretty much done and installation on a
>> clean system works like a charm. This is actually tested every few hours
>> with a Jenkins setup a colleague and I built. We take the sources, compile
>> and package. The packages are added to a repo and chef is used to deploy two
>> new clean CentOS 6.3 boxes. One is configured as database server and another
>> one as CloudStack management server by chef. After installation an ApiKey is
>> created for the admin user. This proves that the package can be installed on
>> a clean system and that the management server starts.
>>
>> With this testing we have found several issues of which a few haven't been
>> resolved yet (hopefully this weekend):
>>
>> * 4.1-new-db-schema.sql is not loaded by cloudstack-setup-databases
>> * userid is null in reponse to a login call with the admin user, expected
>> 2
>> * Excryption initialization is now done in Transaction, this causes the
>> mvn -Pdeveloper -pl developer -D deploydb to fail is db.properties is not in
>> the classpath
>>
>> Next week:
>> * we will continue with the setup and add some real tests to create
>> zones and add hypervisors.
>> *I will also start testing with the agent and usage package, they are
>> created at the moment but not tested for functionality.
>> * Deploy fedora 18 image and extend the test to that
>> * Deploy Ubuntu 12.04 and add packaging scripts for that (check with
>> wido/noa)
>>
>
> We'll sync next week! A lot of the .deb work is already done, but we just
> have to make sure the RPM and DEB packages contain the same files.
>
> Then it will just be tuning and some work in the pre and postinst files, but
> that could be a pain, but we'll just see when we go along.
>
> Wido
>
>
>> Cheers,
>>
>> Hugo
>>
>>> -----Original Message-----
>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>> Sent: Thursday, February 07, 2013 2:39 AM
>>> To: David Nalley
>>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander; cloudstack-
>>> dev@incubator.apache.org
>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>
>>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>>>
>>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com>
>>>
>>> wrote:
>>>>>>
>>>>>> Well, first, Apache CloudStack only releases source code.
>>>>>>
>>>>>> But Wido is kind enough to also host RPM / DEB package repos for
>>>>>> users to take advantage of. Our install guide explains how to
>>>>>> build from source, as well as how to use Wido's repos.
>>>>>>
>>>>>> This was all true for 4.0.0-incubating, and I think it still holds
>>>>>> true for all future releases.
>>>>>>
>>>>> Chip,
>>>>>
>>>>> Can you refresh my memory as to why this is? I look at something like
>>>>> cxf
>>>
>>> or tomcat, they all have binary downloads available.
>>>>>
>>>>>
>>>>> http://cxf.apache.org/download.html
>>>>>
>>>>
>>>> Because providing 'binaries' isn't necessarily problematic, but making
>>>> yum and apt repos work in the ASF mirror system seems a bit more of an
>>>> issue. Plus, Wido stepped up to do the work, no one else has offered
>>>> any other alternatives.
>>>>
>>>> --David
>>>>
>>>
>>> Yup - exactly what David said. We had discussed trying to get ASF Infra
>>> to
>>> help us host package repos somewhere, but I don't think that went
>>> anywhere. And since Wido's doing it, it avoided all sorts of questions
>>> from
>>> the infra team around mirrors, archiving, etc...
>>>
>>> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/08/2013 06:32 PM, Hugo Trippaers wrote:
> Hey guys,
>
> Just a quick note before the weekend with a status update on RPM.
>
> The management server package is pretty much done and installation on a clean system works like a charm. This is actually tested every few hours with a Jenkins setup a colleague and I built. We take the sources, compile and package. The packages are added to a repo and chef is used to deploy two new clean CentOS 6.3 boxes. One is configured as database server and another one as CloudStack management server by chef. After installation an ApiKey is created for the admin user. This proves that the package can be installed on a clean system and that the management server starts.
>
> With this testing we have found several issues of which a few haven't been resolved yet (hopefully this weekend):
>
> * 4.1-new-db-schema.sql is not loaded by cloudstack-setup-databases
> * userid is null in reponse to a login call with the admin user, expected 2
> * Excryption initialization is now done in Transaction, this causes the mvn -Pdeveloper -pl developer -D deploydb to fail is db.properties is not in the classpath
>
> Next week:
> * we will continue with the setup and add some real tests to create zones and add hypervisors.
> *I will also start testing with the agent and usage package, they are created at the moment but not tested for functionality.
> * Deploy fedora 18 image and extend the test to that
> * Deploy Ubuntu 12.04 and add packaging scripts for that (check with wido/noa)
>
We'll sync next week! A lot of the .deb work is already done, but we
just have to make sure the RPM and DEB packages contain the same files.
Then it will just be tuning and some work in the pre and postinst files,
but that could be a pain, but we'll just see when we go along.
Wido
> Cheers,
>
> Hugo
>
>> -----Original Message-----
>> From: Chip Childers [mailto:chip.childers@sungard.com]
>> Sent: Thursday, February 07, 2013 2:39 AM
>> To: David Nalley
>> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander; cloudstack-
>> dev@incubator.apache.org
>> Subject: Re: [DISCUSS] Packaging in 4.1
>>
>> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
>>> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com>
>> wrote:
>>>>> Well, first, Apache CloudStack only releases source code.
>>>>>
>>>>> But Wido is kind enough to also host RPM / DEB package repos for
>>>>> users to take advantage of. Our install guide explains how to
>>>>> build from source, as well as how to use Wido's repos.
>>>>>
>>>>> This was all true for 4.0.0-incubating, and I think it still holds
>>>>> true for all future releases.
>>>>>
>>>> Chip,
>>>>
>>>> Can you refresh my memory as to why this is? I look at something like cxf
>> or tomcat, they all have binary downloads available.
>>>>
>>>> http://cxf.apache.org/download.html
>>>>
>>>
>>> Because providing 'binaries' isn't necessarily problematic, but making
>>> yum and apt repos work in the ASF mirror system seems a bit more of an
>>> issue. Plus, Wido stepped up to do the work, no one else has offered
>>> any other alternatives.
>>>
>>> --David
>>>
>>
>> Yup - exactly what David said. We had discussed trying to get ASF Infra to
>> help us host package repos somewhere, but I don't think that went
>> anywhere. And since Wido's doing it, it avoided all sorts of questions from
>> the infra team around mirrors, archiving, etc...
>>
>> -chip
RE: [DISCUSS] Packaging in 4.1
Posted by Hugo Trippaers <HT...@schubergphilis.com>.
Hey guys,
Just a quick note before the weekend with a status update on RPM.
The management server package is pretty much done and installation on a clean system works like a charm. This is actually tested every few hours with a Jenkins setup a colleague and I built. We take the sources, compile and package. The packages are added to a repo and chef is used to deploy two new clean CentOS 6.3 boxes. One is configured as database server and another one as CloudStack management server by chef. After installation an ApiKey is created for the admin user. This proves that the package can be installed on a clean system and that the management server starts.
With this testing we have found several issues of which a few haven't been resolved yet (hopefully this weekend):
* 4.1-new-db-schema.sql is not loaded by cloudstack-setup-databases
* userid is null in reponse to a login call with the admin user, expected 2
* Excryption initialization is now done in Transaction, this causes the mvn -Pdeveloper -pl developer -D deploydb to fail is db.properties is not in the classpath
Next week:
* we will continue with the setup and add some real tests to create zones and add hypervisors.
*I will also start testing with the agent and usage package, they are created at the moment but not tested for functionality.
* Deploy fedora 18 image and extend the test to that
* Deploy Ubuntu 12.04 and add packaging scripts for that (check with wido/noa)
Cheers,
Hugo
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Thursday, February 07, 2013 2:39 AM
> To: David Nalley
> Cc: Alex Huang; Pradeep Soundararajan; Wido den Hollander; cloudstack-
> dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
> On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
> > On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com>
> wrote:
> > >> Well, first, Apache CloudStack only releases source code.
> > >>
> > >> But Wido is kind enough to also host RPM / DEB package repos for
> > >> users to take advantage of. Our install guide explains how to
> > >> build from source, as well as how to use Wido's repos.
> > >>
> > >> This was all true for 4.0.0-incubating, and I think it still holds
> > >> true for all future releases.
> > >>
> > > Chip,
> > >
> > > Can you refresh my memory as to why this is? I look at something like cxf
> or tomcat, they all have binary downloads available.
> > >
> > > http://cxf.apache.org/download.html
> > >
> >
> > Because providing 'binaries' isn't necessarily problematic, but making
> > yum and apt repos work in the ASF mirror system seems a bit more of an
> > issue. Plus, Wido stepped up to do the work, no one else has offered
> > any other alternatives.
> >
> > --David
> >
>
> Yup - exactly what David said. We had discussed trying to get ASF Infra to
> help us host package repos somewhere, but I don't think that went
> anywhere. And since Wido's doing it, it avoided all sorts of questions from
> the infra team around mirrors, archiving, etc...
>
> -chip
Re: [DISCUSS] Packaging in 4.1
Posted by Chip Childers <ch...@sungard.com>.
On Wed, Feb 06, 2013 at 08:31:14PM -0500, David Nalley wrote:
> On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com> wrote:
> >> Well, first, Apache CloudStack only releases source code.
> >>
> >> But Wido is kind enough to also host RPM / DEB package repos for users
> >> to take advantage of. Our install guide explains how to build from
> >> source, as well as how to use Wido's repos.
> >>
> >> This was all true for 4.0.0-incubating, and I think it still holds true
> >> for all future releases.
> >>
> > Chip,
> >
> > Can you refresh my memory as to why this is? I look at something like cxf or tomcat, they all have binary downloads available.
> >
> > http://cxf.apache.org/download.html
> >
>
> Because providing 'binaries' isn't necessarily problematic, but making
> yum and apt repos work in the ASF mirror system seems a bit more of an
> issue. Plus, Wido stepped up to do the work, no one else has offered
> any other alternatives.
>
> --David
>
Yup - exactly what David said. We had discussed trying to get ASF Infra
to help us host package repos somewhere, but I don't think that went
anywhere. And since Wido's doing it, it avoided all sorts of questions
from the infra team around mirrors, archiving, etc...
-chip
RE: [DISCUSS] Packaging in 4.1
Posted by Rayees Namathponnan <ra...@citrix.com>.
As per Pradeep's suggestion I modified package.sh, (updated "cloud" to "cloudstack" in .../centos63/package.sh)
I am able to create new packages with above changes in Centos6.3, and also able to install MS; but "cloudstack-setup-management" failed with below error
[root@auto_ms1 CloudStack-non-OSS-20-rhel6.3]# cloudstack-setup-management
Starting to configure CloudStack Management Server:
Configure sudoers ... [OK]
Configure Firewall ... [OK]
Configure CloudStack Management Server ...[Failed]
Cannot find /etc/cloud/management/server-nonssl.xml or /etc/cloud/management/tomcat6-nonssl.conf, https enables failed
Try to restore your system:
Restore sudoers ... [OK]
Restore Firewall ... [OK]
Restore CloudStack Management Server ...[OK]
[root@auto_ms1 CloudStack-non-OSS-20-rhel6.3]#
"cloudstack-setup-management " looking for the old path /etc/cloud/management, instead of /etc/cloudstack/management
Hi Wido,
You will be having fixes for this ? is there any workaround for this ?
Regards,
Rayees
-----Original Message-----
From: Chip Childers [mailto:chip.childers@sungard.com]
Sent: Wednesday, February 06, 2013 8:51 AM
To: Pradeep Soundararajan
Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Packaging in 4.1
On Wed, Feb 06, 2013 at 10:05:00PM +0530, Pradeep Soundararajan wrote:
> Wido, how are we planning to setup the install for this new packaging concept?
>
Well, first, Apache CloudStack only releases source code.
But Wido is kind enough to also host RPM / DEB package repos for users to take advantage of. Our install guide explains how to build from source, as well as how to use Wido's repos.
This was all true for 4.0.0-incubating, and I think it still holds true for all future releases.
> Thanks,
> Pradeep S
>
>
> -----Original Message-----
> From: Wido den Hollander [mailto:wido@widodh.nl]
> Sent: Wednesday, February 06, 2013 10:02 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
>
>
> On 02/06/2013 04:57 PM, Chip Childers wrote:
> > On Wed, Feb 06, 2013 at 03:30:04PM +0530, Pradeep Soundararajan wrote:
> >> Thanks Hugo, Wido and Noa for bringing this to some closure :)
> >>
> >> I am able to package rpm using "packaging/centos63/package.sh" after some modification in the package.sh script since cloud.spec is looking for 'cloudstack' Name.
> >>
> >> -------------------------------------------------------------------
> >> --
> >> --- -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
> >> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
> >>
> >>
> >> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> >> $RPMDIR/SOURCES/cloud-$VERSION -x ) -(cd $RPMDIR/SOURCES/; tar -czf
> >> cloud-$VERSION.tgz cloud-$VERSION)
> >> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> >> +$RPMDIR/SOURCES/cloudstack-$VERSION -x ) (cd $RPMDIR/SOURCES/; tar
> >> +-czf cloudstack-$VERSION.tgz cloudstack-$VERSION)
> >> -------------------------------------------------------------------
> >> --
> >> ----
> >>
> >> Packaging went fine after the above modification but I have observed some issues while installing the package. I believe you have changed the installation path from */cloud/* to */cloudstack/* and also observed you have changed all the rpm names from cloud* to cloudstack*. If that is a situation then I feel we cannot upgrade from 4.0 since they were pointing to different rpm names and they were loaded in a different location. I feel, this would raise lot of compatibility issues here and there.
> >>
> >> Noticed you have changed cloud-client to cloudstack-management. I feel, we have to modify install.sh script accordingly in order to satisfy all the changed conditions.
> >
> > Haven't we removed install.sh completely?
>
> wido@wido-laptop:~/repos/cloudstack$ find -name 'install.sh'
> wido@wido-laptop:~/repos/cloudstack$
>
> Says enough I think? :)
>
> Wido
>
> >
> >>
> >> Time being shall we keep all the internals intact with the same name cloud instead of cloudstack?
> >>
> >> Please let us know if any one see any other issues.
> >>
> >> Thanks,
> >> Pradeep S
>
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Wed, Feb 6, 2013 at 8:24 PM, Alex Huang <Al...@citrix.com> wrote:
>> Well, first, Apache CloudStack only releases source code.
>>
>> But Wido is kind enough to also host RPM / DEB package repos for users
>> to take advantage of. Our install guide explains how to build from
>> source, as well as how to use Wido's repos.
>>
>> This was all true for 4.0.0-incubating, and I think it still holds true
>> for all future releases.
>>
> Chip,
>
> Can you refresh my memory as to why this is? I look at something like cxf or tomcat, they all have binary downloads available.
>
> http://cxf.apache.org/download.html
>
Because providing 'binaries' isn't necessarily problematic, but making
yum and apt repos work in the ASF mirror system seems a bit more of an
issue. Plus, Wido stepped up to do the work, no one else has offered
any other alternatives.
--David
RE: [DISCUSS] Packaging in 4.1
Posted by Alex Huang <Al...@citrix.com>.
> Well, first, Apache CloudStack only releases source code.
>
> But Wido is kind enough to also host RPM / DEB package repos for users
> to take advantage of. Our install guide explains how to build from
> source, as well as how to use Wido's repos.
>
> This was all true for 4.0.0-incubating, and I think it still holds true
> for all future releases.
>
Chip,
Can you refresh my memory as to why this is? I look at something like cxf or tomcat, they all have binary downloads available.
http://cxf.apache.org/download.html
--alex
Re: [DISCUSS] Packaging in 4.1
Posted by Chip Childers <ch...@sungard.com>.
On Wed, Feb 06, 2013 at 10:05:00PM +0530, Pradeep Soundararajan wrote:
> Wido, how are we planning to setup the install for this new packaging concept?
>
Well, first, Apache CloudStack only releases source code.
But Wido is kind enough to also host RPM / DEB package repos for users
to take advantage of. Our install guide explains how to build from
source, as well as how to use Wido's repos.
This was all true for 4.0.0-incubating, and I think it still holds true
for all future releases.
> Thanks,
> Pradeep S
>
>
> -----Original Message-----
> From: Wido den Hollander [mailto:wido@widodh.nl]
> Sent: Wednesday, February 06, 2013 10:02 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
>
>
> On 02/06/2013 04:57 PM, Chip Childers wrote:
> > On Wed, Feb 06, 2013 at 03:30:04PM +0530, Pradeep Soundararajan wrote:
> >> Thanks Hugo, Wido and Noa for bringing this to some closure :)
> >>
> >> I am able to package rpm using "packaging/centos63/package.sh" after some modification in the package.sh script since cloud.spec is looking for 'cloudstack' Name.
> >>
> >> ---------------------------------------------------------------------
> >> --- -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
> >> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
> >>
> >>
> >> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> >> $RPMDIR/SOURCES/cloud-$VERSION -x ) -(cd $RPMDIR/SOURCES/; tar -czf
> >> cloud-$VERSION.tgz cloud-$VERSION)
> >> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> >> +$RPMDIR/SOURCES/cloudstack-$VERSION -x ) (cd $RPMDIR/SOURCES/; tar
> >> +-czf cloudstack-$VERSION.tgz cloudstack-$VERSION)
> >> ---------------------------------------------------------------------
> >> ----
> >>
> >> Packaging went fine after the above modification but I have observed some issues while installing the package. I believe you have changed the installation path from */cloud/* to */cloudstack/* and also observed you have changed all the rpm names from cloud* to cloudstack*. If that is a situation then I feel we cannot upgrade from 4.0 since they were pointing to different rpm names and they were loaded in a different location. I feel, this would raise lot of compatibility issues here and there.
> >>
> >> Noticed you have changed cloud-client to cloudstack-management. I feel, we have to modify install.sh script accordingly in order to satisfy all the changed conditions.
> >
> > Haven't we removed install.sh completely?
>
> wido@wido-laptop:~/repos/cloudstack$ find -name 'install.sh'
> wido@wido-laptop:~/repos/cloudstack$
>
> Says enough I think? :)
>
> Wido
>
> >
> >>
> >> Time being shall we keep all the internals intact with the same name cloud instead of cloudstack?
> >>
> >> Please let us know if any one see any other issues.
> >>
> >> Thanks,
> >> Pradeep S
>
RE: [DISCUSS] Packaging in 4.1
Posted by Pradeep Soundararajan <pr...@citrix.com>.
Wido, how are we planning to setup the install for this new packaging concept?
Thanks,
Pradeep S
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: Wednesday, February 06, 2013 10:02 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Packaging in 4.1
On 02/06/2013 04:57 PM, Chip Childers wrote:
> On Wed, Feb 06, 2013 at 03:30:04PM +0530, Pradeep Soundararajan wrote:
>> Thanks Hugo, Wido and Noa for bringing this to some closure :)
>>
>> I am able to package rpm using "packaging/centos63/package.sh" after some modification in the package.sh script since cloud.spec is looking for 'cloudstack' Name.
>>
>> ---------------------------------------------------------------------
>> --- -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
>> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
>>
>>
>> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
>> $RPMDIR/SOURCES/cloud-$VERSION -x ) -(cd $RPMDIR/SOURCES/; tar -czf
>> cloud-$VERSION.tgz cloud-$VERSION)
>> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
>> +$RPMDIR/SOURCES/cloudstack-$VERSION -x ) (cd $RPMDIR/SOURCES/; tar
>> +-czf cloudstack-$VERSION.tgz cloudstack-$VERSION)
>> ---------------------------------------------------------------------
>> ----
>>
>> Packaging went fine after the above modification but I have observed some issues while installing the package. I believe you have changed the installation path from */cloud/* to */cloudstack/* and also observed you have changed all the rpm names from cloud* to cloudstack*. If that is a situation then I feel we cannot upgrade from 4.0 since they were pointing to different rpm names and they were loaded in a different location. I feel, this would raise lot of compatibility issues here and there.
>>
>> Noticed you have changed cloud-client to cloudstack-management. I feel, we have to modify install.sh script accordingly in order to satisfy all the changed conditions.
>
> Haven't we removed install.sh completely?
wido@wido-laptop:~/repos/cloudstack$ find -name 'install.sh'
wido@wido-laptop:~/repos/cloudstack$
Says enough I think? :)
Wido
>
>>
>> Time being shall we keep all the internals intact with the same name cloud instead of cloudstack?
>>
>> Please let us know if any one see any other issues.
>>
>> Thanks,
>> Pradeep S
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/06/2013 04:57 PM, Chip Childers wrote:
> On Wed, Feb 06, 2013 at 03:30:04PM +0530, Pradeep Soundararajan wrote:
>> Thanks Hugo, Wido and Noa for bringing this to some closure :)
>>
>> I am able to package rpm using "packaging/centos63/package.sh" after some modification in the package.sh script since cloud.spec is looking for 'cloudstack' Name.
>>
>> ------------------------------------------------------------------------
>> -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
>> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
>>
>>
>> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C $RPMDIR/SOURCES/cloud-$VERSION -x )
>> -(cd $RPMDIR/SOURCES/; tar -czf cloud-$VERSION.tgz cloud-$VERSION)
>> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C $RPMDIR/SOURCES/cloudstack-$VERSION -x )
>> +(cd $RPMDIR/SOURCES/; tar -czf cloudstack-$VERSION.tgz cloudstack-$VERSION)
>> -------------------------------------------------------------------------
>>
>> Packaging went fine after the above modification but I have observed some issues while installing the package. I believe you have changed the installation path from */cloud/* to */cloudstack/* and also observed you have changed all the rpm names from cloud* to cloudstack*. If that is a situation then I feel we cannot upgrade from 4.0 since they were pointing to different rpm names and they were loaded in a different location. I feel, this would raise lot of compatibility issues here and there.
>>
>> Noticed you have changed cloud-client to cloudstack-management. I feel, we have to modify install.sh script accordingly in order to satisfy all the changed conditions.
>
> Haven't we removed install.sh completely?
wido@wido-laptop:~/repos/cloudstack$ find -name 'install.sh'
wido@wido-laptop:~/repos/cloudstack$
Says enough I think? :)
Wido
>
>>
>> Time being shall we keep all the internals intact with the same name cloud instead of cloudstack?
>>
>> Please let us know if any one see any other issues.
>>
>> Thanks,
>> Pradeep S
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Wednesday, February 6, 2013, Chip Childers wrote:
> On Wed, Feb 06, 2013 at 03:30:04PM +0530, Pradeep Soundararajan wrote:
> > Thanks Hugo, Wido and Noa for bringing this to some closure :)
> >
> > I am able to package rpm using "packaging/centos63/package.sh" after
> some modification in the package.sh script since cloud.spec is looking for
> 'cloudstack' Name.
> >
> > ------------------------------------------------------------------------
> > -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
> > +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
> >
> >
> > -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> $RPMDIR/SOURCES/cloud-$VERSION -x )
> > -(cd $RPMDIR/SOURCES/; tar -czf cloud-$VERSION.tgz cloud-$VERSION)
> > +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> $RPMDIR/SOURCES/cloudstack-$VERSION -x )
> > +(cd $RPMDIR/SOURCES/; tar -czf cloudstack-$VERSION.tgz
> cloudstack-$VERSION)
> > -------------------------------------------------------------------------
> >
> > Packaging went fine after the above modification but I have observed
> some issues while installing the package. I believe you have changed the
> installation path from */cloud/* to */cloudstack/* and also observed you
> have changed all the rpm names from cloud* to cloudstack*. If that is a
> situation then I feel we cannot upgrade from 4.0 since they were pointing
> to different rpm names and they were loaded in a different location. I
> feel, this would raise lot of compatibility issues here and there.
> >
> > Noticed you have changed cloud-client to cloudstack-management. I feel,
> we have to modify install.sh script accordingly in order to satisfy all the
> changed conditions.
>
> Haven't we removed install.sh completely?
>
>
> It was never part of the CloudStack codebase in the first place - it only
existed in the build system and the rendered binary tarballs.
Re: [DISCUSS] Packaging in 4.1
Posted by Chip Childers <ch...@sungard.com>.
On Wed, Feb 06, 2013 at 03:30:04PM +0530, Pradeep Soundararajan wrote:
> Thanks Hugo, Wido and Noa for bringing this to some closure :)
>
> I am able to package rpm using "packaging/centos63/package.sh" after some modification in the package.sh script since cloud.spec is looking for 'cloudstack' Name.
>
> ------------------------------------------------------------------------
> -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
>
>
> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C $RPMDIR/SOURCES/cloud-$VERSION -x )
> -(cd $RPMDIR/SOURCES/; tar -czf cloud-$VERSION.tgz cloud-$VERSION)
> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C $RPMDIR/SOURCES/cloudstack-$VERSION -x )
> +(cd $RPMDIR/SOURCES/; tar -czf cloudstack-$VERSION.tgz cloudstack-$VERSION)
> -------------------------------------------------------------------------
>
> Packaging went fine after the above modification but I have observed some issues while installing the package. I believe you have changed the installation path from */cloud/* to */cloudstack/* and also observed you have changed all the rpm names from cloud* to cloudstack*. If that is a situation then I feel we cannot upgrade from 4.0 since they were pointing to different rpm names and they were loaded in a different location. I feel, this would raise lot of compatibility issues here and there.
>
> Noticed you have changed cloud-client to cloudstack-management. I feel, we have to modify install.sh script accordingly in order to satisfy all the changed conditions.
Haven't we removed install.sh completely?
>
> Time being shall we keep all the internals intact with the same name cloud instead of cloudstack?
>
> Please let us know if any one see any other issues.
>
> Thanks,
> Pradeep S
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/06/2013 02:03 PM, Hugo Trippaers wrote:
> Hey Pradeep,
>
> I'm planning on doing some upgrade tests at the moment. All the packages will list the packages that will be obsoleted by the new install. For users the process should be transparent, but we will create any symlinks if that makes life easier. For installation we should be covered by that and making it clear in the documentation/release notes.
>
> I think it's good to have the name of the product in the name of that package and the installed aritifacts, but we should make it clear for end-users that the name has changes between version 4.0 and 4.1.
>
> I am setting up an environment to do the upgrade tests at the moment and we are in touch with Prassana to align with his work on automated testing as well. So for now I would like to keep the name as cloudstack unless testing proves that this is infeasible.
>
+1
I'll also be setting up a couple of Ubuntu systems to test the 4.0 to
4.1 upgrade.
Wido
> Cheers,
>
> Hugo
>
>> -----Original Message-----
>> From: Pradeep Soundararajan [mailto:pradeep.soundararajan@citrix.com]
>> Sent: Wednesday, February 06, 2013 11:00 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: RE: [DISCUSS] Packaging in 4.1
>> Importance: High
>>
>> Thanks Hugo, Wido and Noa for bringing this to some closure :)
>>
>> I am able to package rpm using "packaging/centos63/package.sh" after some
>> modification in the package.sh script since cloud.spec is looking for
>> 'cloudstack' Name.
>>
>> ------------------------------------------------------------------------
>> -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
>> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
>>
>>
>> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
>> $RPMDIR/SOURCES/cloud-$VERSION -x ) -(cd $RPMDIR/SOURCES/; tar -czf
>> cloud-$VERSION.tgz cloud-$VERSION)
>> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
>> +$RPMDIR/SOURCES/cloudstack-$VERSION -x ) (cd $RPMDIR/SOURCES/; tar
>> -czf
>> +cloudstack-$VERSION.tgz cloudstack-$VERSION)
>> -------------------------------------------------------------------------
>>
>> Packaging went fine after the above modification but I have observed some
>> issues while installing the package. I believe you have changed the
>> installation path from */cloud/* to */cloudstack/* and also observed you
>> have changed all the rpm names from cloud* to cloudstack*. If that is a
>> situation then I feel we cannot upgrade from 4.0 since they were pointing to
>> different rpm names and they were loaded in a different location. I feel, this
>> would raise lot of compatibility issues here and there.
>>
>> Noticed you have changed cloud-client to cloudstack-management. I feel, we
>> have to modify install.sh script accordingly in order to satisfy all the changed
>> conditions.
>>
>> Time being shall we keep all the internals intact with the same name cloud
>> instead of cloudstack?
>>
>> Please let us know if any one see any other issues.
>>
>> Thanks,
>> Pradeep S
>>
>>
>>
>>
>> -----Original Message-----
>> From: Wido den Hollander [mailto:wido@widodh.nl]
>> Sent: Wednesday, February 06, 2013 1:33 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: [DISCUSS] Packaging in 4.1
>>
>>
>>
>> On 02/05/2013 06:33 PM, Noa Resare wrote:
>>> I built the latest from the packaging branch. I think it is shaping up
>>> nicely. Some thoughts:
>>>
>>> How would you feel with postponing the config directory changes until 4.2?
>>> While I agree conceptually that they are an improvement, waiting with
>>> them would keep the diff size down, simplifying merge and the focus of
>>> stabilization for 4.1
>>
>> Yes, I stumbled upon the same while merging master into packaging.
>>
>> It doesn't hurt anybody to keep it in the current shape, the end result will be
>> the same.
>>
>> Wido
>>
>>>
>>> /n
>>>
>>>
>>>
>>>
>>> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl>
>> wrote:
>>>
>>>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>>>
>>>>> Wanted to check when would this be implemented?? Several QA folks
>>>>> depend on the packages and need this working.
>>>>> We have been still building using waf but today that is also not
>>>>> working as some references are removed.
>>>>>
>>>>> Is it possible to accelerate this process or leave old way of
>>>>> packaging in place till you are done with the new changes
>>>>>
>>>>>
>>>> I fully understand. As I told David, I think the DEB work is about
>>>> one day of work, but then again, there is something like $dayjob.
>>>>
>>>> What might be even tougher is to get the RPM and DEB packages fully
>>>> synced. cloudstack-common for example should contain exactly the same
>>>> files in the RPM and DEB version, so Hugo and I will have to keep in touch.
>>>>
>>>> I really want to have the DEB packaging working this week, period.
>>>>
>>>> Wido
>>>>
>>>>
>>>> Thanks
>>>>> /sudha
>>>>>
>>>>> -----Original Message-----
>>>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
>>>>> Behalf Of Rohit Yadav
>>>>> Sent: Sunday, February 03, 2013 5:14 PM
>>>>> To:
>>>>> cloudstack-dev@incubator.**apache.org<cloudstack-
>> dev@incubator.apach
>>>>> e.org>
>>>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>>>
>>>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>>>
>>>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
>> wrote:
>>>>>>
>>>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>>>> ...
>>>>>>>
>>>>>>>>
>>>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>>>> worth than clint (clint is in EPEL, but no new version of
>>>>>>>> pygments in
>>>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>>>
>>>>>>>
>>>>>>> I want people to use pip to install the cli because it's the
>>>>>>> easiest and because rpm/deb packages may have dependency issues
>>>>>>> like you mentioned => may not work on all distros, what we can do
>>>>>>> is when people install cloudstack-cli rpm or deb, it runs a script
>>>>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
>>>>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
>>>>>>> cloudmonkey and its dependencies and install from it. Advise best
>>>>>>> way of doing this?
>>>>>>>
>>>>>>
>>>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>>>
>>>>>> You are assuming that they would have internet access when
>>>>>> installing
>>>>>> - which is not a valid assumption.
>>>>>>
>>>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>>>> installed, and may or may not have installed - may have installed a
>>>>>> compromised package (see rubygems.org compromise recently,
>>>>>> kernel.org, and a number of other site compromises.), or might have
>>>>>> installed packages I didn't know about is a Bad Idea (tm) The
>>>>>> sysadmin doesn't know you are installing some of the dependencies,
>>>>>> there is no record of those packages in the package manager, and
>>>>>> there might potentially be conflicts with system packages, a
>>>>>> security vulnerability in one of those dependencies wouldn't be caught
>> on audit, etc etc.
>>>>>>
>>>>>
>>>>> /facepalm\, it's just a problem of packaging. The package can
>>>>> include cli or any other artifact's dependencies, so in case of cli,
>>>>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
>>>>> of them are pure python so easily installable. The cool people can use
>> pip to install.
>>>>>
>>>>>
>>>>>> And I really don't intend for this to sound like a rant, but the
>>>>>> one of the important benefits behind using packages and a package
>>>>>> manager is that a sysadmin needs (and often is required to have by
>>>>>> government
>>>>>> regulations) a single source of truth about the software installed
>>>>>> on a machine.
>>>>>>
>>>>>
>>>>> No, it's not a rant, I understand.
>>>>>
>>>>> Developers love things like Maven central, pypi, CPAN, and
>>>>> rubygems,
>>>>>> and for good reason, they are fast, flexible, and make their life
>>>>>> easy. To a sysadmin managing machines in production, they are
>>>>>> anathema; they make system state difficult or impossible to
>>>>>> determine, they make audits painful.
>>>>>>
>>>>>
>>>>> I just assumed the sysadmin who would install CloudStack, cli and
>>>>> whatnot won't be stupid, at the same time I want his life to be less
>> miserable.
>>>>>
>>>>> In addition they make troubleshooting
>>>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>>>> there multiple copies of $foo installed on the system? Which one is
>>>>>> actually being called/loaded?
>>>>>>
>>>>>
>>>>> Alright, but I'm talking only about the cli, since most users,
>>>>> admins included, would want it to run on their machines, the
>>>>> installation should be easiest, that's why I said they can use pip,
>>>>> so it works on their windows, osx, linux, bsd boxes and that's why
>>>>> it's pure python (written that way).
>>>>>
>>>>> Regards.
>>>>>
>>>>>>
>>>>>> --David
>>>>>>
>>>>>
>>>>
>>>
>>>
RE: [DISCUSS] Packaging in 4.1
Posted by Hugo Trippaers <HT...@schubergphilis.com>.
Hey Pradeep,
I'm planning on doing some upgrade tests at the moment. All the packages will list the packages that will be obsoleted by the new install. For users the process should be transparent, but we will create any symlinks if that makes life easier. For installation we should be covered by that and making it clear in the documentation/release notes.
I think it's good to have the name of the product in the name of that package and the installed aritifacts, but we should make it clear for end-users that the name has changes between version 4.0 and 4.1.
I am setting up an environment to do the upgrade tests at the moment and we are in touch with Prassana to align with his work on automated testing as well. So for now I would like to keep the name as cloudstack unless testing proves that this is infeasible.
Cheers,
Hugo
> -----Original Message-----
> From: Pradeep Soundararajan [mailto:pradeep.soundararajan@citrix.com]
> Sent: Wednesday, February 06, 2013 11:00 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: [DISCUSS] Packaging in 4.1
> Importance: High
>
> Thanks Hugo, Wido and Noa for bringing this to some closure :)
>
> I am able to package rpm using "packaging/centos63/package.sh" after some
> modification in the package.sh script since cloud.spec is looking for
> 'cloudstack' Name.
>
> ------------------------------------------------------------------------
> -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
>
>
> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> $RPMDIR/SOURCES/cloud-$VERSION -x ) -(cd $RPMDIR/SOURCES/; tar -czf
> cloud-$VERSION.tgz cloud-$VERSION)
> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> +$RPMDIR/SOURCES/cloudstack-$VERSION -x ) (cd $RPMDIR/SOURCES/; tar
> -czf
> +cloudstack-$VERSION.tgz cloudstack-$VERSION)
> -------------------------------------------------------------------------
>
> Packaging went fine after the above modification but I have observed some
> issues while installing the package. I believe you have changed the
> installation path from */cloud/* to */cloudstack/* and also observed you
> have changed all the rpm names from cloud* to cloudstack*. If that is a
> situation then I feel we cannot upgrade from 4.0 since they were pointing to
> different rpm names and they were loaded in a different location. I feel, this
> would raise lot of compatibility issues here and there.
>
> Noticed you have changed cloud-client to cloudstack-management. I feel, we
> have to modify install.sh script accordingly in order to satisfy all the changed
> conditions.
>
> Time being shall we keep all the internals intact with the same name cloud
> instead of cloudstack?
>
> Please let us know if any one see any other issues.
>
> Thanks,
> Pradeep S
>
>
>
>
> -----Original Message-----
> From: Wido den Hollander [mailto:wido@widodh.nl]
> Sent: Wednesday, February 06, 2013 1:33 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
>
>
> On 02/05/2013 06:33 PM, Noa Resare wrote:
> > I built the latest from the packaging branch. I think it is shaping up
> > nicely. Some thoughts:
> >
> > How would you feel with postponing the config directory changes until 4.2?
> > While I agree conceptually that they are an improvement, waiting with
> > them would keep the diff size down, simplifying merge and the focus of
> > stabilization for 4.1
>
> Yes, I stumbled upon the same while merging master into packaging.
>
> It doesn't hurt anybody to keep it in the current shape, the end result will be
> the same.
>
> Wido
>
> >
> > /n
> >
> >
> >
> >
> > On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >
> >> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
> >>
> >>> Wanted to check when would this be implemented?? Several QA folks
> >>> depend on the packages and need this working.
> >>> We have been still building using waf but today that is also not
> >>> working as some references are removed.
> >>>
> >>> Is it possible to accelerate this process or leave old way of
> >>> packaging in place till you are done with the new changes
> >>>
> >>>
> >> I fully understand. As I told David, I think the DEB work is about
> >> one day of work, but then again, there is something like $dayjob.
> >>
> >> What might be even tougher is to get the RPM and DEB packages fully
> >> synced. cloudstack-common for example should contain exactly the same
> >> files in the RPM and DEB version, so Hugo and I will have to keep in touch.
> >>
> >> I really want to have the DEB packaging working this week, period.
> >>
> >> Wido
> >>
> >>
> >> Thanks
> >>> /sudha
> >>>
> >>> -----Original Message-----
> >>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
> >>> Behalf Of Rohit Yadav
> >>> Sent: Sunday, February 03, 2013 5:14 PM
> >>> To:
> >>> cloudstack-dev@incubator.**apache.org<cloudstack-
> dev@incubator.apach
> >>> e.org>
> >>> Subject: Re: [DISCUSS] Packaging in 4.1
> >>>
> >>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
> >>>
> >>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >>>>
> >>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
> >>>>> ...
> >>>>>
> >>>>>>
> >>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
> >>>>>> worth than clint (clint is in EPEL, but no new version of
> >>>>>> pygments in
> >>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
> >>>>>>
> >>>>>
> >>>>> I want people to use pip to install the cli because it's the
> >>>>> easiest and because rpm/deb packages may have dependency issues
> >>>>> like you mentioned => may not work on all distros, what we can do
> >>>>> is when people install cloudstack-cli rpm or deb, it runs a script
> >>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
> >>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
> >>>>> cloudmonkey and its dependencies and install from it. Advise best
> >>>>> way of doing this?
> >>>>>
> >>>>
> >>>> I guess we won't be installing the CLI via RPMs at least for EL6.
> >>>>
> >>>> You are assuming that they would have internet access when
> >>>> installing
> >>>> - which is not a valid assumption.
> >>>>
> >>>> Honestly, the above idea makes me blanch. A package that reports as
> >>>> installed, and may or may not have installed - may have installed a
> >>>> compromised package (see rubygems.org compromise recently,
> >>>> kernel.org, and a number of other site compromises.), or might have
> >>>> installed packages I didn't know about is a Bad Idea (tm) The
> >>>> sysadmin doesn't know you are installing some of the dependencies,
> >>>> there is no record of those packages in the package manager, and
> >>>> there might potentially be conflicts with system packages, a
> >>>> security vulnerability in one of those dependencies wouldn't be caught
> on audit, etc etc.
> >>>>
> >>>
> >>> /facepalm\, it's just a problem of packaging. The package can
> >>> include cli or any other artifact's dependencies, so in case of cli,
> >>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
> >>> of them are pure python so easily installable. The cool people can use
> pip to install.
> >>>
> >>>
> >>>> And I really don't intend for this to sound like a rant, but the
> >>>> one of the important benefits behind using packages and a package
> >>>> manager is that a sysadmin needs (and often is required to have by
> >>>> government
> >>>> regulations) a single source of truth about the software installed
> >>>> on a machine.
> >>>>
> >>>
> >>> No, it's not a rant, I understand.
> >>>
> >>> Developers love things like Maven central, pypi, CPAN, and
> >>> rubygems,
> >>>> and for good reason, they are fast, flexible, and make their life
> >>>> easy. To a sysadmin managing machines in production, they are
> >>>> anathema; they make system state difficult or impossible to
> >>>> determine, they make audits painful.
> >>>>
> >>>
> >>> I just assumed the sysadmin who would install CloudStack, cli and
> >>> whatnot won't be stupid, at the same time I want his life to be less
> miserable.
> >>>
> >>> In addition they make troubleshooting
> >>>> incredibly difficult. Do I have $foo installed - which version? Are
> >>>> there multiple copies of $foo installed on the system? Which one is
> >>>> actually being called/loaded?
> >>>>
> >>>
> >>> Alright, but I'm talking only about the cli, since most users,
> >>> admins included, would want it to run on their machines, the
> >>> installation should be easiest, that's why I said they can use pip,
> >>> so it works on their windows, osx, linux, bsd boxes and that's why
> >>> it's pure python (written that way).
> >>>
> >>> Regards.
> >>>
> >>>>
> >>>> --David
> >>>>
> >>>
> >>
> >
> >
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Wednesday, February 6, 2013, Pradeep Soundararajan wrote:
> Thanks Hugo, Wido and Noa for bringing this to some closure :)
>
> I am able to package rpm using "packaging/centos63/package.sh" after some
> modification in the package.sh script since cloud.spec is looking for
> 'cloudstack' Name.
>
> ------------------------------------------------------------------------
> -mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
> +mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
>
>
> -(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> $RPMDIR/SOURCES/cloud-$VERSION -x )
> -(cd $RPMDIR/SOURCES/; tar -czf cloud-$VERSION.tgz cloud-$VERSION)
> +(cd ../../; tar -c --exclude .git --exclude dist . | tar -C
> $RPMDIR/SOURCES/cloudstack-$VERSION -x )
> +(cd $RPMDIR/SOURCES/; tar -czf cloudstack-$VERSION.tgz
> cloudstack-$VERSION)
> -------------------------------------------------------------------------
>
> Packaging went fine after the above modification but I have observed some
> issues while installing the package. I believe you have changed the
> installation path from */cloud/* to */cloudstack/* and also observed you
> have changed all the rpm names from cloud* to cloudstack*. If that is a
> situation then I feel we cannot upgrade from 4.0 since they were pointing
> to different rpm names and they were loaded in a different location. I
> feel, this would raise lot of compatibility issues here and there.
>
> So we are using things like Obsoletes and Provides in RPM (and their
alternatives in deb land) so that the new packages provide cloud- packages
and also obsolete the old ones (effectively creating a case where the old
packages simply won't be around)
> Noticed you have changed cloud-client to cloudstack-management. I feel, we
> have to modify install.sh script accordingly in order to satisfy all the
> changed conditions.
>
There is no install.sh in Apache CloudStack.
RE: [DISCUSS] Packaging in 4.1
Posted by Pradeep Soundararajan <pr...@citrix.com>.
Thanks Hugo, Wido and Noa for bringing this to some closure :)
I am able to package rpm using "packaging/centos63/package.sh" after some modification in the package.sh script since cloud.spec is looking for 'cloudstack' Name.
------------------------------------------------------------------------
-mkdir -p $RPMDIR/SOURCES/cloud-$VERSION
+mkdir -p $RPMDIR/SOURCES/cloudstack-$VERSION
-(cd ../../; tar -c --exclude .git --exclude dist . | tar -C $RPMDIR/SOURCES/cloud-$VERSION -x )
-(cd $RPMDIR/SOURCES/; tar -czf cloud-$VERSION.tgz cloud-$VERSION)
+(cd ../../; tar -c --exclude .git --exclude dist . | tar -C $RPMDIR/SOURCES/cloudstack-$VERSION -x )
+(cd $RPMDIR/SOURCES/; tar -czf cloudstack-$VERSION.tgz cloudstack-$VERSION)
-------------------------------------------------------------------------
Packaging went fine after the above modification but I have observed some issues while installing the package. I believe you have changed the installation path from */cloud/* to */cloudstack/* and also observed you have changed all the rpm names from cloud* to cloudstack*. If that is a situation then I feel we cannot upgrade from 4.0 since they were pointing to different rpm names and they were loaded in a different location. I feel, this would raise lot of compatibility issues here and there.
Noticed you have changed cloud-client to cloudstack-management. I feel, we have to modify install.sh script accordingly in order to satisfy all the changed conditions.
Time being shall we keep all the internals intact with the same name cloud instead of cloudstack?
Please let us know if any one see any other issues.
Thanks,
Pradeep S
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: Wednesday, February 06, 2013 1:33 AM
To: cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Packaging in 4.1
On 02/05/2013 06:33 PM, Noa Resare wrote:
> I built the latest from the packaging branch. I think it is shaping up
> nicely. Some thoughts:
>
> How would you feel with postponing the config directory changes until 4.2?
> While I agree conceptually that they are an improvement, waiting with
> them would keep the diff size down, simplifying merge and the focus of
> stabilization for 4.1
Yes, I stumbled upon the same while merging master into packaging.
It doesn't hurt anybody to keep it in the current shape, the end result will be the same.
Wido
>
> /n
>
>
>
>
> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>
>>> Wanted to check when would this be implemented?? Several QA folks
>>> depend on the packages and need this working.
>>> We have been still building using waf but today that is also not
>>> working as some references are removed.
>>>
>>> Is it possible to accelerate this process or leave old way of
>>> packaging in place till you are done with the new changes
>>>
>>>
>> I fully understand. As I told David, I think the DEB work is about
>> one day of work, but then again, there is something like $dayjob.
>>
>> What might be even tougher is to get the RPM and DEB packages fully
>> synced. cloudstack-common for example should contain exactly the same
>> files in the RPM and DEB version, so Hugo and I will have to keep in touch.
>>
>> I really want to have the DEB packaging working this week, period.
>>
>> Wido
>>
>>
>> Thanks
>>> /sudha
>>>
>>> -----Original Message-----
>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On
>>> Behalf Of Rohit Yadav
>>> Sent: Sunday, February 03, 2013 5:14 PM
>>> To:
>>> cloudstack-dev@incubator.**apache.org<cloudstack-dev@incubator.apach
>>> e.org>
>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>
>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>
>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>>>>
>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>> ...
>>>>>
>>>>>>
>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>> worth than clint (clint is in EPEL, but no new version of
>>>>>> pygments in
>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>
>>>>>
>>>>> I want people to use pip to install the cli because it's the
>>>>> easiest and because rpm/deb packages may have dependency issues
>>>>> like you mentioned => may not work on all distros, what we can do
>>>>> is when people install cloudstack-cli rpm or deb, it runs a script
>>>>> that installs pip (if unavailable) and cloudmonkey. cloudmonkey is
>>>>> pure python, so the rpm/deb can also ship bundling src tarballs of
>>>>> cloudmonkey and its dependencies and install from it. Advise best
>>>>> way of doing this?
>>>>>
>>>>
>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>
>>>> You are assuming that they would have internet access when
>>>> installing
>>>> - which is not a valid assumption.
>>>>
>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>> installed, and may or may not have installed - may have installed a
>>>> compromised package (see rubygems.org compromise recently,
>>>> kernel.org, and a number of other site compromises.), or might have
>>>> installed packages I didn't know about is a Bad Idea (tm) The
>>>> sysadmin doesn't know you are installing some of the dependencies,
>>>> there is no record of those packages in the package manager, and
>>>> there might potentially be conflicts with system packages, a
>>>> security vulnerability in one of those dependencies wouldn't be caught on audit, etc etc.
>>>>
>>>
>>> /facepalm\, it's just a problem of packaging. The package can
>>> include cli or any other artifact's dependencies, so in case of cli,
>>> you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all
>>> of them are pure python so easily installable. The cool people can use pip to install.
>>>
>>>
>>>> And I really don't intend for this to sound like a rant, but the
>>>> one of the important benefits behind using packages and a package
>>>> manager is that a sysadmin needs (and often is required to have by
>>>> government
>>>> regulations) a single source of truth about the software installed
>>>> on a machine.
>>>>
>>>
>>> No, it's not a rant, I understand.
>>>
>>> Developers love things like Maven central, pypi, CPAN, and
>>> rubygems,
>>>> and for good reason, they are fast, flexible, and make their life
>>>> easy. To a sysadmin managing machines in production, they are
>>>> anathema; they make system state difficult or impossible to
>>>> determine, they make audits painful.
>>>>
>>>
>>> I just assumed the sysadmin who would install CloudStack, cli and
>>> whatnot won't be stupid, at the same time I want his life to be less miserable.
>>>
>>> In addition they make troubleshooting
>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>> there multiple copies of $foo installed on the system? Which one is
>>>> actually being called/loaded?
>>>>
>>>
>>> Alright, but I'm talking only about the cli, since most users,
>>> admins included, would want it to run on their machines, the
>>> installation should be easiest, that's why I said they can use pip,
>>> so it works on their windows, osx, linux, bsd boxes and that's why
>>> it's pure python (written that way).
>>>
>>> Regards.
>>>
>>>>
>>>> --David
>>>>
>>>
>>
>
>
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/05/2013 06:33 PM, Noa Resare wrote:
> I built the latest from the packaging branch. I think it is shaping up
> nicely. Some thoughts:
>
> How would you feel with postponing the config directory changes until 4.2?
> While I agree conceptually that they are an improvement, waiting with them
> would keep the diff size down, simplifying merge and the focus of
> stabilization for 4.1
Yes, I stumbled upon the same while merging master into packaging.
It doesn't hurt anybody to keep it in the current shape, the end result
will be the same.
Wido
>
> /n
>
>
>
>
> On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>>
>>> Wanted to check when would this be implemented?? Several QA folks depend
>>> on the packages and need this working.
>>> We have been still building using waf but today that is also not working
>>> as some references are removed.
>>>
>>> Is it possible to accelerate this process or leave old way of packaging
>>> in place till you are done with the new changes
>>>
>>>
>> I fully understand. As I told David, I think the DEB work is about one day
>> of work, but then again, there is something like $dayjob.
>>
>> What might be even tougher is to get the RPM and DEB packages fully
>> synced. cloudstack-common for example should contain exactly the same files
>> in the RPM and DEB version, so Hugo and I will have to keep in touch.
>>
>> I really want to have the DEB packaging working this week, period.
>>
>> Wido
>>
>>
>> Thanks
>>> /sudha
>>>
>>> -----Original Message-----
>>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On Behalf
>>> Of Rohit Yadav
>>> Sent: Sunday, February 03, 2013 5:14 PM
>>> To: cloudstack-dev@incubator.**apache.org<cl...@incubator.apache.org>
>>> Subject: Re: [DISCUSS] Packaging in 4.1
>>>
>>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>>
>>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>>>>
>>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>>> ...
>>>>>
>>>>>>
>>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>>> worth than clint (clint is in EPEL, but no new version of pygments
>>>>>> in
>>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>>
>>>>>
>>>>> I want people to use pip to install the cli because it's the easiest
>>>>> and because rpm/deb packages may have dependency issues like you
>>>>> mentioned => may not work on all distros, what we can do is when
>>>>> people install cloudstack-cli rpm or deb, it runs a script that
>>>>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>>>>> python, so the rpm/deb can also ship bundling src tarballs of
>>>>> cloudmonkey and its dependencies and install from it. Advise best way
>>>>> of doing this?
>>>>>
>>>>
>>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>>
>>>> You are assuming that they would have internet access when installing
>>>> - which is not a valid assumption.
>>>>
>>>> Honestly, the above idea makes me blanch. A package that reports as
>>>> installed, and may or may not have installed - may have installed a
>>>> compromised package (see rubygems.org compromise recently, kernel.org,
>>>> and a number of other site compromises.), or might have installed
>>>> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
>>>> know you are installing some of the dependencies, there is no record
>>>> of those packages in the package manager, and there might potentially
>>>> be conflicts with system packages, a security vulnerability in one of
>>>> those dependencies wouldn't be caught on audit, etc etc.
>>>>
>>>
>>> /facepalm\, it's just a problem of packaging. The package can include cli
>>> or any other artifact's dependencies, so in case of cli, you bundle both
>>> pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure
>>> python so easily installable. The cool people can use pip to install.
>>>
>>>
>>>> And I really don't intend for this to sound like a rant, but the one
>>>> of the important benefits behind using packages and a package manager
>>>> is that a sysadmin needs (and often is required to have by government
>>>> regulations) a single source of truth about the software installed on
>>>> a machine.
>>>>
>>>
>>> No, it's not a rant, I understand.
>>>
>>> Developers love things like Maven central, pypi, CPAN, and rubygems,
>>>> and for good reason, they are fast, flexible, and make their life
>>>> easy. To a sysadmin managing machines in production, they are
>>>> anathema; they make system state difficult or impossible to determine,
>>>> they make audits painful.
>>>>
>>>
>>> I just assumed the sysadmin who would install CloudStack, cli and whatnot
>>> won't be stupid, at the same time I want his life to be less miserable.
>>>
>>> In addition they make troubleshooting
>>>> incredibly difficult. Do I have $foo installed - which version? Are
>>>> there multiple copies of $foo installed on the system? Which one is
>>>> actually being called/loaded?
>>>>
>>>
>>> Alright, but I'm talking only about the cli, since most users, admins
>>> included, would want it to run on their machines, the installation should
>>> be easiest, that's why I said they can use pip, so it works on their
>>> windows, osx, linux, bsd boxes and that's why it's pure python (written
>>> that way).
>>>
>>> Regards.
>>>
>>>>
>>>> --David
>>>>
>>>
>>
>
>
Re: [DISCUSS] Packaging in 4.1
Posted by Noa Resare <no...@spotify.com>.
I built the latest from the packaging branch. I think it is shaping up
nicely. Some thoughts:
How would you feel with postponing the config directory changes until 4.2?
While I agree conceptually that they are an improvement, waiting with them
would keep the diff size down, simplifying merge and the focus of
stabilization for 4.1
/n
On Mon, Feb 4, 2013 at 11:33 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
>
>> Wanted to check when would this be implemented?? Several QA folks depend
>> on the packages and need this working.
>> We have been still building using waf but today that is also not working
>> as some references are removed.
>>
>> Is it possible to accelerate this process or leave old way of packaging
>> in place till you are done with the new changes
>>
>>
> I fully understand. As I told David, I think the DEB work is about one day
> of work, but then again, there is something like $dayjob.
>
> What might be even tougher is to get the RPM and DEB packages fully
> synced. cloudstack-common for example should contain exactly the same files
> in the RPM and DEB version, so Hugo and I will have to keep in touch.
>
> I really want to have the DEB packaging working this week, period.
>
> Wido
>
>
> Thanks
>> /sudha
>>
>> -----Original Message-----
>> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com**] On Behalf
>> Of Rohit Yadav
>> Sent: Sunday, February 03, 2013 5:14 PM
>> To: cloudstack-dev@incubator.**apache.org<cl...@incubator.apache.org>
>> Subject: Re: [DISCUSS] Packaging in 4.1
>>
>> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>>
>>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>>>
>>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>>> ...
>>>>
>>>>>
>>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>>> worth than clint (clint is in EPEL, but no new version of pygments
>>>>> in
>>>>> EPEL/CentOS-Extras/CentOS-**Plus)
>>>>>
>>>>
>>>> I want people to use pip to install the cli because it's the easiest
>>>> and because rpm/deb packages may have dependency issues like you
>>>> mentioned => may not work on all distros, what we can do is when
>>>> people install cloudstack-cli rpm or deb, it runs a script that
>>>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>>>> python, so the rpm/deb can also ship bundling src tarballs of
>>>> cloudmonkey and its dependencies and install from it. Advise best way
>>>> of doing this?
>>>>
>>>
>>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>>
>>> You are assuming that they would have internet access when installing
>>> - which is not a valid assumption.
>>>
>>> Honestly, the above idea makes me blanch. A package that reports as
>>> installed, and may or may not have installed - may have installed a
>>> compromised package (see rubygems.org compromise recently, kernel.org,
>>> and a number of other site compromises.), or might have installed
>>> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
>>> know you are installing some of the dependencies, there is no record
>>> of those packages in the package manager, and there might potentially
>>> be conflicts with system packages, a security vulnerability in one of
>>> those dependencies wouldn't be caught on audit, etc etc.
>>>
>>
>> /facepalm\, it's just a problem of packaging. The package can include cli
>> or any other artifact's dependencies, so in case of cli, you bundle both
>> pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure
>> python so easily installable. The cool people can use pip to install.
>>
>>
>>> And I really don't intend for this to sound like a rant, but the one
>>> of the important benefits behind using packages and a package manager
>>> is that a sysadmin needs (and often is required to have by government
>>> regulations) a single source of truth about the software installed on
>>> a machine.
>>>
>>
>> No, it's not a rant, I understand.
>>
>> Developers love things like Maven central, pypi, CPAN, and rubygems,
>>> and for good reason, they are fast, flexible, and make their life
>>> easy. To a sysadmin managing machines in production, they are
>>> anathema; they make system state difficult or impossible to determine,
>>> they make audits painful.
>>>
>>
>> I just assumed the sysadmin who would install CloudStack, cli and whatnot
>> won't be stupid, at the same time I want his life to be less miserable.
>>
>> In addition they make troubleshooting
>>> incredibly difficult. Do I have $foo installed - which version? Are
>>> there multiple copies of $foo installed on the system? Which one is
>>> actually being called/loaded?
>>>
>>
>> Alright, but I'm talking only about the cli, since most users, admins
>> included, would want it to run on their machines, the installation should
>> be easiest, that's why I said they can use pip, so it works on their
>> windows, osx, linux, bsd boxes and that's why it's pure python (written
>> that way).
>>
>> Regards.
>>
>>>
>>> --David
>>>
>>
>
--
Engineering Experience, Infrastructure tribe, Spotify
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/04/2013 07:12 AM, Sudha Ponnaganti wrote:
> Wanted to check when would this be implemented?? Several QA folks depend on the packages and need this working.
> We have been still building using waf but today that is also not working as some references are removed.
>
> Is it possible to accelerate this process or leave old way of packaging in place till you are done with the new changes
>
I fully understand. As I told David, I think the DEB work is about one
day of work, but then again, there is something like $dayjob.
What might be even tougher is to get the RPM and DEB packages fully
synced. cloudstack-common for example should contain exactly the same
files in the RPM and DEB version, so Hugo and I will have to keep in touch.
I really want to have the DEB packaging working this week, period.
Wido
> Thanks
> /sudha
>
> -----Original Message-----
> From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com] On Behalf Of Rohit Yadav
> Sent: Sunday, February 03, 2013 5:14 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Packaging in 4.1
>
> On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
>> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>>> ...
>>>>
>>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>>> worth than clint (clint is in EPEL, but no new version of pygments
>>>> in
>>>> EPEL/CentOS-Extras/CentOS-Plus)
>>>
>>> I want people to use pip to install the cli because it's the easiest
>>> and because rpm/deb packages may have dependency issues like you
>>> mentioned => may not work on all distros, what we can do is when
>>> people install cloudstack-cli rpm or deb, it runs a script that
>>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>>> python, so the rpm/deb can also ship bundling src tarballs of
>>> cloudmonkey and its dependencies and install from it. Advise best way
>>> of doing this?
>>
>> I guess we won't be installing the CLI via RPMs at least for EL6.
>>
>> You are assuming that they would have internet access when installing
>> - which is not a valid assumption.
>>
>> Honestly, the above idea makes me blanch. A package that reports as
>> installed, and may or may not have installed - may have installed a
>> compromised package (see rubygems.org compromise recently, kernel.org,
>> and a number of other site compromises.), or might have installed
>> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
>> know you are installing some of the dependencies, there is no record
>> of those packages in the package manager, and there might potentially
>> be conflicts with system packages, a security vulnerability in one of
>> those dependencies wouldn't be caught on audit, etc etc.
>
> /facepalm\, it's just a problem of packaging. The package can include cli or any other artifact's dependencies, so in case of cli, you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure python so easily installable. The cool people can use pip to install.
>
>>
>> And I really don't intend for this to sound like a rant, but the one
>> of the important benefits behind using packages and a package manager
>> is that a sysadmin needs (and often is required to have by government
>> regulations) a single source of truth about the software installed on
>> a machine.
>
> No, it's not a rant, I understand.
>
>> Developers love things like Maven central, pypi, CPAN, and rubygems,
>> and for good reason, they are fast, flexible, and make their life
>> easy. To a sysadmin managing machines in production, they are
>> anathema; they make system state difficult or impossible to determine,
>> they make audits painful.
>
> I just assumed the sysadmin who would install CloudStack, cli and whatnot won't be stupid, at the same time I want his life to be less miserable.
>
>> In addition they make troubleshooting
>> incredibly difficult. Do I have $foo installed - which version? Are
>> there multiple copies of $foo installed on the system? Which one is
>> actually being called/loaded?
>
> Alright, but I'm talking only about the cli, since most users, admins included, would want it to run on their machines, the installation should be easiest, that's why I said they can use pip, so it works on their windows, osx, linux, bsd boxes and that's why it's pure python (written that way).
>
> Regards.
>>
>> --David
RE: [DISCUSS] Packaging in 4.1
Posted by Sudha Ponnaganti <su...@citrix.com>.
Wanted to check when would this be implemented?? Several QA folks depend on the packages and need this working.
We have been still building using waf but today that is also not working as some references are removed.
Is it possible to accelerate this process or leave old way of packaging in place till you are done with the new changes
Thanks
/sudha
-----Original Message-----
From: rohityadav89@gmail.com [mailto:rohityadav89@gmail.com] On Behalf Of Rohit Yadav
Sent: Sunday, February 03, 2013 5:14 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Packaging in 4.1
On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>> ...
>>>
>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>> worth than clint (clint is in EPEL, but no new version of pygments
>>> in
>>> EPEL/CentOS-Extras/CentOS-Plus)
>>
>> I want people to use pip to install the cli because it's the easiest
>> and because rpm/deb packages may have dependency issues like you
>> mentioned => may not work on all distros, what we can do is when
>> people install cloudstack-cli rpm or deb, it runs a script that
>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>> python, so the rpm/deb can also ship bundling src tarballs of
>> cloudmonkey and its dependencies and install from it. Advise best way
>> of doing this?
>
> I guess we won't be installing the CLI via RPMs at least for EL6.
>
> You are assuming that they would have internet access when installing
> - which is not a valid assumption.
>
> Honestly, the above idea makes me blanch. A package that reports as
> installed, and may or may not have installed - may have installed a
> compromised package (see rubygems.org compromise recently, kernel.org,
> and a number of other site compromises.), or might have installed
> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
> know you are installing some of the dependencies, there is no record
> of those packages in the package manager, and there might potentially
> be conflicts with system packages, a security vulnerability in one of
> those dependencies wouldn't be caught on audit, etc etc.
/facepalm\, it's just a problem of packaging. The package can include cli or any other artifact's dependencies, so in case of cli, you bundle both pygments and prettytable in cli's rpm/deb. AFAIK all of them are pure python so easily installable. The cool people can use pip to install.
>
> And I really don't intend for this to sound like a rant, but the one
> of the important benefits behind using packages and a package manager
> is that a sysadmin needs (and often is required to have by government
> regulations) a single source of truth about the software installed on
> a machine.
No, it's not a rant, I understand.
> Developers love things like Maven central, pypi, CPAN, and rubygems,
> and for good reason, they are fast, flexible, and make their life
> easy. To a sysadmin managing machines in production, they are
> anathema; they make system state difficult or impossible to determine,
> they make audits painful.
I just assumed the sysadmin who would install CloudStack, cli and whatnot won't be stupid, at the same time I want his life to be less miserable.
> In addition they make troubleshooting
> incredibly difficult. Do I have $foo installed - which version? Are
> there multiple copies of $foo installed on the system? Which one is
> actually being called/loaded?
Alright, but I'm talking only about the cli, since most users, admins included, would want it to run on their machines, the installation should be easiest, that's why I said they can use pip, so it works on their windows, osx, linux, bsd boxes and that's why it's pure python (written that way).
Regards.
>
> --David
Re: [DISCUSS] Packaging in 4.1
Posted by Rohit Yadav <bh...@apache.org>.
On Mon, Feb 4, 2013 at 4:41 AM, David Nalley <da...@gnsa.us> wrote:
> On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
>> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
>> ...
>>>
>>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>>> worth than clint (clint is in EPEL, but no new version of pygments in
>>> EPEL/CentOS-Extras/CentOS-Plus)
>>
>> I want people to use pip to install the cli because it's the easiest
>> and because rpm/deb packages may have dependency issues like you
>> mentioned => may not work on all distros, what we can do is when
>> people install cloudstack-cli rpm or deb, it runs a script that
>> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
>> python, so the rpm/deb can also ship bundling src tarballs of
>> cloudmonkey and its dependencies and install from it. Advise best way
>> of doing this?
>
> I guess we won't be installing the CLI via RPMs at least for EL6.
>
> You are assuming that they would have internet access when installing
> - which is not a valid assumption.
>
> Honestly, the above idea makes me blanch. A package that reports as
> installed, and may or may not have installed - may have installed a
> compromised package (see rubygems.org compromise recently, kernel.org,
> and a number of other site compromises.), or might have installed
> packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
> know you are installing some of the dependencies, there is no record
> of those packages in the package manager, and there might potentially
> be conflicts with system packages, a security vulnerability in one of
> those dependencies wouldn't be caught on audit, etc etc.
/facepalm\, it's just a problem of packaging. The package can include
cli or any other artifact's dependencies, so in case of cli, you
bundle both pygments and prettytable in cli's rpm/deb. AFAIK all of
them are pure python so easily installable. The cool people can use
pip to install.
>
> And I really don't intend for this to sound like a rant, but the one
> of the important benefits behind using packages and a package manager
> is that a sysadmin needs (and often is required to have by government
> regulations) a single source of truth about the software installed on
> a machine.
No, it's not a rant, I understand.
> Developers love things like Maven central, pypi, CPAN, and
> rubygems, and for good reason, they are fast, flexible, and make their
> life easy. To a sysadmin managing machines in production, they are
> anathema; they make system state difficult or impossible to determine,
> they make audits painful.
I just assumed the sysadmin who would install CloudStack, cli and
whatnot won't be stupid, at the same time I want his life to be less
miserable.
> In addition they make troubleshooting
> incredibly difficult. Do I have $foo installed - which version? Are
> there multiple copies of $foo installed on the system? Which one is
> actually being called/loaded?
Alright, but I'm talking only about the cli, since most users, admins
included, would want it to run on their machines, the installation
should be easiest, that's why I said they can use pip, so it works on
their windows, osx, linux, bsd boxes and that's why it's pure python
(written that way).
Regards.
>
> --David
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Sun, Feb 3, 2013 at 3:58 PM, Rohit Yadav <bh...@apache.org> wrote:
> On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
> ...
>>
>> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
>> worth than clint (clint is in EPEL, but no new version of pygments in
>> EPEL/CentOS-Extras/CentOS-Plus)
>
> I want people to use pip to install the cli because it's the easiest
> and because rpm/deb packages may have dependency issues like you
> mentioned => may not work on all distros, what we can do is when
> people install cloudstack-cli rpm or deb, it runs a script that
> installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
> python, so the rpm/deb can also ship bundling src tarballs of
> cloudmonkey and its dependencies and install from it. Advise best way
> of doing this?
I guess we won't be installing the CLI via RPMs at least for EL6.
You are assuming that they would have internet access when installing
- which is not a valid assumption.
Honestly, the above idea makes me blanch. A package that reports as
installed, and may or may not have installed - may have installed a
compromised package (see rubygems.org compromise recently, kernel.org,
and a number of other site compromises.), or might have installed
packages I didn't know about is a Bad Idea (tm) The sysadmin doesn't
know you are installing some of the dependencies, there is no record
of those packages in the package manager, and there might potentially
be conflicts with system packages, a security vulnerability in one of
those dependencies wouldn't be caught on audit, etc etc.
And I really don't intend for this to sound like a rant, but the one
of the important benefits behind using packages and a package manager
is that a sysadmin needs (and often is required to have by government
regulations) a single source of truth about the software installed on
a machine. Developers love things like Maven central, pypi, CPAN, and
rubygems, and for good reason, they are fast, flexible, and make their
life easy. To a sysadmin managing machines in production, they are
anathema; they make system state difficult or impossible to determine,
they make audits painful. In addition they make troubleshooting
incredibly difficult. Do I have $foo installed - which version? Are
there multiple copies of $foo installed on the system? Which one is
actually being called/loaded?
--David
Re: [DISCUSS] Packaging in 4.1
Posted by Rohit Yadav <bh...@apache.org>.
On Sun, Feb 3, 2013 at 3:07 PM, David Nalley <da...@gnsa.us> wrote:
...
>
> So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
> worth than clint (clint is in EPEL, but no new version of pygments in
> EPEL/CentOS-Extras/CentOS-Plus)
I want people to use pip to install the cli because it's the easiest
and because rpm/deb packages may have dependency issues like you
mentioned => may not work on all distros, what we can do is when
people install cloudstack-cli rpm or deb, it runs a script that
installs pip (if unavailable) and cloudmonkey. cloudmonkey is pure
python, so the rpm/deb can also ship bundling src tarballs of
cloudmonkey and its dependencies and install from it. Advise best way
of doing this?
Regards.
>
> --David
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Sun, Feb 3, 2013 at 4:04 AM, Rohit Yadav <bh...@apache.org> wrote:
> On Sun, Feb 3, 2013 at 2:19 PM, David Nalley <da...@gnsa.us> wrote:
>>> Alright, we will also have to implement upgrade paths and
>>> s/cloud-/cloudstack-/g in a lot of scripts.
>>>
>>>> I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name}
>>>
>>> Why not /usr/share/cloudstack/%{name}? cloudstack-cli would have to be
>>> installed like any other python app, in /usr/*path to python 2.6 or
>>> 2.7 dir*/site-packages/cloudmonkey, or there will be some other format
>>> of installation?
>>>
>>
>> WRT to marvin and cloudmonkey - they will be in site-packages - well
>> that is assuming that we package them. We want to, but at least in
>> RHEL/CentOS, there are dependencies for cloudmonkey (clint) that don't
>
> I got rid of clint, at present cloudmonkey uses pygments for lexical
> parsing and color printing (for future this can also be used to do
> html output stuff), prettytables for tabular output, these are the
> only two external dependencies. cloudmonkey does not depend on marvin
> as well, against a running mgmt server a precache can be created for
> building/packaging cloudmonkey or you do: cloudmonkey sync that
> discovers new apis and generates a cache for the user. Since pygments
> is pretty widely used it may not be a problem for most distros also I
> can write cloudmonkey's own tabular printing so it won't depend on
> prettytable.
>
> Regards.
>
>> exist in the RHEL repos (though it does exist in EPEL). I need to
>> start a thread on that, as essentially cloudstack-cli will not install
>> in a default RHEL/CentOS 6.3 install.
>>
>> I haven't yet looked at Marvin to see what the dependencies are there yet.
>>
>> --David
So EL6 has pygments 1.1.1 - you require 1.5, so in some ways it's
worth than clint (clint is in EPEL, but no new version of pygments in
EPEL/CentOS-Extras/CentOS-Plus)
--David
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
On Sun, Feb 3, 2013 at 4:04 AM, Rohit Yadav <bh...@apache.org> wrote:
> On Sun, Feb 3, 2013 at 2:19 PM, David Nalley <da...@gnsa.us> wrote:
>>> Alright, we will also have to implement upgrade paths and
>>> s/cloud-/cloudstack-/g in a lot of scripts.
>>>
>>>> I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name}
>>>
>>> Why not /usr/share/cloudstack/%{name}? cloudstack-cli would have to be
>>> installed like any other python app, in /usr/*path to python 2.6 or
>>> 2.7 dir*/site-packages/cloudmonkey, or there will be some other format
>>> of installation?
>>>
>>
>> WRT to marvin and cloudmonkey - they will be in site-packages - well
>> that is assuming that we package them. We want to, but at least in
>> RHEL/CentOS, there are dependencies for cloudmonkey (clint) that don't
>
> I got rid of clint, at present cloudmonkey uses pygments for lexical
> parsing and color printing (for future this can also be used to do
> html output stuff), prettytables for tabular output, these are the
> only two external dependencies. cloudmonkey does not depend on marvin
> as well, against a running mgmt server a precache can be created for
> building/packaging cloudmonkey or you do: cloudmonkey sync that
> discovers new apis and generates a cache for the user. Since pygments
> is pretty widely used it may not be a problem for most distros also I
> can write cloudmonkey's own tabular printing so it won't depend on
> prettytable.
>
Doh! - that's what I get for not looking in source and depending on
memory. That's good to know.
--David
Re: [DISCUSS] Packaging in 4.1
Posted by Rohit Yadav <bh...@apache.org>.
On Sun, Feb 3, 2013 at 2:19 PM, David Nalley <da...@gnsa.us> wrote:
>> Alright, we will also have to implement upgrade paths and
>> s/cloud-/cloudstack-/g in a lot of scripts.
>>
>>> I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name}
>>
>> Why not /usr/share/cloudstack/%{name}? cloudstack-cli would have to be
>> installed like any other python app, in /usr/*path to python 2.6 or
>> 2.7 dir*/site-packages/cloudmonkey, or there will be some other format
>> of installation?
>>
>
> WRT to marvin and cloudmonkey - they will be in site-packages - well
> that is assuming that we package them. We want to, but at least in
> RHEL/CentOS, there are dependencies for cloudmonkey (clint) that don't
I got rid of clint, at present cloudmonkey uses pygments for lexical
parsing and color printing (for future this can also be used to do
html output stuff), prettytables for tabular output, these are the
only two external dependencies. cloudmonkey does not depend on marvin
as well, against a running mgmt server a precache can be created for
building/packaging cloudmonkey or you do: cloudmonkey sync that
discovers new apis and generates a cache for the user. Since pygments
is pretty widely used it may not be a problem for most distros also I
can write cloudmonkey's own tabular printing so it won't depend on
prettytable.
Regards.
> exist in the RHEL repos (though it does exist in EPEL). I need to
> start a thread on that, as essentially cloudstack-cli will not install
> in a default RHEL/CentOS 6.3 install.
>
> I haven't yet looked at Marvin to see what the dependencies are there yet.
>
> --David
Re: [DISCUSS] Packaging in 4.1
Posted by David Nalley <da...@gnsa.us>.
> Alright, we will also have to implement upgrade paths and
> s/cloud-/cloudstack-/g in a lot of scripts.
>
>> I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name}
>
> Why not /usr/share/cloudstack/%{name}? cloudstack-cli would have to be
> installed like any other python app, in /usr/*path to python 2.6 or
> 2.7 dir*/site-packages/cloudmonkey, or there will be some other format
> of installation?
>
WRT to marvin and cloudmonkey - they will be in site-packages - well
that is assuming that we package them. We want to, but at least in
RHEL/CentOS, there are dependencies for cloudmonkey (clint) that don't
exist in the RHEL repos (though it does exist in EPEL). I need to
start a thread on that, as essentially cloudstack-cli will not install
in a default RHEL/CentOS 6.3 install.
I haven't yet looked at Marvin to see what the dependencies are there yet.
--David
Re: [DISCUSS] Packaging in 4.1
Posted by Wido den Hollander <wi...@widodh.nl>.
On 02/03/2013 08:34 AM, Rohit Yadav wrote:
> On Sun, Feb 3, 2013 at 12:54 PM, Hugo Trippaers
> <HT...@schubergphilis.com> wrote:
>> Heya all,
>>
>> As Devid already mentioned we met up at Build-A-Cloud-Day in Ghent with Wido, Noa and a few other folks. During this day (with lots of nice talks) we had a chance to sit down and discuss packaging. Something that Noa, Wido and myself were planning to do for a long time. Some other people joined in the discussion and with the help of the available black board (the event was held in a school ;-) ) we managed to sync our ideas on packaging the 4.1 release.
>>
>> With our ideas synced we thought it time to bring our ideas to the list and ask for feedback from the community. We are also using our current time at FOSDEM to sync up with some of the people from the distros to see what they think of the new ideas, it would still be nice to have CloudStack packaged up and shipped with some of the distributions out there.
>>
>> So the main goals of redoing packaging are getting rid of ant and waf completely. A secondary goal is to create a reference set of packages which in itself should be enough to get anyone going with CloudStack, but will hardly depend on the underlying distro. Real distro specific stuff should be handled by packagers from those distros. We just aim to provide a reference implementation.
>>
>> Our goal is to have a reference implementation that will install on the following list of operating systems: CentOS 6.3, Ubuntu 12.04 and Fedora 18. This means that it will probably install and run on a lot more, but this is the set that we will test against (i'm using a jenkins system at the office to automatically build and install and these images will be used for the tests).
>>
>> Next we will remove as much system dependencies as possible, so we will use maven to gather the dependencies and make sure that they are packaged and shipped with the RPMs. This makes for slightly bigger packages, but reduced the overhead of having to check each operating system and run the risk of version mismatched with versions of jar files present on the distro.
>>
>> We also intend to change the name of the packages to cloudstack to make it perfectly clear what somebody is installing, this will also affect the location of several files like configuration files and scripts, but we plan to include symlinks for backwards compatibility. The feasibility of this will obviously be tested in the packaging street my collegues are building for me.
>>
>
> Awesome!
>
>> The planned packages for now are cloudstack-management, cloudstack-agent, cloudstack-common, cloudstack-cli, cloudstack-docs, cloudstack-awsapi and cloudstack-usage. You might already have seen these names in some of the checkins.
>
> Alright, we will also have to implement upgrade paths and
> s/cloud-/cloudstack-/g in a lot of scripts.
>
>> I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name}
>
> Why not /usr/share/cloudstack/%{name}? cloudstack-cli would have to be
> installed like any other python app, in /usr/*path to python 2.6 or
> 2.7 dir*/site-packages/cloudmonkey, or there will be some other format
> of installation?
>
Because of how distros do things. Packages should not share a directory
like /usr/share/cloudstack, so they all need their own directory for stuff.
Otherwise you can get "directory not empty" issues when removing.
>> and the main jar will be located there and any dependencies will be located in the lib directory beneath that location. With the exception of management which will be created as an exploded webapplication archive in the same directory. Scripts will be located in /usr/share/cloudstack-common/scripts and symlinks will be made to the previous locations for backwards compatibility.
>
> +1 I want that
> Why not put eveything under /usr/share/java/cloudstack/* and using
> recursive dir traversal create classpath reusable in all scripts?
>
Then you'll be adding a lot to the classpath which you don't actually need.
So there will indeed be some redundant storage on systems, but the
amount of data is small.
Towards 4.2 we want to make more changes, but we can't do everything at
once.
Now it's just setting up a machine (VM) with 4.0 in it. Try the upgrade,
learn from it, revert the VM to 4.0 and try again until it works.
Snapshots ftw!
Wido
>> I think these are the highlights of what we intend to do for release 4.1. We have a lot of plans for subsequenst release and on how to get us into distros. But for now we thought it prudent to focus on getting packages for the 4.1 release as son as possible and focus on other improvement later.
>
> Awesome, keep us posted.
>
> Regards.
>
>>
>> @Wido, @Noa, @David did i miss anything important?
>>
>> Cheers,
>>
>> Hugo
>>
>>
Re: [DISCUSS] Packaging in 4.1
Posted by Rohit Yadav <bh...@apache.org>.
On Sun, Feb 3, 2013 at 12:54 PM, Hugo Trippaers
<HT...@schubergphilis.com> wrote:
> Heya all,
>
> As Devid already mentioned we met up at Build-A-Cloud-Day in Ghent with Wido, Noa and a few other folks. During this day (with lots of nice talks) we had a chance to sit down and discuss packaging. Something that Noa, Wido and myself were planning to do for a long time. Some other people joined in the discussion and with the help of the available black board (the event was held in a school ;-) ) we managed to sync our ideas on packaging the 4.1 release.
>
> With our ideas synced we thought it time to bring our ideas to the list and ask for feedback from the community. We are also using our current time at FOSDEM to sync up with some of the people from the distros to see what they think of the new ideas, it would still be nice to have CloudStack packaged up and shipped with some of the distributions out there.
>
> So the main goals of redoing packaging are getting rid of ant and waf completely. A secondary goal is to create a reference set of packages which in itself should be enough to get anyone going with CloudStack, but will hardly depend on the underlying distro. Real distro specific stuff should be handled by packagers from those distros. We just aim to provide a reference implementation.
>
> Our goal is to have a reference implementation that will install on the following list of operating systems: CentOS 6.3, Ubuntu 12.04 and Fedora 18. This means that it will probably install and run on a lot more, but this is the set that we will test against (i'm using a jenkins system at the office to automatically build and install and these images will be used for the tests).
>
> Next we will remove as much system dependencies as possible, so we will use maven to gather the dependencies and make sure that they are packaged and shipped with the RPMs. This makes for slightly bigger packages, but reduced the overhead of having to check each operating system and run the risk of version mismatched with versions of jar files present on the distro.
>
> We also intend to change the name of the packages to cloudstack to make it perfectly clear what somebody is installing, this will also affect the location of several files like configuration files and scripts, but we plan to include symlinks for backwards compatibility. The feasibility of this will obviously be tested in the packaging street my collegues are building for me.
>
Awesome!
> The planned packages for now are cloudstack-management, cloudstack-agent, cloudstack-common, cloudstack-cli, cloudstack-docs, cloudstack-awsapi and cloudstack-usage. You might already have seen these names in some of the checkins.
Alright, we will also have to implement upgrade paths and
s/cloud-/cloudstack-/g in a lot of scripts.
> I this the best we to demonstrate is by showing code, instead of a lengthy email ;-) All packages will have a directory in /usr/share/cloudstack-%{name}
Why not /usr/share/cloudstack/%{name}? cloudstack-cli would have to be
installed like any other python app, in /usr/*path to python 2.6 or
2.7 dir*/site-packages/cloudmonkey, or there will be some other format
of installation?
> and the main jar will be located there and any dependencies will be located in the lib directory beneath that location. With the exception of management which will be created as an exploded webapplication archive in the same directory. Scripts will be located in /usr/share/cloudstack-common/scripts and symlinks will be made to the previous locations for backwards compatibility.
+1 I want that
Why not put eveything under /usr/share/java/cloudstack/* and using
recursive dir traversal create classpath reusable in all scripts?
> I think these are the highlights of what we intend to do for release 4.1. We have a lot of plans for subsequenst release and on how to get us into distros. But for now we thought it prudent to focus on getting packages for the 4.1 release as son as possible and focus on other improvement later.
Awesome, keep us posted.
Regards.
>
> @Wido, @Noa, @David did i miss anything important?
>
> Cheers,
>
> Hugo
>
>