rfc: newsystemvm

[Leo Simons <LS...@schubergphilis.com>]

Hey folks,

https://github.com/schubergphilis/cloudstack/commit/f125f1564e8921def00dc0235ecca51470a2a22e
https://github.com/schubergphilis/cloudstack/tree/f125f1564e8921def00dc0235ecca51470a2a22e/tools/appliance

This started out as wanting the systemvm build to take systemvm/patches/debian/{debian,vpn} from the local machine/branch, rather than downloading from the apache git master [1]. In working out how on earth to get veewee to do that cleanly (hint: you can’t, hence resorting to shar usage) I got quite frustrated with the image rebuild times.

It so happens that veewee has a --skip-to-postinstall instruction which is _quite_ useful while debugging these scripts. To get that working requires the post install steps to be retryable/convergent. Of course, our existing scripts weren’t set up for that. So I had to add a bunch of tests whether changes had applied already. Which implied a pretty significant refactor.

I think I was careful enough and I expect this new template will work just as well as the old one. This is a change that we can (and probably should?) merge to master independently of the redundant VPC work (though the `apt-get install chef` would need to be taken out). But, given how big of a chunk of code has changed here, before upstreaming (a version of) this to apache we (I) need to do more testing. So for now I’ve put this change next to the existing definitions rather than replace ‘em, to not block anything else.

Comments/thoughts?


cheers,


Leo


[1] https://github.com/schubergphilis/cloudstack/blob/master/tools/appliance/definitions/systemvmtemplate/postinstall.sh#L228

Begin forwarded message:
...
>    M tools/appliance/build.sh
...
>    A tools/appliance/definitions/newsystemvm/apt_upgrade.sh
>    A tools/appliance/definitions/newsystemvm/authorized_keys.sh
>    A tools/appliance/definitions/newsystemvm/build_time.sh
>    A tools/appliance/definitions/newsystemvm/cleanup.sh
>    A tools/appliance/definitions/newsystemvm/configure_acpid.sh
>    A tools/appliance/definitions/newsystemvm/configure_conntrack.sh
>    A tools/appliance/definitions/newsystemvm/configure_grub.sh
>    A tools/appliance/definitions/newsystemvm/configure_locale.sh
>    A tools/appliance/definitions/newsystemvm/configure_login.sh
>    A tools/appliance/definitions/newsystemvm/configure_networking.sh
>    A tools/appliance/definitions/newsystemvm/configure_systemvm_services.sh
>    A tools/appliance/definitions/newsystemvm/definition.rb
>    A tools/appliance/definitions/newsystemvm/install_systemvm_packages.sh
>    A tools/appliance/definitions/newsystemvm/preseed.cfg
>    A tools/appliance/definitions/newsystemvm/zerodisk.sh
>    A tools/appliance/shar_cloud_scripts.sh
>    A tools/appliance/test.sh
...
> Work in progress to rewrite systemvm box definition in clean code.
> 
> * Refactor build.sh to use functions and pretty logging.
> * Add a new test.sh which tries several different build.sh invocations.
> * Add a new 'debianbase' definition which is just the basic debian box (no systemvm functionality). This is used for testing.
> * Add a new 'newsystemvm' definition which is a work-in-progress replacement of the 'systemvmtemplate' definition:
> ** reduce duplication between preseed.cfg and postinstall.sh
> ** remove duplication between postinstall.sh and cloud-scripts
> ** install cloud-scripts from current working copy instead of tying to a download of the apache master branch
> ** split up postinstall.sh into small utility scripts and clean up that code
> ** clean up preseed.cfg to match current veewee/debian best practices and remove cruft