[jira] [Assigned] (CLOUDSTACK-1019) Fix cloud-setup-database to use DatabaseCreator and make DatabaseCreator do the upgrades

["Abhinandan Prateek (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhinandan Prateek reassigned CLOUDSTACK-1019:
----------------------------------------------

    Assignee:     (was: Abhinandan Prateek)
    
> Fix cloud-setup-database to use DatabaseCreator and make DatabaseCreator do the upgrades
> ----------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-1019
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1019
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>    Affects Versions: 4.2.0
>         Environment: branch: javelin (or master if merge happens)
>            Reporter: Rohit Yadav
>            Priority: Critical
>             Fix For: 4.2.0
>
>
> A new tool/class called DatabaseCreator is introduced in javelin, the idea is that DatabaseCreator can be used by mgmt server, cloud-setup-database script, maven (developer/pom.xml) and possibly by plugins in future to:
> 1. Initialize database using db.properties file
> 2. Run sql scripts
> 3. Run database upgrades
> Presently, this has been fixed in javelin, in cloud-server and is used by maven to deploydb, the task is to fix that same for cloud-setup-databases.
> Current help doc, usage and options:
> DatabaseCreator creates the database schema by removing the 
> previous schema, creating the schema, and running 
> through the database updaters.
> Usage: DatabaseCreator [options] [db.properties file] [schema.sql files] [database upgrade class]
> Options:
>    --database=a,b comma separate databases to initialize, use the db name in db.properties defined as db.xyz.host, xyz should be passed
>    --rootpassword=password, by default it will try with an empty password
>    --dry or -d, this would not run any process, just does a dry run
>    --verbose or -v to print running sql commands, by default it won't print them
>    --help or -h for help
> The issue now is to make cloudstack-setup-databases use DatabaseCreator and remove db upgrading logic from mgmt server and move it to DatabaseCreator.
> Let's talk about the present database deployment and the issues:
> - We've two different routes to deploydb, for developers it's
> DatabaseCreator now (used to be a maven plugin), for sysadmins it's
> cloudstack-setup-database
> - Upgrades are done by mgmt server, this has issues when there is a
> clustered mgmt server, i.e. multiple mgmt servers sharing the same
> database, we do upgrade by:
>      a. Turn down one mgmt server, upgrade it, start it. This mgmt
> server checks in db the cloudstack version, upgrades the db
>      b. Sysadmins meanwhile upgrades all the other mgmt server
>      c. When all mgmt servers are upgraded, they ask the cluster
> manager to run the cleanup. The whole process is automated, an error
> can introduce whole sets of issues.
> How will DatabaseCreator help:
> - Would give a single tool used by both developer and sysadmins. Agony
> of the sysadmins would be much better understood by developers.
> Sysadmins would still use cloudstack-setup-database which would be
> wrapper around DatabaseCreator
> - Upgrade would be driven by sysadmin, giving them explicit power to
> backup database, deploy etc. For example in case of multiple
> (clustered) mgmt server, the upgrade would be like:
>    a. Upgrade database using DatabaseCreator, this step do a backup
> and would just upgrade the db such that it's backward compatible to
> the old database and old mgmt servers can still run on it. When this
> succeeds, system can proceed. Transaction update would provide a way
> to rollback.
>    b. Sysadmins would stop, upgrade and start mgmt server one by one.
>    c. Sysadmins would run the final cleanup operation, this would do a
> final round of cleaning up. If everything goes well, they just did a
> rolling CloudStack upgrade with little to no downtime (no downtime as
> at least one mgmt server was running :)
> TODOs for 4.1:
> - We stick to major db version as 4.0, create-schema is not touched
> unless it's a major version. So, next change is to be done only for
> 5.0.
> - We move out any 4.1 changes to 4041 upgrade sql file and have a
> upgrade path (a upgrading logic/class).
> - Fix cloudstack-setup-databases, fix packaging.
> For 4.2/future:
> - Refactor out code that does upgrading in mgmt server to a separate
> artifact which would also have DatabaseCreator.
> - Implement tool so it supports the operations as discussed above
> - Replace the current pythonic implementation with DatabaseCreator,
> have same routes for both developers and sysadmins
> NOTE TO DEVELOPERS:
> - WE NEVER CHANGE create-schema.sql
> - WE TEST AND IMPLEMENT UPGRADE PATHS from version a to version b
> - WE CLEANUP PROPERLY via our upgrade sql files and upgrade classes

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira