Database compatibility checking step during startup

["Napoleon Esmundo C. Ramirez" <na...@gmail.com>]

Hello everyone,

Deng and I talked about putting a database compatibility check.  I filed
CONTINUUM-1980 to track this new feature.  In summary, it states that
Continuum should be able to detect when it has been started with an
incompatible database and prompt the user to take the appropriate action.
This should work transparently if all goes well, and will just display
recommendations in the UI if any problem occurred.

By intent, the checking should take place before the database connection is
initialized.  But since the database connection happens during the
components' initialization, it is suggested that the JdoFactory for the
continuum database be removed in the application.xml and is instead replaced
with a wrapper component that initializes the database connection
explicitly--not on component initialization.  This new component can be used
by a database-checking interceptor in the webapp or it can be applied to
other modules like the redback users database.

For now, aside from the new interceptor and component suggested to be added
to continuum, the impact I can see is that the following will have to point
to the new component:
- org.apache.maven.continuum.management.StoreUtilities
- org.apache.maven.continuum.management.JdoDataManagementTool
- org.apache.maven.continuum.management.LegacyJdoDataManagementTool

Currently, they require the the continuum JdoFactory:
/**
 * @plexus.requirement role="org.codehaus.plexus.jdo.JdoFactory"
role-hint="continuum"
 */

But I think these components don't need the JdoFactory right away, so I
think we can delay the initialization of the database connection component
to give way to checking during startup.


What do you think?  Any comments are greatly appreciated.

Cheers!
Nap

Re: Database compatibility checking step during startup

["Napoleon Esmundo C. Ramirez" <na...@gmail.com>]

On Wed, Nov 19, 2008 at 12:28 PM, Brett Porter <br...@apache.org> wrote:

> There's actually a version tracked in the modello metadata database table
> for this purpose already.
>
> The tricky part is how to do the conversion - that seems to need to be done
> outside of modello because of the complexity of having multiple versions
> (see what it did to the data management :) I think if you init normally, but
> don't load any data, then check that table for the right version, you can
> decide to upgrade it before loading any data. You'll need to lock the store
> accesses while that occurs. I expect we'd use some other migration tool like
> has been discussed recently for the task.
>
> - Brett
>
> --
> Brett Porter
> brett@apache.org
> http://blogs.exist.com/bporter/
>
>
I saw a ContinuumModelloMetadata class in the package.jdo file, but it isn't
declared as a table thus isn't persisted in the database.  I verified it
with a MySQL database, and that version table indeed isn't there.

The conversion is the real hard part, and I agree it is the job of the
migration tool.  I was thinking the compatibility checking step can be used
to just identify discrepancies when switching databases and to act as some
fail secure mechanism when starting continuum.  It seems that the
initialization of the database will load the data at the same time, so I was
suggesting to defer it until after the schema validation is done.

- Nap

Re: Database compatibility checking step during startup

[Brett Porter <br...@apache.org>]

There's actually a version tracked in the modello metadata database  
table for this purpose already.

The tricky part is how to do the conversion - that seems to need to be  
done outside of modello because of the complexity of having multiple  
versions (see what it did to the data management :) I think if you  
init normally, but don't load any data, then check that table for the  
right version, you can decide to upgrade it before loading any data.  
You'll need to lock the store accesses while that occurs. I expect  
we'd use some other migration tool like has been discussed recently  
for the task.

- Brett

On 19/11/2008, at 12:19 AM, Napoleon Esmundo C. Ramirez wrote:

> Hello everyone,
>
> Deng and I talked about putting a database compatibility check.  I  
> filed
> CONTINUUM-1980 to track this new feature.  In summary, it states that
> Continuum should be able to detect when it has been started with an
> incompatible database and prompt the user to take the appropriate  
> action.
> This should work transparently if all goes well, and will just display
> recommendations in the UI if any problem occurred.
>
> By intent, the checking should take place before the database  
> connection is
> initialized.  But since the database connection happens during the
> components' initialization, it is suggested that the JdoFactory for  
> the
> continuum database be removed in the application.xml and is instead  
> replaced
> with a wrapper component that initializes the database connection
> explicitly--not on component initialization.  This new component can  
> be used
> by a database-checking interceptor in the webapp or it can be  
> applied to
> other modules like the redback users database.
>
> For now, aside from the new interceptor and component suggested to  
> be added
> to continuum, the impact I can see is that the following will have  
> to point
> to the new component:
> - org.apache.maven.continuum.management.StoreUtilities
> - org.apache.maven.continuum.management.JdoDataManagementTool
> - org.apache.maven.continuum.management.LegacyJdoDataManagementTool
>
> Currently, they require the the continuum JdoFactory:
> /**
> * @plexus.requirement role="org.codehaus.plexus.jdo.JdoFactory"
> role-hint="continuum"
> */
>
> But I think these components don't need the JdoFactory right away,  
> so I
> think we can delay the initialization of the database connection  
> component
> to give way to checking during startup.
>
>
> What do you think?  Any comments are greatly appreciated.
>
> Cheers!
> Nap

--
Brett Porter
brett@apache.org
http://blogs.exist.com/bporter/