Schema Versioning

[Joe Bako <jb...@gracenote.com>]

Hi all,

I am curious what techniques are used by others to manage schema changes over time in Cassandra.

Modern RDBMS tools can compare schemas between DDL object definitions and live databases and generate change scripts accordingly.  Older techniques included maintaining a version and script table in the database, storing schema change scripts in a sequential fashion on disk, and iterating over them to apply them against the target database based on whether they had been run previously or not (indicated in the script table).

The former allows you to easily see schema changes over time in your SCM history, the latter keeps a history of actual DDL applied against the database objects.  What options are available for C* and what are people using?

Joe Bako
Software Architect
Gracenote, Inc.
http://www.gracenote.com/

Re: Schema Versioning

[Jonathan Haddad <jo...@jonhaddad.com>]

I wrote most of the cqlengine keyspace & table management pieces of the
Python driver to solve this exact problem.  Instead of working with a
series of statements for creating tables & managing columns, we simply
created classes in Python and sync'ed them to the DB.  It automatically
figured out what was needed and took care of it.

DB Models:
https://datastax.github.io/python-driver/api/cassandra/cqlengine/models.html
Syncing to DB:
https://datastax.github.io/python-driver/api/cassandra/cqlengine/management.html#cassandra.cqlengine.management.sync_table

If you're not using Python, I'm not sure what options you have, I'm not
much help there.

On Wed, Feb 10, 2016 at 12:05 PM Joe Bako <jb...@gracenote.com> wrote:

> Hi all,
>
> I am curious what techniques are used by others to manage schema changes
> over time in Cassandra.
>
> Modern RDBMS tools can compare schemas between DDL object definitions and
> live databases and generate change scripts accordingly.  Older techniques
> included maintaining a version and script table in the database, storing
> schema change scripts in a sequential fashion on disk, and iterating over
> them to apply them against the target database based on whether they had
> been run previously or not (indicated in the script table).
>
> The former allows you to easily see schema changes over time in your SCM
> history, the latter keeps a history of actual DDL applied against the
> database objects.  What options are available for C* and what are people
> using?
>
> Joe Bako
> Software Architect
> Gracenote, Inc.
> http://www.gracenote.com/
>
>

Re: Schema Versioning

[John Sanda <jo...@gmail.com>]

If you are interested in a solution that maintains scripts, there are at
least a few projects available,

https://github.com/comeara/pillar - Runs on the JVM and written in Scala.
Scripts are CQL files.
https://github.com/Contrast-Security-OSS/cassandra-migration - Runs on JVM
and I believe a port of Flyway
https://github.com/hsgubert/cassandra_migrations - Ruby based and similar
to ActiveRecord
https://github.com/jsanda/cassalog - A project I have started. Runs on JVM
and scripts are groovy files.

On Thu, Feb 11, 2016 at 4:57 AM, Carlos Alonso <in...@mrcalonso.com> wrote:

> Here we use the Cassanity gem: https://github.com/jnunemaker/cassanity
> This one suggests using schema migration files that are then registered in
> a column family to keep track of the version.
>
> Carlos Alonso | Software Engineer | @calonso <https://twitter.com/calonso>
>
> On 10 February 2016 at 21:29, Alex Popescu <al...@datastax.com> wrote:
>
>>
>> On Wed, Feb 10, 2016 at 12:05 PM, Joe Bako <jb...@gracenote.com> wrote:
>>
>>> Modern RDBMS tools can compare schemas between DDL object definitions
>>> and live databases and generate change scripts accordingly.  Older
>>> techniques included maintaining a version and script table in the database,
>>> storing schema change scripts in a sequential fashion on disk, and
>>> iterating over them to apply them against the target database based on
>>> whether they had been run previously or not (indicated in the script table).
>>
>>
>> Using DevCenter will give you some of these features (and future versions
>> will add more). Just to give you a quick example, if using DevCenter to
>> make schema changes it will offer the options of saving the final
>> definition or just the set of changes applied (to an existing CQL file or a
>> new one).
>>
>>
>> --
>> Bests,
>>
>> Alex Popescu | @al3xandru
>> Sen. Product Manager @ DataStax
>>
>>
>


-- 

- John

Re: Schema Versioning

[Carlos Alonso <in...@mrcalonso.com>]

Here we use the Cassanity gem: https://github.com/jnunemaker/cassanity
This one suggests using schema migration files that are then registered in
a column family to keep track of the version.

Carlos Alonso | Software Engineer | @calonso <https://twitter.com/calonso>

On 10 February 2016 at 21:29, Alex Popescu <al...@datastax.com> wrote:

>
> On Wed, Feb 10, 2016 at 12:05 PM, Joe Bako <jb...@gracenote.com> wrote:
>
>> Modern RDBMS tools can compare schemas between DDL object definitions and
>> live databases and generate change scripts accordingly.  Older techniques
>> included maintaining a version and script table in the database, storing
>> schema change scripts in a sequential fashion on disk, and iterating over
>> them to apply them against the target database based on whether they had
>> been run previously or not (indicated in the script table).
>
>
> Using DevCenter will give you some of these features (and future versions
> will add more). Just to give you a quick example, if using DevCenter to
> make schema changes it will offer the options of saving the final
> definition or just the set of changes applied (to an existing CQL file or a
> new one).
>
>
> --
> Bests,
>
> Alex Popescu | @al3xandru
> Sen. Product Manager @ DataStax
>
>

Re: Schema Versioning

[Alex Popescu <al...@datastax.com>]

On Wed, Feb 10, 2016 at 12:05 PM, Joe Bako <jb...@gracenote.com> wrote:

> Modern RDBMS tools can compare schemas between DDL object definitions and
> live databases and generate change scripts accordingly.  Older techniques
> included maintaining a version and script table in the database, storing
> schema change scripts in a sequential fashion on disk, and iterating over
> them to apply them against the target database based on whether they had
> been run previously or not (indicated in the script table).


Using DevCenter will give you some of these features (and future versions
will add more). Just to give you a quick example, if using DevCenter to
make schema changes it will offer the options of saving the final
definition or just the set of changes applied (to an existing CQL file or a
new one).


-- 
Bests,

Alex Popescu | @al3xandru
Sen. Product Manager @ DataStax