Cassandra schema migrator

[Jens Rantil <je...@tink.se>]

Hi,


Anyone who is using, or could recommend, a tool for versioning schemas/migrating in Cassandra? My list of requirements is:
 * Support for adding tables.
 * Support for versioning of table properties. All our tables are to be defaulted to LeveledCompactionStrategy.
 * Support for adding non-existing columns.
 * Optional: Support for removing columns.
 * Optional: Support for removing tables.


We are preferably a Java shop, but could potentially integrate something non-Java. I understand I could write a tool that would make these decisions using system.schema_columnfamilies and system.schema_columns, but as always reusing a proven tool would be preferable.


So far I only know of Spring Data Cassandra that handles creating tables and adding columns. However, it does not handle table properties in any way.


Thanks,
Jens

———
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter

Re: Cassandra schema migrator

[Phil Wise <ph...@advancedtelematic.com>]

I've added these as answers to a question I posted on Stack Overflow:

http://stackoverflow.com/questions/26460932/how-to-deploy-changes-to-a-cassandra-cql-schema/27013426

Thank you

Phil

On 05.12.2014 15:23, Brian Sam-Bodden wrote:
> There is also https://github.com/hsgubert/cassandra_migrations
> 
> On Fri, Dec 5, 2014 at 7:49 AM, Ben Hood <0x...@gmail.com>
> wrote:
> 
>> On Tue, Nov 25, 2014 at 12:49 PM, Phil Wise
>> <ph...@advancedtelematic.com> wrote:
>>> https://github.com/advancedtelematic/cql-migrate
>> 
>> Great to see these tools out there!
>> 
>> Just to add to the list
>> 
>> https://github.com/mattes/migrate
>> 
>> Might not be as C* specific as the other tools mentioned earlier
>> in this thread, but it does integrate with Cassandra.
>> 
> 

Re: Cassandra schema migrator

[Brian Sam-Bodden <bs...@integrallis.com>]

There is also https://github.com/hsgubert/cassandra_migrations

On Fri, Dec 5, 2014 at 7:49 AM, Ben Hood <0x...@gmail.com> wrote:

> On Tue, Nov 25, 2014 at 12:49 PM, Phil Wise <ph...@advancedtelematic.com>
> wrote:
> > https://github.com/advancedtelematic/cql-migrate
>
> Great to see these tools out there!
>
> Just to add to the list
>
> https://github.com/mattes/migrate
>
> Might not be as C* specific as the other tools mentioned earlier in
> this thread, but it does integrate with Cassandra.
>

Re: Cassandra schema migrator

[Ben Hood <0x...@gmail.com>]

On Tue, Nov 25, 2014 at 12:49 PM, Phil Wise <ph...@advancedtelematic.com> wrote:
> https://github.com/advancedtelematic/cql-migrate

Great to see these tools out there!

Just to add to the list

https://github.com/mattes/migrate

Might not be as C* specific as the other tools mentioned earlier in
this thread, but it does integrate with Cassandra.

Re: Cassandra schema migrator

[Phil Wise <ph...@advancedtelematic.com>]

On 25.11.2014 10:22, Jens Rantil wrote:

> Anyone who is using, or could recommend, a tool for versioning 
> schemas/migrating in Cassandra?

I've recently written a tool to solve schema migration at our company
which may be useful:

https://github.com/advancedtelematic/cql-migrate

> My list of requirements is: * Support for adding tables.

Yep, this works.

> * Support for versioning of table properties. All our tables are
> to be defaulted to LeveledCompactionStrategy.

My parser/statement splitter doesn't currently recognise the WITH ...
syntax on CREATE tables and ALTER TABLE, but it could easily be added.

Do you need to modify the properties after the table has been created?
If so the changes would need to be expressed as ALTER TABLE statements.

> * Support for adding non-existing columns.

Supported. cql-migrate works by getting the user to express the
upgrade as an ALTER TABLE ... ADD <column> statement.

> * Optional: Support for removing columns. * Optional: Support for 
> removing tables.

Not supported, although it wouldn't be hard to add.

> We are preferably a Java shop, but could potentially integrate 
> something non-Java. I understand I could write a tool that would
> make these decisions using system.schema_columnfamilies and 
> system.schema_columns, but as always reusing a proven tool would
> be preferable.

It is implemented in python, and can be built into a Debian package
for deployment.

Rather than try and fully parse the CQL, cql-migrate works by
splitting a .CQL script into individual statements, executing them in
turn and ignoring errors that relate to tables etc already exiting.
Doing it that way means there is very little chance that the schema
created will differ from what running the same script through cqlsh
would have produced.

Let me know if you need a hand getting going with it.

Cheers,

Phil

Re: Cassandra schema migrator

[Jan Kesten <j....@enercast.de>]

Hi Jens,

maybe you should have a look at mutagen for cassandra:

https://github.com/toddfast/mutagen-cassandra

It is a litte quiet around this for some months, but maybe still worth it.

Cheers,
Jan

Am 25.11.2014 um 10:22 schrieb Jens Rantil:
> Hi,
>
> Anyone who is using, or could recommend, a tool for versioning 
> schemas/migrating in Cassandra? My list of requirements is:
>  * Support for adding tables.
>  * Support for versioning of table properties. All our tables are to 
> be defaulted to LeveledCompactionStrategy.
>  * Support for adding non-existing columns.
>  * Optional: Support for removing columns.
>  * Optional: Support for removing tables.
>
> We are preferably a Java shop, but could potentially integrate 
> something non-Java. I understand I could write a tool that would make 
> these decisions using system.schema_columnfamilies and 
> system.schema_columns, but as always reusing a proven tool would be 
> preferable.
>
> So far I only know of Spring Data Cassandra that handles creating 
> tables and adding columns. However, it does not handle table 
> properties in any way.
>
> Thanks,
> Jens
>
> ——— Jens Rantil Backend engineer Tink AB Email: jens.rantil@tink.se 
> Phone: +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter