dynamic tables in cassandra sink

[Meghashyam Sandeep V <vr...@gmail.com>]

Hi there,

I have a flink stream from kafka writing to Cassandra. We use monthly
tables in Cassandra to avoid TTL and tombstones that come with it. Tables
would be like table_05_2018, table_06_2018 and so on. How do I dynamically
register this table name in the following snippet?

CassandraSink
.addSink(dataStream)
.setQuery("INSERT INTO table_05_2018(id, text) VALUES (?,?);")
.setHost("127.0.0.1")
.build();

Re: dynamic tables in cassandra sink

[Michael Latta <ml...@technomage.com>]

If you restart the job each month you can build the string dynamically. If you want data to flow into the log based on a date in the record you will need to do something fancier. I have not used the casandra connector so I can’t help on the details. Can you subclass the connector and build the query dynamically for each data item? Or build a custom sink that just writes to casandra?

Michael

Sent from my iPad

> On May 3, 2018, at 5:31 PM, Meghashyam Sandeep V <vr...@gmail.com> wrote:
> 
> Hi there,
> 
> I have a flink stream from kafka writing to Cassandra. We use monthly tables in Cassandra to avoid TTL and tombstones that come with it. Tables would be like table_05_2018, table_06_2018 and so on. How do I dynamically register this table name in the following snippet? 
> 
>   CassandraSink
> .addSink(dataStream)
> .setQuery("INSERT INTO table_05_2018(id, text) VALUES (?,?);")
> .setHost("127.0.0.1")
> .build();
> 
> 
> 
> 
>