Inserts with a dynamic datamodel using Datastax java driver

["Raveendran, Varsha IN BLR STS" <va...@siemens.com>]

Hello,

I am building a write client in java to insert records into  Cassandra 2.0.5.  I am using the Datastax java driver.

Problem :  The datamodel is dynamic. By dynamic, I mean that the number of columns and the datatype of columns will be given as an input by the user.  It has only 1 keyspace and 1 column family.

For inserting records bound statements seems the way to go.  But the bind() function accepts only a sequence of Objects  ( column values) .
How do I bind the values when the number and datatype of columns is given as input? Any suggestions?

 Thanks & Regards,
Varsha

RE: Inserts with a dynamic datamodel using Datastax java driver

["Raveendran, Varsha IN BLR STS" <va...@siemens.com>]

Hi,

Thanks for replying.

I dint quite get what you meant by "use clustering columns in CQL3 with blob/text type".

I have elaborated my problem statement below.
Assume the schema of the keyspace to which random records need to be inserted is given in the following format :
KeySpace Name :   KS_1
ColumnFamilyName : CF_1
Columns: [Column1 : uuid , Column2: varint, Column3: timestamp, .... ... ColumnN:text]


So I parse this file to get the schema.  Also, the data/value for each column should be generated randomly depending on the datatype of the column.
My question is how do I insert the records ?


1.      I created a prepared statement depending on the number of columns (using a for loop).  Then for each record I called methods like setDate() or setVarint()  to bind the values.

But this was taking too much time because data was being generated for each column , then set in the prepared statement  and then inserted.  And the number of records = 1 billion!!



2.      The executeAsync () function seemed more likely to be faster. But the problem is that the bind() function takes a sequence of values.  Since the number of columns is variable I am not able to make this code generic (i.e to cater to any schema given by the user) .



I am not sure if there is another way to approach this problem.


Thanks & Regards,
Varsha


From: DuyHai Doan [mailto:doanduyhai@gmail.com]
Sent: Wednesday, April 02, 2014 4:05 PM
To: user@cassandra.apache.org
Subject: Re: Inserts with a dynamic datamodel using Datastax java driver

Hello Varsha

 Your best bet is to go with blob type by serializing all data into bytes. Another alternative is to use text and serialize to JSON.

 For the dynamic columns, use clustering columns in CQL3 with blob/text type

 Regards

 Duy Hai DOAN

On Wed, Apr 2, 2014 at 11:21 AM, Raveendran, Varsha IN BLR STS <va...@siemens.com>> wrote:
Hello,

I am building a write client in java to insert records into  Cassandra 2.0.5.  I am using the Datastax java driver.

Problem : The datamodel is dynamic. By dynamic, I mean that the number of columns and the datatype of columns will be given as an input by the user.  It has only 1 keyspace and 1 column family.

For inserting records bound statements seems the way to go.  But the bind() function accepts only a sequence of Objects  ( column values) .
How do I bind the values when the number and datatype of columns is given as input? Any suggestions?

Thanks & Regards,
Varsha

Re: Inserts with a dynamic datamodel using Datastax java driver

[DuyHai Doan <do...@gmail.com>]

Hello Varsha

 Your best bet is to go with blob type by serializing all data into bytes.
Another alternative is to use text and serialize to JSON.

 For the dynamic columns, use clustering columns in CQL3 with blob/text type

 Regards

 Duy Hai DOAN


On Wed, Apr 2, 2014 at 11:21 AM, Raveendran, Varsha IN BLR STS <
varsha.raveendran@siemens.com> wrote:

>  Hello,
>
> I am building a write client in java to insert records into  Cassandra
> 2.0.5.  I am using the Datastax java driver.
>
> *Problem** : * The datamodel is dynamic. By dynamic, I mean that the
> number of columns and the datatype of columns will be given as an input by
> the user.  It has only 1 keyspace and 1 column family.
>
> For inserting records bound statements seems the way to go.  But the
> bind() function accepts only a sequence of Objects  ( column values) .
> How do I bind the values when the number and datatype of columns is given
> as input? Any suggestions?
>
>  Thanks & Regards,
> Varsha
>
>
>