Moving data

["Morey, Gary" <Ga...@xerox.com>]

I have several large SQL Server 2005 tables.  I need to load the data in
these tables into Cassandra.  FYI, the Cassandra installation is on a
linux server running CentOS.

 

Can anyone suggest the best way to accomplish this?  I am a newbie to
Cassandra, so any advice would be greatly appreciated.

 

Best,

 

Gary

Re: Moving data

[buddhasystem <po...@bnl.gov>]

FWIW, I'm working on migrating a large amount of data out of Oracle into my
test cluster. The data has been warehoused as CSV files on Amazon S3. Having
that in place allows me to not put extra load on the production service when
doing many repeated tests. I then parse the data using CSV Python module
and, as Jonathan says, use threads to batch upload data into Cassandra.
Notable points: since the data is relatively sparse (i.e. many zeros for
integers and empty strings for strings etc), I establish a default value
dictionary, and don't write these to Cassandra at all -- they can be
reconstructed as needed when reading back.

Also, make sure you wrap Cassandra writes etc into exceptions. When load is
high, you might get timeouts at TSocket level etc.

-- 
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Moving-data-tp5992669p5993443.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.

Re: Moving data

[Jonathan Ellis <jb...@gmail.com>]

I'm afraid there is no short answer.

The long answer is,

1) Read about Cassandra data modeling at
http://wiki.apache.org/cassandra/ArticlesAndPresentations.  It is not
as simple as "one table equals one columnfamily."
2) Write a program to read your data out of SQL Server and write it
into Cassandra, preferably with multiple threads

On Fri, Feb 4, 2011 at 6:00 AM, Morey, Gary <Ga...@xerox.com> wrote:
> I have several large SQL Server 2005 tables.  I need to load the data in
> these tables into Cassandra.  FYI, the Cassandra installation is on a linux
> server running CentOS.
>
>
>
> Can anyone suggest the best way to accomplish this?  I am a newbie to
> Cassandra, so any advice would be greatly appreciated.
>
>
>
> Best,
>
>
>
> Gary



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com