You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by 严超 <ya...@gmail.com> on 2014/12/04 09:26:50 UTC
[Import csv to Cassandra] Taking too much time
Hi, Everyone:
I'm importing a CSV file into Cassandra, and it always get
error:"Request did not complete within rpc_timeout ", then I have to
continue my COPY command of cql again. And the CSV file is 2.2 G . It is
taking a long time.
How can I speed up csv file importing ? Is there another way except
COPY command of cql ?
Thank you for any help .
*Best Regards！*
*Chao Yan--------------**My twitter：Andy Yan @yanchao727
<https://twitter.com/yanchao727>*
*My Weibo：http://weibo.com/herewearenow
<http://weibo.com/herewearenow>--------------*
Re: [Import csv to Cassandra] Taking too much time
Posted by Yuki Morishita <mo...@gmail.com>.
Here's blog post about writing SSTables from CSV and using
SSTableLoader to load them.
http://www.datastax.com/dev/blog/using-the-cassandra-bulk-loader-updated
On Thu, Dec 4, 2014 at 5:57 AM, 严超 <ya...@gmail.com> wrote:
> Thank you very much for your advice.
> Can you give me more advice for using SSTableLoader to import csv ?
> What is the best practice to use SStableLoader importing csv in Cassandra ?
>
> Best Regards！
> Chao Yan
> --------------
> My twitter：Andy Yan @yanchao727
> My Weibo：http://weibo.com/herewearenow
> --------------
>
> 2014-12-04 18:58 GMT+08:00 Akshay Ballarpure <ak...@tcs.com>:
>>
>> Hello Chao Yan,
>> CSV data import using Copy command in cassandra is always painful for
>> large size file (say > 1Gig).
>> CQL tool is not developed for performing such heavy operations instead try
>> using SSTableLoader to import.
>>
>>
>> Best Regards
>> Akshay
>>
>>
>>
>> From: 严超 <ya...@gmail.com>
>> To: user@cassandra.apache.org
>> Date: 12/04/2014 02:11 PM
>> Subject: [Import csv to Cassandra] Taking too much time
>> ________________________________
>>
>>
>>
>> Hi, Everyone:
>> I'm importing a CSV file into Cassandra, and it always get
>> error:"Request did not complete within rpc_timeout ", then I have to
>> continue my COPY command of cql again. And the CSV file is 2.2 G . It is
>> taking a long time.
>> How can I speed up csv file importing ? Is there another way
>> except COPY command of cql ?
>> Thank you for any help .
>>
>> Best Regards！
>> Chao Yan
>> --------------
>> My twitter：Andy Yan @yanchao727
>> My Weibo：http://weibo.com/herewearenow
>> --------------
>>
>> =====-----=====-----=====
>> Notice: The information contained in this e-mail
>> message and/or attachments to it may contain
>> confidential or privileged information. If you are
>> not the intended recipient, any dissemination, use,
>> review, distribution, printing or copying of the
>> information contained in this e-mail message
>> and/or attachments to it are strictly prohibited. If
>> you have received this communication in error,
>> please notify us by reply e-mail or telephone and
>> immediately and permanently delete the message
>> and any attachments. Thank you
>
>
--
Yuki Morishita
t:yukim (http://twitter.com/yukim)
Re: [Import csv to Cassandra] Taking too much time
Posted by 严超 <ya...@gmail.com>.
Thank you very much for your advice.
Can you give me more advice for using SSTableLoader to import csv ?
What is the best practice to use SStableLoader importing csv in Cassandra ?
*Best Regards！*
*Chao Yan--------------**My twitter：Andy Yan @yanchao727
<https://twitter.com/yanchao727>*
*My Weibo：http://weibo.com/herewearenow
<http://weibo.com/herewearenow>--------------*
2014-12-04 18:58 GMT+08:00 Akshay Ballarpure <ak...@tcs.com>:
> Hello Chao Yan,
> CSV data import using Copy command in cassandra is always painful for
> large size file (say > 1Gig).
> CQL tool is not developed for performing such heavy operations instead try
> using SSTableLoader to import.
>
>
> Best Regards
> Akshay
>
>
>
> From: 严超 <ya...@gmail.com>
> To: user@cassandra.apache.org
> Date: 12/04/2014 02:11 PM
> Subject: [Import csv to Cassandra] Taking too much time
> ------------------------------
>
>
>
> Hi, Everyone:
> I'm importing a CSV file into Cassandra, and it always get
> error:"Request did not complete within rpc_timeout ", then I have to
> continue my COPY command of cql again. And the CSV file is 2.2 G . It is
> taking a long time.
> How can I speed up csv file importing ? Is there another way
> except COPY command of cql ?
> Thank you for any help .
>
> *Best Regards！*
>
>
> *Chao Yan -------------- My twitter：Andy Yan **@yanchao727*
> <https://twitter.com/yanchao727>
> * My Weibo：**http://weibo.com/herewearenow*
> <http://weibo.com/herewearenow>
> * --------------*
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
Re: [Import csv to Cassandra] Taking too much time
Posted by Akshay Ballarpure <ak...@tcs.com>.
Hello Chao Yan,
CSV data import using Copy command in cassandra is always painful for
large size file (say > 1Gig).
CQL tool is not developed for performing such heavy operations instead try
using SSTableLoader to import.
Best Regards
Akshay
From: 严超 <ya...@gmail.com>
To: user@cassandra.apache.org
Date: 12/04/2014 02:11 PM
Subject: [Import csv to Cassandra] Taking too much time
Hi, Everyone:
I'm importing a CSV file into Cassandra, and it always get
error:"Request did not complete within rpc_timeout ", then I have to
continue my COPY command of cql again. And the CSV file is 2.2 G . It is
taking a long time.
How can I speed up csv file importing ? Is there another way
except COPY command of cql ?
Thank you for any help .
Best Regards！
Chao Yan
--------------
My twitter：Andy Yan @yanchao727
My Weibo：http://weibo.com/herewearenow
--------------
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you