You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by varun allampalli <vs...@gmail.com> on 2013/12/11 23:27:18 UTC
Bulkoutputformat
Hi All,
I want to bulk insert data into cassandra. I was wondering of using
BulkOutputformat in hadoop. Is it the best way or using driver and doing
batch insert is the better way.
Are there any disandvantages of using bulkoutputformat.
Thanks for helping
Varun
Re: Bulkoutputformat
Posted by Aaron Morton <aa...@thelastpickle.com>.
> Request did not complete within rpc_timeout.
The node is overloaded and did not return in time.
Check the logs for errors or excessive JVM GC and try selecting less data.
Cheers
-----------------
Aaron Morton
New Zealand
@aaronmorton
Co-Founder & Principal Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
On 14/12/2013, at 10:06 am, varun allampalli <vs...@gmail.com> wrote:
> Thanks Rahul..article was insightful
>
>
> On Fri, Dec 13, 2013 at 12:25 AM, Rahul Menon <ra...@apigee.com> wrote:
> Here you go
>
> http://thelastpickle.com/blog/2013/01/11/primary-keys-in-cql.html
>
>
> On Fri, Dec 13, 2013 at 7:19 AM, varun allampalli <vs...@gmail.com> wrote:
> Hi Aaron,
>
> It seems like you answered the question here.
>
> https://groups.google.com/forum/#!topic/nosql-databases/vjZA5vdycWA
>
> Can you give me the link to the blog which you mentioned
>
> http://thelastpickle.com/2013/01/11/primary-keys-in-cql/
>
> Thanks in advance
> Varun
>
>
> On Thu, Dec 12, 2013 at 3:36 PM, varun allampalli <vs...@gmail.com> wrote:
> Thanks Aaron, I was able to generate sstables and load using sstableloader. But after loading the tables when I do a select query I get this, the table has only one record. Is there anything I am missing or any logs I can look at.
>
> Request did not complete within rpc_timeout.
>
>
> On Wed, Dec 11, 2013 at 7:58 PM, Aaron Morton <aa...@thelastpickle.com> wrote:
> If you don’t need to use Hadoop then try the SSTableSimpleWriter and sstableloader , this post is a little old but still relevant http://www.datastax.com/dev/blog/bulk-loading
>
> Otherwise AFAIK BulkOutputFormat is what you want from hadoop http://www.datastax.com/docs/1.1/cluster_architecture/hadoop_integration
>
> Cheers
>
> -----------------
> Aaron Morton
> New Zealand
> @aaronmorton
>
> Co-Founder & Principal Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> On 12/12/2013, at 11:27 am, varun allampalli <vs...@gmail.com> wrote:
>
>> Hi All,
>>
>> I want to bulk insert data into cassandra. I was wondering of using BulkOutputformat in hadoop. Is it the best way or using driver and doing batch insert is the better way.
>>
>> Are there any disandvantages of using bulkoutputformat.
>>
>> Thanks for helping
>>
>> Varun
>
>
>
>
>
Re: Bulkoutputformat
Posted by varun allampalli <vs...@gmail.com>.
Thanks Rahul..article was insightful
On Fri, Dec 13, 2013 at 12:25 AM, Rahul Menon <ra...@apigee.com> wrote:
> Here you go
>
> http://thelastpickle.com/blog/2013/01/11/primary-keys-in-cql.html
>
>
> On Fri, Dec 13, 2013 at 7:19 AM, varun allampalli <
> vshoori.office@gmail.com> wrote:
>
>> Hi Aaron,
>>
>> It seems like you answered the question here.
>>
>> https://groups.google.com/forum/#!topic/nosql-databases/vjZA5vdycWA
>>
>> Can you give me the link to the blog which you mentioned
>>
>> http://thelastpickle.com/2013/01/11/primary-keys-in-cql/
>>
>> Thanks in advance
>> Varun
>>
>>
>> On Thu, Dec 12, 2013 at 3:36 PM, varun allampalli <
>> vshoori.office@gmail.com> wrote:
>>
>>> Thanks Aaron, I was able to generate sstables and load using
>>> sstableloader. But after loading the tables when I do a select query I get
>>> this, the table has only one record. Is there anything I am missing or any
>>> logs I can look at.
>>>
>>> Request did not complete within rpc_timeout.
>>>
>>>
>>> On Wed, Dec 11, 2013 at 7:58 PM, Aaron Morton <aa...@thelastpickle.com>wrote:
>>>
>>>> If you don’t need to use Hadoop then try the SSTableSimpleWriter and
>>>> sstableloader , this post is a little old but still relevant
>>>> http://www.datastax.com/dev/blog/bulk-loading
>>>>
>>>> Otherwise AFAIK BulkOutputFormat is what you want from hadoop
>>>> http://www.datastax.com/docs/1.1/cluster_architecture/hadoop_integration
>>>>
>>>> Cheers
>>>>
>>>> -----------------
>>>> Aaron Morton
>>>> New Zealand
>>>> @aaronmorton
>>>>
>>>> Co-Founder & Principal Consultant
>>>> Apache Cassandra Consulting
>>>> http://www.thelastpickle.com
>>>>
>>>> On 12/12/2013, at 11:27 am, varun allampalli <vs...@gmail.com>
>>>> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> I want to bulk insert data into cassandra. I was wondering of using
>>>> BulkOutputformat in hadoop. Is it the best way or using driver and doing
>>>> batch insert is the better way.
>>>>
>>>> Are there any disandvantages of using bulkoutputformat.
>>>>
>>>> Thanks for helping
>>>>
>>>> Varun
>>>>
>>>>
>>>>
>>>
>>
>
Re: Bulkoutputformat
Posted by Rahul Menon <ra...@apigee.com>.
Here you go
http://thelastpickle.com/blog/2013/01/11/primary-keys-in-cql.html
On Fri, Dec 13, 2013 at 7:19 AM, varun allampalli
<vs...@gmail.com>wrote:
> Hi Aaron,
>
> It seems like you answered the question here.
>
> https://groups.google.com/forum/#!topic/nosql-databases/vjZA5vdycWA
>
> Can you give me the link to the blog which you mentioned
>
> http://thelastpickle.com/2013/01/11/primary-keys-in-cql/
>
> Thanks in advance
> Varun
>
>
> On Thu, Dec 12, 2013 at 3:36 PM, varun allampalli <
> vshoori.office@gmail.com> wrote:
>
>> Thanks Aaron, I was able to generate sstables and load using
>> sstableloader. But after loading the tables when I do a select query I get
>> this, the table has only one record. Is there anything I am missing or any
>> logs I can look at.
>>
>> Request did not complete within rpc_timeout.
>>
>>
>> On Wed, Dec 11, 2013 at 7:58 PM, Aaron Morton <aa...@thelastpickle.com>wrote:
>>
>>> If you don’t need to use Hadoop then try the SSTableSimpleWriter and
>>> sstableloader , this post is a little old but still relevant
>>> http://www.datastax.com/dev/blog/bulk-loading
>>>
>>> Otherwise AFAIK BulkOutputFormat is what you want from hadoop
>>> http://www.datastax.com/docs/1.1/cluster_architecture/hadoop_integration
>>>
>>> Cheers
>>>
>>> -----------------
>>> Aaron Morton
>>> New Zealand
>>> @aaronmorton
>>>
>>> Co-Founder & Principal Consultant
>>> Apache Cassandra Consulting
>>> http://www.thelastpickle.com
>>>
>>> On 12/12/2013, at 11:27 am, varun allampalli <vs...@gmail.com>
>>> wrote:
>>>
>>> Hi All,
>>>
>>> I want to bulk insert data into cassandra. I was wondering of using
>>> BulkOutputformat in hadoop. Is it the best way or using driver and doing
>>> batch insert is the better way.
>>>
>>> Are there any disandvantages of using bulkoutputformat.
>>>
>>> Thanks for helping
>>>
>>> Varun
>>>
>>>
>>>
>>
>
Re: Bulkoutputformat
Posted by varun allampalli <vs...@gmail.com>.
Hi Aaron,
It seems like you answered the question here.
https://groups.google.com/forum/#!topic/nosql-databases/vjZA5vdycWA
Can you give me the link to the blog which you mentioned
http://thelastpickle.com/2013/01/11/primary-keys-in-cql/
Thanks in advance
Varun
On Thu, Dec 12, 2013 at 3:36 PM, varun allampalli
<vs...@gmail.com>wrote:
> Thanks Aaron, I was able to generate sstables and load using
> sstableloader. But after loading the tables when I do a select query I get
> this, the table has only one record. Is there anything I am missing or any
> logs I can look at.
>
> Request did not complete within rpc_timeout.
>
>
> On Wed, Dec 11, 2013 at 7:58 PM, Aaron Morton <aa...@thelastpickle.com>wrote:
>
>> If you don’t need to use Hadoop then try the SSTableSimpleWriter and
>> sstableloader , this post is a little old but still relevant
>> http://www.datastax.com/dev/blog/bulk-loading
>>
>> Otherwise AFAIK BulkOutputFormat is what you want from hadoop
>> http://www.datastax.com/docs/1.1/cluster_architecture/hadoop_integration
>>
>> Cheers
>>
>> -----------------
>> Aaron Morton
>> New Zealand
>> @aaronmorton
>>
>> Co-Founder & Principal Consultant
>> Apache Cassandra Consulting
>> http://www.thelastpickle.com
>>
>> On 12/12/2013, at 11:27 am, varun allampalli <vs...@gmail.com>
>> wrote:
>>
>> Hi All,
>>
>> I want to bulk insert data into cassandra. I was wondering of using
>> BulkOutputformat in hadoop. Is it the best way or using driver and doing
>> batch insert is the better way.
>>
>> Are there any disandvantages of using bulkoutputformat.
>>
>> Thanks for helping
>>
>> Varun
>>
>>
>>
>
Re: Bulkoutputformat
Posted by varun allampalli <vs...@gmail.com>.
Thanks Aaron, I was able to generate sstables and load using sstableloader.
But after loading the tables when I do a select query I get this, the table
has only one record. Is there anything I am missing or any logs I can look
at.
Request did not complete within rpc_timeout.
On Wed, Dec 11, 2013 at 7:58 PM, Aaron Morton <aa...@thelastpickle.com>wrote:
> If you don’t need to use Hadoop then try the SSTableSimpleWriter and
> sstableloader , this post is a little old but still relevant
> http://www.datastax.com/dev/blog/bulk-loading
>
> Otherwise AFAIK BulkOutputFormat is what you want from hadoop
> http://www.datastax.com/docs/1.1/cluster_architecture/hadoop_integration
>
> Cheers
>
> -----------------
> Aaron Morton
> New Zealand
> @aaronmorton
>
> Co-Founder & Principal Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> On 12/12/2013, at 11:27 am, varun allampalli <vs...@gmail.com>
> wrote:
>
> Hi All,
>
> I want to bulk insert data into cassandra. I was wondering of using
> BulkOutputformat in hadoop. Is it the best way or using driver and doing
> batch insert is the better way.
>
> Are there any disandvantages of using bulkoutputformat.
>
> Thanks for helping
>
> Varun
>
>
>
Re: Bulkoutputformat
Posted by Aaron Morton <aa...@thelastpickle.com>.
If you don’t need to use Hadoop then try the SSTableSimpleWriter and sstableloader , this post is a little old but still relevant http://www.datastax.com/dev/blog/bulk-loading
Otherwise AFAIK BulkOutputFormat is what you want from hadoop http://www.datastax.com/docs/1.1/cluster_architecture/hadoop_integration
Cheers
-----------------
Aaron Morton
New Zealand
@aaronmorton
Co-Founder & Principal Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
On 12/12/2013, at 11:27 am, varun allampalli <vs...@gmail.com> wrote:
> Hi All,
>
> I want to bulk insert data into cassandra. I was wondering of using BulkOutputformat in hadoop. Is it the best way or using driver and doing batch insert is the better way.
>
> Are there any disandvantages of using bulkoutputformat.
>
> Thanks for helping
>
> Varun