You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Umang Shah <sh...@gmail.com> on 2014/10/16 07:46:02 UTC
What will be system configuration for retrieving few "GB" of data
Hi,
I am facing many problem after storing certain limit of records in
cassandra, and giving outofmemoryerror.
I have 8GB of RAM in my system, so how much records i can expect to
retrieve by using select query?
and what will be the configuration for those people who are retrieving
15-20 GB of data?
Can somebody explain me how to improve read performance then it will be
great help, i tried
http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_tune_jvm_c.html
such things but no help.
--
Regards,
Umang Shah
shahumang4@gmail.com
Re: What will be system configuration for retrieving few "GB" of data
Posted by Umang Shah <sh...@gmail.com>.
Thanks Mohammed.. This the Answer which i am looking for.
Regards,
Umang Shah
On Sat, Oct 18, 2014 at 5:11 AM, Mohammed Guller <mo...@glassbeam.com>
wrote:
> With 8GB RAM, the default heap size is 2GB, so you will quickly start
> running out of heap space if you do large reads. What is a large read? It
> depends on the number of columns in each row and data in each column. It
> could 100,000 rows for some and 300,000 for others. In addition, remember
> that Java adds a lot of overhead to data in memory, so a 8 character string
> will not occupy just 8 bytes in memory, but a lot more.
>
>
>
> In general, avoid large reads in C*. If it is absolutely must and you
> cannot repartition the data, then use a driver that supports paging.
>
>
>
> Mohammed
>
>
>
> *From:* Umang Shah [mailto:shahumang4@gmail.com]
> *Sent:* Wednesday, October 15, 2014 10:46 PM
> *To:* user@cassandra.apache.org
> *Subject:* What will be system configuration for retrieving few "GB" of
> data
>
>
>
> Hi,
>
>
>
> I am facing many problem after storing certain limit of records in
> cassandra, and giving outofmemoryerror.
>
>
>
> I have 8GB of RAM in my system, so how much records i can expect to
> retrieve by using select query?
>
>
>
> and what will be the configuration for those people who are retrieving
> 15-20 GB of data?
>
>
>
> Can somebody explain me how to improve read performance then it will be
> great help, i tried
>
>
> http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_tune_jvm_c.html
>
>
>
> such things but no help.
>
>
>
>
>
> --
>
> Regards,
>
> Umang Shah
>
> shahumang4@gmail.com
>
--
Regards,
Umang V.Shah
+919886829019
RE: What will be system configuration for retrieving few "GB" of
data
Posted by Mohammed Guller <mo...@glassbeam.com>.
With 8GB RAM, the default heap size is 2GB, so you will quickly start running out of heap space if you do large reads. What is a large read? It depends on the number of columns in each row and data in each column. It could 100,000 rows for some and 300,000 for others. In addition, remember that Java adds a lot of overhead to data in memory, so a 8 character string will not occupy just 8 bytes in memory, but a lot more.
In general, avoid large reads in C*. If it is absolutely must and you cannot repartition the data, then use a driver that supports paging.
Mohammed
From: Umang Shah [mailto:shahumang4@gmail.com]
Sent: Wednesday, October 15, 2014 10:46 PM
To: user@cassandra.apache.org
Subject: What will be system configuration for retrieving few "GB" of data
Hi,
I am facing many problem after storing certain limit of records in cassandra, and giving outofmemoryerror.
I have 8GB of RAM in my system, so how much records i can expect to retrieve by using select query?
and what will be the configuration for those people who are retrieving 15-20 GB of data?
Can somebody explain me how to improve read performance then it will be great help, i tried
http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_tune_jvm_c.html
such things but no help.
--
Regards,
Umang Shah
shahumang4@gmail.com<ma...@gmail.com>
Re: What will be system configuration for retrieving few "GB" of data
Posted by Robert Coli <rc...@eventbrite.com>.
On Wed, Oct 15, 2014 at 10:46 PM, Umang Shah <sh...@gmail.com> wrote:
> I am facing many problem after storing certain limit of records in
> cassandra, and giving outofmemoryerror.
>
This description is too vague. Be more specific.
=Rob
http://twitter.com/rcolidba