You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Aditya Narayan <ad...@gmail.com> on 2011/02/10 11:56:35 UTC
Calculating the size of rows in KBs
How can I get or calculate the size of rows/ columns ? what are the
any overheads on memory for each column/row ?
Re: Calculating the size of rows in KBs
Posted by Aklin_81 <as...@gmail.com>.
I think it does not deserialize the entire list of columns in the
row(though it is the case with subcolumns in a supercolumn). In case
of standard columns, only the blocks on the disk containing the
columns values of the columns being asked for, are read and
deserailized to get the values.
On Sat, Feb 12, 2011 at 10:51 AM, Stu Hood <st...@gmail.com> wrote:
>> Does it also mean that the whole row will be deserialized when a query
>> comes
>> just for one column?
> No, it does not mean that: at most column_index_size_in_kb will be read to
> read a single column, independent of where that column is in the row.
> On the other hand, with the row cache enabled, it is as if everything in the
> row is "needed", so the entire row will be read. There is talk of improving
> this limitation of the row cache:
> see https://issues.apache.org/jira/browse/CASSANDRA-1956
>
> On Fri, Feb 11, 2011 at 6:00 PM, buddhasystem <po...@bnl.gov> wrote:
>>
>> Does it also mean that the whole row will be deserialized when a query
>> comes
>> just for one column?
>>
>> --
>> View this message in context:
>> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Calculating-the-size-of-rows-in-KBs-tp6011243p6017870.html
>> Sent from the cassandra-user@incubator.apache.org mailing list archive at
>> Nabble.com.
>
>
Re: Calculating the size of rows in KBs
Posted by Stu Hood <st...@gmail.com>.
> Does it also mean that the whole row will be deserialized when a query
comes
> just for one column?
No, it does not mean that: at most column_index_size_in_kb will be read to
read a single column, independent of where that column is in the row.
On the other hand, with the row cache enabled, it is as if everything in the
row is "needed", so the entire row will be read. There is talk of improving
this limitation of the row cache: see
https://issues.apache.org/jira/browse/CASSANDRA-1956
On Fri, Feb 11, 2011 at 6:00 PM, buddhasystem <po...@bnl.gov> wrote:
>
> Does it also mean that the whole row will be deserialized when a query
> comes
> just for one column?
>
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Calculating-the-size-of-rows-in-KBs-tp6011243p6017870.html
> Sent from the cassandra-user@incubator.apache.org mailing list archive at
> Nabble.com.
>
Re: Calculating the size of rows in KBs
Posted by buddhasystem <po...@bnl.gov>.
Does it also mean that the whole row will be deserialized when a query comes
just for one column?
--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Calculating-the-size-of-rows-in-KBs-tp6011243p6017870.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: Calculating the size of rows in KBs
Posted by Robert Coli <rc...@digg.com>.
On Thu, Feb 10, 2011 at 12:24 PM, Aaron Morton <aa...@thelastpickle.com> wrote:
> In general the entire row only exists in memory when it is contained in the first Memtable it's written to.
Or, somewhat importantly considering the serialization penalty paid to
load it there, when it is in the Row Cache.
As a simple example, reading a single column from a row will result in
the entire row being loaded into cache.
=Rob
Re: Calculating the size of rows in KBs
Posted by Aaron Morton <aa...@thelastpickle.com>.
If you are thinking about column_index_size_in_kb in Cassandra.yaml then yes.
Aaron
On 11/02/2011, at 9:39 AM, Aditya Narayan <ad...@gmail.com> wrote:
> Thank you Aaron!!
>
> But, If you are reading partial rows(that otherwise contain several
> thousands of **valueless** columns) then do the column indexes help in
> making the reads faster & more efficient than if they were not
> valueless?
> Perhaps, because they would only need to look up whether the asked
> column names exist in indexes for that row/key & they dont need to
> deserialize the blocks in SST tables searching for column values. Am I
> thinking right way??
>
>
> -Aditya
>
>
>
> On Fri, Feb 11, 2011 at 1:54 AM, Aaron Morton <aa...@thelastpickle.com> wrote:
>> If you want to get the byte size of a particular row you will need to read it all back.
>>
>> If you connect with JConsole at look at you column families, there are attributes for the max, min and mean row sizes.
>>
>> In general the entire row only exists in memory when it is contained in the first Memtable it's written to. It may then be partially or fully read from disk during subsequent reads or compactions.
>>
>> On disk format described here may help http://wiki.apache.org/cassandra/ArchitectureSSTable
>>
>> Hope that helps
>> Aaron
>> On 10/02/2011, at 11:56 PM, Aditya Narayan <ad...@gmail.com> wrote:
>>
>>> How can I get or calculate the size of rows/ columns ? what are the
>>> any overheads on memory for each column/row ?
>>
Re: Calculating the size of rows in KBs
Posted by Aditya Narayan <ad...@gmail.com>.
Thank you Aaron!!
But, If you are reading partial rows(that otherwise contain several
thousands of **valueless** columns) then do the column indexes help in
making the reads faster & more efficient than if they were not
valueless?
Perhaps, because they would only need to look up whether the asked
column names exist in indexes for that row/key & they dont need to
deserialize the blocks in SST tables searching for column values. Am I
thinking right way??
-Aditya
On Fri, Feb 11, 2011 at 1:54 AM, Aaron Morton <aa...@thelastpickle.com> wrote:
> If you want to get the byte size of a particular row you will need to read it all back.
>
> If you connect with JConsole at look at you column families, there are attributes for the max, min and mean row sizes.
>
> In general the entire row only exists in memory when it is contained in the first Memtable it's written to. It may then be partially or fully read from disk during subsequent reads or compactions.
>
> On disk format described here may help http://wiki.apache.org/cassandra/ArchitectureSSTable
>
> Hope that helps
> Aaron
> On 10/02/2011, at 11:56 PM, Aditya Narayan <ad...@gmail.com> wrote:
>
>> How can I get or calculate the size of rows/ columns ? what are the
>> any overheads on memory for each column/row ?
>
Re: Calculating the size of rows in KBs
Posted by Aaron Morton <aa...@thelastpickle.com>.
If you want to get the byte size of a particular row you will need to read it all back.
If you connect with JConsole at look at you column families, there are attributes for the max, min and mean row sizes.
In general the entire row only exists in memory when it is contained in the first Memtable it's written to. It may then be partially or fully read from disk during subsequent reads or compactions.
On disk format described here may help http://wiki.apache.org/cassandra/ArchitectureSSTable
Hope that helps
Aaron
On 10/02/2011, at 11:56 PM, Aditya Narayan <ad...@gmail.com> wrote:
> How can I get or calculate the size of rows/ columns ? what are the
> any overheads on memory for each column/row ?