You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ertio Lew <er...@gmail.com> on 2012/01/02 06:25:16 UTC
Composite column names: How much space do they occupy ?
I am storing composite column names which are made up of two integer
components. However I am shocked after seeing the storage overhead of these.
I just tried out a composite name (with single integer component):
Composite composite = new Composite();
composite.addComponent(-165376575,is);
System.out.println(CS.toByteBuffer( composite ).array().length); // the
result is 256
After writing & then reading back this composite column from cassandra:
System.out.println(CS.toByteBuffer( readColumn.getName() ).array().length);
// the result is 91
How much is the storage overhead, as I am quite sure that I'm making a
mistake in realizing the actual values ?
Re: Composite column names: How much space do they occupy ?
Posted by Ertio Lew <er...@gmail.com>.
Yes that makes a lot of sense! on using remaining() method I see the
proper expected sizes.
On Mon, Jan 2, 2012 at 5:26 PM, Sylvain Lebresne <sy...@datastax.com>wrote:
> I am not familiar enough with Hector to tell you if it is doing something
> special here, but note that:
>
> 1) you may have better luck getting that kind of question answered
> quickly by using the Hector mailing list.
>
> 2) that may or may not change what you're seeing (since again I don't
> know what Hector is actually doing), but "bb.array().length" is not a
> reliable way to get the effective length of a ByteBuffer, as it is
> perfectly
> legit to have a byte buffer only use parts of it's underlying array. You
> should use the remaining() method instead.
>
> --
> Sylvain
>
> On Mon, Jan 2, 2012 at 12:29 PM, Ertio Lew <er...@gmail.com> wrote:
> > Sorry I forgot to tell that I'm using Hector to communicate with
> Cassandra.
> > CS.toByteBuffer is to convert the composite type name to ByteBuffer.
> >
> > Can anyone aware of Hector API enlighten me why am I seeing this size for
> > the composite type names.
> >
> >
> > On Mon, Jan 2, 2012 at 2:52 PM, aaron morton <aa...@thelastpickle.com>
> > wrote:
> >>
> >> What is the definition of the composite type and what is CS.toByteBuffer
> >> ?
> >>
> >> CompositeTypes have a small overhead
> >> see
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/marshal/CompositeType.java
> >>
> >> Hope that helps.
> >> Aaron
> >>
> >> -----------------
> >> Aaron Morton
> >> Freelance Developer
> >> @aaronmorton
> >> http://www.thelastpickle.com
> >>
> >> On 2/01/2012, at 6:25 PM, Ertio Lew wrote:
> >>
> >> I am storing composite column names which are made up of two integer
> >> components. However I am shocked after seeing the storage overhead of
> these.
> >>
> >> I just tried out a composite name (with single integer component):
> >>
> >> Composite composite = new Composite();
> >> composite.addComponent(-165376575,is);
> >>
> >> System.out.println(CS.toByteBuffer( composite ).array().length); // the
> >> result is 256
> >>
> >>
> >> After writing & then reading back this composite column from cassandra:
> >>
> >>
> >>
> System.out.println(CS.toByteBuffer( readColumn.getName() ).array().length);
> >> // the result is 91
> >>
> >>
> >> How much is the storage overhead, as I am quite sure that I'm making a
> >> mistake in realizing the actual values ?
> >>
> >>
> >
>
Re: Composite column names: How much space do they occupy ?
Posted by Sylvain Lebresne <sy...@datastax.com>.
I am not familiar enough with Hector to tell you if it is doing something
special here, but note that:
1) you may have better luck getting that kind of question answered
quickly by using the Hector mailing list.
2) that may or may not change what you're seeing (since again I don't
know what Hector is actually doing), but "bb.array().length" is not a
reliable way to get the effective length of a ByteBuffer, as it is perfectly
legit to have a byte buffer only use parts of it's underlying array. You
should use the remaining() method instead.
--
Sylvain
On Mon, Jan 2, 2012 at 12:29 PM, Ertio Lew <er...@gmail.com> wrote:
> Sorry I forgot to tell that I'm using Hector to communicate with Cassandra.
> CS.toByteBuffer is to convert the composite type name to ByteBuffer.
>
> Can anyone aware of Hector API enlighten me why am I seeing this size for
> the composite type names.
>
>
> On Mon, Jan 2, 2012 at 2:52 PM, aaron morton <aa...@thelastpickle.com>
> wrote:
>>
>> What is the definition of the composite type and what is CS.toByteBuffer
>> ?
>>
>> CompositeTypes have a small overhead
>> see https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/marshal/CompositeType.java
>>
>> Hope that helps.
>> Aaron
>>
>> -----------------
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 2/01/2012, at 6:25 PM, Ertio Lew wrote:
>>
>> I am storing composite column names which are made up of two integer
>> components. However I am shocked after seeing the storage overhead of these.
>>
>> I just tried out a composite name (with single integer component):
>>
>> Composite composite = new Composite();
>> composite.addComponent(-165376575,is);
>>
>> System.out.println(CS.toByteBuffer( composite ).array().length); // the
>> result is 256
>>
>>
>> After writing & then reading back this composite column from cassandra:
>>
>>
>> System.out.println(CS.toByteBuffer( readColumn.getName() ).array().length);
>> // the result is 91
>>
>>
>> How much is the storage overhead, as I am quite sure that I'm making a
>> mistake in realizing the actual values ?
>>
>>
>
Re: Composite column names: How much space do they occupy ?
Posted by Ertio Lew <er...@gmail.com>.
Sorry I forgot to tell that I'm using Hector to communicate with
Cassandra. CS.toByteBuffer is to convert the composite type name to
ByteBuffer.
Can anyone aware of Hector API enlighten me why am I seeing this size for
the composite type names.
On Mon, Jan 2, 2012 at 2:52 PM, aaron morton <aa...@thelastpickle.com>wrote:
> What is the definition of the composite type and what is CS.toByteBuffer ?
>
> CompositeTypes have a small overhead see
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/marshal/CompositeType.java
>
> Hope that helps.
> Aaron
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 2/01/2012, at 6:25 PM, Ertio Lew wrote:
>
> I am storing composite column names which are made up of two integer
> components. However I am shocked after seeing the storage overhead of these.
>
> I just tried out a composite name (with single integer component):
>
> Composite composite = new Composite();
> composite.addComponent(-165376575,is);
>
> System.out.println(CS.toByteBuffer( composite ).array().length); // the
> result is 256
>
>
> After writing & then reading back this composite column from cassandra:
>
> System.out.println(CS.toByteBuffer( readColumn.getName() ).array().length);
> // the result is 91
>
>
> How much is the storage overhead, as I am quite sure that I'm making a
> mistake in realizing the actual values ?
>
>
>
Re: Composite column names: How much space do they occupy ?
Posted by aaron morton <aa...@thelastpickle.com>.
What is the definition of the composite type and what is CS.toByteBuffer ?
CompositeTypes have a small overhead see https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/marshal/CompositeType.java
Hope that helps.
Aaron
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 2/01/2012, at 6:25 PM, Ertio Lew wrote:
> I am storing composite column names which are made up of two integer components. However I am shocked after seeing the storage overhead of these.
>
> I just tried out a composite name (with single integer component):
>
> Composite composite = new Composite();
> composite.addComponent(-165376575,is);
>
> System.out.println(CS.toByteBuffer( composite ).array().length); // the result is 256
>
>
> After writing & then reading back this composite column from cassandra:
>
> System.out.println(CS.toByteBuffer( readColumn.getName() ).array().length); // the result is 91
>
>
> How much is the storage overhead, as I am quite sure that I'm making a mistake in realizing the actual values ?