You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jonathan Colby <jo...@gmail.com> on 2011/06/23 09:58:04 UTC
Decorator Algorithm
Hi -
I'd like to understand more how the token is hashed with the key to determine on which node the data is stored - called decorating in cassandra speak.
Can anyone share any documentation on this or describe this more in detail? Yes, I could look at the code, but I was hoping to be able to read more about how it works first.
thanks.
Re: Decorator Algorithm
Posted by Sylvain Lebresne <sy...@datastax.com>.
On Mon, Jun 27, 2011 at 12:02 PM, Philippe <wa...@gmail.com> wrote:
> a quick followup on this : when using Byte ordered partitioner. how does a
> short key get mapped to the 128bit token ? what about keys longer than 128 ?
> does Cassandra just pad and truncate ?
Token are not limited to 128 bits. With the ordered partitioner, the
token is the key and has thus the same limitation than row key (i.e,
are limited to be 65535 bytes long).
--
Sylvain
>
> thanks
>
> Le 24 juin 2011 04:53, "Maki Watanabe" <wa...@gmail.com> a écrit :
>> A little addendum
>>
>> Key := Your data to identify a row
>> Token := Index on the ring calculated from Key. The calculation is
>> defined in replication strategy.
>>
>> You can lookup responsible nodes (endpoints) for a specific key with
>> JMX getNaturalEndpoints interface.
>>
>> maki
>>
>>
>> 2011/6/24 aaron morton <aa...@thelastpickle.com>:
>>> Various places in the code call IPartitioner.decorateKey() which returns
>>> a DecoratedKey<T> which contains both the original key and the Token<T>
>>>
>>> The RandomPartitioner md5 to hash the key ByteBuffer and create a
>>> BigInteger. OPP converts the key into utf8 encoded String.
>>>
>>> Using the token to find which endpoints contain replicas is done by the
>>> AbstractReplicationStrategy.calculateNaturalEndpoints() implementations.
>>>
>>> Does that help?
>>>
>>> -----------------
>>> Aaron Morton
>>> Freelance Cassandra Developer
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>>
>>> On 23 Jun 2011, at 19:58, Jonathan Colby wrote:
>>>
>>>> Hi -
>>>>
>>>> I'd like to understand more how the token is hashed with the key to
>>>> determine on which node the data is stored - called decorating in cassandra
>>>> speak.
>>>>
>>>> Can anyone share any documentation on this or describe this more in
>>>> detail? Yes, I could look at the code, but I was hoping to be able to read
>>>> more about how it works first.
>>>>
>>>> thanks.
>>>
>>>
>>
>>
>>
>> --
>> w3m
>
Re: Decorator Algorithm
Posted by Philippe <wa...@gmail.com>.
a quick followup on this : when using Byte ordered partitioner. how does a
short key get mapped to the 128bit token ? what about keys longer than 128 ?
does Cassandra just pad and truncate ?
thanks
Le 24 juin 2011 04:53, "Maki Watanabe" <wa...@gmail.com> a écrit :
> A little addendum
>
> Key := Your data to identify a row
> Token := Index on the ring calculated from Key. The calculation is
> defined in replication strategy.
>
> You can lookup responsible nodes (endpoints) for a specific key with
> JMX getNaturalEndpoints interface.
>
> maki
>
>
> 2011/6/24 aaron morton <aa...@thelastpickle.com>:
>> Various places in the code call IPartitioner.decorateKey() which returns
a DecoratedKey<T> which contains both the original key and the Token<T>
>>
>> The RandomPartitioner md5 to hash the key ByteBuffer and create a
BigInteger. OPP converts the key into utf8 encoded String.
>>
>> Using the token to find which endpoints contain replicas is done by the
AbstractReplicationStrategy.calculateNaturalEndpoints() implementations.
>>
>> Does that help?
>>
>> -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 23 Jun 2011, at 19:58, Jonathan Colby wrote:
>>
>>> Hi -
>>>
>>> I'd like to understand more how the token is hashed with the key to
determine on which node the data is stored - called decorating in cassandra
speak.
>>>
>>> Can anyone share any documentation on this or describe this more in
detail? Yes, I could look at the code, but I was hoping to be able to read
more about how it works first.
>>>
>>> thanks.
>>
>>
>
>
>
> --
> w3m
Re: Decorator Algorithm
Posted by Jonathan Colby <jo...@gmail.com>.
thanks guys. That clears things up.
On Jun 24, 2011, at 4:53 AM, Maki Watanabe wrote:
> A little addendum
>
> Key := Your data to identify a row
> Token := Index on the ring calculated from Key. The calculation is
> defined in replication strategy.
>
> You can lookup responsible nodes (endpoints) for a specific key with
> JMX getNaturalEndpoints interface.
>
> maki
>
>
> 2011/6/24 aaron morton <aa...@thelastpickle.com>:
>> Various places in the code call IPartitioner.decorateKey() which returns a DecoratedKey<T> which contains both the original key and the Token<T>
>>
>> The RandomPartitioner md5 to hash the key ByteBuffer and create a BigInteger. OPP converts the key into utf8 encoded String.
>>
>> Using the token to find which endpoints contain replicas is done by the AbstractReplicationStrategy.calculateNaturalEndpoints() implementations.
>>
>> Does that help?
>>
>> -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 23 Jun 2011, at 19:58, Jonathan Colby wrote:
>>
>>> Hi -
>>>
>>> I'd like to understand more how the token is hashed with the key to determine on which node the data is stored - called decorating in cassandra speak.
>>>
>>> Can anyone share any documentation on this or describe this more in detail? Yes, I could look at the code, but I was hoping to be able to read more about how it works first.
>>>
>>> thanks.
>>
>>
>
>
>
> --
> w3m
Re: Decorator Algorithm
Posted by Maki Watanabe <wa...@gmail.com>.
A little addendum
Key := Your data to identify a row
Token := Index on the ring calculated from Key. The calculation is
defined in replication strategy.
You can lookup responsible nodes (endpoints) for a specific key with
JMX getNaturalEndpoints interface.
maki
2011/6/24 aaron morton <aa...@thelastpickle.com>:
> Various places in the code call IPartitioner.decorateKey() which returns a DecoratedKey<T> which contains both the original key and the Token<T>
>
> The RandomPartitioner md5 to hash the key ByteBuffer and create a BigInteger. OPP converts the key into utf8 encoded String.
>
> Using the token to find which endpoints contain replicas is done by the AbstractReplicationStrategy.calculateNaturalEndpoints() implementations.
>
> Does that help?
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 23 Jun 2011, at 19:58, Jonathan Colby wrote:
>
>> Hi -
>>
>> I'd like to understand more how the token is hashed with the key to determine on which node the data is stored - called decorating in cassandra speak.
>>
>> Can anyone share any documentation on this or describe this more in detail? Yes, I could look at the code, but I was hoping to be able to read more about how it works first.
>>
>> thanks.
>
>
--
w3m
Re: Decorator Algorithm
Posted by aaron morton <aa...@thelastpickle.com>.
Various places in the code call IPartitioner.decorateKey() which returns a DecoratedKey<T> which contains both the original key and the Token<T>
The RandomPartitioner md5 to hash the key ByteBuffer and create a BigInteger. OPP converts the key into utf8 encoded String.
Using the token to find which endpoints contain replicas is done by the AbstractReplicationStrategy.calculateNaturalEndpoints() implementations.
Does that help?
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 23 Jun 2011, at 19:58, Jonathan Colby wrote:
> Hi -
>
> I'd like to understand more how the token is hashed with the key to determine on which node the data is stored - called decorating in cassandra speak.
>
> Can anyone share any documentation on this or describe this more in detail? Yes, I could look at the code, but I was hoping to be able to read more about how it works first.
>
> thanks.