You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Piavlo <pi...@cs.bgu.ac.il> on 2011/11/24 10:55:22 UTC
need help with choosing correct tokens for ByteOrderedPartitioner
Hi,
We need help with choosing correct tokens for ByteOrderedPartitioner
Originally the key where supposed to be member_id-yyyymmdd
but since we need to male rage scans on same member_id and varying date
ranges yyyymmdd
we decided to use ByteOrderedPartitioner, so we need that same member
will be assigned to same token range.
So we decided that the keys will be md5(member_id)yyyymmdd
Since md5 on member_id should give even distribution or member_id across
tokens.
We have 4 nodes, and don't understand how to choose the tokens.
We tried the following tokens
# ./tokengentool 4
token 0: 0
token 1: 42535295865117307932921825928971026432
token 2: 85070591730234615865843651857942052864
token 3: 127605887595351923798765477786913079296
and appended 29991231
so we ended up with the following tokens
token 0: 0
token 1: 4253529586511730793292182592897102643229991231
token 2: 8507059173023461586584365185794205286429991231
token 3: 12760588759535192379876547778691307929629991231
But the key end up not evenly distributed.
So any help is appreciated.
Thanks
Alex
Re: need help with choosing correct tokens for ByteOrderedPartitioner
Posted by Piavlo <lo...@gmail.com>.
On 11/29/2011 02:51 AM, Benoit Perroud wrote:
> You may want to add 29991231 instead of appending.
Do you mean generate keys like this
md5(member_id) + yyyymmdd
instead of
md5(member_id)yyyymmdd
And use the normal 128bit token range ?
Hmm - this way there will obviously be overlaps of members ranges - then
two md5(member_id1) & md5(member_id2) end up very close so
using md5(member_id1)+yyyymmdd & md5(member_id2)+yyyymmdd will cause
range overlaps with ByteOrderedPartitioner.
Thanks
Alex
>
> Le lundi 28 novembre 2011, Piavlo <lolitushka@gmail.com
> <ma...@gmail.com>> a écrit :
> > Anyone can help with this?
> >
> > Thanks
> >
> > On 11/24/2011 11:55 AM, Piavlo wrote:
> >>
> >> Hi,
> >>
> >> We need help with choosing correct tokens for ByteOrderedPartitioner
> >> Originally the key where supposed to be member_id-yyyymmdd
> >> but since we need to male rage scans on same member_id and varying
> date ranges yyyymmdd
> >> we decided to use ByteOrderedPartitioner, so we need that same
> member will be assigned to same token range.
> >> So we decided that the keys will be md5(member_id)yyyymmdd
> >> Since md5 on member_id should give even distribution or member_id
> across tokens.
> >>
> >> We have 4 nodes, and don't understand how to choose the tokens.
> >> We tried the following tokens
> >>
> >> # ./tokengentool 4
> >> token 0: 0
> >> token 1: 42535295865117307932921825928971026432
> >> token 2: 85070591730234615865843651857942052864
> >> token 3: 127605887595351923798765477786913079296
> >>
> >> and appended 29991231
> >>
> >> so we ended up with the following tokens
> >>
> >> token 0: 0
> >> token 1: 4253529586511730793292182592897102643229991231
> >> token 2: 8507059173023461586584365185794205286429991231
> >> token 3: 12760588759535192379876547778691307929629991231
> >>
> >> But the key end up not evenly distributed.
> >>
> >> So any help is appreciated.
> >>
> >> Thanks
> >> Alex
> >
> >
>
> --
> sent from my Nokia 3210
Re: need help with choosing correct tokens for ByteOrderedPartitioner
Posted by Benoit Perroud <be...@noisette.ch>.
You may want to add 29991231 instead of appending.
Le lundi 28 novembre 2011, Piavlo <lo...@gmail.com> a écrit :
> Anyone can help with this?
>
> Thanks
>
> On 11/24/2011 11:55 AM, Piavlo wrote:
>>
>> Hi,
>>
>> We need help with choosing correct tokens for ByteOrderedPartitioner
>> Originally the key where supposed to be member_id-yyyymmdd
>> but since we need to male rage scans on same member_id and varying date
ranges yyyymmdd
>> we decided to use ByteOrderedPartitioner, so we need that same member
will be assigned to same token range.
>> So we decided that the keys will be md5(member_id)yyyymmdd
>> Since md5 on member_id should give even distribution or member_id across
tokens.
>>
>> We have 4 nodes, and don't understand how to choose the tokens.
>> We tried the following tokens
>>
>> # ./tokengentool 4
>> token 0: 0
>> token 1: 42535295865117307932921825928971026432
>> token 2: 85070591730234615865843651857942052864
>> token 3: 127605887595351923798765477786913079296
>>
>> and appended 29991231
>>
>> so we ended up with the following tokens
>>
>> token 0: 0
>> token 1: 4253529586511730793292182592897102643229991231
>> token 2: 8507059173023461586584365185794205286429991231
>> token 3: 12760588759535192379876547778691307929629991231
>>
>> But the key end up not evenly distributed.
>>
>> So any help is appreciated.
>>
>> Thanks
>> Alex
>
>
--
sent from my Nokia 3210
Re: need help with choosing correct tokens for ByteOrderedPartitioner
Posted by Piavlo <lo...@gmail.com>.
Anyone can help with this?
Thanks
On 11/24/2011 11:55 AM, Piavlo wrote:
> Hi,
>
> We need help with choosing correct tokens for ByteOrderedPartitioner
> Originally the key where supposed to be member_id-yyyymmdd
> but since we need to male rage scans on same member_id and varying
> date ranges yyyymmdd
> we decided to use ByteOrderedPartitioner, so we need that same member
> will be assigned to same token range.
> So we decided that the keys will be md5(member_id)yyyymmdd
> Since md5 on member_id should give even distribution or member_id
> across tokens.
>
> We have 4 nodes, and don't understand how to choose the tokens.
> We tried the following tokens
>
> # ./tokengentool 4
> token 0: 0
> token 1: 42535295865117307932921825928971026432
> token 2: 85070591730234615865843651857942052864
> token 3: 127605887595351923798765477786913079296
>
> and appended 29991231
>
> so we ended up with the following tokens
>
> token 0: 0
> token 1: 4253529586511730793292182592897102643229991231
> token 2: 8507059173023461586584365185794205286429991231
> token 3: 12760588759535192379876547778691307929629991231
>
> But the key end up not evenly distributed.
>
> So any help is appreciated.
>
> Thanks
> Alex