You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Richard grossman <ri...@gmail.com> on 2009/11/10 16:38:04 UTC
Strange Data distribution in 0.4.2
Hi
I've build the 0.4.2 from the tags in SVN.
I've made exactly the same cluster with same configuration as 0.4.1
I've delete all the data in all server
Now I send the data to first server and the data is not more distributed
across the other server as previously.
I've configured replica to 1
here is my storage-conf.xml
<Storage>
<ClusterName>BeeCluster</ClusterName>
<Keyspaces>
<Keyspace Name="Keyspace1">
<ColumnFamily CompareWith="BytesType"
Name="channelShowLink"
FlushPeriodInMinutes="15"/>
<ColumnFamily CompareWith="BytesType"
Name="channelShow"
FlushPeriodInMinutes="15"/>
<ColumnFamily CompareWith="BytesType"
Name="userAction"
FlushPeriodInMinutes="15"/>
<ColumnFamily CompareWith="BytesType"
Name="headends"
FlushPeriodInMinutes="15"/>
<ColumnFamily CompareWith="BytesType"
Name="similarity"
FlushPeriodInMinutes="500"/>
</Keyspace>
</Keyspaces>
<Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
<InitialToken></InitialToken>
<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
<ReplicationFactor>1</ReplicationFactor>
<CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
<DataFileDirectories>
<DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
</DataFileDirectories>
<CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
<BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
<StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
<Seeds>
<Seed>192.168.249.200</Seed>
<Seed>192.168.249.222</Seed>
<Seed>192.168.249.95</Seed>
</Seeds>
<!-- Miscellaneous -->
<!-- Time to wait for a reply from other nodes before failing the command
-->
<RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
<!-- Size to allow commitlog to grow to before creating a new segment -->
<CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
<ListenAddress>192.168.249.200</ListenAddress>
<!-- TCP port, for commands and data -->
<StoragePort>7000</StoragePort>
<!-- UDP port, for membership communications (gossip) -->
<ControlPort>7001</ControlPort>
<ThriftAddress>0.0.0.0</ThriftAddress>
<!-- Thrift RPC port (the port clients connect to). -->
<ThriftPort>9160</ThriftPort>
<!--
~ Whether or not to use a framed transport for Thrift. If this option
~ is set to true then you must also use a framed transport on the
~ client-side, (framed and non-framed transports are not compatible).
-->
<ThriftFramedTransport>false</ThriftFramedTransport>
<SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
<FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
<FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
<ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
<MemtableSizeInMB>64</MemtableSizeInMB>
<MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
<ConcurrentReads>8</ConcurrentReads>
<ConcurrentWrites>32</ConcurrentWrites>
<CommitLogSync>periodic</CommitLogSync>
<CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
<!-- <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS> -->
<GCGraceSeconds>864000</GCGraceSeconds>
<FlushMinThreads>1</FlushMinThreads>
<FlushMaxThreads>1</FlushMaxThreads>
<BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
</Storage>
Is there anyone with the same problem ??
Thanks
Re: Strange Data distribution in 0.4.2
Posted by Jonathan Ellis <jb...@gmail.com>.
... of course this will change over time, which is why we're adding
active load balancing in 0.5, but the closer you can get to a
reasonable starting point, the better.
On Tue, Nov 10, 2009 at 11:08 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> yup, so you need to decide what dates are actually going to appear in your data
>
> On Tue, Nov 10, 2009 at 10:59 AM, Richard grossman <ri...@gmail.com> wrote:
>> ok but If it's not defined ? for example 1 field is a date there is no end
>> for date ?
>>
>> On Tue, Nov 10, 2009 at 6:57 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>>
>>> no.
>>>
>>> for randompartitioner, you use integers from 0 to 2**127, but for OPP
>>> you use strings from your key universe.
>>>
>>> On Tue, Nov 10, 2009 at 10:37 AM, Richard grossman <ri...@gmail.com>
>>> wrote:
>>> > If I understand good if I transform my String key <data>:<data>:<data>
>>> > to
>>> > some long value and If I've 3 server then I put on the first server
>>> > initialToken : 0
>>> > second : Long.max() /2
>>> > third : Long.max()
>>> >
>>> > Is it correct ?? or there is something better ?
>>> >
>>> > thanks
>>> >
>>> > On Tue, Nov 10, 2009 at 6:01 PM, Jonathan Ellis <jb...@gmail.com>
>>> > wrote:
>>> >>
>>> >> for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
>>> >>
>>> >> On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <ri...@gmail.com>
>>> >> wrote:
>>> >> > hi
>>> >> >
>>> >> > I've understand this but I don't know what to write into initialtoken
>>> >> > is
>>> >> > it
>>> >> > "1" or "a" or something else ?
>>> >> > as I've said in a previous post My keys are build like
>>> >> > <data>:<data>:<data>
>>> >> > Is there any link ?
>>> >> >
>>> >> > Thanks,
>>> >> >
>>> >> > On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> if you're not specifying initialtoken, every time you wipe your
>>> >> >> installation it will generate new tokens. for a small number of
>>> >> >> machines you'll definitely see some random tokens better balanced
>>> >> >> than
>>> >> >> others.
>>> >> >>
>>> >> >> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman
>>> >> >> <ri...@gmail.com>
>>> >> >> wrote:
>>> >> >> > Hi
>>> >> >> >
>>> >> >> > I've build the 0.4.2 from the tags in SVN.
>>> >> >> > I've made exactly the same cluster with same configuration as
>>> >> >> > 0.4.1
>>> >> >> > I've delete all the data in all server
>>> >> >> >
>>> >> >> > Now I send the data to first server and the data is not more
>>> >> >> > distributed
>>> >> >> > across the other server as previously.
>>> >> >> > I've configured replica to 1
>>> >> >> >
>>> >> >> > here is my storage-conf.xml
>>> >> >> >
>>> >> >> > <Storage>
>>> >> >> >
>>> >> >> > <ClusterName>BeeCluster</ClusterName>
>>> >> >> >
>>> >> >> >
>>> >> >> > <Keyspaces>
>>> >> >> > <Keyspace Name="Keyspace1">
>>> >> >> >
>>> >> >> > <ColumnFamily CompareWith="BytesType"
>>> >> >> > Name="channelShowLink"
>>> >> >> > FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> > <ColumnFamily CompareWith="BytesType"
>>> >> >> > Name="channelShow"
>>> >> >> > FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> > <ColumnFamily CompareWith="BytesType"
>>> >> >> > Name="userAction"
>>> >> >> > FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> > <ColumnFamily CompareWith="BytesType"
>>> >> >> > Name="headends"
>>> >> >> > FlushPeriodInMinutes="15"/>
>>> >> >> >
>>> >> >> > <ColumnFamily CompareWith="BytesType"
>>> >> >> > Name="similarity"
>>> >> >> > FlushPeriodInMinutes="500"/>
>>> >> >> > </Keyspace>
>>> >> >> > </Keyspaces>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
>>> >> >> >
>>> >> >> > <InitialToken></InitialToken>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>> >> >> >
>>> >> >> > <ReplicationFactor>1</ReplicationFactor>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
>>> >> >> > <DataFileDirectories>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
>>> >> >> > </DataFileDirectories>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <Seeds>
>>> >> >> > <Seed>192.168.249.200</Seed>
>>> >> >> > <Seed>192.168.249.222</Seed>
>>> >> >> > <Seed>192.168.249.95</Seed>
>>> >> >> > </Seeds>
>>> >> >> >
>>> >> >> >
>>> >> >> > <!-- Miscellaneous -->
>>> >> >> >
>>> >> >> > <!-- Time to wait for a reply from other nodes before failing
>>> >> >> > the
>>> >> >> > command
>>> >> >> > -->
>>> >> >> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
>>> >> >> > <!-- Size to allow commitlog to grow to before creating a new
>>> >> >> > segment
>>> >> >> > -->
>>> >> >> >
>>> >> >> >
>>> >> >> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <ListenAddress>192.168.249.200</ListenAddress>
>>> >> >> > <!-- TCP port, for commands and data -->
>>> >> >> > <StoragePort>7000</StoragePort>
>>> >> >> > <!-- UDP port, for membership communications (gossip) -->
>>> >> >> > <ControlPort>7001</ControlPort>
>>> >> >> >
>>> >> >> >
>>> >> >> > <ThriftAddress>0.0.0.0</ThriftAddress>
>>> >> >> > <!-- Thrift RPC port (the port clients connect to). -->
>>> >> >> > <ThriftPort>9160</ThriftPort>
>>> >> >> > <!--
>>> >> >> > ~ Whether or not to use a framed transport for Thrift. If this
>>> >> >> > option
>>> >> >> > ~ is set to true then you must also use a framed transport on
>>> >> >> > the
>>> >> >> > ~ client-side, (framed and non-framed transports are not
>>> >> >> > compatible).
>>> >> >> > -->
>>> >> >> > <ThriftFramedTransport>false</ThriftFramedTransport>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>> >> >> >
>>> >> >> >
>>> >> >> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
>>> >> >> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>>> >> >> >
>>> >> >> >
>>> >> >> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>> >> >> >
>>> >> >> >
>>> >> >> > <MemtableSizeInMB>64</MemtableSizeInMB>
>>> >> >> >
>>> >> >> >
>>> >> >> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>>> >> >> >
>>> >> >> >
>>> >> >> > <ConcurrentReads>8</ConcurrentReads>
>>> >> >> > <ConcurrentWrites>32</ConcurrentWrites>
>>> >> >> >
>>> >> >> >
>>> >> >> > <CommitLogSync>periodic</CommitLogSync>
>>> >> >> >
>>> >> >> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>>> >> >> >
>>> >> >> > <!--
>>> >> >> > <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
>>> >> >> > -->
>>> >> >> >
>>> >> >> >
>>> >> >> > <GCGraceSeconds>864000</GCGraceSeconds>
>>> >> >> >
>>> >> >> >
>>> >> >> > <FlushMinThreads>1</FlushMinThreads>
>>> >> >> > <FlushMaxThreads>1</FlushMaxThreads>
>>> >> >> >
>>> >> >> >
>>> >> >> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>>> >> >> >
>>> >> >> > </Storage>
>>> >> >> >
>>> >> >> > Is there anyone with the same problem ??
>>> >> >> >
>>> >> >> > Thanks
>>> >> >> >
>>> >> >
>>> >> >
>>> >
>>> >
>>
>>
>
Re: Strange Data distribution in 0.4.2
Posted by Jonathan Ellis <jb...@gmail.com>.
yup, so you need to decide what dates are actually going to appear in your data
On Tue, Nov 10, 2009 at 10:59 AM, Richard grossman <ri...@gmail.com> wrote:
> ok but If it's not defined ? for example 1 field is a date there is no end
> for date ?
>
> On Tue, Nov 10, 2009 at 6:57 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> no.
>>
>> for randompartitioner, you use integers from 0 to 2**127, but for OPP
>> you use strings from your key universe.
>>
>> On Tue, Nov 10, 2009 at 10:37 AM, Richard grossman <ri...@gmail.com>
>> wrote:
>> > If I understand good if I transform my String key <data>:<data>:<data>
>> > to
>> > some long value and If I've 3 server then I put on the first server
>> > initialToken : 0
>> > second : Long.max() /2
>> > third : Long.max()
>> >
>> > Is it correct ?? or there is something better ?
>> >
>> > thanks
>> >
>> > On Tue, Nov 10, 2009 at 6:01 PM, Jonathan Ellis <jb...@gmail.com>
>> > wrote:
>> >>
>> >> for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
>> >>
>> >> On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <ri...@gmail.com>
>> >> wrote:
>> >> > hi
>> >> >
>> >> > I've understand this but I don't know what to write into initialtoken
>> >> > is
>> >> > it
>> >> > "1" or "a" or something else ?
>> >> > as I've said in a previous post My keys are build like
>> >> > <data>:<data>:<data>
>> >> > Is there any link ?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> if you're not specifying initialtoken, every time you wipe your
>> >> >> installation it will generate new tokens. for a small number of
>> >> >> machines you'll definitely see some random tokens better balanced
>> >> >> than
>> >> >> others.
>> >> >>
>> >> >> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman
>> >> >> <ri...@gmail.com>
>> >> >> wrote:
>> >> >> > Hi
>> >> >> >
>> >> >> > I've build the 0.4.2 from the tags in SVN.
>> >> >> > I've made exactly the same cluster with same configuration as
>> >> >> > 0.4.1
>> >> >> > I've delete all the data in all server
>> >> >> >
>> >> >> > Now I send the data to first server and the data is not more
>> >> >> > distributed
>> >> >> > across the other server as previously.
>> >> >> > I've configured replica to 1
>> >> >> >
>> >> >> > here is my storage-conf.xml
>> >> >> >
>> >> >> > <Storage>
>> >> >> >
>> >> >> > <ClusterName>BeeCluster</ClusterName>
>> >> >> >
>> >> >> >
>> >> >> > <Keyspaces>
>> >> >> > <Keyspace Name="Keyspace1">
>> >> >> >
>> >> >> > <ColumnFamily CompareWith="BytesType"
>> >> >> > Name="channelShowLink"
>> >> >> > FlushPeriodInMinutes="15"/>
>> >> >> >
>> >> >> > <ColumnFamily CompareWith="BytesType"
>> >> >> > Name="channelShow"
>> >> >> > FlushPeriodInMinutes="15"/>
>> >> >> >
>> >> >> > <ColumnFamily CompareWith="BytesType"
>> >> >> > Name="userAction"
>> >> >> > FlushPeriodInMinutes="15"/>
>> >> >> >
>> >> >> > <ColumnFamily CompareWith="BytesType"
>> >> >> > Name="headends"
>> >> >> > FlushPeriodInMinutes="15"/>
>> >> >> >
>> >> >> > <ColumnFamily CompareWith="BytesType"
>> >> >> > Name="similarity"
>> >> >> > FlushPeriodInMinutes="500"/>
>> >> >> > </Keyspace>
>> >> >> > </Keyspaces>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
>> >> >> >
>> >> >> > <InitialToken></InitialToken>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>> >> >> >
>> >> >> > <ReplicationFactor>1</ReplicationFactor>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
>> >> >> > <DataFileDirectories>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
>> >> >> > </DataFileDirectories>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <Seeds>
>> >> >> > <Seed>192.168.249.200</Seed>
>> >> >> > <Seed>192.168.249.222</Seed>
>> >> >> > <Seed>192.168.249.95</Seed>
>> >> >> > </Seeds>
>> >> >> >
>> >> >> >
>> >> >> > <!-- Miscellaneous -->
>> >> >> >
>> >> >> > <!-- Time to wait for a reply from other nodes before failing
>> >> >> > the
>> >> >> > command
>> >> >> > -->
>> >> >> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
>> >> >> > <!-- Size to allow commitlog to grow to before creating a new
>> >> >> > segment
>> >> >> > -->
>> >> >> >
>> >> >> >
>> >> >> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <ListenAddress>192.168.249.200</ListenAddress>
>> >> >> > <!-- TCP port, for commands and data -->
>> >> >> > <StoragePort>7000</StoragePort>
>> >> >> > <!-- UDP port, for membership communications (gossip) -->
>> >> >> > <ControlPort>7001</ControlPort>
>> >> >> >
>> >> >> >
>> >> >> > <ThriftAddress>0.0.0.0</ThriftAddress>
>> >> >> > <!-- Thrift RPC port (the port clients connect to). -->
>> >> >> > <ThriftPort>9160</ThriftPort>
>> >> >> > <!--
>> >> >> > ~ Whether or not to use a framed transport for Thrift. If this
>> >> >> > option
>> >> >> > ~ is set to true then you must also use a framed transport on
>> >> >> > the
>> >> >> > ~ client-side, (framed and non-framed transports are not
>> >> >> > compatible).
>> >> >> > -->
>> >> >> > <ThriftFramedTransport>false</ThriftFramedTransport>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>> >> >> >
>> >> >> >
>> >> >> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
>> >> >> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>> >> >> >
>> >> >> >
>> >> >> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>> >> >> >
>> >> >> >
>> >> >> > <MemtableSizeInMB>64</MemtableSizeInMB>
>> >> >> >
>> >> >> >
>> >> >> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>> >> >> >
>> >> >> >
>> >> >> > <ConcurrentReads>8</ConcurrentReads>
>> >> >> > <ConcurrentWrites>32</ConcurrentWrites>
>> >> >> >
>> >> >> >
>> >> >> > <CommitLogSync>periodic</CommitLogSync>
>> >> >> >
>> >> >> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>> >> >> >
>> >> >> > <!--
>> >> >> > <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
>> >> >> > -->
>> >> >> >
>> >> >> >
>> >> >> > <GCGraceSeconds>864000</GCGraceSeconds>
>> >> >> >
>> >> >> >
>> >> >> > <FlushMinThreads>1</FlushMinThreads>
>> >> >> > <FlushMaxThreads>1</FlushMaxThreads>
>> >> >> >
>> >> >> >
>> >> >> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>> >> >> >
>> >> >> > </Storage>
>> >> >> >
>> >> >> > Is there anyone with the same problem ??
>> >> >> >
>> >> >> > Thanks
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>
Re: Strange Data distribution in 0.4.2
Posted by Richard grossman <ri...@gmail.com>.
ok but If it's not defined ? for example 1 field is a date there is no end
for date ?
On Tue, Nov 10, 2009 at 6:57 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> no.
>
> for randompartitioner, you use integers from 0 to 2**127, but for OPP
> you use strings from your key universe.
>
> On Tue, Nov 10, 2009 at 10:37 AM, Richard grossman <ri...@gmail.com>
> wrote:
> > If I understand good if I transform my String key <data>:<data>:<data> to
> > some long value and If I've 3 server then I put on the first server
> > initialToken : 0
> > second : Long.max() /2
> > third : Long.max()
> >
> > Is it correct ?? or there is something better ?
> >
> > thanks
> >
> > On Tue, Nov 10, 2009 at 6:01 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> >>
> >> for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
> >>
> >> On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <ri...@gmail.com>
> >> wrote:
> >> > hi
> >> >
> >> > I've understand this but I don't know what to write into initialtoken
> is
> >> > it
> >> > "1" or "a" or something else ?
> >> > as I've said in a previous post My keys are build like
> >> > <data>:<data>:<data>
> >> > Is there any link ?
> >> >
> >> > Thanks,
> >> >
> >> > On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com>
> >> > wrote:
> >> >>
> >> >> if you're not specifying initialtoken, every time you wipe your
> >> >> installation it will generate new tokens. for a small number of
> >> >> machines you'll definitely see some random tokens better balanced
> than
> >> >> others.
> >> >>
> >> >> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman <
> richiesgr@gmail.com>
> >> >> wrote:
> >> >> > Hi
> >> >> >
> >> >> > I've build the 0.4.2 from the tags in SVN.
> >> >> > I've made exactly the same cluster with same configuration as 0.4.1
> >> >> > I've delete all the data in all server
> >> >> >
> >> >> > Now I send the data to first server and the data is not more
> >> >> > distributed
> >> >> > across the other server as previously.
> >> >> > I've configured replica to 1
> >> >> >
> >> >> > here is my storage-conf.xml
> >> >> >
> >> >> > <Storage>
> >> >> >
> >> >> > <ClusterName>BeeCluster</ClusterName>
> >> >> >
> >> >> >
> >> >> > <Keyspaces>
> >> >> > <Keyspace Name="Keyspace1">
> >> >> >
> >> >> > <ColumnFamily CompareWith="BytesType"
> >> >> > Name="channelShowLink"
> >> >> > FlushPeriodInMinutes="15"/>
> >> >> >
> >> >> > <ColumnFamily CompareWith="BytesType"
> >> >> > Name="channelShow"
> >> >> > FlushPeriodInMinutes="15"/>
> >> >> >
> >> >> > <ColumnFamily CompareWith="BytesType"
> >> >> > Name="userAction"
> >> >> > FlushPeriodInMinutes="15"/>
> >> >> >
> >> >> > <ColumnFamily CompareWith="BytesType"
> >> >> > Name="headends"
> >> >> > FlushPeriodInMinutes="15"/>
> >> >> >
> >> >> > <ColumnFamily CompareWith="BytesType"
> >> >> > Name="similarity"
> >> >> > FlushPeriodInMinutes="500"/>
> >> >> > </Keyspace>
> >> >> > </Keyspaces>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
> >> >> >
> >> >> > <InitialToken></InitialToken>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
> >> >> >
> >> >> > <ReplicationFactor>1</ReplicationFactor>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
> >> >> > <DataFileDirectories>
> >> >> >
> >> >> >
> >> >> >
> <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
> >> >> > </DataFileDirectories>
> >> >> >
> >> >> >
> >> >> >
> <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
> >> >> >
> >> >> >
> >> >> >
> >> >> > <Seeds>
> >> >> > <Seed>192.168.249.200</Seed>
> >> >> > <Seed>192.168.249.222</Seed>
> >> >> > <Seed>192.168.249.95</Seed>
> >> >> > </Seeds>
> >> >> >
> >> >> >
> >> >> > <!-- Miscellaneous -->
> >> >> >
> >> >> > <!-- Time to wait for a reply from other nodes before failing the
> >> >> > command
> >> >> > -->
> >> >> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
> >> >> > <!-- Size to allow commitlog to grow to before creating a new
> >> >> > segment
> >> >> > -->
> >> >> >
> >> >> >
> <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
> >> >> >
> >> >> >
> >> >> >
> >> >> > <ListenAddress>192.168.249.200</ListenAddress>
> >> >> > <!-- TCP port, for commands and data -->
> >> >> > <StoragePort>7000</StoragePort>
> >> >> > <!-- UDP port, for membership communications (gossip) -->
> >> >> > <ControlPort>7001</ControlPort>
> >> >> >
> >> >> >
> >> >> > <ThriftAddress>0.0.0.0</ThriftAddress>
> >> >> > <!-- Thrift RPC port (the port clients connect to). -->
> >> >> > <ThriftPort>9160</ThriftPort>
> >> >> > <!--
> >> >> > ~ Whether or not to use a framed transport for Thrift. If this
> >> >> > option
> >> >> > ~ is set to true then you must also use a framed transport on
> the
> >> >> > ~ client-side, (framed and non-framed transports are not
> >> >> > compatible).
> >> >> > -->
> >> >> > <ThriftFramedTransport>false</ThriftFramedTransport>
> >> >> >
> >> >> >
> >> >> >
> >> >> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
> >> >> >
> >> >> >
> >> >> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
> >> >> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
> >> >> >
> >> >> >
> >> >> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
> >> >> >
> >> >> >
> >> >> > <MemtableSizeInMB>64</MemtableSizeInMB>
> >> >> >
> >> >> >
> <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
> >> >> >
> >> >> >
> >> >> > <ConcurrentReads>8</ConcurrentReads>
> >> >> > <ConcurrentWrites>32</ConcurrentWrites>
> >> >> >
> >> >> >
> >> >> > <CommitLogSync>periodic</CommitLogSync>
> >> >> >
> >> >> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
> >> >> >
> >> >> > <!--
> <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
> >> >> > -->
> >> >> >
> >> >> >
> >> >> > <GCGraceSeconds>864000</GCGraceSeconds>
> >> >> >
> >> >> >
> >> >> > <FlushMinThreads>1</FlushMinThreads>
> >> >> > <FlushMaxThreads>1</FlushMaxThreads>
> >> >> >
> >> >> >
> >> >> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
> >> >> >
> >> >> > </Storage>
> >> >> >
> >> >> > Is there anyone with the same problem ??
> >> >> >
> >> >> > Thanks
> >> >> >
> >> >
> >> >
> >
> >
>
Re: Strange Data distribution in 0.4.2
Posted by Jonathan Ellis <jb...@gmail.com>.
no.
for randompartitioner, you use integers from 0 to 2**127, but for OPP
you use strings from your key universe.
On Tue, Nov 10, 2009 at 10:37 AM, Richard grossman <ri...@gmail.com> wrote:
> If I understand good if I transform my String key <data>:<data>:<data> to
> some long value and If I've 3 server then I put on the first server
> initialToken : 0
> second : Long.max() /2
> third : Long.max()
>
> Is it correct ?? or there is something better ?
>
> thanks
>
> On Tue, Nov 10, 2009 at 6:01 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
>>
>> On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <ri...@gmail.com>
>> wrote:
>> > hi
>> >
>> > I've understand this but I don't know what to write into initialtoken is
>> > it
>> > "1" or "a" or something else ?
>> > as I've said in a previous post My keys are build like
>> > <data>:<data>:<data>
>> > Is there any link ?
>> >
>> > Thanks,
>> >
>> > On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com>
>> > wrote:
>> >>
>> >> if you're not specifying initialtoken, every time you wipe your
>> >> installation it will generate new tokens. for a small number of
>> >> machines you'll definitely see some random tokens better balanced than
>> >> others.
>> >>
>> >> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman <ri...@gmail.com>
>> >> wrote:
>> >> > Hi
>> >> >
>> >> > I've build the 0.4.2 from the tags in SVN.
>> >> > I've made exactly the same cluster with same configuration as 0.4.1
>> >> > I've delete all the data in all server
>> >> >
>> >> > Now I send the data to first server and the data is not more
>> >> > distributed
>> >> > across the other server as previously.
>> >> > I've configured replica to 1
>> >> >
>> >> > here is my storage-conf.xml
>> >> >
>> >> > <Storage>
>> >> >
>> >> > <ClusterName>BeeCluster</ClusterName>
>> >> >
>> >> >
>> >> > <Keyspaces>
>> >> > <Keyspace Name="Keyspace1">
>> >> >
>> >> > <ColumnFamily CompareWith="BytesType"
>> >> > Name="channelShowLink"
>> >> > FlushPeriodInMinutes="15"/>
>> >> >
>> >> > <ColumnFamily CompareWith="BytesType"
>> >> > Name="channelShow"
>> >> > FlushPeriodInMinutes="15"/>
>> >> >
>> >> > <ColumnFamily CompareWith="BytesType"
>> >> > Name="userAction"
>> >> > FlushPeriodInMinutes="15"/>
>> >> >
>> >> > <ColumnFamily CompareWith="BytesType"
>> >> > Name="headends"
>> >> > FlushPeriodInMinutes="15"/>
>> >> >
>> >> > <ColumnFamily CompareWith="BytesType"
>> >> > Name="similarity"
>> >> > FlushPeriodInMinutes="500"/>
>> >> > </Keyspace>
>> >> > </Keyspaces>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
>> >> >
>> >> > <InitialToken></InitialToken>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>> >> >
>> >> > <ReplicationFactor>1</ReplicationFactor>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
>> >> > <DataFileDirectories>
>> >> >
>> >> >
>> >> > <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
>> >> > </DataFileDirectories>
>> >> >
>> >> >
>> >> > <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
>> >> >
>> >> >
>> >> >
>> >> > <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
>> >> >
>> >> >
>> >> >
>> >> > <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
>> >> >
>> >> >
>> >> >
>> >> > <Seeds>
>> >> > <Seed>192.168.249.200</Seed>
>> >> > <Seed>192.168.249.222</Seed>
>> >> > <Seed>192.168.249.95</Seed>
>> >> > </Seeds>
>> >> >
>> >> >
>> >> > <!-- Miscellaneous -->
>> >> >
>> >> > <!-- Time to wait for a reply from other nodes before failing the
>> >> > command
>> >> > -->
>> >> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
>> >> > <!-- Size to allow commitlog to grow to before creating a new
>> >> > segment
>> >> > -->
>> >> >
>> >> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>> >> >
>> >> >
>> >> >
>> >> > <ListenAddress>192.168.249.200</ListenAddress>
>> >> > <!-- TCP port, for commands and data -->
>> >> > <StoragePort>7000</StoragePort>
>> >> > <!-- UDP port, for membership communications (gossip) -->
>> >> > <ControlPort>7001</ControlPort>
>> >> >
>> >> >
>> >> > <ThriftAddress>0.0.0.0</ThriftAddress>
>> >> > <!-- Thrift RPC port (the port clients connect to). -->
>> >> > <ThriftPort>9160</ThriftPort>
>> >> > <!--
>> >> > ~ Whether or not to use a framed transport for Thrift. If this
>> >> > option
>> >> > ~ is set to true then you must also use a framed transport on the
>> >> > ~ client-side, (framed and non-framed transports are not
>> >> > compatible).
>> >> > -->
>> >> > <ThriftFramedTransport>false</ThriftFramedTransport>
>> >> >
>> >> >
>> >> >
>> >> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>> >> >
>> >> >
>> >> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
>> >> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>> >> >
>> >> >
>> >> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>> >> >
>> >> >
>> >> > <MemtableSizeInMB>64</MemtableSizeInMB>
>> >> >
>> >> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>> >> >
>> >> >
>> >> > <ConcurrentReads>8</ConcurrentReads>
>> >> > <ConcurrentWrites>32</ConcurrentWrites>
>> >> >
>> >> >
>> >> > <CommitLogSync>periodic</CommitLogSync>
>> >> >
>> >> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>> >> >
>> >> > <!-- <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
>> >> > -->
>> >> >
>> >> >
>> >> > <GCGraceSeconds>864000</GCGraceSeconds>
>> >> >
>> >> >
>> >> > <FlushMinThreads>1</FlushMinThreads>
>> >> > <FlushMaxThreads>1</FlushMaxThreads>
>> >> >
>> >> >
>> >> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>> >> >
>> >> > </Storage>
>> >> >
>> >> > Is there anyone with the same problem ??
>> >> >
>> >> > Thanks
>> >> >
>> >
>> >
>
>
Re: Strange Data distribution in 0.4.2
Posted by Richard grossman <ri...@gmail.com>.
If I understand good if I transform my String key <data>:<data>:<data> to
some long value and If I've 3 server then I put on the first server
initialToken : 0
second : Long.max() /2
third : Long.max()
Is it correct ?? or there is something better ?
thanks
On Tue, Nov 10, 2009 at 6:01 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
>
> On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <ri...@gmail.com>
> wrote:
> > hi
> >
> > I've understand this but I don't know what to write into initialtoken is
> it
> > "1" or "a" or something else ?
> > as I've said in a previous post My keys are build like
> <data>:<data>:<data>
> > Is there any link ?
> >
> > Thanks,
> >
> > On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> >>
> >> if you're not specifying initialtoken, every time you wipe your
> >> installation it will generate new tokens. for a small number of
> >> machines you'll definitely see some random tokens better balanced than
> >> others.
> >>
> >> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman <ri...@gmail.com>
> >> wrote:
> >> > Hi
> >> >
> >> > I've build the 0.4.2 from the tags in SVN.
> >> > I've made exactly the same cluster with same configuration as 0.4.1
> >> > I've delete all the data in all server
> >> >
> >> > Now I send the data to first server and the data is not more
> distributed
> >> > across the other server as previously.
> >> > I've configured replica to 1
> >> >
> >> > here is my storage-conf.xml
> >> >
> >> > <Storage>
> >> >
> >> > <ClusterName>BeeCluster</ClusterName>
> >> >
> >> >
> >> > <Keyspaces>
> >> > <Keyspace Name="Keyspace1">
> >> >
> >> > <ColumnFamily CompareWith="BytesType"
> >> > Name="channelShowLink"
> >> > FlushPeriodInMinutes="15"/>
> >> >
> >> > <ColumnFamily CompareWith="BytesType"
> >> > Name="channelShow"
> >> > FlushPeriodInMinutes="15"/>
> >> >
> >> > <ColumnFamily CompareWith="BytesType"
> >> > Name="userAction"
> >> > FlushPeriodInMinutes="15"/>
> >> >
> >> > <ColumnFamily CompareWith="BytesType"
> >> > Name="headends"
> >> > FlushPeriodInMinutes="15"/>
> >> >
> >> > <ColumnFamily CompareWith="BytesType"
> >> > Name="similarity"
> >> > FlushPeriodInMinutes="500"/>
> >> > </Keyspace>
> >> > </Keyspaces>
> >> >
> >> >
> >> >
> >> >
> >> >
> <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
> >> >
> >> > <InitialToken></InitialToken>
> >> >
> >> >
> >> >
> >> >
> >> >
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> >> >
> >> >
> >> >
> >> >
> >> >
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
> >> >
> >> > <ReplicationFactor>1</ReplicationFactor>
> >> >
> >> >
> >> >
> >> >
> >> >
> <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
> >> > <DataFileDirectories>
> >> >
> >> >
> <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
> >> > </DataFileDirectories>
> >> >
> >> >
> <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
> >> >
> >> >
> >> >
> <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
> >> >
> >> >
> >> >
> <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
> >> >
> >> >
> >> >
> >> > <Seeds>
> >> > <Seed>192.168.249.200</Seed>
> >> > <Seed>192.168.249.222</Seed>
> >> > <Seed>192.168.249.95</Seed>
> >> > </Seeds>
> >> >
> >> >
> >> > <!-- Miscellaneous -->
> >> >
> >> > <!-- Time to wait for a reply from other nodes before failing the
> >> > command
> >> > -->
> >> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
> >> > <!-- Size to allow commitlog to grow to before creating a new
> segment
> >> > -->
> >> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
> >> >
> >> >
> >> >
> >> > <ListenAddress>192.168.249.200</ListenAddress>
> >> > <!-- TCP port, for commands and data -->
> >> > <StoragePort>7000</StoragePort>
> >> > <!-- UDP port, for membership communications (gossip) -->
> >> > <ControlPort>7001</ControlPort>
> >> >
> >> >
> >> > <ThriftAddress>0.0.0.0</ThriftAddress>
> >> > <!-- Thrift RPC port (the port clients connect to). -->
> >> > <ThriftPort>9160</ThriftPort>
> >> > <!--
> >> > ~ Whether or not to use a framed transport for Thrift. If this
> option
> >> > ~ is set to true then you must also use a framed transport on the
> >> > ~ client-side, (framed and non-framed transports are not
> compatible).
> >> > -->
> >> > <ThriftFramedTransport>false</ThriftFramedTransport>
> >> >
> >> >
> >> >
> >> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
> >> >
> >> >
> >> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
> >> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
> >> >
> >> >
> >> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
> >> >
> >> >
> >> > <MemtableSizeInMB>64</MemtableSizeInMB>
> >> >
> >> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
> >> >
> >> >
> >> > <ConcurrentReads>8</ConcurrentReads>
> >> > <ConcurrentWrites>32</ConcurrentWrites>
> >> >
> >> >
> >> > <CommitLogSync>periodic</CommitLogSync>
> >> >
> >> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
> >> >
> >> > <!-- <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
> >> > -->
> >> >
> >> >
> >> > <GCGraceSeconds>864000</GCGraceSeconds>
> >> >
> >> >
> >> > <FlushMinThreads>1</FlushMinThreads>
> >> > <FlushMaxThreads>1</FlushMaxThreads>
> >> >
> >> >
> >> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
> >> >
> >> > </Storage>
> >> >
> >> > Is there anyone with the same problem ??
> >> >
> >> > Thanks
> >> >
> >
> >
>
Re: Strange Data distribution in 0.4.2
Posted by Jonathan Ellis <jb...@gmail.com>.
for OPP, tokens are equivalent to keys so pick keys evenly spaced apart
On Tue, Nov 10, 2009 at 9:55 AM, Richard grossman <ri...@gmail.com> wrote:
> hi
>
> I've understand this but I don't know what to write into initialtoken is it
> "1" or "a" or something else ?
> as I've said in a previous post My keys are build like <data>:<data>:<data>
> Is there any link ?
>
> Thanks,
>
> On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> if you're not specifying initialtoken, every time you wipe your
>> installation it will generate new tokens. for a small number of
>> machines you'll definitely see some random tokens better balanced than
>> others.
>>
>> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman <ri...@gmail.com>
>> wrote:
>> > Hi
>> >
>> > I've build the 0.4.2 from the tags in SVN.
>> > I've made exactly the same cluster with same configuration as 0.4.1
>> > I've delete all the data in all server
>> >
>> > Now I send the data to first server and the data is not more distributed
>> > across the other server as previously.
>> > I've configured replica to 1
>> >
>> > here is my storage-conf.xml
>> >
>> > <Storage>
>> >
>> > <ClusterName>BeeCluster</ClusterName>
>> >
>> >
>> > <Keyspaces>
>> > <Keyspace Name="Keyspace1">
>> >
>> > <ColumnFamily CompareWith="BytesType"
>> > Name="channelShowLink"
>> > FlushPeriodInMinutes="15"/>
>> >
>> > <ColumnFamily CompareWith="BytesType"
>> > Name="channelShow"
>> > FlushPeriodInMinutes="15"/>
>> >
>> > <ColumnFamily CompareWith="BytesType"
>> > Name="userAction"
>> > FlushPeriodInMinutes="15"/>
>> >
>> > <ColumnFamily CompareWith="BytesType"
>> > Name="headends"
>> > FlushPeriodInMinutes="15"/>
>> >
>> > <ColumnFamily CompareWith="BytesType"
>> > Name="similarity"
>> > FlushPeriodInMinutes="500"/>
>> > </Keyspace>
>> > </Keyspaces>
>> >
>> >
>> >
>> >
>> > <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
>> >
>> > <InitialToken></InitialToken>
>> >
>> >
>> >
>> >
>> > <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>> >
>> >
>> >
>> >
>> > <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>> >
>> > <ReplicationFactor>1</ReplicationFactor>
>> >
>> >
>> >
>> >
>> > <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
>> > <DataFileDirectories>
>> >
>> > <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
>> > </DataFileDirectories>
>> >
>> > <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
>> >
>> >
>> > <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
>> >
>> >
>> > <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
>> >
>> >
>> >
>> > <Seeds>
>> > <Seed>192.168.249.200</Seed>
>> > <Seed>192.168.249.222</Seed>
>> > <Seed>192.168.249.95</Seed>
>> > </Seeds>
>> >
>> >
>> > <!-- Miscellaneous -->
>> >
>> > <!-- Time to wait for a reply from other nodes before failing the
>> > command
>> > -->
>> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
>> > <!-- Size to allow commitlog to grow to before creating a new segment
>> > -->
>> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>> >
>> >
>> >
>> > <ListenAddress>192.168.249.200</ListenAddress>
>> > <!-- TCP port, for commands and data -->
>> > <StoragePort>7000</StoragePort>
>> > <!-- UDP port, for membership communications (gossip) -->
>> > <ControlPort>7001</ControlPort>
>> >
>> >
>> > <ThriftAddress>0.0.0.0</ThriftAddress>
>> > <!-- Thrift RPC port (the port clients connect to). -->
>> > <ThriftPort>9160</ThriftPort>
>> > <!--
>> > ~ Whether or not to use a framed transport for Thrift. If this option
>> > ~ is set to true then you must also use a framed transport on the
>> > ~ client-side, (framed and non-framed transports are not compatible).
>> > -->
>> > <ThriftFramedTransport>false</ThriftFramedTransport>
>> >
>> >
>> >
>> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>> >
>> >
>> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
>> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>> >
>> >
>> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>> >
>> >
>> > <MemtableSizeInMB>64</MemtableSizeInMB>
>> >
>> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>> >
>> >
>> > <ConcurrentReads>8</ConcurrentReads>
>> > <ConcurrentWrites>32</ConcurrentWrites>
>> >
>> >
>> > <CommitLogSync>periodic</CommitLogSync>
>> >
>> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>> >
>> > <!-- <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS>
>> > -->
>> >
>> >
>> > <GCGraceSeconds>864000</GCGraceSeconds>
>> >
>> >
>> > <FlushMinThreads>1</FlushMinThreads>
>> > <FlushMaxThreads>1</FlushMaxThreads>
>> >
>> >
>> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>> >
>> > </Storage>
>> >
>> > Is there anyone with the same problem ??
>> >
>> > Thanks
>> >
>
>
Re: Strange Data distribution in 0.4.2
Posted by Richard grossman <ri...@gmail.com>.
hi
I've understand this but I don't know what to write into initialtoken is it
"1" or "a" or something else ?
as I've said in a previous post My keys are build like <data>:<data>:<data>
Is there any link ?
Thanks,
On Tue, Nov 10, 2009 at 5:48 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> if you're not specifying initialtoken, every time you wipe your
> installation it will generate new tokens. for a small number of
> machines you'll definitely see some random tokens better balanced than
> others.
>
> On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman <ri...@gmail.com>
> wrote:
> > Hi
> >
> > I've build the 0.4.2 from the tags in SVN.
> > I've made exactly the same cluster with same configuration as 0.4.1
> > I've delete all the data in all server
> >
> > Now I send the data to first server and the data is not more distributed
> > across the other server as previously.
> > I've configured replica to 1
> >
> > here is my storage-conf.xml
> >
> > <Storage>
> >
> > <ClusterName>BeeCluster</ClusterName>
> >
> >
> > <Keyspaces>
> > <Keyspace Name="Keyspace1">
> >
> > <ColumnFamily CompareWith="BytesType"
> > Name="channelShowLink"
> > FlushPeriodInMinutes="15"/>
> >
> > <ColumnFamily CompareWith="BytesType"
> > Name="channelShow"
> > FlushPeriodInMinutes="15"/>
> >
> > <ColumnFamily CompareWith="BytesType"
> > Name="userAction"
> > FlushPeriodInMinutes="15"/>
> >
> > <ColumnFamily CompareWith="BytesType"
> > Name="headends"
> > FlushPeriodInMinutes="15"/>
> >
> > <ColumnFamily CompareWith="BytesType"
> > Name="similarity"
> > FlushPeriodInMinutes="500"/>
> > </Keyspace>
> > </Keyspaces>
> >
> >
> >
> >
> <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
> >
> > <InitialToken></InitialToken>
> >
> >
> >
> >
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> >
> >
> >
> >
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
> >
> > <ReplicationFactor>1</ReplicationFactor>
> >
> >
> >
> >
> <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
> > <DataFileDirectories>
> >
> > <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
> > </DataFileDirectories>
> >
> <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
> >
> >
> <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
> >
> >
> <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
> >
> >
> >
> > <Seeds>
> > <Seed>192.168.249.200</Seed>
> > <Seed>192.168.249.222</Seed>
> > <Seed>192.168.249.95</Seed>
> > </Seeds>
> >
> >
> > <!-- Miscellaneous -->
> >
> > <!-- Time to wait for a reply from other nodes before failing the
> command
> > -->
> > <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
> > <!-- Size to allow commitlog to grow to before creating a new segment
> -->
> > <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
> >
> >
> >
> > <ListenAddress>192.168.249.200</ListenAddress>
> > <!-- TCP port, for commands and data -->
> > <StoragePort>7000</StoragePort>
> > <!-- UDP port, for membership communications (gossip) -->
> > <ControlPort>7001</ControlPort>
> >
> >
> > <ThriftAddress>0.0.0.0</ThriftAddress>
> > <!-- Thrift RPC port (the port clients connect to). -->
> > <ThriftPort>9160</ThriftPort>
> > <!--
> > ~ Whether or not to use a framed transport for Thrift. If this option
> > ~ is set to true then you must also use a framed transport on the
> > ~ client-side, (framed and non-framed transports are not compatible).
> > -->
> > <ThriftFramedTransport>false</ThriftFramedTransport>
> >
> >
> >
> > <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
> >
> >
> > <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
> > <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
> >
> >
> > <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
> >
> >
> > <MemtableSizeInMB>64</MemtableSizeInMB>
> >
> > <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
> >
> >
> > <ConcurrentReads>8</ConcurrentReads>
> > <ConcurrentWrites>32</ConcurrentWrites>
> >
> >
> > <CommitLogSync>periodic</CommitLogSync>
> >
> > <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
> >
> > <!-- <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS> -->
> >
> >
> > <GCGraceSeconds>864000</GCGraceSeconds>
> >
> >
> > <FlushMinThreads>1</FlushMinThreads>
> > <FlushMaxThreads>1</FlushMaxThreads>
> >
> >
> > <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
> >
> > </Storage>
> >
> > Is there anyone with the same problem ??
> >
> > Thanks
> >
>
Re: Strange Data distribution in 0.4.2
Posted by Jonathan Ellis <jb...@gmail.com>.
if you're not specifying initialtoken, every time you wipe your
installation it will generate new tokens. for a small number of
machines you'll definitely see some random tokens better balanced than
others.
On Tue, Nov 10, 2009 at 9:38 AM, Richard grossman <ri...@gmail.com> wrote:
> Hi
>
> I've build the 0.4.2 from the tags in SVN.
> I've made exactly the same cluster with same configuration as 0.4.1
> I've delete all the data in all server
>
> Now I send the data to first server and the data is not more distributed
> across the other server as previously.
> I've configured replica to 1
>
> here is my storage-conf.xml
>
> <Storage>
>
> <ClusterName>BeeCluster</ClusterName>
>
>
> <Keyspaces>
> <Keyspace Name="Keyspace1">
>
> <ColumnFamily CompareWith="BytesType"
> Name="channelShowLink"
> FlushPeriodInMinutes="15"/>
>
> <ColumnFamily CompareWith="BytesType"
> Name="channelShow"
> FlushPeriodInMinutes="15"/>
>
> <ColumnFamily CompareWith="BytesType"
> Name="userAction"
> FlushPeriodInMinutes="15"/>
>
> <ColumnFamily CompareWith="BytesType"
> Name="headends"
> FlushPeriodInMinutes="15"/>
>
> <ColumnFamily CompareWith="BytesType"
> Name="similarity"
> FlushPeriodInMinutes="500"/>
> </Keyspace>
> </Keyspaces>
>
>
>
> <Partitioner>org.apache.cassandra.dht.OrderPreservingPartitioner</Partitioner>
>
> <InitialToken></InitialToken>
>
>
>
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>
>
>
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>
> <ReplicationFactor>1</ReplicationFactor>
>
>
>
> <CommitLogDirectory>/home/beecloud/cassandrapart/commitlog</CommitLogDirectory>
> <DataFileDirectories>
>
> <DataFileDirectory>/home/beecloud/cassandrapart/data</DataFileDirectory>
> </DataFileDirectories>
> <CalloutLocation>/home/beecloud/cassandrapart/callouts</CalloutLocation>
>
> <BootstrapFileDirectory>/home/beecloud/cassandrapart/bootstrap</BootstrapFileDirectory>
>
> <StagingFileDirectory>/home/beecloud/cassandrapart/staging</StagingFileDirectory>
>
>
>
> <Seeds>
> <Seed>192.168.249.200</Seed>
> <Seed>192.168.249.222</Seed>
> <Seed>192.168.249.95</Seed>
> </Seeds>
>
>
> <!-- Miscellaneous -->
>
> <!-- Time to wait for a reply from other nodes before failing the command
> -->
> <RpcTimeoutInMillis>50000</RpcTimeoutInMillis>
> <!-- Size to allow commitlog to grow to before creating a new segment -->
> <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>
>
>
>
> <ListenAddress>192.168.249.200</ListenAddress>
> <!-- TCP port, for commands and data -->
> <StoragePort>7000</StoragePort>
> <!-- UDP port, for membership communications (gossip) -->
> <ControlPort>7001</ControlPort>
>
>
> <ThriftAddress>0.0.0.0</ThriftAddress>
> <!-- Thrift RPC port (the port clients connect to). -->
> <ThriftPort>9160</ThriftPort>
> <!--
> ~ Whether or not to use a framed transport for Thrift. If this option
> ~ is set to true then you must also use a framed transport on the
> ~ client-side, (framed and non-framed transports are not compatible).
> -->
> <ThriftFramedTransport>false</ThriftFramedTransport>
>
>
>
> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>
>
> <FlushDataBufferSizeInMB>32</FlushDataBufferSizeInMB>
> <FlushIndexBufferSizeInMB>8</FlushIndexBufferSizeInMB>
>
>
> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>
>
> <MemtableSizeInMB>64</MemtableSizeInMB>
>
> <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
>
>
> <ConcurrentReads>8</ConcurrentReads>
> <ConcurrentWrites>32</ConcurrentWrites>
>
>
> <CommitLogSync>periodic</CommitLogSync>
>
> <CommitLogSyncPeriodInMS>1000</CommitLogSyncPeriodInMS>
>
> <!-- <CommitLogSyncBatchWindowInMS>1</CommitLogSyncBatchWindowInMS> -->
>
>
> <GCGraceSeconds>864000</GCGraceSeconds>
>
>
> <FlushMinThreads>1</FlushMinThreads>
> <FlushMaxThreads>1</FlushMaxThreads>
>
>
> <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>
>
> </Storage>
>
> Is there anyone with the same problem ??
>
> Thanks
>