You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by vasu gunja <va...@gmail.com> on 2017/06/07 02:50:38 UTC

Local_serial >> Adding nodes

Hi All,

We are having 2 DC setup each consists of 20 odd nodes and recently we
decided to add 6 more nodes to DC1.  We are using LWT's,  application
dirvers are configuared to use LOCAL_SERIAL.
As we are adding multiple nodes at a time we used option
"-Dcassandra.consistent.rangemovement=false"
we added all nodes with gap of 10 mins each

We are facing lot of timeouts more 30k transactions over 8 hours of period
. is anyone ran into same issue ? are we doing something.wrong ?



Thanks,
vasu

RE: Local_serial >> Adding nodes

Posted by "ZAIDI, ASAD A" <az...@att.com>.
Please share exact timeout error message text to get idea what type of timeout you're facing.


From: Nitan Kainth [mailto:nitan@bamlabs.com]
Sent: Wednesday, June 07, 2017 7:24 AM
To: vasu gunja <va...@gmail.com>
Cc: user@cassandra.apache.org
Subject: Re: Local_serial >> Adding nodes

What is in system log?
Does it show new tokens allocated?
nodetool netstats

On Jun 7, 2017, at 6:50 AM, vasu gunja <va...@gmail.com>> wrote:

Yeah all of them are in UJ state..

Thanks,
Vasu

On Jun 7, 2017, at 6:32 AM, Nitan Kainth <ni...@bamlabs.com>> wrote:
Is the streaming still going on?
Timeouts are in reads or writes?

Sent from my iPhone

On Jun 6, 2017, at 9:50 PM, vasu gunja <va...@gmail.com>> wrote:
Hi All,

We are having 2 DC setup each consists of 20 odd nodes and recently we decided to add 6 more nodes to DC1.  We are using LWT's,  application dirvers are configuared to use LOCAL_SERIAL.
As we are adding multiple nodes at a time we used option  "-Dcassandra.consistent.rangemovement=false" we added all nodes with gap of 10 mins each

We are facing lot of timeouts more 30k transactions over 8 hours of period . is anyone ran into same issue ? are we doing something.wrong ?



Thanks,
vasu



Re: Local_serial >> Adding nodes

Posted by Nitan Kainth <ni...@bamlabs.com>.
What is in system log? 
Does it show new tokens allocated?
nodetool netstats

> On Jun 7, 2017, at 6:50 AM, vasu gunja <va...@gmail.com> wrote:
> 
> Yeah all of them are in UJ state..
> 
> Thanks,
> Vasu
> 
> On Jun 7, 2017, at 6:32 AM, Nitan Kainth <nitan@bamlabs.com <ma...@bamlabs.com>> wrote:
> 
>> Is the streaming still going on?
>> Timeouts are in reads or writes?
>> 
>> Sent from my iPhone
>> 
>> On Jun 6, 2017, at 9:50 PM, vasu gunja <vasu.nosql@gmail.com <ma...@gmail.com>> wrote:
>> 
>>> Hi All,
>>> 
>>> We are having 2 DC setup each consists of 20 odd nodes and recently we decided to add 6 more nodes to DC1.  We are using LWT's,  application dirvers are configuared to use LOCAL_SERIAL.
>>> As we are adding multiple nodes at a time we used option  "-Dcassandra.consistent.rangemovement=false" we added all nodes with gap of 10 mins each
>>> 
>>> We are facing lot of timeouts more 30k transactions over 8 hours of period . is anyone ran into same issue ? are we doing something.wrong ? 
>>> 
>>> 
>>> 
>>> Thanks,
>>> vasu
>>> 


Re: Local_serial >> Adding nodes

Posted by vasu gunja <va...@gmail.com>.
Yeah all of them are in UJ state..

Thanks,
Vasu

> On Jun 7, 2017, at 6:32 AM, Nitan Kainth <ni...@bamlabs.com> wrote:
> 
> Is the streaming still going on?
> Timeouts are in reads or writes?
> 
> Sent from my iPhone
> 
>> On Jun 6, 2017, at 9:50 PM, vasu gunja <va...@gmail.com> wrote:
>> 
>> Hi All,
>> 
>> We are having 2 DC setup each consists of 20 odd nodes and recently we decided to add 6 more nodes to DC1.  We are using LWT's,  application dirvers are configuared to use LOCAL_SERIAL.
>> As we are adding multiple nodes at a time we used option  "-Dcassandra.consistent.rangemovement=false" we added all nodes with gap of 10 mins each
>> 
>> We are facing lot of timeouts more 30k transactions over 8 hours of period . is anyone ran into same issue ? are we doing something.wrong ? 
>> 
>> 
>> 
>> Thanks,
>> vasu
>> 

Re: Local_serial >> Adding nodes

Posted by Nitan Kainth <ni...@bamlabs.com>.
Is the streaming still going on?
Timeouts are in reads or writes?

Sent from my iPhone

> On Jun 6, 2017, at 9:50 PM, vasu gunja <va...@gmail.com> wrote:
> 
> Hi All,
> 
> We are having 2 DC setup each consists of 20 odd nodes and recently we decided to add 6 more nodes to DC1.  We are using LWT's,  application dirvers are configuared to use LOCAL_SERIAL.
> As we are adding multiple nodes at a time we used option  "-Dcassandra.consistent.rangemovement=false" we added all nodes with gap of 10 mins each
> 
> We are facing lot of timeouts more 30k transactions over 8 hours of period . is anyone ran into same issue ? are we doing something.wrong ? 
> 
> 
> 
> Thanks,
> vasu
>