You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Niteesh kumar <ni...@directi.com> on 2012/10/02 11:31:47 UTC
Persistent connection among nodes to communicate and redirect request
while looking at netstat table i observed that my cluster nodes not
using persistent connection to talk among themselves on port 9160 to
redirect request. I also observed that local write latency is around
30-40 microsecond, while its takes around .5 miliseconds if the chosen
node is not the node responsible for the key for 50K QPS. I think this
attributes to connection making time among servers as my servers are on
same rack.
how can i configure my servers to use persistent connection on port 9160
thus exclude connection making time for each request that is redirected...
Re: Persistent connection among nodes to communicate and redirect
request
Posted by Niteesh kumar <ni...@directi.com>.
m sry, all connection among servers on port 9160 was some another
application... thanx for support
regard
Niteesh
On Wednesday 03 October 2012 02:59 PM, mdione.ext@orange.com wrote:
> De : Hiller, Dean [mailto:Dean.Hiller@nrel.gov]
>> Can you just use netstat and dig into the process id and do a ps -ef |
>> grep <pid> to clear up all the confusion.
> A better way is to use:
>
> sudo netstat --tcp --programs
>
> Optionally, add --numeric to avoid hostname and port resolution.
>
> --
> Marcos Dione
> SysAdmin
> Astek Sud-Est
> pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo
> 04 97 12 62 45 - mdione.ext@orange.com
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
>
RE: Persistent connection among nodes to communicate and redirect
request
Posted by md...@orange.com.
De : Hiller, Dean [mailto:Dean.Hiller@nrel.gov]
> Can you just use netstat and dig into the process id and do a ps -ef |
> grep <pid> to clear up all the confusion.
A better way is to use:
sudo netstat --tcp --programs
Optionally, add --numeric to avoid hostname and port resolution.
--
Marcos Dione
SysAdmin
Astek Sud-Est
pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo
04 97 12 62 45 - mdione.ext@orange.com
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
Re: Persistent connection among nodes to communicate and redirect request
Posted by aaron morton <aa...@thelastpickle.com>.
May end up with a local 9160 connection if you are using cassandra-cli, cqlsh or hadoop on the node.
> I also observed that local write latency is around 30-40 microsecond, while its takes around .5 miliseconds if the chosen node is not the node responsible for the key for 50K QPS
What CL ?
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 3/10/2012, at 4:57 AM, "Hiller, Dean" <De...@nrel.gov> wrote:
> Can you just use netstat and dig into the process id and do a ps -ef |
> grep <pid> to clear up all the confusion. Doing so you can tell which
> process communicates with which process(I am assuming you are on linuxŠ.on
> MAC or windows it is different commands).
>
> Then, just paste all that in the email to this list so we can see it.
>
> MY GUESS is someone is running a tool that is talking thrift on your
> cassandra node???? Maybe another server like a web server?
>
> Later,
> Dean
>
> On 10/2/12 9:50 AM, "Nick Bailey" <ni...@datastax.com> wrote:
>
>> The comments here so far are correct. Cassandra itself will never open
>> a thrift connection. Thrift is only for clients. Not sure what exactly
>> you are seeing but I don't think it's cassandra.
>>
>> On Tue, Oct 2, 2012 at 10:48 AM, Niteesh kumar <ni...@directi.com>
>> wrote:
>>> not only a node make connection to other nodes. i can also see nodes
>>> making
>>> connection to itself on port 9160.
>>>
>>>
>>>
>>> On Tuesday 02 October 2012 07:42 PM, Viktor Jevdokimov wrote:
>>>>
>>>> not a thrift por
>>>
>>>
>
Re: Persistent connection among nodes to communicate and redirect
request
Posted by "Hiller, Dean" <De...@nrel.gov>.
Can you just use netstat and dig into the process id and do a ps -ef |
grep <pid> to clear up all the confusion. Doing so you can tell which
process communicates with which process(I am assuming you are on linuxŠ.on
MAC or windows it is different commands).
Then, just paste all that in the email to this list so we can see it.
MY GUESS is someone is running a tool that is talking thrift on your
cassandra node???? Maybe another server like a web server?
Later,
Dean
On 10/2/12 9:50 AM, "Nick Bailey" <ni...@datastax.com> wrote:
>The comments here so far are correct. Cassandra itself will never open
>a thrift connection. Thrift is only for clients. Not sure what exactly
>you are seeing but I don't think it's cassandra.
>
>On Tue, Oct 2, 2012 at 10:48 AM, Niteesh kumar <ni...@directi.com>
>wrote:
>> not only a node make connection to other nodes. i can also see nodes
>>making
>> connection to itself on port 9160.
>>
>>
>>
>> On Tuesday 02 October 2012 07:42 PM, Viktor Jevdokimov wrote:
>>>
>>> not a thrift por
>>
>>
Re: Persistent connection among nodes to communicate and redirect request
Posted by Nick Bailey <ni...@datastax.com>.
The comments here so far are correct. Cassandra itself will never open
a thrift connection. Thrift is only for clients. Not sure what exactly
you are seeing but I don't think it's cassandra.
On Tue, Oct 2, 2012 at 10:48 AM, Niteesh kumar <ni...@directi.com> wrote:
> not only a node make connection to other nodes. i can also see nodes making
> connection to itself on port 9160.
>
>
>
> On Tuesday 02 October 2012 07:42 PM, Viktor Jevdokimov wrote:
>>
>> not a thrift por
>
>
Re: Persistent connection among nodes to communicate and redirect
request
Posted by Niteesh kumar <ni...@directi.com>.
not only a node make connection to other nodes. i can also see nodes
making connection to itself on port 9160.
On Tuesday 02 October 2012 07:42 PM, Viktor Jevdokimov wrote:
> not a thrift por
RE: Persistent connection among nodes to communicate and redirect
request
Posted by Viktor Jevdokimov <Vi...@adform.com>.
Never seen connections between nodes on 9160 port, 7000 only.
>From the source code, for example, thrift request goes to rpc port 9160 (org.apache.cassandra.thrift.CassandraDaemon, org.apache.cassandra.thrift.CassandraServer), then to StorageProxy (org.apache.cassandra.service.StorageProxy), which forward request (if needed) to other endpoints via MessagingService (org.apache.cassandra.net.MessagingService), which uses storage_port from yaml, not a thrift port (rpc_port in yaml). What else could be wrong? Wiki or source code?
Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer
Email: Viktor.Jevdokimov@adform.com
Phone: +370 5 212 3063
Fax: +370 5 261 0453
J. Jasinskio 16C,
LT-01112 Vilnius,
Lithuania
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.> -----Original Message-----
> From: rohit bhatia [mailto:rohit2412@gmail.com]
> Sent: Tuesday, October 02, 2012 14:35
> To: user@cassandra.apache.org
> Subject: Re: Persistent connection among nodes to communicate and
> redirect request
>
> i guess 7000 is only for gossip protocol. Cassandra still uses 9160 for RPC even
> among nodes....
> Also, I see Connections over port 9160 among various cassandra Nodes in my
> cluster.
> Please correct me if i am wrong..
>
> PS: mentioned Here http://wiki.apache.org/cassandra/CloudConfig
>
> On Tue, Oct 2, 2012 at 4:56 PM, Viktor Jevdokimov
> <Vi...@adform.com> wrote:
> > 9160 is a client port. Nodes are using messaging service on storage_port
> (7000) for intra-node communication.
> >
> >
> > Best regards / Pagarbiai
> >
> > Viktor Jevdokimov
> > Senior Developer
> >
> > Email: Viktor.Jevdokimov@adform.com
> > Phone: +370 5 212 3063
> > Fax: +370 5 261 0453
> >
> > J. Jasinskio 16C,
> > LT-01112 Vilnius,
> > Lithuania
> >
> >
> >
> > Disclaimer: The information contained in this message and attachments
> > is intended solely for the attention and use of the named addressee
> > and may be confidential. If you are not the intended recipient, you
> > are reminded that the information remains the property of the sender.
> > You must not use, disclose, distribute, copy, print or rely on this
> > e-mail. If you have received this message in error, please contact the
> > sender immediately and irrevocably delete this message and any
> > copies.> -----Original Message-----
> >> From: Niteesh kumar [mailto:niteesh.k@directi.com]
> >> Sent: Tuesday, October 02, 2012 12:32
> >> To: user@cassandra.apache.org
> >> Subject: Persistent connection among nodes to communicate and
> >> redirect request
> >>
> >> while looking at netstat table i observed that my cluster nodes not
> >> using persistent connection to talk among themselves on port 9160 to
> >> redirect request. I also observed that local write latency is around
> >> 30-40 microsecond, while its takes around .5 miliseconds if the
> >> chosen node is not the node responsible for the key for 50K QPS. I
> >> think this attributes to connection making time among servers as my
> servers are on same rack.
> >>
> >> how can i configure my servers to use persistent connection on port
> >> 9160 thus exclude connection making time for each request that is
> redirected...
Re: Persistent connection among nodes to communicate and redirect request
Posted by rohit bhatia <ro...@gmail.com>.
i guess 7000 is only for gossip protocol. Cassandra still uses 9160
for RPC even among nodes....
Also, I see Connections over port 9160 among various cassandra Nodes
in my cluster.
Please correct me if i am wrong..
PS: mentioned Here http://wiki.apache.org/cassandra/CloudConfig
On Tue, Oct 2, 2012 at 4:56 PM, Viktor Jevdokimov
<Vi...@adform.com> wrote:
> 9160 is a client port. Nodes are using messaging service on storage_port (7000) for intra-node communication.
>
>
> Best regards / Pagarbiai
>
> Viktor Jevdokimov
> Senior Developer
>
> Email: Viktor.Jevdokimov@adform.com
> Phone: +370 5 212 3063
> Fax: +370 5 261 0453
>
> J. Jasinskio 16C,
> LT-01112 Vilnius,
> Lithuania
>
>
>
> Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.> -----Original Message-----
>> From: Niteesh kumar [mailto:niteesh.k@directi.com]
>> Sent: Tuesday, October 02, 2012 12:32
>> To: user@cassandra.apache.org
>> Subject: Persistent connection among nodes to communicate and redirect
>> request
>>
>> while looking at netstat table i observed that my cluster nodes not using
>> persistent connection to talk among themselves on port 9160 to redirect
>> request. I also observed that local write latency is around
>> 30-40 microsecond, while its takes around .5 miliseconds if the chosen node
>> is not the node responsible for the key for 50K QPS. I think this attributes to
>> connection making time among servers as my servers are on same rack.
>>
>> how can i configure my servers to use persistent connection on port 9160
>> thus exclude connection making time for each request that is redirected...
RE: Persistent connection among nodes to communicate and redirect
request
Posted by Viktor Jevdokimov <Vi...@adform.com>.
9160 is a client port. Nodes are using messaging service on storage_port (7000) for intra-node communication.
Best regards / Pagarbiai
Viktor Jevdokimov
Senior Developer
Email: Viktor.Jevdokimov@adform.com
Phone: +370 5 212 3063
Fax: +370 5 261 0453
J. Jasinskio 16C,
LT-01112 Vilnius,
Lithuania
Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.> -----Original Message-----
> From: Niteesh kumar [mailto:niteesh.k@directi.com]
> Sent: Tuesday, October 02, 2012 12:32
> To: user@cassandra.apache.org
> Subject: Persistent connection among nodes to communicate and redirect
> request
>
> while looking at netstat table i observed that my cluster nodes not using
> persistent connection to talk among themselves on port 9160 to redirect
> request. I also observed that local write latency is around
> 30-40 microsecond, while its takes around .5 miliseconds if the chosen node
> is not the node responsible for the key for 50K QPS. I think this attributes to
> connection making time among servers as my servers are on same rack.
>
> how can i configure my servers to use persistent connection on port 9160
> thus exclude connection making time for each request that is redirected...