You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@thrift.apache.org by Dinesh Pathak <di...@gmail.com> on 2011/06/15 11:43:38 UTC
Closing Non Blocking socket in Async Thrift Client
Hi,
I am new to these mailing lists, so please let me know if this is the
right place to ask Thrift related questions. I am writing Async Thrift
Client, using Non Blocking Socket. When the client is sending a lot of
requests to server, then it dies out saying "too many open files." It seems
like AsyncClientManager does not close the socket. Is there any way for the
client to close socket in onComplete of callback. Also, would like to
confirm that client can send only one service at a time. Thanks in advance.
RE: Closing Non Blocking socket in Async Thrift Client
Posted by Surbhi Bhasin <su...@komli.com>.
This problem is there in sync version also
-----Original Message-----
From: Dinesh Pathak [mailto:dinesh.pathak@gmail.com]
Sent: Wednesday, June 15, 2011 3:14 PM
To: user@thrift.apache.org
Subject: Closing Non Blocking socket in Async Thrift Client
Hi,
I am new to these mailing lists, so please let me know if this is the
right place to ask Thrift related questions. I am writing Async Thrift
Client, using Non Blocking Socket. When the client is sending a lot of
requests to server, then it dies out saying "too many open files." It
seems like AsyncClientManager does not close the socket. Is there any
way for the client to close socket in onComplete of callback. Also,
would like to confirm that client can send only one service at a time.
Thanks in advance.
Re: How to do AUTHENTICATION with thift
Posted by Rory McGuire <rm...@clearformat.com>.
Assuming that all clients speak proper Thrift protocol to your server you
will be able to have a authentication mechanism as long as you define your
own.
What I was saying is that if you are trying to make your server secure you
need to check if thrift still has the same problem where it crashes if you
send data that is not formatted in valid Thrift protocol. If it does still
crash your server then you'll need to think of some way around this.
I believe most people use Thrift in private networks so they do not need
to be as concerned about client validation.
-Rory
PS: By bogus I just meant that the data is not valid in the context
On Fri, 17 Jun 2011 15:22:48 +0200, Ma Chao <ch...@yunrang.com> wrote:
> So you mean bogus could achieve this:
> Our requirement is:
> Assume there are two clients called CLIENT-A and CLIENT-B and a server
> SERVER.
>
> Now, we hope only CLIENT-A which registered before in SERVER could
> invoke the RPC server (SERVER). CLIENT-B can't.
>
> Could bogus achieve this?
>
> Thanks,
>
> -Chao
>
> On Fri, 2011-06-17 at 09:03 +0200, Rory McGuire wrote:
>> Thrift also used to be easy to break the server by sending bogus data
>> to
>> it so you might want to check if thats
>> changed.
>> Having authentication doesn't mean much if just anyone can crash it
>> with
>> telnet.
>>
>> Enjoy
>> -Rory
>>
>> On Fri, 17 Jun 2011 03:55:51 +0200, Ma Chao <ch...@yunrang.com>
>> wrote:
>>
>> > OK. I see. I will try to modifying our interface. Thank you very
>> much~
>> > On Thu, 2011-06-16 at 09:30 -0700, Bryan Duxbury wrote:
>> >> Thrift does not have authentication support. You need to add it
>> into
>> >> your
>> >> own interfaces manually.
>> >>
>> >> On Wed, Jun 15, 2011 at 6:47 PM, Ma Chao <ch...@yunrang.com>
>> wrote:
>> >>
>> >> > Hi guys,
>> >> >
>> >> > This is Chao Ma, a engineer in a small company and using thrift
>> as our
>> >> > RPC generator.
>> >> >
>> >> > We want to give authentication to the RPC client to invoke our
>> RPC
>> >> > server. That means only some authenticated clients can invoke our
>> RPC
>> >> > server.
>> >> > We don't want to change the interface of RPC servers/clients what
>> we
>> >> > have. So the best thing is thrift supports it natively.
>> >> >
>> >> > I don't know how to achieve this with thrift. Could you guys help
>> me?
>> >> > Any solution and idea is appreciated :-)
>> >> >
>> >> > Thank you very much!
>> >> >
>> >> > Chao.
>> >> >
>> >> >
Re: How to do AUTHENTICATION with thift
Posted by Ma Chao <ch...@yunrang.com>.
So you mean bogus could achieve this:
Our requirement is:
Assume there are two clients called CLIENT-A and CLIENT-B and a server
SERVER.
Now, we hope only CLIENT-A which registered before in SERVER could
invoke the RPC server (SERVER). CLIENT-B can't.
Could bogus achieve this?
Thanks,
-Chao
On Fri, 2011-06-17 at 09:03 +0200, Rory McGuire wrote:
> Thrift also used to be easy to break the server by sending bogus data
> to
> it so you might want to check if thats
> changed.
> Having authentication doesn't mean much if just anyone can crash it
> with
> telnet.
>
> Enjoy
> -Rory
>
> On Fri, 17 Jun 2011 03:55:51 +0200, Ma Chao <ch...@yunrang.com>
> wrote:
>
> > OK. I see. I will try to modifying our interface. Thank you very
> much~
> > On Thu, 2011-06-16 at 09:30 -0700, Bryan Duxbury wrote:
> >> Thrift does not have authentication support. You need to add it
> into
> >> your
> >> own interfaces manually.
> >>
> >> On Wed, Jun 15, 2011 at 6:47 PM, Ma Chao <ch...@yunrang.com>
> wrote:
> >>
> >> > Hi guys,
> >> >
> >> > This is Chao Ma, a engineer in a small company and using thrift
> as our
> >> > RPC generator.
> >> >
> >> > We want to give authentication to the RPC client to invoke our
> RPC
> >> > server. That means only some authenticated clients can invoke our
> RPC
> >> > server.
> >> > We don't want to change the interface of RPC servers/clients what
> we
> >> > have. So the best thing is thrift supports it natively.
> >> >
> >> > I don't know how to achieve this with thrift. Could you guys help
> me?
> >> > Any solution and idea is appreciated :-)
> >> >
> >> > Thank you very much!
> >> >
> >> > Chao.
> >> >
> >> >
Re: How to do AUTHENTICATION with thift
Posted by Rory McGuire <rm...@clearformat.com>.
Thrift also used to be easy to break the server by sending bogus data to
it so you might want to check if thats
changed.
Having authentication doesn't mean much if just anyone can crash it with
telnet.
Enjoy
-Rory
On Fri, 17 Jun 2011 03:55:51 +0200, Ma Chao <ch...@yunrang.com> wrote:
> OK. I see. I will try to modifying our interface. Thank you very much~
> On Thu, 2011-06-16 at 09:30 -0700, Bryan Duxbury wrote:
>> Thrift does not have authentication support. You need to add it intoyour
>> own interfaces manually.
>>
>> On Wed, Jun 15, 2011 at 6:47 PM, Ma Chao <ch...@yunrang.com> wrote:
>>
>> > Hi guys,
>> >
>> > This is Chao Ma, a engineer in a small company and using thrift as our
>> > RPC generator.
>> >
>> > We want to give authentication to the RPC client to invoke our RPC
>> > server. That means only some authenticated clients can invoke our RPC
>> > server.
>> > We don't want to change the interface of RPC servers/clients what we
>> > have. So the best thing is thrift supports it natively.
>> >
>> > I don't know how to achieve this with thrift. Could you guys help me?
>> > Any solution and idea is appreciated :-)
>> >
>> > Thank you very much!
>> >
>> > Chao.
>> >
>> >
Re: How to do AUTHENTICATION with thift
Posted by Ma Chao <ch...@yunrang.com>.
OK. I see. I will try to modifying our interface. Thank you very much~
On Thu, 2011-06-16 at 09:30 -0700, Bryan Duxbury wrote:
> Thrift does not have authentication support. You need to add it into your
> own interfaces manually.
>
> On Wed, Jun 15, 2011 at 6:47 PM, Ma Chao <ch...@yunrang.com> wrote:
>
> > Hi guys,
> >
> > This is Chao Ma, a engineer in a small company and using thrift as our
> > RPC generator.
> >
> > We want to give authentication to the RPC client to invoke our RPC
> > server. That means only some authenticated clients can invoke our RPC
> > server.
> > We don't want to change the interface of RPC servers/clients what we
> > have. So the best thing is thrift supports it natively.
> >
> > I don't know how to achieve this with thrift. Could you guys help me?
> > Any solution and idea is appreciated :-)
> >
> > Thank you very much!
> >
> > Chao.
> >
> >
Re: How to do AUTHENTICATION with thift
Posted by Bryan Duxbury <br...@rapleaf.com>.
Thrift does not have authentication support. You need to add it into your
own interfaces manually.
On Wed, Jun 15, 2011 at 6:47 PM, Ma Chao <ch...@yunrang.com> wrote:
> Hi guys,
>
> This is Chao Ma, a engineer in a small company and using thrift as our
> RPC generator.
>
> We want to give authentication to the RPC client to invoke our RPC
> server. That means only some authenticated clients can invoke our RPC
> server.
> We don't want to change the interface of RPC servers/clients what we
> have. So the best thing is thrift supports it natively.
>
> I don't know how to achieve this with thrift. Could you guys help me?
> Any solution and idea is appreciated :-)
>
> Thank you very much!
>
> Chao.
>
>
How to do AUTHENTICATION with thift
Posted by Ma Chao <ch...@yunrang.com>.
Hi guys,
This is Chao Ma, a engineer in a small company and using thrift as our
RPC generator.
We want to give authentication to the RPC client to invoke our RPC
server. That means only some authenticated clients can invoke our RPC
server.
We don't want to change the interface of RPC servers/clients what we
have. So the best thing is thrift supports it natively.
I don't know how to achieve this with thrift. Could you guys help me?
Any solution and idea is appreciated :-)
Thank you very much!
Chao.
Re: Closing Non Blocking socket in Async Thrift Client
Posted by Bryan Duxbury <br...@rapleaf.com>.
Yes, I am suggesting socket pooling.
I'd accept a patch that made the transport available through the existing
interfaces.
On Thu, Jun 16, 2011 at 4:30 AM, Dinesh Pathak <di...@gmail.com>wrote:
> Thanks Bryan. All the clients are sending requests to the server
> simultaneously. Are you talking about socket pooling between clients?
> Regarding implementing my own scheme, my Async Client, which is the class
> generated from .thrift file, cant access transport, because both
> TAsyncClient and TAsyncMethodCall have these parameters as protected. Now,
> I
> am creating 'final' objects in the callback interface, but I was expecting
> to get it from the generated Thrift classes.
>
> On Thu, Jun 16, 2011 at 10:16 AM, Surbhi Bhasin <surbhi.bhasin@komli.com
> >wrote:
>
> > I tried this option also, but even this is not working. Giving:
> > apr_socket_recv: Connection reset by peer (104)
> >
> > -----Original Message-----
> > From: Bryan Duxbury [mailto:bryan@rapleaf.com]
> > Sent: Wednesday, June 15, 2011 10:12 PM
> > To: user@thrift.apache.org
> > Subject: Re: Closing Non Blocking socket in Async Thrift Client
> >
> > You're not supposed to close your sockets after every request. You
> > should reuse them. Reusing them will have a significantly positive
> > effect on performance.
> >
> > If you really want to close after every request, then it's up to you to
> > implement a scheme that allows the onComplete method to close the
> > associated transport.
> >
> > On Wed, Jun 15, 2011 at 2:43 AM, Dinesh Pathak
> > <di...@gmail.com>wrote:
> >
> > > Hi,
> > > I am new to these mailing lists, so please let me know if this is
> > > the right place to ask Thrift related questions. I am writing Async
> > > Thrift Client, using Non Blocking Socket. When the client is sending
> > > a lot of requests to server, then it dies out saying "too many open
> > > files." It seems like AsyncClientManager does not close the socket.
> > > Is there any way for the client to close socket in onComplete of
> > > callback. Also, would like to confirm that client can send only one
> > > service at a time. Thanks in advance.
> > >
> >
>
Re: Closing Non Blocking socket in Async Thrift Client
Posted by Dinesh Pathak <di...@gmail.com>.
Thanks Bryan. All the clients are sending requests to the server
simultaneously. Are you talking about socket pooling between clients?
Regarding implementing my own scheme, my Async Client, which is the class
generated from .thrift file, cant access transport, because both
TAsyncClient and TAsyncMethodCall have these parameters as protected. Now, I
am creating 'final' objects in the callback interface, but I was expecting
to get it from the generated Thrift classes.
On Thu, Jun 16, 2011 at 10:16 AM, Surbhi Bhasin <su...@komli.com>wrote:
> I tried this option also, but even this is not working. Giving:
> apr_socket_recv: Connection reset by peer (104)
>
> -----Original Message-----
> From: Bryan Duxbury [mailto:bryan@rapleaf.com]
> Sent: Wednesday, June 15, 2011 10:12 PM
> To: user@thrift.apache.org
> Subject: Re: Closing Non Blocking socket in Async Thrift Client
>
> You're not supposed to close your sockets after every request. You
> should reuse them. Reusing them will have a significantly positive
> effect on performance.
>
> If you really want to close after every request, then it's up to you to
> implement a scheme that allows the onComplete method to close the
> associated transport.
>
> On Wed, Jun 15, 2011 at 2:43 AM, Dinesh Pathak
> <di...@gmail.com>wrote:
>
> > Hi,
> > I am new to these mailing lists, so please let me know if this is
> > the right place to ask Thrift related questions. I am writing Async
> > Thrift Client, using Non Blocking Socket. When the client is sending
> > a lot of requests to server, then it dies out saying "too many open
> > files." It seems like AsyncClientManager does not close the socket.
> > Is there any way for the client to close socket in onComplete of
> > callback. Also, would like to confirm that client can send only one
> > service at a time. Thanks in advance.
> >
>
RE: Closing Non Blocking socket in Async Thrift Client
Posted by Surbhi Bhasin <su...@komli.com>.
I tried this option also, but even this is not working. Giving:
apr_socket_recv: Connection reset by peer (104)
-----Original Message-----
From: Bryan Duxbury [mailto:bryan@rapleaf.com]
Sent: Wednesday, June 15, 2011 10:12 PM
To: user@thrift.apache.org
Subject: Re: Closing Non Blocking socket in Async Thrift Client
You're not supposed to close your sockets after every request. You
should reuse them. Reusing them will have a significantly positive
effect on performance.
If you really want to close after every request, then it's up to you to
implement a scheme that allows the onComplete method to close the
associated transport.
On Wed, Jun 15, 2011 at 2:43 AM, Dinesh Pathak
<di...@gmail.com>wrote:
> Hi,
> I am new to these mailing lists, so please let me know if this is
> the right place to ask Thrift related questions. I am writing Async
> Thrift Client, using Non Blocking Socket. When the client is sending
> a lot of requests to server, then it dies out saying "too many open
> files." It seems like AsyncClientManager does not close the socket.
> Is there any way for the client to close socket in onComplete of
> callback. Also, would like to confirm that client can send only one
> service at a time. Thanks in advance.
>
Re: Closing Non Blocking socket in Async Thrift Client
Posted by Bryan Duxbury <br...@rapleaf.com>.
You're not supposed to close your sockets after every request. You should
reuse them. Reusing them will have a significantly positive effect on
performance.
If you really want to close after every request, then it's up to you to
implement a scheme that allows the onComplete method to close the associated
transport.
On Wed, Jun 15, 2011 at 2:43 AM, Dinesh Pathak <di...@gmail.com>wrote:
> Hi,
> I am new to these mailing lists, so please let me know if this is the
> right place to ask Thrift related questions. I am writing Async Thrift
> Client, using Non Blocking Socket. When the client is sending a lot of
> requests to server, then it dies out saying "too many open files." It
> seems
> like AsyncClientManager does not close the socket. Is there any way for the
> client to close socket in onComplete of callback. Also, would like to
> confirm that client can send only one service at a time. Thanks in advance.
>