You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Jens Geyer <je...@hotmail.com> on 2014/05/31 12:54:37 UTC

Re: Multithreaded servers in Apache Thrift for Go

Hi Sergey,

implying your agreement ;-) I’m moving this topic onto the dev mailing list as this seems a better place to discuss this. 

The Go language part has been gone through some major updates in the last monts, especially thanks to the massive contributions from Travis, Ben and Aleksey. I think everybody will agree when I say, that the Go language part it is in a much better shape now than it was a year ago. That said, there is surely more work needed to add some still missing pieces, or make some stuff better that we are not fully satisfied with yet. 

So the answer is twofold: 
- Yes, there is only the simple server (yet), 
- and yes, we are of course accepting quality contributions. 

Could that be an option for you?

Thanks + have fun,
JensG




From: Sergey Serebryakov 
Sent: Saturday, May 31, 2014 5:25 AM
To: jensg@apache.org 
Subject: Multithreaded servers in Apache Thrift for Go

Hi Jens, 

I'm trying out various languages to implement a simple IP lookup Thrift-based server in. So far I've tried official library for Java and Twitter's Scrooge+Finagle for Scala. I'm trying Go now, and I've noticed that the official Thrift library for Go only contains SimpleServer implementation. It works alright, but it's purpose is just testing and I need something more performant and mature. In Java, I've used TThreadPoolServer and was satisfied with its performance. Is there anything similar for Go, perhaps making use of goroutines? 
Thank you!

Best regards, 
Sergey Serebryakov

Re: Multithreaded servers in Apache Thrift for Go

Posted by Aleksey Pesternikov <ap...@gmail.com>.
Sergey,
Which version are you using? If it is not current most recent version from
the source repository you might want to try it.
Second, how are you running your go application? Most probably you are not
changing GOMAXPROCS which means your application is using 1 CPU core.
If you are ready to share some contained source code illustrating the
problem I can take a look.


On Mon, Jun 2, 2014 at 1:13 AM, Sergey Serebryakov <he...@gmail.com>
wrote:

> Thank you for your answers! I have compared TThreadPoolServer.java and
> simple_server.go and they indeed look similar, except that Go uses
> goroutines instead of a thread pool. I'm wondering about why does my Go
> server perform 2.5 to 3 times worse than the equivalent Java thread-pool
> server with no handling logic on my Mac OS X while being queried from a
> multithreaded Thrift client on the same machine (20 000 qps vs. 55 000
> qps). Can it be because of how goroutines are managed in Go implementation
> on Mac? I can prepare a minimal sample for you to look at.
> Thanks again!
>
> Best regards,
> Sergey Serebryakov
>
>
> On Sat, May 31, 2014 at 4:12 AM, Chris Bannister <c....@gmail.com>
> wrote:
>
> > The TSimpleServer in Go is making use of goroutines, as there inst really
> > any reason not to. Every request is handled in its own goroutine, unlike
> > Java there is no need to have a specific thread pool implementation to
> > handle requests, and requests are handled in the same manner that the
> > standard library http server works.
> >
> >
> > On 31 May 2014 11:54, Jens Geyer <je...@hotmail.com> wrote:
> >
> >> Hi Sergey,
> >>
> >> implying your agreement ;-) I’m moving this topic onto the dev mailing
> >> list as this seems a better place to discuss this.
> >>
> >> The Go language part has been gone through some major updates in the
> last
> >> monts, especially thanks to the massive contributions from Travis, Ben
> and
> >> Aleksey. I think everybody will agree when I say, that the Go language
> part
> >> it is in a much better shape now than it was a year ago. That said,
> there
> >> is surely more work needed to add some still missing pieces, or make
> some
> >> stuff better that we are not fully satisfied with yet.
> >>
> >> So the answer is twofold:
> >> - Yes, there is only the simple server (yet),
> >> - and yes, we are of course accepting quality contributions.
> >>
> >> Could that be an option for you?
> >>
> >> Thanks + have fun,
> >> JensG
> >>
> >>
> >>
> >>
> >> From: Sergey Serebryakov
> >> Sent: Saturday, May 31, 2014 5:25 AM
> >> To: jensg@apache.org
> >> Subject: Multithreaded servers in Apache Thrift for Go
> >>
> >> Hi Jens,
> >>
> >> I'm trying out various languages to implement a simple IP lookup
> >> Thrift-based server in. So far I've tried official library for Java and
> >> Twitter's Scrooge+Finagle for Scala. I'm trying Go now, and I've noticed
> >> that the official Thrift library for Go only contains SimpleServer
> >> implementation. It works alright, but it's purpose is just testing and I
> >> need something more performant and mature. In Java, I've used
> >> TThreadPoolServer and was satisfied with its performance. Is there
> anything
> >> similar for Go, perhaps making use of goroutines?
> >> Thank you!
> >>
> >> Best regards,
> >> Sergey Serebryakov
> >
> >
> >
>

Re: Multithreaded servers in Apache Thrift for Go

Posted by Sergey Serebryakov <he...@gmail.com>.
Thank you for your answers! I have compared TThreadPoolServer.java and
simple_server.go and they indeed look similar, except that Go uses
goroutines instead of a thread pool. I'm wondering about why does my Go
server perform 2.5 to 3 times worse than the equivalent Java thread-pool
server with no handling logic on my Mac OS X while being queried from a
multithreaded Thrift client on the same machine (20 000 qps vs. 55 000
qps). Can it be because of how goroutines are managed in Go implementation
on Mac? I can prepare a minimal sample for you to look at.
Thanks again!

Best regards,
Sergey Serebryakov


On Sat, May 31, 2014 at 4:12 AM, Chris Bannister <c....@gmail.com>
wrote:

> The TSimpleServer in Go is making use of goroutines, as there inst really
> any reason not to. Every request is handled in its own goroutine, unlike
> Java there is no need to have a specific thread pool implementation to
> handle requests, and requests are handled in the same manner that the
> standard library http server works.
>
>
> On 31 May 2014 11:54, Jens Geyer <je...@hotmail.com> wrote:
>
>> Hi Sergey,
>>
>> implying your agreement ;-) I’m moving this topic onto the dev mailing
>> list as this seems a better place to discuss this.
>>
>> The Go language part has been gone through some major updates in the last
>> monts, especially thanks to the massive contributions from Travis, Ben and
>> Aleksey. I think everybody will agree when I say, that the Go language part
>> it is in a much better shape now than it was a year ago. That said, there
>> is surely more work needed to add some still missing pieces, or make some
>> stuff better that we are not fully satisfied with yet.
>>
>> So the answer is twofold:
>> - Yes, there is only the simple server (yet),
>> - and yes, we are of course accepting quality contributions.
>>
>> Could that be an option for you?
>>
>> Thanks + have fun,
>> JensG
>>
>>
>>
>>
>> From: Sergey Serebryakov
>> Sent: Saturday, May 31, 2014 5:25 AM
>> To: jensg@apache.org
>> Subject: Multithreaded servers in Apache Thrift for Go
>>
>> Hi Jens,
>>
>> I'm trying out various languages to implement a simple IP lookup
>> Thrift-based server in. So far I've tried official library for Java and
>> Twitter's Scrooge+Finagle for Scala. I'm trying Go now, and I've noticed
>> that the official Thrift library for Go only contains SimpleServer
>> implementation. It works alright, but it's purpose is just testing and I
>> need something more performant and mature. In Java, I've used
>> TThreadPoolServer and was satisfied with its performance. Is there anything
>> similar for Go, perhaps making use of goroutines?
>> Thank you!
>>
>> Best regards,
>> Sergey Serebryakov
>
>
>

Re: Multithreaded servers in Apache Thrift for Go

Posted by Chris Bannister <c....@gmail.com>.
The TSimpleServer in Go is making use of goroutines, as there inst really
any reason not to. Every request is handled in its own goroutine, unlike
Java there is no need to have a specific thread pool implementation to
handle requests, and requests are handled in the same manner that the
standard library http server works.


On 31 May 2014 11:54, Jens Geyer <je...@hotmail.com> wrote:

> Hi Sergey,
>
> implying your agreement ;-) I’m moving this topic onto the dev mailing
> list as this seems a better place to discuss this.
>
> The Go language part has been gone through some major updates in the last
> monts, especially thanks to the massive contributions from Travis, Ben and
> Aleksey. I think everybody will agree when I say, that the Go language part
> it is in a much better shape now than it was a year ago. That said, there
> is surely more work needed to add some still missing pieces, or make some
> stuff better that we are not fully satisfied with yet.
>
> So the answer is twofold:
> - Yes, there is only the simple server (yet),
> - and yes, we are of course accepting quality contributions.
>
> Could that be an option for you?
>
> Thanks + have fun,
> JensG
>
>
>
>
> From: Sergey Serebryakov
> Sent: Saturday, May 31, 2014 5:25 AM
> To: jensg@apache.org
> Subject: Multithreaded servers in Apache Thrift for Go
>
> Hi Jens,
>
> I'm trying out various languages to implement a simple IP lookup
> Thrift-based server in. So far I've tried official library for Java and
> Twitter's Scrooge+Finagle for Scala. I'm trying Go now, and I've noticed
> that the official Thrift library for Go only contains SimpleServer
> implementation. It works alright, but it's purpose is just testing and I
> need something more performant and mature. In Java, I've used
> TThreadPoolServer and was satisfied with its performance. Is there anything
> similar for Go, perhaps making use of goroutines?
> Thank you!
>
> Best regards,
> Sergey Serebryakov