Re: Add SASL support for non blocking server

[XU Qinghui <qi...@gmail.com>]

Hello, folks

I propose a PR (https://github.com/apache/thrift/pull/1892) for a
nonblocking thrift server (java) implementation with sasl support. The idea
is that some of our services using `TThreadPoolServer` are facing some
scaling issues with increasing number of connections, and we would like to
refactor them with a nonblocking server while still being able to secure
them with sasl. One example of such services is hive metastore, and hive
community seems agreed with the idea of using a nonblocking server (
https://issues.apache.org/jira/browse/HIVE-22306)

In my PR, I provide `TNonblockingSaslServer` which has one thread listening
for new incoming connections, and separate thread pools for network io,
sasl authentication, and processing. For each (nonblocking) connection,
there is a state machine `NonblockingSaslHandler` managing the lifecycle,
which is event driven by network io.

It would be great if I can get some feedback and code review on my PR.

Cheers,



Le ven. 19 juil. 2019 à 12:41, XU Qinghui <qi...@gmail.com> a écrit :

> Thrift folks,
>
> Recently we are facing a scaling issue with the thrift server
> (TThreadPoolServer) when there are a lot of connections. We would like to
> use the non blocking server implementations, but it seems it miss SASL
> support there (as far as I understand, SASL is only added for blocking
> servers in THRIFT-876).
> So I would like to implement SASL for non blocking servers (THRIFT-4889).
> Do you have some advices?
>
> Best regards,
> Qinghui
>

Fwd: Add SASL support for non blocking server

[XU Qinghui <qi...@gmail.com>]

Hive folks,

We currently see some scalability issues with Hive Metastore (
https://issues.apache.org/jira/browse/HIVE-22306), and it turns out to be
because of the use of thrfit (blocking) thread pool server.
I proposed a PR (https://github.com/apache/thrift/pull/1892) to thrift for
a secure nonblocking thrfit server implementation, which I would like to
use for HIVE-22306 <https://issues.apache.org/jira/browse/HIVE-22306> once
merged. It seems that hive community used to have strong influence and
contribution on thrift project, so I'd like to ask some help from the
community to review the PR.

Best regards,
Qinghui

---------- Forwarded message ---------
De : XU Qinghui <qi...@gmail.com>
Date: ven. 11 oct. 2019 à 14:36
Subject: Re: Add SASL support for non blocking server
To: <de...@thrift.apache.org>


Hello, folks

I propose a PR (https://github.com/apache/thrift/pull/1892) for a
nonblocking thrift server (java) implementation with sasl support. The idea
is that some of our services using `TThreadPoolServer` are facing some
scaling issues with increasing number of connections, and we would like to
refactor them with a nonblocking server while still being able to secure
them with sasl. One example of such services is hive metastore, and hive
community seems agreed with the idea of using a nonblocking server (
https://issues.apache.org/jira/browse/HIVE-22306)

In my PR, I provide `TNonblockingSaslServer` which has one thread listening
for new incoming connections, and separate thread pools for network io,
sasl authentication, and processing. For each (nonblocking) connection,
there is a state machine `NonblockingSaslHandler` managing the lifecycle,
which is event driven by network io.

It would be great if I can get some feedback and code review on my PR.

Cheers,



Le ven. 19 juil. 2019 à 12:41, XU Qinghui <qi...@gmail.com> a écrit :

> Thrift folks,
>
> Recently we are facing a scaling issue with the thrift server
> (TThreadPoolServer) when there are a lot of connections. We would like to
> use the non blocking server implementations, but it seems it miss SASL
> support there (as far as I understand, SASL is only added for blocking
> servers in THRIFT-876).
> So I would like to implement SASL for non blocking servers (THRIFT-4889).
> Do you have some advices?
>
> Best regards,
> Qinghui
>