You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/06/02 16:22:00 UTC
[jira] [Commented] (IMPALA-12114) SSL Thrift connections disconnect if idle more than ~150 seconds
[ https://issues.apache.org/jira/browse/IMPALA-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728816#comment-17728816 ]
ASF subversion and git services commented on IMPALA-12114:
----------------------------------------------------------
Commit e9fb8e717c32a9a72dc81b85084f982efc25a2f8 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e9fb8e717 ]
IMPALA-12114: Pull in fix for THRIFT-5705 and add test
This pulls in a new toolchain to get a Thrift with
the patch for THRIFT-5705. This fixes an issue where
idle clients using TLS are needlessly disconnected due
to a bug in the read retry count logic inside Thrift.
Tests:
- This modifies test_thrift_socket.py to make it do
more idle polls and check that ImpalaShell is not
disconnected. It fails without the THRIFT-5705 patch
and passes now.
Change-Id: Ifc7704cba032a91b9fd0d5d54d1e0a7e17fb10bb
Reviewed-on: http://gerrit.cloudera.org:8080/19962
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Daniel Becker <da...@cloudera.com>
Reviewed-by: Andrew Sherman <as...@cloudera.com>
> SSL Thrift connections disconnect if idle more than ~150 seconds
> ----------------------------------------------------------------
>
> Key: IMPALA-12114
> URL: https://issues.apache.org/jira/browse/IMPALA-12114
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.3.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
>
> A test cluster ran into issues with idle connections being disconnected when using SSL.
> This reproduces on my development environment with these steps:
> # Start Impala with SSL enabled
> {noformat}
> bin/start-impala-cluster.py --impalad_args="--ssl_client_ca_certificate=${IMPALA_HOME}/be/src/testutil/server-cert.pem --ssl_server_certificate=${IMPALA_HOME}/be/src/testutil/server-cert.pem --ssl_private_key=${IMPALA_HOME}/be/src/testutil/server-key.pem --hostname=localhost --idle_client_poll_period_s=30 -v=2" --state_store_args="--ssl_client_ca_certificate=${IMPALA_HOME}/be/src/testutil/server-cert.pem --ssl_server_certificate=${IMPALA_HOME}/be/src/testutil/server-cert.pem --ssl_private_key=${IMPALA_HOME}/be/src/testutil/server-key.pem --hostname=localhost" --catalogd_args="--ssl_client_ca_certificate=${IMPALA_HOME}/be/src/testutil/server-cert.pem --ssl_server_certificate=${IMPALA_HOME}/be/src/testutil/server-cert.pem --ssl_private_key=${IMPALA_HOME}/be/src/testutil/server-key.pem --hostname=localhost" --cluster_size=1{noformat}
> # Connect with impala-shell
> {noformat}
> impala-shell --ssl{noformat}
> # Leave this idle for 150+ seconds
> In the Impalad logs will be a statement like this:
> {noformat}
> I0503 22:11:53.233147 206554 impala-server.cc:2488] Connection 20470cb275a1d256:3d68601942f3179f from client 172.27.100.70:42540 to server hiveserver2-frontend closed. The connection had 2 associated session(s).{noformat}
> # Run a statement in impala-shell and will show that it needs to reconnect
> {noformat}
> default> show tables;
> Caught exception TSocket read 0 bytes, type=<class 'thrift.transport.TTransport.TTransportException'> in PingImpalaHS2Service.
> Caught exception [Errno 32] Broken pipe, type=<class 'socket.error'> in CloseSession.
> Warning: close session RPC failed: [Errno 32] Broken pipe, <class 'socket.error'>
> Connection lost, reconnecting...
> ... then it retries and succeeds{noformat}
> Tracing through the code, it appears that this peek() call returns false:
> {noformat}
> try {
> bytes_pending = input_->getTransport()->peek();
> break;
> } catch (const TTransportException& ttx) {{noformat}
> bytes_pending is false, and this causes the connection to be closed.
> This doesn't seem to impact Impala with older Thrift versions, so maybe something changed in Thrift 0.16.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org