You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/11/24 17:29:00 UTC
[jira] [Commented] (HADOOP-14621) S3A client raising
ConnectionPoolTimeoutException
[ https://issues.apache.org/jira/browse/HADOOP-14621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265452#comment-16265452 ]
Steve Loughran commented on HADOOP-14621:
-----------------------------------------
This is possibly now remapped to ConnectionTimeoutException and so retried...we'd need to experiment
ConnectionPoolTimeoutException is a sign of a serious problem, as it can be raised if the pool of http threads has been used up by files not being closed in user code (SPARK-22526), elsewhere (HIVE-13216). At the same time, completely spurious if it is raised early and often.
Proposed
# Make sure translation preserves it as its own exception (interruptedIOE with text)
# add commentary in troubleshooting doc
# review timeout value and pool size, consider recommending values (2 per CPU?)
# could we make queue size a gauge in metrics ?
# have a test which creates a very small pool and timeout and triggers a failure (GET on small file and read() one byte should be enough to use up a connection.
Look @ other object store code and see if the same policy applies
> S3A client raising ConnectionPoolTimeoutException
> -------------------------------------------------
>
> Key: HADOOP-14621
> URL: https://issues.apache.org/jira/browse/HADOOP-14621
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.0.0-beta1
> Environment: Home network with 2+ other users on high bandwidth activities
> Reporter: Steve Loughran
> Priority: Minor
>
> Parallel test with threads = 12 triggering connection pool timeout. Hypothesis? Congested network triggering pool timeout.
> Fix? For tests, could increase pool size
> For retry logic, this should be considered retriable, even on idempotent calls (as its a failure to acquire a connection
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org