You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/04/22 13:44:13 UTC
[jira] [Commented] (HADOOP-12844) Recover when S3A fails on
IOException in read()
[ https://issues.apache.org/jira/browse/HADOOP-12844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253785#comment-15253785 ]
Steve Loughran commented on HADOOP-12844:
-----------------------------------------
I've taken this over. It's actually simpler than this patch, as all that's needed is the existing clause catching some socket exceptions to be expanded to catch any IOE, and then log and retry ... this is common code which can be shared in both read operations.
what is important is to catch and respond to EOF exceptions before the generic IOE clause
> Recover when S3A fails on IOException in read()
> -----------------------------------------------
>
> Key: HADOOP-12844
> URL: https://issues.apache.org/jira/browse/HADOOP-12844
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.7.1, 2.7.2
> Reporter: Pieter Reuse
> Assignee: Pieter Reuse
> Attachments: HADOOP-12844.001.patch
>
>
> This simple patch catches IOExceptions in S3AInputStream.read(byte[] buf, int off, int len) and reopens the connection on the same location as it was before the exception.
> This is similar to the functionality introduced in S3N in [HADOOP-6254|https://issues.apache.org/jira/browse/HADOOP-6254], for exactly the same reason.
> Patch developed in cooperation with [~emres].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)