You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2020/10/19 11:03:00 UTC
[jira] [Commented] (HADOOP-17312) S3AInputStream to be resilient to
faiures in abort(); translate AWS Exceptions
[ https://issues.apache.org/jira/browse/HADOOP-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216626#comment-17216626 ]
Steve Loughran commented on HADOOP-17312:
-----------------------------------------
{code}
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:198)
at org.apache.http.impl.io.ContentLengthInputStream.close(ContentLengthInputStream.java:101)
at org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:180)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:180)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:180)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:180)
at com.amazonaws.services.s3.model.S3ObjectInputStream.abort(S3ObjectInputStream.java:90)
at org.apache.hadoop.fs.s3a.S3AInputStream.close(S3AInputStream.java:199)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:180)
at org.apache.hadoop.util.LineReader.close(LineReader.java:150)
{code}
> S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions
> ------------------------------------------------------------------------------
>
> Key: HADOOP-17312
> URL: https://issues.apache.org/jira/browse/HADOOP-17312
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0, 3.2.1
> Reporter: Steve Loughran
> Priority: Major
>
> Stack overflow issue complaining about ConnectionClosedException during S3AInputStream close(), seems triggered by an EOF exception in abort. That is: we are trying to close the stream and it is failing because the stream is closed. oops.
> https://stackoverflow.com/questions/64412010/pyspark-org-apache-http-connectionclosedexception-premature-end-of-content-leng
> Looking @ the stack, we aren't translating AWS exceptions in abort() to IOEs, which may be a factor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org