You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/01/19 13:37:00 UTC
[jira] [Work logged] (HADOOP-16155) S3AInputStream read(bytes[]) to not retry on read failure: pass action up
[ https://issues.apache.org/jira/browse/HADOOP-16155?focusedWorklogId=711381&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-711381 ]
ASF GitHub Bot logged work on HADOOP-16155:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 19/Jan/22 13:36
Start Date: 19/Jan/22 13:36
Worklog Time Spent: 10m
Work Description: ahmarsuhail opened a new pull request #3905:
URL: https://github.com/apache/hadoop/pull/3905
### Description of PR
Jira: https://issues.apache.org/jira/browse/HADOOP-16155
When an error happens on read, do not reopen the stream.
For the unit tests I've changed the order of super.read() and trigger failure. This is because previously the failure would happen after the read actually executed and so the buffer would not be empty and we couldn't assert on the buffer being empty in case of failures.
I've also added a new variable triggerGetObjectFailure. This will not trigger a get object failure in case of testInputStreamReadFullyRetryForException. This is because since we are removing retries from read(bytes[]), readFully() will call read(bytes[]) after each failure which will then call lazySeek() every time. The get object failure ends up being thrown in lazySeek() which does not retry and so readFully does not complete.
### How was this patch tested?
Tested in eu-west-1 by running
```
mvn -Dparallel-tests -DtestsThreadCount=16 clean verify
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 711381)
Remaining Estimate: 0h
Time Spent: 10m
> S3AInputStream read(bytes[]) to not retry on read failure: pass action up
> -------------------------------------------------------------------------
>
> Key: HADOOP-16155
> URL: https://issues.apache.org/jira/browse/HADOOP-16155
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The S3AInputStream reacts to read(byte[]) failure by reopening the stream, just as for the single byte read(). We shouldn't need to do that. Instead just close the stream, return 0 and let the caller decided what to do.
> why so?
> # its in the contract of InputStream.read(bytes[]),
> # readFully() can handle the 0 in its loop
> # other apps can decided what to do.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org