You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/02/16 17:07:00 UTC

[jira] [Resolved] (HADOOP-14606) S3AInputStream: Handle http stream skip(n) skipping < n bytes in a forward seek

     [ https://issues.apache.org/jira/browse/HADOOP-14606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved HADOOP-14606.
-------------------------------------
       Resolution: Invalid
    Fix Version/s: 3.1.0

Reviewing the code  here, the problem doesn't exist. When we seek forwards we look for the return value, update our position, then, if it it doesn't match the expected number, close the stream and do a new GET.

> S3AInputStream: Handle http stream skip(n) skipping < n bytes in a forward seek
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-14606
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14606
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.1
>            Reporter: Steve Loughran
>            Priority: Critical
>             Fix For: 3.1.0
>
>
> There's some hints in the InputStream docs that {{skip(n)}} may skip <n bytes. Codepaths only seem to do this if read() returns -1, meaning end of stream is reached.
> If that happens when doing a forward seek via skip, then we have got our numbers wrong and are in trouble. Look for a negative response, log @ ERROR and revert to a close/reopen seek to an absolute position.
> *I have no evidence of this acutally occurring*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org