[jira] [Commented] (HADOOP-18175) test failures with prefetching s3a input stream

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516870#comment-17516870 ] 

Steve Loughran commented on HADOOP-18175:
-----------------------------------------

code}

[ERROR] testStatistics(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics)  Time elapsed: 1.591 s  <<< ERROR!
java.io.EOFException: Attempted to seek or read past the end of the file 512
	at org.apache.hadoop.fs.s3a.read.S3InputStream.throwIfInvalidSeek(S3InputStream.java:440)
	at org.apache.hadoop.fs.s3a.read.S3InputStream.seek(S3InputStream.java:235)
	at org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.seek(S3PrefetchingInputStream.java:162)
	at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:85)
	at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:98)
	at org.apache.hadoop.fs.FCStatisticsBaseTest.testStatistics(FCStatisticsBaseTest.java:107)


{code}


it's happening in the seek(oldPos) code in the default implementation of positioned read, because the previous unpositioned read 


{code}

    byte[] buf = new byte[blockSize];
    int bytesRead = fstr.read(buf, 0, blockSize);
    fstr.read(0, buf, 0, blockSize);

{code}


This is quite a good little failure; I'm impressed but who ever managed to think at this corner case. pos is at EOF, so returning to it is raising the EOF. 
We will probably want seek() to not do the checks, and have read() do the validation, which is mostly what s3a does , as it only rejects negative offsets.

> test failures with prefetching s3a input stream
> -----------------------------------------------
>
>                 Key: HADOOP-18175
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18175
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: 3.4.0
>            Reporter: Steve Loughran
>            Priority: Major
>
> identify and fix all test regressions from the prefetching s3a input stream



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org