[jira] [Work logged] (HADOOP-18231) tests in ITestS3AInputStreamPerformance are failing

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-18231?focusedWorklogId=769538&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769538 ]

ASF GitHub Bot logged work on HADOOP-18231:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 12/May/22 10:57
            Start Date: 12/May/22 10:57
    Worklog Time Spent: 10m 
      Work Description: ahmarsuhail opened a new pull request, #4305:
URL: https://github.com/apache/hadoop/pull/4305

   ### Description of PR
   
   `ITestS3AInputStreamPerformance` was failing when prefetching is enabled. This PR disables prefetching when running `ITestS3AInputStreamPerformance`. 
   
   It also adds in a new test class `ITestS3PrefetchingInputStream` with tests specific for prefetching. Once more stats are added to prefetching iostats, we can add in more tests + assertions. For eg it would be good to assert on if a file has been read via the cache or not on a backward seek. 
   
   ### How was this patch tested?
   
   Tested in eu-west-1 by running
   
   `mvn -Dparallel-tests -DtestsThreadCount=16 clean verify`
   
   ### For code changes:
   
   - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   




Issue Time Tracking
-------------------

            Worklog Id:     (was: 769538)
    Remaining Estimate: 0h
            Time Spent: 10m

> tests in ITestS3AInputStreamPerformance are failing 
> ----------------------------------------------------
>
>                 Key: HADOOP-18231
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18231
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Ahmar Suhail
>            Assignee: Ahmar Suhail
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The following tests are failing when prefetching is enabled:
> testRandomIORandomPolicy - expects stream to be opened 4 times (once for every random read), but prefetching will only open twice. 
> testDecompressionSequential128K - expects stream to be opened once, but prefetching will open once for each block the file has. landsat file used in the test has size 42MB, prefetching block size = 8MB, expected open count is 6.
>  testReadWithNormalPolicy - same as above. 
> testRandomIONormalPolicy - executes random IO, but with a normal policy. S3AInputStream will abort the stream and change the policy, prefetching handles random IO by caching blocks so doesn't do any of that. 
> testRandomReadOverBuffer - multiple assertions failing here, also depends a lot on readAhead values, not very relevant for prefetching



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org