[GitHub] [hadoop] adoroszlai opened a new pull request #1349: HDDS-2026. Overlapping chunk region cannot be read concurrently

[GitBox <gi...@apache.org>]

adoroszlai opened a new pull request #1349: HDDS-2026. Overlapping chunk region cannot be read concurrently
URL: https://github.com/apache/hadoop/pull/1349
 
 
   ## What changes were proposed in this pull request?
   
   Only allow a single read/write operation for the same path in `ChunkUtils`, to avoid `OverlappingFileLockException` due to concurrent reads.  This allows concurrent reads/writes of separate files (as opposed to simply synchronizing the methods).  It might be improved later by storing and reusing the file lock.
   
   Use plain `FileChannel` instead of `AsynchronousFileChannel` for reading, too, since it was used in synchronous fashion (by calling `.get()`) anyway.
   
   https://issues.apache.org/jira/browse/HDDS-2026
   
   ## How was this patch tested?
   
   Added unit test.
   
   Used improved Freon tool from #1341 to perform read of same key from multiple threads (which revealed the bug in the first place).
   
   ```
   $ ozone freon ockg -n 1 -p asdf
   $ ozone sh key list vol1/bucket1
   [ {
     "version" : 0,
     "size" : 10240,
     "keyName" : "asdf/0"
     ...
   } ]
   
   $ ozone freon ocokr -k 'asdf/0'
   ...
            mean rate = 164.39 calls/second
                     ...
                 mean = 53.75 milliseconds
               stddev = 42.11 milliseconds
               median = 44.05 milliseconds
   ...
   Total execution time (sec): 6
   Failures: 0
   Successful executions: 1000
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org