[jira] [Created] (HDFS-15183) For AzureNativeFS, when BlockCompaction is enabled, FileSystem.create(path).close() would throw exception.

["Xiaolei Liu (Jira)" <ji...@apache.org>]

Xiaolei Liu created HDFS-15183:
----------------------------------

             Summary: For AzureNativeFS, when BlockCompaction is enabled, FileSystem.create(path).close() would throw exception.
                 Key: HDFS-15183
                 URL: https://issues.apache.org/jira/browse/HDFS-15183
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: fs/azure
    Affects Versions: 3.2.1, 2.9.2
         Environment: macOS Mojave 10.14.6

 
            Reporter: Xiaolei Liu


For AzureNativeFS, when BlockCompaction is enabled, FileSystem.create(path).close() would throw blob not existed exception.

Block Compaction Setting: fs.azure.block.blob.with.compaction.dir
Exception is thrown from close(), this would happen when no write happened. When actually write any content in the file, same context close() won't trigger the exception. 

When BlockCompaction is not enabled, this issue won't happen. 

Call Stack:

org.apache.hadoop.fs.azure.AzureException: Source blob _$azuretmpfolder$/956457df-4a3e-4285-bc68-29f68b9b36c4test1911.log does not exist.
org.apache.hadoop.fs.azure.AzureException: Source blob _$azuretmpfolder$/956457df-4a3e-4285-bc68-29f68b9b36c4test1911.log does not exist. 
at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2648) 
at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2608) 
at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsOutputStream.restoreKey(NativeAzureFileSystem.java:1199) 
at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsOutputStream.close(NativeAzureFileSystem.java:1068) 
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) 
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org