You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Jesse Schoen (Jira)" <ji...@apache.org> on 2022/03/07 19:00:00 UTC
[jira] [Commented] (HADOOP-16874) For AzureNativeFS, when BlockCompaction is enabled, FileSystem.create(path).close() would throw exception.
[ https://issues.apache.org/jira/browse/HADOOP-16874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502506#comment-17502506 ]
Jesse Schoen commented on HADOOP-16874:
---------------------------------------
Is there a workaround for this currently, or do we turn of block compaction? What are the downsides to disabling this flag?
> For AzureNativeFS, when BlockCompaction is enabled, FileSystem.create(path).close() would throw exception.
> ----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-16874
> URL: https://issues.apache.org/jira/browse/HADOOP-16874
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 2.9.2, 3.2.1
> Environment: macOS Mojave 10.14.6
>
> Reporter: Xiaolei Liu
> Priority: Minor
>
> For AzureNativeFS, when BlockCompaction is enabled, FileSystem.create(path).close() would throw blob not existed exception.
> Block Compaction Setting: fs.azure.block.blob.with.compaction.dir
> Exception is thrown from close(), this would happen when no write happened. When actually write any content in the file, same context close() won't trigger the exception.
> When BlockCompaction is not enabled, this issue won't happen.
> Call Stack:
> org.apache.hadoop.fs.azure.AzureException: Source blob _$azuretmpfolder$/956457df-4a3e-4285-bc68-29f68b9b36c4test1911.log does not exist.
> org.apache.hadoop.fs.azure.AzureException: Source blob _$azuretmpfolder$/956457df-4a3e-4285-bc68-29f68b9b36c4test1911.log does not exist.
> at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2648)
> at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2608)
> at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsOutputStream.restoreKey(NativeAzureFileSystem.java:1199)
> at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsOutputStream.close(NativeAzureFileSystem.java:1068)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org