You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Marton Elek (Jira)" <ji...@apache.org> on 2020/10/09 10:59:00 UTC
[jira] [Commented] (HDDS-4209) S3A Filesystem does not work with
Ozone S3 in file system compat mode
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210831#comment-17210831 ]
Marton Elek commented on HDDS-4209:
-----------------------------------
If not, It might be better to add this information to the documentation.
There is an open pull request for this jira. Shall we close it?
> S3A Filesystem does not work with Ozone S3 in file system compat mode
> ---------------------------------------------------------------------
>
> Key: HDDS-4209
> URL: https://issues.apache.org/jira/browse/HDDS-4209
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Blocker
> Labels: OzoneS3, S3A, pull-request-available
>
> When *ozone.om.enable.filesystem.paths* is enabled
>
> hdfs dfs -mkdir -p s3a://b12345/d11/d12 -> Success
> hdfs dfs -put /tmp/file1 s3a://b12345/d11/d12/file1 -> fails with below error
>
> {code:java}
> 2020-09-04 03:53:51,377 ERROR org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest: Key creation failed. Volume:s3v, Bucket:b1234, Keyd11/d12/file1._COPYING_. Exception:{}
> NOT_A_FILE org.apache.hadoop.ozone.om.exceptions.OMException: Can not create file: cp/k1._COPYING_ as there is already file in the given path
> at org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest.validateAndUpdateCache(OMKeyCreateRequest.java:256)
> at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:227)
> at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.runCommand(OzoneManagerStateMachine.java:428)
> at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.lambda$applyTransaction$1(OzoneManagerStateMachine.java:246)
> at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> *Reason for this*
> S3A filesystem when create directory creates an empty file
> *Now entries in Ozone KeyTable after create directory*
> d11/
> d11/d12
> Because of this in OMFileRequest.VerifyInFilesPath fails with FILE_EXISTS_IN_GIVEN_PATH because d11/d12 is considered as file not a directory. (As in ozone currently, directories end with trailing "/")
> So, when d11/d12/file is created, we check parent exists, now d11/d12 is considered as file and fails with NOT_A_FILE
> When disabled it works fine, as when disabled during key create we do not check any filesystem semantics and also does not create intermediate directories.
> {code:java}
> [root@bvoz-1 ~]# hdfs dfs -mkdir -p s3a://b12345/d11/d12
> [root@bvoz-1 ~]# hdfs dfs -put /etc/hadoop/conf/ozone-site.xml s3a://b12345/d11/d12/k1
> [root@bvoz-1 ~]# hdfs dfs -ls s3a://b12345/d11/d12
> Found 1 items
> -rw-rw-rw- 1 systest systest 2373 2020-09-04 04:45 s3a://b12345/d11/d12/k1
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org