You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2020/10/07 10:30:00 UTC
[jira] [Commented] (HDDS-4243) Get API not working from S3A
filesystem with Ozone S3
[ https://issues.apache.org/jira/browse/HDDS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209452#comment-17209452 ]
Steve Loughran commented on HDDS-4243:
--------------------------------------
all we like is an etag which can be returned on GET so that while we read around a stream in multiple GET requests, we fail if the etag has changed...its how we detect updated files.
> Get API not working from S3A filesystem with Ozone S3
> -----------------------------------------------------
>
> Key: HDDS-4243
> URL: https://issues.apache.org/jira/browse/HDDS-4243
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: S3
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: OzoneS3, S3A
>
> TroubleShooting S3A mentions S3 compatible servers that donot support Etags will see this server
> Refer [link|https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/troubleshooting_s3a.html] and look for below section content.
> Using a third-party S3 implementation that doesn’t support eTags might result in the following error.
> org.apache.hadoop.fs.s3a.NoVersionAttributeException: `s3a://my-bucket/test/file.txt':
> Change detection policy requires ETag
> at org.apache.hadoop.fs.s3a.impl.ChangeTracker.processResponse(ChangeTracker.java:153)
> at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:200)
> at org.apache.hadoop.fs.s3a.S3AInputStream.lambda$lazySeek$1(S3AInputStream.java:346)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$2(Invoker.java:195)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:193)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:215)
> at org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:339)
> at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:372)
> {code:java}
> org.apache.hadoop.fs.s3a.NoVersionAttributeException: `s3a://sept14/dir1/dir2/dir3/key1': Change detection policy requires ETag
> at org.apache.hadoop.fs.s3a.impl.ChangeTracker.processNewRevision(ChangeTracker.java:275)
> at org.apache.hadoop.fs.s3a.impl.ChangeTracker.processMetadata(ChangeTracker.java:261)
> at org.apache.hadoop.fs.s3a.impl.ChangeTracker.processResponse(ChangeTracker.java:195)
> at org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:208)
> at org.apache.hadoop.fs.s3a.S3AInputStream.lambda$lazySeek$1(S3AInputStream.java:359)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$maybeRetry$3(Invoker.java:223)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:110)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$maybeRetry$5(Invoker.java:347)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:407)
> at org.apache.hadoop.fs.s3a.Invoker.maybeRetry(Invoker.java:343)
> at org.apache.hadoop.fs.s3a.Invoker.maybeRetry(Invoker.java:221)
> at org.apache.hadoop.fs.s3a.Invoker.maybeRetry(Invoker.java:265)
> at org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:351)
> at org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:464)
> at java.io.DataInputStream.read(DataInputStream.java:100)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:94)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:129)
> at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.writeStreamToFile(CommandWithDestination.java:494)
> at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:416)
> at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:351)
> at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:286)
> at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:271)
> at org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:367)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331)
> at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:304)
> at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:266)
> at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:286)
> at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
> at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:237)
> at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:391)
> get: `s3a://sept14/dir1/dir2/dir3/key1': Change detection policy requires ETag
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org