[jira] [Created] (HADOOP-18738) Error message when multipart disabled and buffering is array not useful

["Steve Loughran (Jira)" <ji...@apache.org>]

Steve Loughran created HADOOP-18738:
---------------------------------------

             Summary: Error message when multipart disabled and buffering is array not useful
                 Key: HADOOP-18738
                 URL: https://issues.apache.org/jira/browse/HADOOP-18738
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs/s3
    Affects Versions: 3.3.9
            Reporter: Steve Loughran
            Assignee: Harshit Gupta


testing branch-3.3 on a bucket with array buffering, I get a fairly useless error message.
Problem here is that AbstractCommitITest forces fs.s3a.fast.upload.buffer to array and somehow multipart is disabled.

# error text needs fixing (move to %s)
# s3a fs should fail in init if the threshold is isn't allowed for the store type
# then determine why the test is failing, and fix.


The tests do work in trunk, so something in branch-3.3 is stopping this


Also 

* correct the name of the constant STORE_CAPABILITY_DIRECTORY_MARKER_MULTIPART_UPLOAD_ENABLED to match the actual string "fs.s3a.capability.multipart.uploads.enabled"; accidental copy there

* capability FS_MULTIPART_UPLOADER must declare itself as disabled when multipart is off
    

{code}

[ERROR] testRevertCommit(org.apache.hadoop.fs.s3a.commit.ITestCommitOperations)  Time elapsed: 0.688 s  <<< ERROR!
java.lang.IllegalArgumentException: Invalid block size: %d [-1]
        at org.apache.hadoop.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:205)
        at org.apache.hadoop.fs.s3a.S3ADataBlocks$ArrayBlockFactory.create(S3ADataBlocks.java:397)
        at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.createBlockIfNeeded(S3ABlockOutputStream.java:235)
        at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.<init>(S3ABlockOutputStream.java:217)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.innerCreateFile(S3AFileSystem.java:1887)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$create$7(S3AFileSystem.java:1789)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
        at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2479)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2498)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:1788)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1233)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1210)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1091)
        at org.apache.hadoop.fs.contract.ContractTestUtils.createFile(ContractTestUtils.java:650)
        at org.apache.hadoop.fs.contract.ContractTestUtils.touch(ContractTestUtils.java:686)
        at org.apache.hadoop.fs.s3a.commit.ITestCommitOperations.testRevertCommit(ITestCommitOperations.java:543)
{code}






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org