You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by st...@apache.org on 2019/04/25 18:05:47 UTC
[hadoop] branch branch-2 updated (313608e -> 8900c7f)
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a change to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from 313608e HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji.
new 1b64b0f HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
new 37426c2 HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. Contributed by Da Zhou.
new b537d8c HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. Contributed by Steve Loughran and Da Zhou.
new 5da4bb8 HADOOP-15659. Code changes for bug fix and new tests. Contributed by Da Zhou.
new e320b64 Fixing issue due to commit 2b2399d6 after rebase onto trunk.
new 92e37a0 HADOOP-15669. ABFS: Improve HTTPS Performance. Contributed by Vishwajeet Dusane.
new 9d47230 HADOOP-15660. ABFS: Add support for OAuth Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
new e727c87 HADOOP-15661. ABFS: Add support for ACL. Contributed by Junhua Gu and Da Zhou.
new 08d3d50 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. Contributed by Sean Mackrory.
new b59741f HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. Contributed by Thomas Marquardt.
new 92f3f9c HADOOP-15692. ABFS: extensible support for custom oauth. Contributed by Junhua Gu and Rajeev Bansal.
new 3bd089d HADOOP-15663. ABFS: Simplify configuration. Contributed by Da Zhou.
new 12c621e HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. Contributed by Da Zhou.
new 305e750 HADOOP-15703. ABFS - Implement client-side throttling. Contributed by Sneha Varma and Thomas Marquardt.
new 8bb713e Fixing findbugs and license issues related to:
new a192d85 HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation for storage account with no namespace feature.
new 36b2bab HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. Contributed by Sneha Varma.
new e52f4a1 HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. Contributed by Sean Mackrory.
new 263094f HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" Contributed by Da Zhou.
new 3599cd3 HADOOP-15757. ABFS: remove dependency on common-codec Base64. Contributed by Da Zhou.
new 94d0de2 HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. Contributed by Steve Loughran.
new 26ce4e9 HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. Contributed by Da Zhou
new 8acab6c HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. Contributed by Da Zhou.
new 1efa1b7 HADOOP-15719. Fail-fast when using OAuth over http. Contributed by Da Zhou.
new 10e149d HADOOP-15769. ABFS: distcp tests are always skipped. Contributed by Steve Loughran
new 920b77c HADOOP-15773. Fixing checkstyle and other issues raised by Yetus.
new 6c7c8b8 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. Contributed by Steve Loughran.
new 4f3cc1d HADOOP-15778. ABFS: Fix client side throttling for read. Contributed by Sneha Varma.
new bde00d5 HADOOP-15723. ABFS: Ranger Support. Contributed by Yuan Gao.
new ee0fa0d HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. Contributed by Da Zhou.
new 0c5074a HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization Contributed by Yuan Gao.
new d5e4c40 HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. Contributed by Da Zhou
new e3033a5 HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. Contributed by Abhishek Modi.
new 1b08e8d HADOOP-15795. Make HTTPS the default protocol for ABFS. Contributed by Da Zhou.
new 73bbada HADOOP-15809. ABFS: better exception handling when making getAccessToken call. Contributed by Da Zhou
new 885c726 HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. Contributed by Da Zhou.
new 8330f18 HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI (Contributed by Da Zhou via Daniel Templeton)
new d370981 HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries.
new 9f220c4 HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console.
new 9e1d39b HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore
new afa5705 HADOOP-15872. ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2.
new 9cf1fb6 HADOOP-15940. ABFS: For HNS account, avoid unnecessary get call when doing Rename.
new 6f200a6 HADOOP-15968. ABFS: add try catch for UGI failure when initializing ABFS.
new 06e6a67 HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking user access thru ACLs.
new 38a36d4 HADOOP-15972 ABFS: reduce list page size to to 500.
new eaaf283 HADOOP-16004. ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException.
new 175a69e HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). Contributed by Shweta Yakkali.
new 4a17610 HADOOP-16040. ABFS: Bug fix for tolerateOobAppends configuration.
new a0b99f4 HADOOP-15662. Better exception handling of DNS errors.
new 34dbac8 HADOOP-15975. ABFS: remove timeout check for DELETE and RENAME.
new 00b990d HADOOP-16044. ABFS: Better exception handling of DNS errors followup
new f799c46 HADOOP-16048. ABFS: Fix Date format parser.
new 0bbbb9c HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. Contributed by Shweta Yakkali.
new 80c8b77 HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
new c2d7429 HADOOP-16136. ABFS: Should only transform username to short name
new 024a694 HADOOP-16169. ABFS: Bug fix for getPathProperties.
new 662a75d HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file.
new 18f66a0 HADOOP-16205 Backport ABFS driver from trunk to branch 2.0: Fix build and test failures.
new 8900c7f HADOOP-16205 Backport ABFS driver from trunk to branch 2.
The 59 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 +
.../src/main/resources/core-default.xml | 37 +
.../src/site/markdown/filesystem/filesystem.md | 11 +-
.../hadoop/conf/TestCommonConfigurationFields.java | 3 +
.../fs/contract/AbstractContractConcatTest.java | 50 +-
.../AbstractContractGetFileStatusTest.java | 42 +-
.../fs/contract/AbstractContractMkdirTest.java | 8 +
.../hadoop/fs/contract/AbstractFSContract.java | 2 -
.../hadoop/fs/contract/ContractTestUtils.java | 19 +-
hadoop-project/pom.xml | 18 +
hadoop-tools/hadoop-azure/pom.xml | 418 +++++-
.../src/config/checkstyle-suppressions.xml | 49 +
.../hadoop/fs/azure/ClientThrottlingAnalyzer.java | 2 +-
.../java/org/apache/hadoop/fs/azurebfs/Abfs.java} | 44 +-
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 638 +++++++++
.../java/org/apache/hadoop/fs/azurebfs/Abfss.java | 46 +
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 1086 +++++++++++++++
.../fs/azurebfs/AzureBlobFileSystemStore.java | 1041 +++++++++++++++
.../fs/azurebfs/SecureAzureBlobFileSystem.java} | 37 +-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 95 ++
.../fs/azurebfs/constants/ConfigurationKeys.java | 114 ++
.../constants/FileSystemConfigurations.java | 71 +
.../azurebfs/constants/FileSystemUriSchemes.java | 42 +
.../constants/HttpHeaderConfigurations.java | 63 +
.../fs/azurebfs/constants/HttpQueryParams.java | 42 +
.../hadoop/fs/azurebfs/constants/package-info.java | 22 +
.../ConfigurationValidationAnnotations.java | 104 ++
.../contracts/annotations/package-info.java | 22 +
.../diagnostics/ConfigurationValidator.java} | 33 +-
.../contracts/diagnostics/package-info.java | 22 +
.../exceptions/AbfsRestOperationException.java | 103 ++
.../exceptions/AzureBlobFileSystemException.java | 56 +
.../ConfigurationPropertyNotFoundException.java} | 32 +-
.../FileSystemOperationUnhandledException.java | 33 +
.../InvalidAbfsRestOperationException.java | 40 +
.../exceptions/InvalidAclOperationException.java | 33 +
.../InvalidConfigurationValueException.java} | 35 +-
.../InvalidFileSystemPropertyException.java | 33 +
.../exceptions/InvalidUriAuthorityException.java | 33 +
.../contracts/exceptions/InvalidUriException.java} | 31 +-
.../exceptions/KeyProviderException.java} | 40 +-
.../contracts/exceptions/TimeoutException.java | 33 +
.../exceptions/TokenAccessProviderException.java | 36 +
.../contracts/exceptions/package-info.java | 22 +
.../hadoop/fs/azurebfs/contracts/package-info.java | 22 +
.../contracts/services/AzureServiceErrorCode.java | 115 ++
.../contracts/services/ListResultEntrySchema.java | 239 ++++
.../contracts/services/ListResultSchema.java | 58 +
.../contracts/services/ReadBufferStatus.java | 29 +
.../azurebfs/contracts/services/package-info.java | 22 +
.../Base64StringConfigurationBasicValidator.java | 50 +
.../BooleanConfigurationBasicValidator.java | 50 +
.../diagnostics/ConfigurationBasicValidator.java | 67 +
.../IntegerConfigurationBasicValidator.java | 68 +
.../LongConfigurationBasicValidator.java | 63 +
.../StringConfigurationBasicValidator.java | 43 +
.../fs/azurebfs/diagnostics/package-info.java | 22 +
.../extensions/AbfsAuthorizationException.java} | 39 +-
.../fs/azurebfs/extensions/AbfsAuthorizer.java | 57 +
.../extensions/CustomDelegationTokenManager.java | 70 +
.../extensions/CustomTokenProviderAdaptee.java | 75 ++
.../fs/azurebfs/extensions/package-info.java | 32 +
.../fs/azurebfs/oauth2/AccessTokenProvider.java | 98 ++
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 346 +++++
.../hadoop/fs/azurebfs/oauth2/AzureADToken.java | 47 +
.../azurebfs/oauth2/ClientCredsTokenProvider.java | 62 +
.../oauth2/CustomTokenProviderAdapter.java | 58 +
.../fs/azurebfs/oauth2/IdentityTransformer.java | 279 ++++
.../fs/azurebfs/oauth2/MsiTokenProvider.java | 48 +
.../hadoop/fs/azurebfs/oauth2/QueryParams.java | 69 +
.../oauth2/RefreshTokenBasedTokenProvider.java | 57 +
.../azurebfs/oauth2/UserPasswordTokenProvider.java | 56 +
.../hadoop/fs/azurebfs/oauth2/package-info.java | 18 +
.../org/apache/hadoop/fs/azurebfs/package.html | 31 +
.../security/AbfsDelegationTokenIdentifier.java} | 51 +-
.../security/AbfsDelegationTokenManager.java | 88 ++
.../fs/azurebfs/security/AbfsTokenRenewer.java | 96 ++
.../hadoop/fs/azurebfs/security/package-info.java | 23 +
.../hadoop/fs/azurebfs/services/AbfsAclHelper.java | 278 ++++
.../hadoop/fs/azurebfs/services/AbfsClient.java | 589 ++++++++
.../services/AbfsClientThrottlingAnalyzer.java} | 82 +-
.../services/AbfsClientThrottlingIntercept.java | 135 ++
.../fs/azurebfs/services/AbfsHttpHeader.java | 40 +
.../fs/azurebfs/services/AbfsHttpOperation.java | 446 +++++++
.../fs/azurebfs/services/AbfsInputStream.java | 391 ++++++
.../fs/azurebfs/services/AbfsOutputStream.java | 398 ++++++
.../fs/azurebfs/services/AbfsPermission.java | 114 ++
.../fs/azurebfs/services/AbfsRestOperation.java | 208 +++
.../azurebfs/services/AbfsRestOperationType.java | 43 +
.../fs/azurebfs/services/AbfsUriQueryBuilder.java | 64 +
.../hadoop/fs/azurebfs/services/AuthType.java | 27 +
.../azurebfs/services/ExponentialRetryPolicy.java | 144 ++
.../hadoop/fs/azurebfs/services/KeyProvider.java | 43 +
.../hadoop/fs/azurebfs/services/ReadBuffer.java | 139 ++
.../fs/azurebfs/services/ReadBufferManager.java | 395 ++++++
.../fs/azurebfs/services/ReadBufferWorker.java | 72 +
.../fs/azurebfs/services/SharedKeyCredentials.java | 510 +++++++
.../services/ShellDecryptionKeyProvider.java | 71 +
.../fs/azurebfs/services/SimpleKeyProvider.java | 54 +
.../hadoop/fs/azurebfs/services/package-info.java | 22 +
.../apache/hadoop/fs/azurebfs/utils/Base64.java | 329 +++++
.../fs/azurebfs/utils/SSLSocketFactoryEx.java | 241 ++++
.../apache/hadoop/fs/azurebfs/utils/UriUtils.java | 78 ++
.../hadoop/fs/azurebfs/utils/package-info.java | 22 +
...rg.apache.hadoop.security.token.TokenIdentifier | 1 +
.../org.apache.hadoop.security.token.TokenRenewer | 1 +
.../hadoop-azure/src/site/markdown/abfs.md | 82 ++
.../src/site/markdown/testing_azure.md | 230 +++-
.../fs/azure/AzureBlobStorageTestAccount.java | 22 +-
.../ITestFileSystemOperationExceptionMessage.java | 3 +-
.../fs/azure/ITestNativeFileSystemStatistics.java | 99 ++
.../fs/azure/ITestWasbUriAndConfiguration.java | 26 -
.../fs/azure/NativeAzureFileSystemBaseTest.java | 78 +-
.../fs/azure/integration/AzureTestConstants.java | 6 +-
.../fs/azure/integration/AzureTestUtils.java | 69 +
.../fs/azure/metrics/TestRollingWindowAverage.java | 4 +-
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 345 +++++
.../hadoop/fs/azurebfs/AbstractAbfsScaleTest.java | 59 +
.../fs/azurebfs/AbstractAbfsTestWithTimeout.java | 70 +
.../apache/hadoop/fs/azurebfs/ITestAbfsClient.java | 82 ++
.../fs/azurebfs/ITestAbfsIdentityTransformer.java | 304 +++++
.../fs/azurebfs/ITestAbfsReadWriteAndSeek.java | 89 ++
.../azurebfs/ITestAbfsRestOperationException.java | 75 ++
.../azurebfs/ITestAzureBlobFileSystemAppend.java | 79 ++
.../ITestAzureBlobFileSystemAuthorization.java | 418 ++++++
.../ITestAzureBlobFileSystemBackCompat.java | 86 ++
.../fs/azurebfs/ITestAzureBlobFileSystemCopy.java | 96 ++
.../azurebfs/ITestAzureBlobFileSystemCreate.java | 107 ++
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 144 ++
.../fs/azurebfs/ITestAzureBlobFileSystemE2E.java | 242 ++++
.../azurebfs/ITestAzureBlobFileSystemE2EScale.java | 120 ++
.../ITestAzureBlobFileSystemFileStatus.java | 125 ++
.../azurebfs/ITestAzureBlobFileSystemFinalize.java | 62 +
.../fs/azurebfs/ITestAzureBlobFileSystemFlush.java | 382 ++++++
.../ITestAzureBlobFileSystemInitAndCreate.java | 52 +
.../ITestAzureBlobFileSystemListStatus.java | 245 ++++
.../ITestAzureBlobFileSystemMkDir.java} | 44 +-
.../fs/azurebfs/ITestAzureBlobFileSystemOauth.java | 178 +++
.../ITestAzureBlobFileSystemPermission.java | 108 ++
.../ITestAzureBlobFileSystemRandomRead.java | 586 ++++++++
.../azurebfs/ITestAzureBlobFileSystemRename.java | 152 +++
.../ITestAzureBlobFileSystemRenameUnicode.java | 98 ++
.../fs/azurebfs/ITestAzureBlobFilesystemAcl.java | 1410 ++++++++++++++++++++
.../hadoop/fs/azurebfs/ITestClientUrlScheme.java | 101 ++
.../fs/azurebfs/ITestFileSystemInitialization.java | 77 ++
.../fs/azurebfs/ITestFileSystemProperties.java | 119 ++
.../fs/azurebfs/ITestFileSystemRegistration.java | 113 ++
.../fs/azurebfs/ITestGetNameSpaceEnabled.java | 103 ++
.../fs/azurebfs/ITestOauthOverAbfsScheme.java | 63 +
.../fs/azurebfs/ITestWasbAbfsCompatibility.java | 194 +++
.../TestAbfsConfigurationFieldsValidation.java | 179 +++
.../fs/azurebfs/TestAccountConfiguration.java | 285 ++++
.../azurebfs/constants/TestConfigurationKeys.java | 42 +
.../hadoop/fs/azurebfs/constants/package-info.java | 22 +
.../azurebfs/contract/ABFSContractTestBinding.java | 67 +
.../azurebfs/contract/AbfsFileSystemContract.java | 64 +
.../ITestAbfsFileSystemContractAppend.java | 61 +
.../ITestAbfsFileSystemContractConcat.java | 51 +
.../ITestAbfsFileSystemContractCreate.java | 52 +
.../ITestAbfsFileSystemContractDelete.java | 52 +
.../ITestAbfsFileSystemContractDistCp.java | 52 +
.../ITestAbfsFileSystemContractGetFileStatus.java | 51 +
.../contract/ITestAbfsFileSystemContractMkdir.java | 52 +
.../contract/ITestAbfsFileSystemContractOpen.java | 52 +
.../ITestAbfsFileSystemContractRename.java | 52 +
.../ITestAbfsFileSystemContractRootDirectory.java | 57 +
.../ITestAbfsFileSystemContractSecureDistCp.java | 49 +
.../contract/ITestAbfsFileSystemContractSeek.java | 52 +
.../ITestAbfsFileSystemContractSetTimes.java | 51 +
.../contract/ITestAzureBlobFileSystemBasics.java | 105 ++
.../hadoop/fs/azurebfs/contract/package-info.java | 22 +
.../diagnostics/TestConfigurationValidators.java | 121 ++
.../fs/azurebfs/diagnostics/package-info.java | 22 +
.../fs/azurebfs/extensions/MockAbfsAuthorizer.java | 90 ++
.../fs/azurebfs/extensions/package-info.java | 22 +
.../apache/hadoop/fs/azurebfs/package-info.java | 22 +
.../fs/azurebfs/services/TestAbfsClient.java | 94 ++
.../services/TestAbfsClientThrottlingAnalyzer.java | 177 +++
.../fs/azurebfs/services/TestQueryParams.java | 72 +
.../services/TestShellDecryptionKeyProvider.java | 92 ++
.../hadoop/fs/azurebfs/services/package-info.java | 22 +
.../hadoop/fs/azurebfs/utils/AbfsTestUtils.java | 85 ++
.../hadoop/fs/azurebfs/utils/AclTestHelpers.java | 119 ++
.../hadoop/fs/azurebfs/utils/Parallelized.java | 60 +
.../hadoop/fs/azurebfs/utils/TestUriUtils.java | 48 +
.../hadoop/fs/azurebfs/utils/package-info.java | 22 +
.../hadoop-azure/src/test/resources/abfs.xml | 64 +
.../hadoop-azure/src/test/resources/azure-test.xml | 56 +-
.../src/test/resources/log4j.properties | 35 +
189 files changed, 21605 insertions(+), 462 deletions(-)
create mode 100644 hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/Abfs.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/Abfss.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/SecureAzureBlobFileSystem.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsHttpConstants.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/ConfigurationKeys.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemUriSchemes.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpHeaderConfigurations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpQueryParams.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/annotations/ConfigurationValidationAnnotations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/annotations/package-info.java
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/contracts/diagnostics/ConfigurationValidator.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/diagnostics/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsRestOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AzureBlobFileSystemException.java
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/ConfigurationPropertyNotFoundException.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/FileSystemOperationUnhandledException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidAbfsRestOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidAclOperationException.java
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidFileSystemPropertyException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidUriAuthorityException.java
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidUriException.java} (50%)
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/KeyProviderException.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/TimeoutException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/TokenAccessProviderException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/AzureServiceErrorCode.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ListResultEntrySchema.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ListResultSchema.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ReadBufferStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/Base64StringConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/BooleanConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/ConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/IntegerConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/LongConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/StringConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/package-info.java
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizationException.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/CustomDelegationTokenManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/CustomTokenProviderAdaptee.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AccessTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADToken.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/ClientCredsTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/CustomTokenProviderAdapter.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/IdentityTransformer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/MsiTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/QueryParams.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/RefreshTokenBasedTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/UserPasswordTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/package.html
copy hadoop-tools/hadoop-azure/src/{test/java/org/apache/hadoop/fs/azure/metrics/TestRollingWindowAverage.java => main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDelegationTokenIdentifier.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDelegationTokenManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsTokenRenewer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsAclHelper.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
copy hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/{azure/ClientThrottlingAnalyzer.java => azurebfs/services/AbfsClientThrottlingAnalyzer.java} (81%)
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpHeader.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsPermission.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperationType.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AuthType.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/KeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBuffer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferWorker.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SharedKeyCredentials.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ShellDecryptionKeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SimpleKeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/Base64.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/SSLSocketFactoryEx.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestNativeFileSystemStatistics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsScaleTest.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsTestWithTimeout.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsIdentityTransformer.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsReadWriteAndSeek.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsRestOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAppend.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemBackCompat.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCopy.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCreate.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelete.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemE2E.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemE2EScale.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFileStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFinalize.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFlush.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemInitAndCreate.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemListStatus.java
copy hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/{azure/metrics/TestRollingWindowAverage.java => azurebfs/ITestAzureBlobFileSystemMkDir.java} (50%)
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemOauth.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemPermission.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRandomRead.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRenameUnicode.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFilesystemAcl.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestClientUrlScheme.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemInitialization.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemProperties.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemRegistration.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestGetNameSpaceEnabled.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestOauthOverAbfsScheme.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestWasbAbfsCompatibility.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAccountConfiguration.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ABFSContractTestBinding.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/AbfsFileSystemContract.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractAppend.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractConcat.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractCreate.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractDelete.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractDistCp.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractGetFileStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractMkdir.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractOpen.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractRename.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractRootDirectory.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSecureDistCp.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSeek.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSetTimes.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAzureBlobFileSystemBasics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/TestConfigurationValidators.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockAbfsAuthorizer.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClientThrottlingAnalyzer.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestQueryParams.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AbfsTestUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AclTestHelpers.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/Parallelized.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/TestUriUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/resources/abfs.xml
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org