You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by ey...@apache.org on 2020/05/20 16:28:18 UTC
[hadoop] reference refs/remotes/origin/branch-3.3 updated (4bd37f2
-> e452163)
This is an automated email from the ASF dual-hosted git repository.
eyang pushed a change to reference refs/remotes/origin/branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from 4bd37f2 YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja
add 94da630 HADOOP-16465 listLocatedStatus() optimisation (#1943)
add b4ba9be HADOOP-16985. Handle release package related issues (#1957)
add aaad947 HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell.
add 96d7ceb HADOOP-13873. log DNS addresses on s3a initialization.
add 49ae9b2 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
add de5d433 HADOOP-16972. Ignore AuthenticationFilterInitializer for KMSWebServer. (#1961)
add a6c718f HDFS-15218. RBF: MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. Contributed by Surendra Singh Lilhore.
add 9c81b17 HADOOP-16959. Resolve hadoop-cos dependency conflict. Contributed by Yang Yu.
add de9a6b4 HADOOP-16986. S3A to not need wildfly on the classpath. (#1948)
add 0982f56 HADOOP-16953. tuning s3guard disabled warnings (#1962)
add 5459dd6 HDFS-14742. RBF: TestRouterFaultTolerant tests are flaky
add 54a64e5 HADOOP-16944. Use Yetus 0.12.0 in GitHub PR (#1917)
add f74a571 HADOOP-16910 : ABFS Streams to update FileSystem.Statistics counters on IO. (#1918). Contributed by Mehakmeet Singh.
add 98fdbb8 HADOOP-16965. Refactor abfs stream configuration. (#1956)
add f53ded6 HADOOP-16922. ABFS: Change User-Agent header (#1938)
add 5b92d73 HADOOP-17001. The suffix name of the unified compression class. Contributed by bianqi
add 47b330d HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina
add 1340518 YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
add 32fb174 Hadoop 16857. ABFS: Stop CustomTokenProvider retry logic to depend on AbfsRestOp retry policy
add 2471ba8 HADOOP-16914 Adding Output Stream Counters in ABFS (#1899)
add 497c7a1 YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939)
add 61ca459 YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
add 7a3f190 YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
add d6c48f8 HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun.
add c276938 HADOOP-16886. Add hadoop.http.idle_timeout.ms to core-default.xml. Contributed by Lisheng Sun.
add 1c19107 HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964)
add 1db1992 HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena
add 9e79cca HADOOP-17007. hadoop-cos fails to build. Contributed by Yang Yu.
add c0b7b38 YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
add 68d8802 HDFS-1820. FTPFileSystem attempts to close the outputstream even when it is not initialised. (#1952)
add 741fcf2 MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles).
add 62ee597 YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
add d504574 HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
add e45faae YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori
add 08dbe32 HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.(#1980)
add 75433a6 YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
add 8ffe1f3 YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
add a34174f HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
add dfa7f16 Preparing for 3.3.1 development
add ccebc9d HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena.
add 0f27c04 SPNEGO TLS verification
add e32e138 HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter (#1992)
add df16146 Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
add 1d83500 YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
add 8809b43 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
add 78c9790 HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. (#1998)
add 4e0d99c HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.
add 7aade93 YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
add edf52d2 HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.
add 6a0aeea HADOOP-17025. Fix invalid metastore configuration in S3GuardTool tests. (#1994)
add 8178141 HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.
add 763a799 HDFS-15343. TestConfiguredFailoverProxyProvider is failing. (#2001)
add 6ccb1cd HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest.
add 282427f HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. (#2003)
add 61a4cd5 HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.
add d1e5e39 HDFS-15270. Account for *env == NULL in hdfsThreadDestructor (#1951)
add 95e4ed1 YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
add ef41229 HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei.
add 433aaee HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.
add 564d321 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T
add ee58c11 Revert "SPNEGO TLS verification"
add 74887ab YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
add 7cef710 YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
add 7f2cf33 Revert "HADOOP-8143. Change distcp to have -pb on by default."
add f2f7273 Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."
add 77587ff HADOOP-17042. Hadoop distcp throws 'ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found'. Contributed by Aki Tanaka.
add ad9a6a0 HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016)
add acae31a HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
add 032ccba HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.
add 53d22fd Revert "HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He."
add 289f8ac Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016)"
add 240cba7 HDFS-15202 Boost short circuit cache (rebase PR-1884) (#2016)
add 3915d1a HDFS-15356. Unify configuration `dfs.ha.allow.stale.reads` to DFSConfigKeys. Contributed by Xiaoqiao He.
add eb045ea HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
add e452163 HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang
No new revisions were added by this update.
Summary of changes:
.gitignore | 1 +
Jenkinsfile | 7 +-
dev-support/bin/create-release | 4 +-
hadoop-assemblies/pom.xml | 4 +-
.../src/main/resources/assemblies/hadoop-src.xml | 1 +
hadoop-build-tools/pom.xml | 2 +-
hadoop-client-modules/hadoop-client-api/pom.xml | 4 +-
.../hadoop-client-check-invariants/pom.xml | 4 +-
.../hadoop-client-check-test-invariants/pom.xml | 4 +-
.../hadoop-client-integration-tests/pom.xml | 4 +-
.../hadoop-client-minicluster/pom.xml | 4 +-
.../hadoop-client-runtime/pom.xml | 4 +-
hadoop-client-modules/hadoop-client/pom.xml | 4 +-
hadoop-client-modules/pom.xml | 2 +-
.../hadoop-cloud-storage/pom.xml | 9 +-
.../hadoop-cos/dev-support/findbugs-exclude.xml | 5 +
hadoop-cloud-storage-project/hadoop-cos/pom.xml | 22 +-
.../java/org/apache/hadoop/fs/cosn/BufferPool.java | 39 +-
.../apache/hadoop/fs/cosn/CosNFileReadTask.java | 1 -
.../java/org/apache/hadoop/fs/cosn/CosNUtils.java | 46 ++-
.../hadoop/fs/cosn/CosNativeFileSystemStore.java | 28 +-
...er.java => AbstractCOSCredentialsProvider.java} | 44 +--
...erList.java => COSCredentialsProviderList.java} | 42 ++-
...=> EnvironmentVariableCredentialsProvider.java} | 32 +-
...rovider.java => SimpleCredentialsProvider.java} | 34 +-
.../src/site/markdown/cloud-storage/index.md | 13 +-
.../apache/hadoop/fs/cosn/TestCosCredentials.java | 134 +++++++
hadoop-cloud-storage-project/pom.xml | 4 +-
hadoop-common-project/hadoop-annotations/pom.xml | 4 +-
hadoop-common-project/hadoop-auth-examples/pom.xml | 4 +-
hadoop-common-project/hadoop-auth/pom.xml | 4 +-
hadoop-common-project/hadoop-common/pom.xml | 9 +-
.../hadoop-common/src/main/bin/hadoop-functions.sh | 2 +-
.../hadoop/fs/CommonConfigurationKeysPublic.java | 9 +
.../java/org/apache/hadoop/fs/FsConstants.java | 2 +
.../org/apache/hadoop/fs/ftp/FTPFileSystem.java | 27 +-
.../apache/hadoop/fs/viewfs/ViewFileSystem.java | 49 ++-
.../fs/viewfs/ViewFileSystemOverloadScheme.java | 204 ++++++++++
.../org/apache/hadoop/ha/ZKFailoverController.java | 3 +-
.../org/apache/hadoop/io/compress/BZip2Codec.java | 2 +-
.../apache/hadoop/io/compress/CodecConstants.java | 68 ++++
.../apache/hadoop/io/compress/DefaultCodec.java | 2 +-
.../org/apache/hadoop/io/compress/GzipCodec.java | 2 +-
.../org/apache/hadoop/io/compress/Lz4Codec.java | 2 +-
.../hadoop/io/compress/PassthroughCodec.java | 3 +-
.../org/apache/hadoop/io/compress/SnappyCodec.java | 2 +-
.../apache/hadoop/io/compress/ZStandardCodec.java | 2 +-
.../org/apache/hadoop/net/NetworkTopology.java | 6 +-
.../security/ssl/DelegatingSSLSocketFactory.java | 142 +++----
.../src/main/resources/core-default.xml | 8 +
.../hadoop/fs/FileSystemContractBaseTest.java | 3 +-
.../org/apache/hadoop/fs/ftp/FtpTestServer.java | 99 +++++
.../apache/hadoop/fs/ftp/TestFTPFileSystem.java | 84 ++++-
...iewFileSystemOverloadSchemeLocalFileSystem.java | 149 ++++++++
.../hadoop/fs/viewfs/ViewFileSystemBaseTest.java | 4 +-
.../io/compress/CompressDecompressTester.java | 72 ++--
.../snappy/TestSnappyCompressorDecompressor.java | 102 +++--
.../apache/hadoop/net/MockDomainNameResolver.java | 4 +-
hadoop-common-project/hadoop-kms/pom.xml | 4 +-
.../hadoop/crypto/key/kms/server/KMSWebServer.java | 20 +
.../hadoop/crypto/key/kms/server/TestKMS.java | 42 +++
hadoop-common-project/hadoop-minikdc/pom.xml | 4 +-
hadoop-common-project/hadoop-nfs/pom.xml | 4 +-
hadoop-common-project/hadoop-registry/pom.xml | 4 +-
hadoop-common-project/pom.xml | 4 +-
hadoop-dist/pom.xml | 4 +-
hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 4 +-
.../java/org/apache/hadoop/hdfs/ClientContext.java | 20 +-
.../org/apache/hadoop/hdfs/DFSInputStream.java | 26 +-
.../apache/hadoop/hdfs/DistributedFileSystem.java | 3 +-
.../hadoop/hdfs/client/HdfsClientConfigKeys.java | 5 +
.../hdfs/client/impl/BlockReaderFactory.java | 12 +-
.../hadoop/hdfs/client/impl/DfsClientConf.java | 34 +-
.../apache/hadoop/hdfs/protocol/LocatedBlock.java | 3 +-
hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 4 +-
.../hadoop-hdfs-native-client/pom.xml | 4 +-
.../src/main/native/libhdfs/hdfs.c | 2 +-
.../native/libhdfs/os/posix/thread_local_storage.c | 2 +-
.../libhdfs/os/windows/thread_local_storage.c | 4 +-
hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 4 +-
hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 4 +-
.../router/MountTableRefresherService.java | 9 +-
.../router/MountTableRefresherThread.java | 16 +-
.../server/federation/FederationTestUtils.java | 5 +-
.../hdfs/server/federation/MockNamenode.java | 7 +-
.../federation/router/TestRouterFaultTolerant.java | 1 +
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 4 +-
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 11 +
.../main/java/org/apache/hadoop/hdfs/HAUtil.java | 7 +-
.../hdfs/qjournal/client/QuorumJournalManager.java | 4 +-
.../hadoop/hdfs/server/balancer/Balancer.java | 39 +-
.../hdfs/server/balancer/NameNodeConnector.java | 85 +++++
.../AvailableSpaceBlockPlacementPolicy.java | 4 +-
.../server/blockmanagement/DatanodeManager.java | 26 +-
.../common/HostRestrictingAuthorizationFilter.java | 11 +-
.../hadoop/hdfs/server/datanode/BlockScanner.java | 6 +
.../hadoop/hdfs/server/datanode/VolumeScanner.java | 22 ++
.../datanode/erasurecode/ErasureCodingWorker.java | 6 +-
.../erasurecode/StripedBlockReconstructor.java | 6 +-
.../datanode/erasurecode/StripedReconstructor.java | 4 +
.../hadoop/hdfs/server/namenode/CacheManager.java | 14 +
.../hadoop/hdfs/server/namenode/FSDirConcatOp.java | 2 +-
.../hadoop/hdfs/server/namenode/FSDirectory.java | 13 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 1 +
.../apache/hadoop/hdfs/server/namenode/INode.java | 2 +-
.../hdfs/server/namenode/INodeDirectory.java | 2 +-
.../hadoop/hdfs/server/namenode/ImageServlet.java | 39 +-
.../hdfs/server/namenode/ha/EditLogTailer.java | 24 +-
.../hadoop/hdfs/tools/DFSZKFailoverController.java | 28 +-
.../src/main/resources/hdfs-default.xml | 52 +++
.../hadoop/fs/TestEnhancedByteBufferAccess.java | 6 +-
...SystemOverloadSchemeHdfsFileSystemContract.java | 126 +++++++
...ViewFileSystemOverloadSchemeWithHdfsScheme.java | 410 +++++++++++++++++++++
.../org/apache/hadoop/hdfs/TestDFSInputStream.java | 68 ++++
.../hadoop/hdfs/TestReconstructStripedFile.java | 6 +
.../hdfs/client/impl/TestBlockReaderFactory.java | 14 +-
.../hdfs/client/impl/TestBlockReaderLocal.java | 189 +++++++---
.../balancer/TestBalancerWithHANameNodes.java | 56 ++-
.../blockmanagement/TestDatanodeManager.java | 241 ++++++++++++
.../TestHostRestrictingAuthorizationFilter.java | 25 ++
.../hdfs/server/datanode/BlockReportTestBase.java | 3 +-
.../hdfs/server/datanode/TestBlockScanner.java | 38 ++
.../hdfs/server/namenode/TestCacheDirectives.java | 6 +-
.../hdfs/server/namenode/TestCheckpoint.java | 53 ++-
.../hdfs/server/namenode/TestFileTruncate.java | 34 ++
.../hdfs/server/namenode/TestHDFSConcat.java | 39 ++
.../hdfs/server/namenode/TestListOpenFiles.java | 26 ++
.../namenode/ha/TestStandbyInProgressTail.java | 28 ++
.../hdfs/shortcircuit/TestShortCircuitCache.java | 19 +-
.../hdfs/tools/TestDFSZKFailoverController.java | 24 ++
.../src/test/resources/testHDFSConf.xml | 48 +--
hadoop-hdfs-project/pom.xml | 4 +-
.../hadoop-mapreduce-client-app/pom.xml | 4 +-
.../hadoop-mapreduce-client-common/pom.xml | 4 +-
.../hadoop-mapreduce-client-core/pom.xml | 4 +-
.../java/org/apache/hadoop/mapred/IndexCache.java | 95 +++--
.../org/apache/hadoop/mapred/TestIndexCache.java | 56 +--
.../hadoop-mapreduce-client-hs-plugins/pom.xml | 4 +-
.../hadoop-mapreduce-client-hs/pom.xml | 4 +-
.../mapreduce/v2/hs/webapp/HsWebServices.java | 18 +-
.../hadoop-mapreduce-client-jobclient/pom.xml | 4 +-
.../hadoop-mapreduce-client-nativetask/pom.xml | 4 +-
.../hadoop-mapreduce-client-shuffle/pom.xml | 4 +-
.../hadoop-mapreduce-client-uploader/pom.xml | 4 +-
.../hadoop-mapreduce-client/pom.xml | 4 +-
.../hadoop-mapreduce-examples/pom.xml | 4 +-
hadoop-mapreduce-project/pom.xml | 4 +-
hadoop-maven-plugins/pom.xml | 2 +-
hadoop-minicluster/pom.xml | 4 +-
hadoop-project-dist/pom.xml | 4 +-
hadoop-project/pom.xml | 18 +-
hadoop-tools/hadoop-aliyun/pom.xml | 2 +-
hadoop-tools/hadoop-archive-logs/pom.xml | 4 +-
hadoop-tools/hadoop-archives/pom.xml | 4 +-
hadoop-tools/hadoop-aws/pom.xml | 4 +-
.../java/org/apache/hadoop/fs/s3a/Constants.java | 10 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 92 +++--
.../apache/hadoop/fs/s3a/impl/NetworkBinding.java | 66 +++-
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 38 +-
.../site/markdown/tools/hadoop-aws/performance.md | 36 +-
.../src/site/markdown/tools/hadoop-aws/s3guard.md | 29 ++
.../tools/hadoop-aws/troubleshooting_s3a.md | 13 +
.../hadoop/fs/s3a/ITestS3AFileOperationCost.java | 57 +++
.../fs/s3a/TestWildflyAndOpenSSLBinding.java | 152 ++++++++
.../s3a/s3guard/AbstractS3GuardToolTestBase.java | 3 +-
.../apache/hadoop/fs/s3a/s3guard/TestS3Guard.java | 14 +-
hadoop-tools/hadoop-azure-datalake/pom.xml | 2 +-
hadoop-tools/hadoop-azure/pom.xml | 2 +-
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 34 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 4 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 58 ++-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 3 +
.../fs/azurebfs/constants/ConfigurationKeys.java | 3 +
.../constants/FileSystemConfigurations.java | 6 +
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 21 +-
.../oauth2/CustomTokenProviderAdapter.java | 46 ++-
.../hadoop/fs/azurebfs/services/AbfsClient.java | 70 ++--
.../fs/azurebfs/services/AbfsInputStream.java | 32 +-
.../azurebfs/services/AbfsInputStreamContext.java | 70 ++++
.../fs/azurebfs/services/AbfsOutputStream.java | 88 ++++-
.../azurebfs/services/AbfsOutputStreamContext.java | 80 ++++
.../services/AbfsOutputStreamStatistics.java | 77 ++++
.../services/AbfsOutputStreamStatisticsImpl.java | 176 +++++++++
.../fs/azurebfs/services/AbfsStreamContext.java | 13 +-
.../hadoop-azure/src/site/markdown/abfs.md | 2 +
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 26 ++
.../fs/azurebfs/AbstractAbfsTestWithTimeout.java | 59 +++
.../azurebfs/ITestAbfsOutputStreamStatistics.java | 229 ++++++++++++
.../azurebfs/ITestAbfsRestOperationException.java | 13 +-
.../fs/azurebfs/ITestAbfsStreamStatistics.java | 157 ++++++++
.../fs/azurebfs/ITestAzureBlobFileSystemOauth.java | 11 +-
.../azurebfs/TestAbfsOutputStreamStatistics.java | 176 +++++++++
.../fs/azurebfs/services/TestAbfsClient.java | 224 +++++++++--
.../services/TestExponentialRetryPolicy.java | 88 +++++
hadoop-tools/hadoop-datajoin/pom.xml | 4 +-
hadoop-tools/hadoop-distcp/pom.xml | 4 +-
.../org/apache/hadoop/tools/OptionsParser.java | 3 -
.../hadoop-distcp/src/site/markdown/DistCp.md.vm | 2 +-
.../org/apache/hadoop/tools/TestOptionsParser.java | 2 +-
.../hadoop-dynamometer-blockgen/pom.xml | 4 +-
.../hadoop-dynamometer-dist/pom.xml | 4 +-
.../hadoop-dynamometer-infra/pom.xml | 4 +-
.../hadoop-dynamometer-workload/pom.xml | 4 +-
hadoop-tools/hadoop-dynamometer/pom.xml | 4 +-
hadoop-tools/hadoop-extras/pom.xml | 4 +-
hadoop-tools/hadoop-fs2img/pom.xml | 4 +-
hadoop-tools/hadoop-gridmix/pom.xml | 4 +-
hadoop-tools/hadoop-kafka/pom.xml | 4 +-
hadoop-tools/hadoop-openstack/pom.xml | 4 +-
hadoop-tools/hadoop-pipes/pom.xml | 4 +-
hadoop-tools/hadoop-resourceestimator/pom.xml | 2 +-
hadoop-tools/hadoop-rumen/pom.xml | 4 +-
hadoop-tools/hadoop-sls/pom.xml | 4 +-
hadoop-tools/hadoop-streaming/pom.xml | 4 +-
hadoop-tools/hadoop-tools-dist/pom.xml | 4 +-
hadoop-tools/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 4 +-
.../api/records/ApplicationSubmissionContext.java | 4 +-
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 16 +-
.../pom.xml | 2 +-
.../pom.xml | 2 +-
.../hadoop-yarn-applications-catalog/pom.xml | 2 +-
.../pom.xml | 4 +-
.../hadoop-yarn-applications-mawo-core/pom.xml | 2 +-
.../hadoop-yarn-applications-mawo/pom.xml | 2 +-
.../pom.xml | 4 +-
.../hadoop-yarn-services-api/pom.xml | 2 +-
.../hadoop-yarn-services-core/pom.xml | 2 +-
.../hadoop-yarn-services/pom.xml | 2 +-
.../hadoop-yarn/hadoop-yarn-applications/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 4 +-
.../pb/ApplicationSubmissionContextPBImpl.java | 20 -
.../src/main/resources/yarn-default.xml | 13 +
.../hadoop/yarn/webapp/WebServicesTestUtils.java | 18 +
.../hadoop-yarn/hadoop-yarn-csi/pom.xml | 2 +-
.../hadoop-yarn/hadoop-yarn-registry/pom.xml | 4 +-
.../pom.xml | 4 +-
.../webapp/AHSWebServices.java | 22 +-
.../hadoop-yarn-server-common/pom.xml | 14 +-
.../hadoop/yarn/server/webapp/LogServlet.java | 53 ++-
.../hadoop/yarn/server/webapp/LogWebService.java | 18 +-
.../yarn/server/webapp/YarnWebServiceParams.java | 1 +
.../hadoop-yarn-server-nodemanager/pom.xml | 4 +-
.../scheduler/TestContainerSchedulerQueuing.java | 52 +--
.../hadoop-yarn-server-resourcemanager/pom.xml | 4 +-
.../server/resourcemanager/ClientRMService.java | 27 ++
.../yarn/server/resourcemanager/DBManager.java | 131 +++++++
.../yarn/server/resourcemanager/RMAppManager.java | 85 ++---
.../recovery/LeveldbRMStateStore.java | 176 ++-------
.../yarn/server/resourcemanager/rmapp/RMApp.java | 4 -
.../server/resourcemanager/rmapp/RMAppImpl.java | 10 -
.../scheduler/capacity/AbstractCSQueue.java | 4 -
.../scheduler/capacity/CapacityScheduler.java | 6 +-
.../capacity/CapacitySchedulerConfigValidator.java | 18 +-
.../capacity/CapacitySchedulerConfiguration.java | 3 +-
.../conf/FSSchedulerConfigurationStore.java | 1 +
.../capacity/conf/InMemoryConfigurationStore.java | 6 +
.../capacity/conf/LeveldbConfigurationStore.java | 105 +-----
.../QueueAdminConfigurationMutationACLPolicy.java | 60 ++-
.../capacity/conf/YarnConfigurationStore.java | 2 +-
.../capacity/conf/ZKConfigurationStore.java | 8 +
.../webapp/dao/CapacitySchedulerQueueInfo.java | 29 ++
.../webapp/dao/LeafQueueTemplateInfo.java | 91 +++++
.../server/resourcemanager/AppManagerTestBase.java | 31 --
.../server/resourcemanager/TestAppManager.java | 120 +-----
.../resourcemanager/TestClientRMService.java | 45 +++
.../applicationsmanager/MockAsm.java | 11 -
.../recovery/TestLeveldbRMStateStore.java | 21 +-
.../server/resourcemanager/rmapp/MockRMApp.java | 20 -
.../capacity/TestApplicationPriorityACLs.java | 1 +
.../scheduler/capacity/TestContainerResizing.java | 6 +-
.../scheduler/capacity/TestQueueState.java | 17 +
.../capacity/conf/ConfigurationStoreBaseTest.java | 21 +-
.../conf/PersistentConfigurationStoreBaseTest.java | 4 +
.../conf/TestFSSchedulerConfigurationStore.java | 162 ++++----
.../webapp/TestRMWebServicesCapacitySched.java | 51 ++-
.../hadoop-yarn-server-router/pom.xml | 4 +-
.../hadoop-yarn-server-sharedcachemanager/pom.xml | 4 +-
.../hadoop-yarn-server-tests/pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 2 +-
.../pom.xml | 4 +-
.../pom.xml | 2 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../hadoop-yarn-server-timelineservice/pom.xml | 4 +-
.../hadoop-yarn-server-web-proxy/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-server/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-site/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-ui/pom.xml | 4 +-
.../main/webapp/app/adapters/yarn-app-jhs-log.js | 10 +-
...app-jhs-log.js => yarn-app-jhs-redirect-log.js} | 10 +-
.../src/main/webapp/app/adapters/yarn-app-log.js | 10 +-
...arn-app-jhs-log.js => yarn-app-redirect-log.js} | 9 +-
.../src/main/webapp/app/adapters/yarn-jhs-log.js | 12 +-
.../{yarn-jhs-log.js => yarn-jhs-redirect-log.js} | 8 +-
.../src/main/webapp/app/adapters/yarn-log.js | 24 +-
.../yarn-redirect-log.js} | 15 +-
.../main/webapp/app/controllers/yarn-app/logs.js | 42 ++-
.../log-adapter-helper.js} | 19 +-
.../src/main/webapp/app/models/yarn-app-jhs-log.js | 7 +-
...arn-app-log.js => yarn-app-jhs-redirect-log.js} | 7 +-
.../src/main/webapp/app/models/yarn-app-log.js | 7 +-
.../{yarn-app-log.js => yarn-app-redirect-log.js} | 7 +-
.../src/main/webapp/app/models/yarn-jhs-log.js | 11 +-
...arn-app-jhs-log.js => yarn-jhs-redirect-log.js} | 9 +-
.../src/main/webapp/app/models/yarn-log.js | 11 +-
.../{yarn-app-jhs-log.js => yarn-redirect-log.js} | 9 +-
.../webapp/app/serializers/yarn-app-jhs-log.js | 5 +-
...app-jhs-log.js => yarn-app-jhs-redirect-log.js} | 6 +-
.../main/webapp/app/serializers/yarn-app-log.js | 5 +-
...arn-app-jhs-log.js => yarn-app-redirect-log.js} | 6 +-
.../main/webapp/app/serializers/yarn-jhs-log.js | 1 +
.../{yarn-jhs-log.js => yarn-jhs-redirect-log.js} | 1 +
.../{yarn-log.js => yarn-redirect-log.js} | 0
hadoop-yarn-project/hadoop-yarn/pom.xml | 4 +-
hadoop-yarn-project/pom.xml | 4 +-
pom.xml | 4 +-
322 files changed, 6474 insertions(+), 1743 deletions(-)
copy hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/{SimpleCredentialProvider.java => AbstractCOSCredentialsProvider.java} (50%)
rename hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/{COSCredentialProviderList.java => COSCredentialsProviderList.java} (80%)
rename hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/{EnvironmentVariableCredentialProvider.java => EnvironmentVariableCredentialsProvider.java} (70%)
rename hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/{SimpleCredentialProvider.java => SimpleCredentialsProvider.java} (66%)
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/TestCosCredentials.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystemOverloadScheme.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/CodecConstants.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/ftp/FtpTestServer.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeLocalFileSystem.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeHdfsFileSystemContract.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemOverloadSchemeWithHdfsScheme.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestWildflyAndOpenSSLBinding.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStreamContext.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStreamContext.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStreamStatistics.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStreamStatisticsImpl.java
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-app-log.js => hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsStreamContext.java (79%)
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsOutputStreamStatistics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsStreamStatistics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsOutputStreamStatistics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestExponentialRetryPolicy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/DBManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/LeafQueueTemplateInfo.java
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/{yarn-app-jhs-log.js => yarn-app-jhs-redirect-log.js} (86%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/{yarn-app-jhs-log.js => yarn-app-redirect-log.js} (86%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/{yarn-jhs-log.js => yarn-jhs-redirect-log.js} (82%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/{models/yarn-app-log.js => adapters/yarn-redirect-log.js} (78%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/{models/yarn-app-log.js => helpers/log-adapter-helper.js} (76%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/{yarn-app-log.js => yarn-app-jhs-redirect-log.js} (80%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/{yarn-app-log.js => yarn-app-redirect-log.js} (80%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/{yarn-app-jhs-log.js => yarn-jhs-redirect-log.js} (73%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/{yarn-app-jhs-log.js => yarn-redirect-log.js} (73%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/{yarn-app-jhs-log.js => yarn-app-jhs-redirect-log.js} (88%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/{yarn-app-jhs-log.js => yarn-app-redirect-log.js} (88%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/{yarn-jhs-log.js => yarn-jhs-redirect-log.js} (98%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/{yarn-log.js => yarn-redirect-log.js} (100%)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org