You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by br...@apache.org on 2019/06/23 11:34:56 UTC
[hadoop] branch HDFS-13891 updated (caa285b -> 02597b6)
This is an automated email from the ASF dual-hosted git repository.
brahma pushed a change to branch HDFS-13891
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
discard caa285b HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena.
discard bee9fff HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao.
discard f3e25bb HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma.
discard 3344c95 HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
discard d60e686 HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma.
discard 0bcbdd6 HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena.
discard f3a8e62 HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota.
discard b4ee2e3 HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota.
discard 58a22e9 HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri.
discard 04977cc HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma.
discard 377d7bf HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena.
discard 4c0c9ff HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka.
discard 8f1f042 HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena.
discard 4a16a08 HDFS-13995. RBF: Security documentation. Contributed by CR Hota.
discard a1a28a6 HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.
discard 66f235e HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena.
discard f4101bb HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena.
discard 6f46691 HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li.
discard 9bbc4de HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri.
discard 5b3f123 HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao.
discard 0156a0e HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota.
discard 7e5d043 HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri.
discard bf44a11 HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka.
discard 54d44bc HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.
discard 9c38a6e HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
discard 4f66992 HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
discard 3def419 HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
discard 786a1ce HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
discard 0555771 HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
discard abb0e82 HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
discard fc05a96 HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
discard 1e50408 HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
discard b2fcb25 HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
discard d3acc68 HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
discard 4dd0dd0 HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
discard 200d457 HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
discard 60b8e6e HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
discard 5550323 HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
discard 0a6b4dd HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
discard ad0bcc1 HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
discard 38a12a3 HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
discard 7868115 HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
discard 6631b8d HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
discard 48eb588 HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.
discard abcead9 HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
discard aa8c5ce HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
discard dd7f681 HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
discard ba0c4d2 HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
discard 3c2f611 HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
discard c50f00f HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
discard 499c5a3 HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
discard 3e2f308 HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
discard 255d207 HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
discard 7728424 HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
discard 76f632b HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
discard 1717a96 HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
discard dc094cc HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
discard 52d3ac6 HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
discard a531ee4 HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
discard f6f18b5 HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
discard 3de14b3 HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
discard 6ec52bc HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
discard 3654241 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
discard 06edf6b Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
discard a208772 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
discard 09b76cc HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
discard 0505586 HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
discard 737b47a HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
discard a125863 HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
discard 992c059 HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
discard a26a967 HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
discard 308b9c9 HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
discard 50567f0 HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
discard 8d589ab HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
discard 95b65f1 HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
discard d5370fd HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
discard a55917b HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
add 5906268 HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors
add a85451c YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph.
add ef1cc72 HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828)
add 77c49f2 HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714)
add 9dff6ef HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles
add 67f9a7b MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event.
add 2fc6f85 HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825)
add a315913 HDDS-1517. AllocateBlock call fails with ContainerNotFoundException (#826). Contributed by Shashikant Banerjee.
add 9c61494 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834)
add 5565f2c MAPREDUCE-7198. mapreduce.task.timeout=0 configuration used to disable timeout doesn't work.
add 03aa70f SUBMARINE-83. Refine the documents of submarine targeting 0.2.0 release. Contributed by Zhankun Tang.
add a771e2a HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang.
add ea0b1d8 HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation. Contributed by Prabhu Joseph
add 7b03072 YARN-9080. Added clean up of bucket directories. Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
add e2b7991 YARN-9469. Fix typo in YarnConfiguration.
add f96a2df HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.
add 20a4ec3 HDDS-700. Support rack awared node placement policy based on network topology. Contributed by Sammi Chen.
add 869a1ab HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka
add 64c3985 Fixed checkstyle issues.
add 456bb8a verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion
add 9da62f3 Merge branch 'trunk' into HDDS-1535
add 72bef0f fixed merge error. adapted to new signature of BlockUtils getDB
add ca93760 fixed checkstyle issue post merge
add b4b9120 HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka (#832)
add 6a0e7dd YARN-9482. DistributedShell job with localization fails in unsecure cluster. Contributed by Prabhu Joseph.
add 4b099b8 HDDS-1501 : Create a Recon task interface to update internal DB on updates from OM. (#819)
add 460ba7f YARN-9558. Fixed LogAggregation test cases. Contributed by Prabhu Joseph
add 93d7630 HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta.
add c9393dd Revert "HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong."
add 6d0e79c HDDS-1512. Implement DoubleBuffer in OzoneManager. (#810)
add c31b7b8 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko.
add 55e0c13 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.
add 37900c5 HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
add 9f056d9 YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
add 9f933e6 HADOOP-16323. https everywhere in Maven settings.
add a3745c5 HDDS-1588. Update Ozone dist image to use jdk11. Contributed by Eric Yang.
add f0e44b3 HDDS-1584. Fix TestFailureHandlingByClient tests. Contributed by Shashikant Banerjee (#845).
add 83549db HDDS-1509. TestBlockOutputStreamWithFailures#test2DatanodesFailure fails intermittently. Contributed by Shashikant Banerjee (#805).
add ec92ca6 HDDS-1598. Fix Ozone checkstyle issues on trunk. Contributed by Elek, Marton. (#854)
add b70d1be HDDS-1559. Fix TestReplicationManager. Contributed by Bharat Viswanadham. (#856)
add 72dd790 HDDS-1534. freon should return non-zero exit code on failure. Contributed by Nilotpal Nandi.
add 9078e28 YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
add 4a692bc YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang.
add 9f0d341 HDDS-1533. JVM exit on TestHddsDatanodeService
add 06fa4d2 HDDS-1565. Rename k8s-dev and k8s-dev-push profiles to docker and docker-push.
add d1ec1c5 HDDS-1580. Obtain Handler reference in ContainerScrubber (#842)
add d8b18e8 HDDS-1602. Fix TestContainerPersistence#testDeleteBlockTwice. (#858)
add fb0b39f HDDS-1536. testSCMSafeModeRestrictedOp is failing consistently. Contributed by Xiaoyu Yao. (#865)
add d78854b HDFS-14434. Ignore user.name query parameter in secure WebHDFS. Contributed by KWON BYUNGCHANG
add 79d14d0 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862)
add 0c73dba HADOOP-16332. Remove S3A dependency on http core.
add 7f2e87a HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866)
add 346c2b7 HDDS-1559. Include committedBytes to determine Out of Space in VolumeChoosingPolicy. Contributed by Supratim Deka (#841)
add afd8440 HADOOP-16331. Fix ASF License check in pom.xml
add 544876f YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
add 3c63551 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph.
add abf76ac YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
add 751f0df HDDS-1231. Add ChillMode metrics. Contributed by Bharat Viswanadham.
add 0ead209 HDDS-1542. Create Radix tree to support ozone prefix ACLs. Contributed by Xiaoyu Yao.
add 9ad7cad HDDS-1530. Freon support big files larger than 2GB and add --bufferSize and --validateWrites options. Contributed by Xudong Cao. (#830)
add c1caab4 HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena.
add d23e8e9 HADOOP-16334. Fix yetus-wrapper not working when HADOOP_YETUS_VERSION >= 0.9.0
add 12be6ff HDDS-1539. Implement addAcl,removeAcl,setAcl,getAcl for Volume. Contributed Ajay Kumar. (#847)
add 2b303e9 SUBMARINE-86. Bump TonY version to pick up a docker related fix & Update documentation. Contributed by Keqiu Hu.
add 18c1eeb HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833).
add 8c8cb2d HDDS-1502. Fix the compilation issue with commit 18c1eebc08f93055ffdef1812247b439c8404163.
add 1b041d4 HDDS-1497. Refactor blockade Tests. Contributed by Nilotpal Nandi.
add 0b907bc0 HDDS-1581. Atleast one of the metadata dir config property must be tagged as REQUIRED.
add 33419a9 HDDS-1568 : Add RocksDB metrics to OM. Contributed by Aravindan Vijayan
add baee715 HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)
add 30c6dd9 YARN-9452. Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures. Contributed by Prabhu Joseph.
add f1552f6 YARN-9553. Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines. Contributed by Prabhu Joseph.
add 6f5a36c HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta.
add 33c62f8 HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao.
add c965f7f HDDS-1551. Implement Bucket Write Requests to use Cache and DoubleBuffer. (#850)
add 9122b9b HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee
add 219e286 HDDS-1608. Support Ozone Prefix ACLs in OM metadata table. Contributed by Xiaoyu Yao. (#875)
add 35f1014 HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default.
add 7861a5e YARN-9033. ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled. Contributed by Zhankun Tang.
add e49162f YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
add 52128e3 YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B.
add 1ae062c HDDS-1382. Create customized CSI server for Ozone
add 4cb559e YARN-9027. Fixed LevelDBCacheTimelineStore initialization. Contributed by Prabhu Joseph
add c1d2d92 HDDS-1540. Implement addAcl,removeAcl,setAcl,getAcl for Bucket. Contributed by Ajay Kumar. (#874)
add 2210897 YARN-9592. Use Logger format in ContainersMonitorImpl. Contributed by Inigo Goiri.
add 08363db SUBMARINE-87. Add an apache rat check script for submarine. Contributed by Zhankun Tang.
add 2185249 Preparing for submarine-0.3.0 development
add 4530f45 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T.
add 2a97a37 Revert "HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)"
add 7f46dda YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB.
add 59719dc YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB.
add bd2590d YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
add f327112 HDDS-1558. IllegalArgumentException while processing container Reports.
add 21de9af HDDS-1625 : ConcurrentModificationException when SCM has containers of different owners. (#883)
add 277e9a8 Opening of rocksDB in datanode fails with "No locks available"
add 606061a YARN-9595. FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource(). Contributed by Peter Bacsko.
add 1fc359f HDDS-1607. Create smoketest for non-secure mapreduce example (#869)
add e140a45 HDDS-1629. Tar file creation can be optional for non-dist builds. Contributed by Elek, Marton. (#887)
add 5d5081e HDDS-1631. Fix auditparser smoketests (#892)
add 7991159 SUBMARINE-82. Fix english grammar mistakes in documentation. Contributed by Szilard Nemeth.
add 50909a7 HDDS-1641. Csi server fails because transitive Netty dependencies (#906)
add 827a847 HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer -follow-on patch.
add ea3b0a1 HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
add e4b4fec HDDS-1224. Restructure code to validate the response from server in the Read path (#806)
add 97607f3 HDDS-1510. Classpath files are deployed to the maven repository as pom/jar files. Contributed by Elek, Marton. (#900)
add 580b639 HDDS-1600. Add userName and IPAddress as part of OMRequest. (#857)
add 1a78794 HDDS-1624 : Refactor operations inside the bucket lock in OM key write. (#882)
add d45669c YARN-7537. Add ability to load hbase config from distributed file system. Contributed by Prabhu Joseph
add e7e30a5 MAPREDUCE-7210. Replace `mapreduce.job.counters.limit` with `mapreduce.job.counters.max` in mapred-default.xml
add 96d818a SUBMARINE-88. rat.sh regex pattern not working issue while using lower version of grep in ubuntu16.04. Contributed by Zhankun Tang.
add cd17cc2 SUBMARINE-89. Add submarine-src profile to generate source package. Contributed by Sunil Govindan.
add ef2da40 HDDS-1640. Reduce the size of recon jar file
add 433e97c YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang.
add 42cd861 HDDS-1628. Fix the execution and return code of smoketest executor shell script
add 7724d80 Revert "HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors"
add 309501c Revert "HADOOP-16050: s3a SSL connections should use OpenSSL"
add d1aad44 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
add 0b1e288 HDDS-1637. Fix random test failure TestSCMContainerPlacementRackAware. Contributed by Sammi Chen. (#904)
add 3b1c257 HDDS-1541. Implement addAcl,removeAcl,setAcl,getAcl for Key. Contributed by Ajay Kumat. (#885)
add 294695d HADOOP-16314. Make sure all web end points are covered by the same authentication filter. Contributed by Prabhu Joseph
add 73954c1 HDDS-1612. Add 'scmcli printTopology' shell command to print datanode topology. Contributed by Sammi Chen.(#910)
add ec26c43 HADOOP-16117. Update AWS SDK to 1.11.563.
add 649666e YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal.
add f7c77b3 HDDS-1458. Create a maven profile to run fault injection tests. Contributed by Eric Yang.
add 829848b HADOOP-16344. Make DurationInfo public unstable.
add 9fded67 HDDS-1621. writeData in ChunkUtils should not use AsynchronousFileChannel. Contributed by Supratim Deka (#917)
add 0976392 YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
add 36757ad HDDS-1645. Change the version of Pico CLI to the latest 3.x release - 3.9.6
add 1f244b4 Revert "HADOOP-16344. Make DurationInfo public unstable."
add 0b8a260 HDDS-1605. Implement AuditLogging for OM HA Bucket write requests. (#867)
add 944adc6 HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao.
add c8276f3 HDDS-1201. Reporting Corruptions in Containers to SCM (#912)
add fe06957 HDDS-1647 : Recon config tag does not show up on Ozone UI. (#914)
add 76c0183 HDDS-1652. HddsDispatcher should not shutdown volumeSet. Contributed by Xiaoyu Yao. (#916)
add c7e6f07 HDDS-1650. Fix Ozone tests leaking volume checker thread. Contributed by Xiaoyu Yao. (#915)
add e1dfc06 HDFS-14486. The exception classes in some throw statements do not accurately describe why they are thrown. Contributed by Ayush Saxena.
add 8ca58ef HDDS-1490. Support configurable container placement policy through 'o… (#903)
add cb9bc6e YARN-9581. Fixed yarn logs cli to access RM2. Contributed by Prabhu Joseph
add 3ea4f41 MAPREDUCE-6794. Remove unused properties from TTConfig.java
add 0b115b6 HDDS-1371. OMSnapshotProvider to download DB checkpoint from leader OM. (#703)
add a91d24f HDDS-1496. Support partial chunk reads and checksum verification (#804)
add 14552d1 HADOOP-16344. Make DurationInfo public unstable.
add 8547957 HDDS-1622. Use picocli for StorageContainerManager
add 4e38daf HADOOP-15563. S3Guard to support creating on-demand DDB tables.
add 76b94c2 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen.
add 46b23c1 HDDS-1636. Tracing id is not propagated via async datanode grpc call (#895)
add 9deac3b HDDS-1657. Fix parallelStream usage in volume and key native acl. Contributed by Ajay Kumar. (#926)
add fcfe7a3 HDDS-1655. Redundant toString() call for metaDataPath in KeyValueContainerCheck (#932)
add d6d95d2 HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
add e94e643 YARN-9471. Cleanup in TestLogAggregationIndexFileController. Contributed by Adam Antal.
add 7217494 HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru.
add 0d160a0 HDFS-10210. Remove the defunct startKdc profile from hdfs. Contributed by Wei-Chiu Chuang, Akira Ajisaka.
add 9191e08 YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
add bd46bdf HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao.
add 101d5b5 HDFS-14234. Limit WebHDFS to specifc user, host, directory triples. Contributed by Clay B.
add b417a4c MAPREDUCE-7214. Remove unused pieces related to `mapreduce.job.userlog.retain.hours`
add b057479 HDDS-1662. Missing test resources of integrataion-test project in target directory after compile
add f7df55f YARN-9602. Use logger format in Container Executor. Contributed by Abhishek Modi.
add 6d80b9b YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie.
add f918e3f HDDS-1669. SCM startup is failing if network-topology-default.xml is part of a jar
add 60c95e9 YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
add 2263ead YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T.
add 3c9a5e7 HDDS-1635. Maintain docker entrypoint and envtoconf inside ozone project (#894)
add 96e1e41 HDDS-1627. Make the version of the used hadoop-runner configurable (#886)
add e997f2a HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)
add 5740eea HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.
add 4fecc2a HADOOP-16263. Update BUILDING.txt with macOS native build instructions. Contributed by Siyao Meng.
add 4ea6c2f HADOOP-16354. Enable AuthFilter as default for WebHDFS. Contributed by Prabhu Joseph
add 23c0379 Revert "HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)"
add 3b31694 HDDS-1545. Cli to add,remove,get and delete acls for Ozone objects. Contributed by Ajay Kumar. (#920)
add 50de087 HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell.
add cf84881 HADOOP-16365. Upgrade jackson-databind to 2.9.9. Contributed by Shweta Yakkali.
add 1732312 HDDS-1663. Add datanode to network topology cluster during node regis… (#937)
add 205dd2d HADOOP-16367. Fixed MiniYarnCluster AuthenticationFilter initialization. Contributed by Prabhu Joseph
add a43f444 HDDS-1543. Implement addAcl,removeAcl,setAcl,getAcl for Prefix. Contr… (#927)
add 88c53d5 HDDS-1620. Implement Volume Write Requests to use Cache and DoubleBuffer. (#884)
add 970b0b0 YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
add 940bcf0 HDDS-1587. Support dynamically adding delegated classes from to isolated class loader. Contributed by Junjie Chen. (#942)
add bcfd228 YARN-9599. TestContainerSchedulerQueuing#testQueueShedding fails intermittently. Contributed by Abhishek Modi.
add 89acbbc HDDS-1677. Auditparser robot test shold use a world writable working directory (#958)
add e094b3b HDDS-1680. Create missing parent directories during the creation of HddsVolume dirs (#961)
add 54f9f75 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.
add 4f45529 HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell.
add 9ebbda3 HADOOP-16372. Fix typo in DFSUtil getHttpPolicy method
add 3ba090f HADOOP-16366. Fixed ProxyUserAuthenticationFilterInitializer for timeline server. Contributed by Prabhu Joseph
add ae4143a HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
add cdc5de6 HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
add b24efa1 HDFS-14549. EditLogTailer shouldn't output full stack trace when interrupted. Contributed by Inigo Goiri.
add c7554ff HADOOP-16373. Fix typo in FileSystemShell#test documentation (#968)
add cda9f33 YARN-8499 ATSv2 Generalize TimelineStorageMonitor. Contributed by Prabhu Joseph
add 0766186 HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm.
add 8370a0a HDFS-14203. Refactor OIV Delimited output entry building mechanism. Contributed by Adam Antal.
add e70aeb4 HDDS-1601. Implement updating lastAppliedIndex after buffer flush to OM DB. (#972)
add f9cc9e1 HADOOP-16279. S3Guard: Implement time-based (TTL) expiry for entries (and tombstones).
add 1da09bd HADOOP-16376. ABFS: Override access() to no-op.
add cc16302 HDFS-14556: Spelling Mistake "gloablly" (#938)
add ba681bb HDDS-1692. RDBTable#iterator should disabled caching of the keys during iterator. (#975)
add 304a47e YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
add 22b36dd HDDS-1660. Use Picocli for Ozone Manager
add 3d020e9 HDDS-1532. Improve the concurrent testing framework of Freon. (#957)
add f9a7b44 HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai.
add a50c35b Revert "HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
add 7314185 Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
add 6822193 HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
add a95e87d HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.
add 1e92db5 HDFS-11949. Add testcase for ensuring that FsShell cann't move file to the target directory that file exists. Contributed by legend.
add 10311c3 HADOOP-14807. should prevent the possibility of NPE about ReconfigurableBase.java. Contributed by hu xiaodong.
add 62ad988 HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.
add 67414a1 YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.
add e317222 HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang.
add f1c239c HADOOP-9157. Better option for curl in hadoop-auth-examples. Contributed by Andras Bokor.
add 098c325 HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar.
add eb6be46 YARN-9630. [UI2] Add a link in docs's top page
add 56c621c HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal.
add d41310a HDFS-12315. Use Path instead of String to check closedFiles set. Contributed by Oleg Danilov.
add 54cdde3 HDFS-12314. Typo in the TestDataNodeHotSwapVolumes.testAddOneNewVolume(). Contributed by Oleg Danilov.
add dd4a763 HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali.
add e188bb1 HDDS-1694. TestNodeReportHandler is failing with NPE
add 335c1c9 HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti.
add fb1ce0d HDDS-1670. Add limit support to /api/containers and /api/containers/{id} endpoints (#954)
add 3ab77d9 HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.
add 3c1a1ce HDFS-14487. Missing Space in Client Error Message (Contributed by Shweta Yakkali via Daniel Templeton)
add b14f056 YARN-9574. Update hadoop-yarn-applications-mawo artifactId to match directory name. Contributed by Wanqiang Ji
add 71edfce HDDS-1699. Update RocksDB version to 6.0.1 (#980)
add 81ec909 HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton.
add 37bd5bb HDDS-1702. Optimize Ozone Recon build time (#982)
add 2dfa932 HDDS-1684. OM should create Ratis related dirs only if ratis is enabled (#965)
add f9ee97d HDFS-14398. Update HAState.java to fix typos. Contributed by Nikhil Navadiya.
add a1c3868 HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti.
add 48e564f HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.
add d3ac516 Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti."
add 9d68425 HDDS-1454. GC other system pause events can trigger pipeline destroy for all the nodes in the cluster. Contributed by Supratim Deka (#852)
add 450c070 HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations.
add f5ecc0b HDFS-14103. Review Logging of BlockPlacementPolicyDefault. Contributed by David Mollitor.
add 71ecd2e HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
add 5bfdf62 YARN-9631. Added ability to select JavaScript test or skip JavaScript tests for YARN application catalog. Contributed by Eric Yang
add 28291a9 HADOOP-16379: S3AInputStream.unbuffer should merge input stream stats into fs-wide stats
add e02eb24 HADOOP-15183. S3Guard store becomes inconsistent after partial failure of rename.
add 9c4b15d HDDS-1508. Provide example k8s deployment files for the new CSI server
add 5962a51 HDFS-14581. Appending to EC files crashes NameNode. Contributed by Surendra Singh Lilhore.
add d9a9e99 HDDS-1579. Create OMDoubleBuffer metrics. (#871)
add 840d02c HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang.
add 98d2065 HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma.
add 272b96d HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain.
add b95a58e HDDS-1674. Make ScmBlockLocationProtocol message type based.
add 83dcb9d YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
add 256bdd6 HDDS-1678. Default image name for kubernetes examples should be ozone and not hadoop
add cba13c7 HDDS-1715. Update the Intellij runner definitition of SCM to use the new class name
add 8194a11 HDDS-1690. ContainerController should provide a way to retrieve containers per volume (#986)
add 371452e HADOOP-15989. Synchronized at CompositeService#removeService is not required. Contributed by Prabhu Joseph.
add 1524e2e HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi.
add 645d67b HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu.
add b28ddb2 HDDS-1706. Replication Manager thread running too frequently. Contributed by Nilotpal Nandi.
new 8dfd2e5 HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
new 7ac5e76 HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
new ab38e37 HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
new d2d2148 HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
new da154a6 HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
new 84b33ee HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
new 7d2a1f8 HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
new 0635de6 HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
new 650b0f5 HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
new 5df703f HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
new 5b24f0f HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
new d49ebf3 HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
new acafc7a HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
new 88fd500 Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
new dbe0139 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
new 71cec6a HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
new a505876 HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
new 7d8cc5d HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
new cd73cb8 HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
new a75d1fc HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
new 53791b9 HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
new ea3e7b8 HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
new 3bb5752 HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
new 5fcfc3c HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
new 85f2d54 HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
new b990ba5 HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
new 58c9bc1 HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
new b20c5fa HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
new b9e0b02 HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
new 6b5f63c HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
new d9c09ed HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
new 0c47bac HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
new 4c4e8df HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
new c1345bc HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.
new b4eb949 HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
new 5757a02 HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
new bdacc8c HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
new d8d6c9d HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
new a2c8633 HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
new 9c46012 HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
new ef1aaa7 HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
new a701c13 HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
new 8b31975 HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
new 0f43b36 HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
new 7400a0a HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
new 0e97ed1 HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
new 55b499d HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
new d79685a HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
new c99f62f HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
new 6c68625 HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
new 5cb7a4d HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
new 5664b3e HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
new 86a3cd5 HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
new 1fc3857 HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.
new 5cd42d4 HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka.
new e7e48a4 HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri.
new 506d073 HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota.
new b4e852e HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao.
new 97b672d HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri.
new 326ec17 HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li.
new 4afe588 HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena.
new a53d678 HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena.
new dc32bf0 HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.
new ec1b79e HDFS-13995. RBF: Security documentation. Contributed by CR Hota.
new 0512084 HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena.
new 68d4df4 HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka.
new 395312b HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena.
new 6cf674c HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma.
new 8edfb8a HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri.
new 90f4887 HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota.
new f544121 HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota.
new 0c21e81 HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena.
new 1579136 HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma.
new 812256b HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
new e2a900b HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma.
new 8a9281a HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao.
new 02597b6 HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (caa285b)
\
N -- N -- N refs/heads/HDFS-13891 (02597b6)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 77 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
BUILDING.txt | 72 ++
NOTICE.txt | 15 +-
dev-support/bin/dist-copynativelibs | 8 +
dev-support/bin/yetus-wrapper | 24 +-
hadoop-assemblies/pom.xml | 2 +-
...-src-with-hdds.xml => hadoop-src-submarine.xml} | 0
hadoop-build-tools/pom.xml | 2 +-
hadoop-client-modules/hadoop-client-api/pom.xml | 2 +-
.../hadoop-client-check-invariants/pom.xml | 2 +-
.../hadoop-client-check-test-invariants/pom.xml | 2 +-
.../hadoop-client-integration-tests/pom.xml | 2 +-
.../hadoop-client-minicluster/pom.xml | 2 +-
.../hadoop-client-runtime/pom.xml | 2 +-
hadoop-client-modules/hadoop-client/pom.xml | 2 +-
hadoop-client-modules/pom.xml | 2 +-
.../hadoop-cloud-storage/pom.xml | 2 +-
hadoop-cloud-storage-project/pom.xml | 2 +-
hadoop-common-project/hadoop-annotations/pom.xml | 2 +-
hadoop-common-project/hadoop-auth-examples/pom.xml | 2 +-
hadoop-common-project/hadoop-auth/pom.xml | 2 +-
.../hadoop-auth/src/site/markdown/Examples.md | 6 +-
hadoop-common-project/hadoop-common/pom.xml | 113 +-
.../hadoop-common/src/CMakeLists.txt | 21 +
.../hadoop-common/src/config.h.cmake | 1 +
.../hadoop-common/src/main/bin/hadoop | 4 +
.../hadoop-common/src/main/bin/hadoop.cmd | 5 +
.../org/apache/hadoop/conf/ReconfigurableBase.java | 3 +-
.../main/java/org/apache/hadoop/fs/FileSystem.java | 6 +
.../main/java/org/apache/hadoop/fs/FsShell.java | 4 +-
.../apache/hadoop/fs/impl/FunctionsRaisingIOE.java | 69 +
.../org/apache/hadoop/fs/impl/FutureIOSupport.java | 48 +-
.../java/org/apache/hadoop/fs/shell/Delete.java | 16 +-
.../java/org/apache/hadoop/http/HttpServer2.java | 48 +-
.../java/org/apache/hadoop/http/WebServlet.java | 59 +
.../io/compress/zstd/ZStandardCompressor.java | 4 +-
.../org/apache/hadoop/io/nativeio/NativeIO.java | 135 +-
.../org/apache/hadoop/ipc/CallQueueManager.java | 5 +-
.../org/apache/hadoop/ipc/DecayRpcScheduler.java | 12 +-
.../org/apache/hadoop/ipc/DefaultRpcScheduler.java | 4 +-
.../java/org/apache/hadoop/ipc/ExternalCall.java | 5 +
.../java/org/apache/hadoop/ipc/FairCallQueue.java | 16 +-
.../org/apache/hadoop/ipc/ProcessingDetails.java | 96 ++
.../org/apache/hadoop/ipc/ProtobufRpcEngine.java | 31 +-
.../java/org/apache/hadoop/ipc/RpcScheduler.java | 41 +-
.../main/java/org/apache/hadoop/ipc/Server.java | 171 ++-
.../org/apache/hadoop/ipc/WritableRpcEngine.java | 20 +-
.../hadoop/ipc/metrics/RpcDetailedMetrics.java | 6 +-
.../org/apache/hadoop/ipc/metrics/RpcMetrics.java | 63 +-
.../java/org/apache/hadoop/net/InnerNodeImpl.java | 44 +-
.../server/ProxyUserAuthenticationFilter.java | 199 +++
.../ProxyUserAuthenticationFilterInitializer.java | 60 +
.../authentication/server/package-info.java | 22 +
.../hadoop/security/ssl/OpenSSLSocketFactory.java | 248 ----
.../apache/hadoop/service/CompositeService.java | 3 +-
.../java/org/apache/hadoop/util/DurationInfo.java | 8 +-
.../org/apache/hadoop/util/HttpExceptionUtils.java | 12 +-
.../hadoop/io/compress/zstd/ZStandardCompressor.c | 2 +-
.../src/org/apache/hadoop/io/nativeio/NativeIO.c | 252 ++++
.../src/org/apache/hadoop/io/nativeio/pmdk_load.c | 106 ++
.../src/org/apache/hadoop/io/nativeio/pmdk_load.h | 95 ++
.../src/main/resources/core-default.xml | 59 +-
.../src/site/markdown/DeprecatedProperties.md | 21 -
.../src/site/markdown/FileSystemShell.md | 13 +-
.../src/site/markdown/HttpAuthentication.md | 10 +
.../hadoop-common/src/site/markdown/Metrics.md | 9 +
.../java/org/apache/hadoop/fs/TestFsShellCopy.java | 47 +-
.../test/java/org/apache/hadoop/fs/TestTrash.java | 326 ++---
.../fs/contract/AbstractContractRenameTest.java | 2 +-
.../apache/hadoop/fs/viewfs/TestViewFsTrash.java | 2 +-
.../org/apache/hadoop/http/TestGlobalFilter.java | 4 +-
.../hadoop/http/TestHttpServerWithSpnego.java | 238 ++++
.../org/apache/hadoop/http/TestPathFilter.java | 2 -
.../org/apache/hadoop/http/TestServletFilter.java | 1 -
.../apache/hadoop/io/nativeio/TestNativeIO.java | 153 +++
.../apache/hadoop/ipc/TestProcessingDetails.java | 61 +
.../org/apache/hadoop/ipc/TestProtoBufRpc.java | 9 +-
.../test/java/org/apache/hadoop/ipc/TestRPC.java | 18 +-
.../java/org/apache/hadoop/ipc/TestRpcBase.java | 28 +
.../java/org/apache/hadoop/log/TestLogLevel.java | 9 +
.../hadoop/security/TestUGIWithSecurityOn.java | 117 --
.../server/TestProxyUserAuthenticationFilter.java | 125 ++
.../security/ssl/TestOpenSSLSocketFactory.java | 53 -
.../src/test/proto/test_rpc_service.proto | 1 +
.../src/test/resources/kdc/killKdc.sh | 19 -
.../src/test/resources/kdc/ldif/users.ldif | 78 --
.../src/test/resources/kdc/server.xml | 258 ----
hadoop-common-project/hadoop-kms/pom.xml | 2 +-
hadoop-common-project/hadoop-minikdc/pom.xml | 2 +-
hadoop-common-project/hadoop-nfs/pom.xml | 2 +-
hadoop-common-project/hadoop-registry/pom.xml | 2 +-
hadoop-common-project/pom.xml | 2 +-
hadoop-dist/pom.xml | 2 +-
hadoop-hdds/client/pom.xml | 4 +-
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 95 +-
.../hdds/scm/client/ContainerOperationClient.java | 14 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 649 ++++------
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 18 +-
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 544 ++++++++
.../hdds/scm/storage/TestBlockInputStream.java | 251 ++--
.../hdds/scm/storage/TestChunkInputStream.java | 222 ++++
hadoop-hdds/common/pom.xml | 4 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 9 +-
.../hadoop/hdds/protocol/DatanodeDetails.java | 39 +-
.../SCMSecurityProtocolClientSideTranslatorPB.java | 53 +-
.../SCMSecurityProtocolServerSideTranslatorPB.java | 7 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 4 +
.../apache/hadoop/hdds/scm/XceiverClientSpi.java | 17 +-
.../algorithms/ContainerPlacementPolicy.java | 45 +
.../placement/algorithms/package-info.java | 21 +
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 2 +-
.../java/org/apache/hadoop/hdds/scm/net/Node.java | 12 +
.../org/apache/hadoop/hdds/scm/net/NodeImpl.java | 36 +-
.../hadoop/hdds/scm/net/NodeSchemaLoader.java | 117 +-
.../hadoop/hdds/scm/net/NodeSchemaManager.java | 17 +-
...lockLocationProtocolClientSideTranslatorPB.java | 43 +-
.../hadoop/hdds/scm/storage/CheckedBiFunction.java | 31 +
.../hdds/scm/storage/ContainerProtocolCalls.java | 146 +--
.../x509/certificate/client/CertificateClient.java | 13 +
.../client/DefaultCertificateClient.java | 25 +-
.../apache/hadoop/hdds/tracing/StringCodec.java | 2 +-
.../java/org/apache/hadoop/ozone/OzoneAcl.java | 233 ----
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 14 +-
.../org/apache/hadoop/ozone/audit/AuditLogger.java | 10 +
.../org/apache/hadoop/ozone/common/Checksum.java | 29 +-
.../apache/hadoop/ozone/common/ChecksumData.java | 37 +-
...lockLocationProtocolServerSideTranslatorPB.java | 65 +-
.../java/org/apache/hadoop/utils/RocksDBStore.java | 3 +-
.../org/apache/hadoop/utils/RocksDBStoreMBean.java | 79 +-
.../java/org/apache/hadoop/utils/UniqueId.java | 69 +
.../org/apache/hadoop/utils/db/DBCheckpoint.java | 16 +
.../java/org/apache/hadoop/utils/db/DBStore.java | 14 +
.../org/apache/hadoop/utils/db/DBStoreBuilder.java | 25 +-
.../java/org/apache/hadoop/utils/db/RDBStore.java | 43 +-
.../java/org/apache/hadoop/utils/db/RDBTable.java | 11 +
.../apache/hadoop/utils/db/RocksDBCheckpoint.java | 11 +
.../java/org/apache/hadoop/utils/db/Table.java | 20 +
.../org/apache/hadoop/utils/db/TypedTable.java | 14 +-
.../org/apache/hadoop/utils/db/cache/CacheKey.java | 2 +-
.../apache/hadoop/utils/db/cache/CacheValue.java | 2 +-
.../hadoop/utils/db/cache/PartialTableCache.java | 23 +-
.../apache/hadoop/utils/db/cache/TableCache.java | 9 +
.../src/main/proto/SCMSecurityProtocol.proto | 1 +
.../src/main/proto/ScmBlockLocationProtocol.proto | 76 +-
hadoop-hdds/common/src/main/proto/hdds.proto | 1 +
.../common/src/main/resources/ozone-default.xml | 108 +-
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 42 +
.../hadoop/hdds/scm/net/TestNodeSchemaLoader.java | 8 +-
.../hadoop/hdds/scm/net/TestNodeSchemaManager.java | 2 +-
.../hadoop/hdds/scm/net/TestYamlSchemaLoader.java | 10 +-
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 141 ---
.../apache/hadoop/utils/TestRocksDBStoreMBean.java | 164 ++-
.../org/apache/hadoop/utils/db/TestRDBStore.java | 43 -
.../apache/hadoop/utils/db/TestRDBTableStore.java | 21 +-
.../hadoop/utils/db/TestTypedRDBTableStore.java | 37 +-
.../utils/db/cache/TestPartialTableCache.java | 8 +-
hadoop-hdds/config/pom.xml | 2 +-
hadoop-hdds/container-service/pom.xml | 2 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 28 +-
.../ozone/container/common/impl/ContainerSet.java | 19 +
.../container/common/impl/HddsDispatcher.java | 17 +-
.../ozone/container/common/interfaces/Handler.java | 9 +
.../common/statemachine/DatanodeStateMachine.java | 5 +
.../commandhandler/DeleteBlocksCommandHandler.java | 90 +-
.../common/transport/server/ratis/CSMMetrics.java | 58 +-
.../server/ratis/ContainerStateMachine.java | 16 +-
.../container/common/utils/ContainerCache.java | 55 +-
.../container/common/utils/ReferenceCountedDB.java | 97 ++
.../ozone/container/common/volume/HddsVolume.java | 5 +-
.../volume/RoundRobinVolumeChoosingPolicy.java | 4 +-
.../container/keyvalue/KeyValueBlockIterator.java | 15 +-
.../container/keyvalue/KeyValueContainer.java | 19 +-
.../container/keyvalue/KeyValueContainerCheck.java | 64 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 18 +-
.../container/keyvalue/helpers/BlockUtils.java | 4 +-
.../container/keyvalue/helpers/ChunkUtils.java | 22 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 36 +-
.../container/keyvalue/impl/BlockManagerImpl.java | 206 +--
.../background/BlockDeletingService.java | 116 +-
.../container/ozoneimpl/ContainerController.java | 17 +
.../ozone/container/ozoneimpl/ContainerReader.java | 91 +-
.../container/ozoneimpl/ContainerScrubber.java | 31 +-
.../ozone/container/ozoneimpl/OzoneContainer.java | 2 +-
.../replication/ReplicationSupervisor.java | 8 +
.../hadoop/ozone/TestHddsDatanodeService.java | 2 +-
.../ozone/container/common/TestContainerCache.java | 128 ++
.../container/common/impl/TestContainerSet.java | 44 +
.../container/common/impl/TestHddsDispatcher.java | 11 +-
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 13 +-
.../container/common/volume/TestVolumeSet.java | 3 +-
.../common/volume/TestVolumeSetDiskChecks.java | 3 +
.../keyvalue/TestKeyValueBlockIterator.java | 212 ++--
.../container/keyvalue/TestKeyValueContainer.java | 44 +-
.../keyvalue/TestKeyValueContainerCheck.java | 88 +-
.../container/keyvalue/TestKeyValueHandler.java | 7 +-
.../container/ozoneimpl/TestOzoneContainer.java | 138 +-
hadoop-hdds/docs/pom.xml | 2 +-
hadoop-hdds/framework/pom.xml | 2 +-
.../hadoop/hdds/server/PrometheusMetricsSink.java | 9 +
.../hdds/server/TestPrometheusMetricsSink.java | 5 +
hadoop-hdds/pom.xml | 28 +-
hadoop-hdds/server-scm/pom.xml | 10 +-
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 45 +-
.../hdds/scm/container/ContainerStateManager.java | 10 +-
.../hdds/scm/container/ReplicationManager.java | 4 +-
.../hdds/scm/container/SCMContainerManager.java | 24 +-
.../algorithms/ContainerPlacementPolicy.java | 44 -
.../ContainerPlacementPolicyFactory.java | 67 +
.../placement/algorithms/SCMCommonPolicy.java | 10 +-
.../algorithms/SCMContainerPlacementCapacity.java | 13 +-
.../algorithms/SCMContainerPlacementRackAware.java | 342 +++++
.../algorithms/SCMContainerPlacementRandom.java | 12 +-
.../hadoop/hdds/scm/exceptions/SCMException.java | 3 +-
.../apache/hadoop/hdds/scm/node/NodeManager.java | 9 +
.../hadoop/hdds/scm/node/NodeStateManager.java | 167 ++-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 117 +-
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 4 +
.../hadoop/hdds/scm/pipeline/PipelineProvider.java | 1 +
.../hdds/scm/pipeline/RatisPipelineProvider.java | 117 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 73 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 5 +-
.../hdds/scm/pipeline/SimplePipelineProvider.java | 5 +
.../hdds/scm/safemode/ContainerSafeModeRule.java | 4 +
.../scm/safemode/HealthyPipelineSafeModeRule.java | 5 +-
.../safemode/OneReplicaPipelineSafeModeRule.java | 5 +
.../hdds/scm/safemode/SCMSafeModeManager.java | 15 +
.../hadoop/hdds/scm/safemode/SafeModeExitRule.java | 4 +
.../hadoop/hdds/scm/safemode/SafeModeMetrics.java | 111 ++
.../hdds/scm/server/SCMStarterInterface.java | 37 +
.../hdds/scm/server/StorageContainerManager.java | 224 +---
.../scm/server/StorageContainerManagerStarter.java | 153 +++
.../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 2 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 29 +-
.../hadoop/hdds/scm/block/TestBlockManager.java | 43 +
.../hadoop/hdds/scm/container/MockNodeManager.java | 5 +
.../hdds/scm/container/TestReplicationManager.java | 3 +-
.../scm/container/TestSCMContainerManager.java | 47 +-
.../algorithms/TestContainerPlacementFactory.java | 142 +++
.../TestSCMContainerPlacementCapacity.java | 6 +-
.../TestSCMContainerPlacementRackAware.java | 292 +++++
.../TestSCMContainerPlacementRandom.java | 6 +-
.../hdds/scm/node/TestContainerPlacement.java | 8 +-
.../hdds/scm/node/TestNodeReportHandler.java | 9 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 204 +++
.../scm/pipeline/MockRatisPipelineProvider.java | 5 +
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 60 +-
.../server/TestStorageContainerManagerStarter.java | 166 +++
.../placement/TestContainerPlacement.java | 10 +-
.../testutils/ReplicationNodeManagerMock.java | 5 +
.../src/test/resources/nodegroup-mapping | 24 +
.../server-scm/src/test/resources/rack-mapping | 24 +
hadoop-hdds/tools/pom.xml | 2 +-
.../org/apache/hadoop/hdds/scm/cli/SCMCLI.java | 3 +-
.../hadoop/hdds/scm/cli/TopologySubcommand.java | 80 ++
hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 2 +-
.../org/apache/hadoop/hdfs/DFSOutputStream.java | 2 +-
.../hdfs/client/impl/BlockReaderFactory.java | 3 +-
.../org/apache/hadoop/hdfs/net/BasicInetPeer.java | 4 +-
.../org/apache/hadoop/hdfs/net/NioInetPeer.java | 4 +-
.../apache/hadoop/hdfs/web/WebHdfsFileSystem.java | 16 +-
.../hadoop-hdfs-client/src/main/proto/hdfs.proto | 2 +-
hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 3 +-
.../src/main/conf/httpfs-signature.secret | 1 -
.../fs/http/server/HttpFSAuthenticationFilter.java | 46 +-
.../src/main/resources/httpfs-default.xml | 3 +
.../src/site/markdown/ServerSetup.md.vm | 3 +-
.../TestHttpFSServerWebServerWithRandomSecret.java | 58 +
.../hadoop-hdfs-native-client/pom.xml | 2 +-
.../src/main/native/libhdfs/CMakeLists.txt | 2 +-
hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 2 +-
.../hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java | 2 +-
hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 4 +-
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 84 +-
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 22 +-
.../main/java/org/apache/hadoop/hdfs/DFSUtil.java | 29 +-
.../hadoop/hdfs/qjournal/server/JNStorage.java | 30 +-
.../hadoop/hdfs/qjournal/server/Journal.java | 26 +-
.../hdfs/server/blockmanagement/BlockManager.java | 108 +-
.../blockmanagement/BlockPlacementPolicy.java | 12 +
.../BlockPlacementPolicyDefault.java | 88 +-
.../BlockPlacementPolicyRackFaultTolerant.java | 4 +-
.../blockmanagement/DatanodeAdminManager.java | 11 +-
.../common/HostRestrictingAuthorizationFilter.java | 443 +++++++
.../hadoop/hdfs/server/common/JspHelper.java | 10 +-
.../hadoop/hdfs/server/datanode/BlockReceiver.java | 3 +-
.../hadoop/hdfs/server/datanode/DataNode.java | 3 +-
.../hadoop/hdfs/server/datanode/DiskBalancer.java | 5 +
.../datanode/checker/DatasetVolumeChecker.java | 10 +-
.../datanode/checker/StorageLocationChecker.java | 8 +-
.../datanode/checker/ThrottledAsyncChecker.java | 4 +-
.../datanode/erasurecode/ErasureCodingWorker.java | 4 -
.../erasurecode/StripedBlockReconstructor.java | 6 +-
.../datanode/erasurecode/StripedReconstructor.java | 4 -
.../datanode/fsdataset/impl/FsDatasetCache.java | 22 +
.../datanode/fsdataset/impl/FsDatasetImpl.java | 12 +-
.../datanode/fsdataset/impl/FsDatasetUtil.java | 22 +
.../datanode/fsdataset/impl/FsVolumeImpl.java | 13 +-
.../datanode/fsdataset/impl/MappableBlock.java | 6 +
.../fsdataset/impl/MappableBlockLoader.java | 66 +-
.../fsdataset/impl/MappableBlockLoaderFactory.java | 4 +
.../fsdataset/impl/MemoryMappableBlockLoader.java | 67 +-
.../datanode/fsdataset/impl/MemoryMappedBlock.java | 5 +
.../impl/NativePmemMappableBlockLoader.java | 191 +++
.../fsdataset/impl/NativePmemMappedBlock.java | 85 ++
.../fsdataset/impl/PmemMappableBlockLoader.java | 116 +-
.../datanode/fsdataset/impl/PmemMappedBlock.java | 5 +
.../server/datanode/web/DatanodeHttpServer.java | 246 ++--
.../HostRestrictingAuthorizationFilterHandler.java | 240 ++++
.../web/RestCsrfPreventionFilterHandler.java | 64 +-
.../hdfs/server/datanode/web/package-info.java | 21 +
.../hadoop/hdfs/server/namenode/FSDirAppendOp.java | 11 +-
.../hadoop/hdfs/server/namenode/FSDirAttrOp.java | 6 +-
.../hadoop/hdfs/server/namenode/FSImage.java | 25 +
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 28 +-
.../hdfs/server/namenode/FSNamesystemLock.java | 66 +-
.../hdfs/server/namenode/GlobalStateIdContext.java | 6 +-
.../hadoop/hdfs/server/namenode/NameNode.java | 77 +-
.../hdfs/server/namenode/NameNodeHttpServer.java | 62 -
.../hdfs/server/namenode/NameNodeRpcServer.java | 34 +-
.../hdfs/server/namenode/ha/EditLogTailer.java | 2 +-
.../hadoop/hdfs/server/namenode/ha/HAState.java | 6 +-
.../namenode/startupprogress/StartupProgress.java | 35 +-
.../org/apache/hadoop/hdfs/tools/AdminHelper.java | 10 +-
.../org/apache/hadoop/hdfs/tools/DFSAdmin.java | 13 +-
.../apache/hadoop/hdfs/tools/DiskBalancerCLI.java | 8 +
.../PBImageDelimitedTextWriter.java | 155 ++-
.../org/apache/hadoop/hdfs/web/AuthFilter.java | 115 +-
.../hadoop/hdfs/web/AuthFilterInitializer.java | 69 +
.../src/main/resources/hdfs-default.xml | 25 +
.../src/main/webapps/hdfs/dfshealth.html | 4 +
.../hadoop-hdfs/src/main/webapps/static/hadoop.css | 11 +-
.../hadoop-hdfs/src/site/markdown/HDFSCommands.md | 2 +-
.../site/markdown/HDFSHighAvailabilityWithNFS.md | 12 +
.../site/markdown/HDFSHighAvailabilityWithQJM.md | 12 +
.../hadoop-hdfs/src/site/markdown/WebHDFS.md | 61 +
.../TestDFSInotifyEventInputStreamKerberized.java | 9 +
.../java/org/apache/hadoop/hdfs/TestDFSShell.java | 3 -
.../hadoop/hdfs/TestDistributedFileSystem.java | 29 +
.../java/org/apache/hadoop/hdfs/TestHDFSTrash.java | 2 +-
.../java/org/apache/hadoop/hdfs/TestHdfsAdmin.java | 2 +-
.../apache/hadoop/hdfs/TestMaintenanceState.java | 1 +
.../hadoop/hdfs/TestReconstructStripedFile.java | 6 -
.../apache/hadoop/hdfs/TestStripedFileAppend.java | 35 +
.../hadoop/hdfs/qjournal/TestSecureNNWithQJM.java | 8 +
.../blockmanagement/TestBlockReportLease.java | 159 +++
.../blockmanagement/TestRedundancyMonitor.java | 108 ++
.../TestHostRestrictingAuthorizationFilter.java | 275 ++++
.../hadoop/hdfs/server/common/TestJspHelper.java | 88 +-
.../datanode/TestDataNodeHotSwapVolumes.java | 2 +-
.../TestDataNodeVolumeFailureToleration.java | 4 +-
.../hdfs/server/datanode/TestDatanodeRegister.java | 12 +
.../hdfs/server/datanode/TestDirectoryScanner.java | 68 +
.../datanode/checker/TestDatasetVolumeChecker.java | 37 +
.../checker/TestStorageLocationChecker.java | 33 +-
.../datanode/fsdataset/impl/TestProvidedImpl.java | 8 +
...tHostRestrictingAuthorizationFilterHandler.java | 178 +++
.../command/TestDiskBalancerCommand.java | 11 +
.../server/namenode/NNThroughputBenchmark.java | 5 +
.../hdfs/server/namenode/TestCheckpoint.java | 2 -
.../hadoop/hdfs/server/namenode/TestMetaSave.java | 60 +
.../TestRefreshNamenodeReplicationConfig.java | 143 +++
.../namenode/ha/TestConsistentReadsObserver.java | 5 -
.../hdfs/server/namenode/ha/TestHASafeMode.java | 31 +
.../hdfs/server/namenode/ha/TestNNHealthCheck.java | 30 +
.../startupprogress/TestStartupProgress.java | 59 +-
.../apache/hadoop/hdfs/tools/TestAdminHelper.java | 50 +
.../org/apache/hadoop/hdfs/tools/TestDFSAdmin.java | 2 +-
.../offlineImageViewer/TestOfflineImageViewer.java | 113 +-
.../org/apache/hadoop/hdfs/web/TestAuthFilter.java | 125 +-
.../apache/hadoop/hdfs/web/TestWebHdfsTokens.java | 225 ++--
.../org/apache/hadoop/hdfs/web/TestWebHdfsUrl.java | 47 +-
.../web/TestWebHdfsWithAuthenticationFilter.java | 18 +-
.../TestWebHdfsWithRestCsrfPreventionFilter.java | 20 +-
hadoop-hdfs-project/pom.xml | 2 +-
.../hadoop-mapreduce-client-app/pom.xml | 2 +-
.../mapreduce/v2/app/TaskHeartbeatHandler.java | 5 +-
.../mapreduce/v2/app/rm/RMContainerAllocator.java | 20 +-
.../mapreduce/v2/app/TestTaskHeartbeatHandler.java | 43 +-
.../v2/app/rm/TestRMContainerAllocator.java | 22 +
.../hadoop-mapreduce-client-common/pom.xml | 2 +-
.../hadoop-mapreduce-client-core/pom.xml | 2 +-
.../org/apache/hadoop/mapreduce/MRJobConfig.java | 2 -
.../mapreduce/server/tasktracker/TTConfig.java | 73 +-
.../apache/hadoop/mapreduce/util/ConfigUtil.java | 43 -
.../src/main/resources/mapred-default.xml | 5 +-
.../hadoop-mapreduce-client-hs-plugins/pom.xml | 2 +-
.../hadoop-mapreduce-client-hs/pom.xml | 2 +-
.../test/resources/job_1329348432655_0001_conf.xml | 14 -
.../hadoop-mapreduce-client-jobclient/pom.xml | 2 +-
.../hadoop-mapreduce-client-nativetask/pom.xml | 2 +-
.../hadoop-mapreduce-client-shuffle/pom.xml | 2 +-
.../hadoop-mapreduce-client-uploader/pom.xml | 4 +-
.../hadoop-mapreduce-client/pom.xml | 4 +-
.../hadoop-mapreduce-examples/pom.xml | 2 +-
hadoop-mapreduce-project/pom.xml | 4 +-
hadoop-maven-plugins/pom.xml | 2 +-
hadoop-minicluster/pom.xml | 2 +-
hadoop-ozone/.gitignore | 20 +
hadoop-ozone/client/pom.xml | 4 +-
.../apache/hadoop/ozone/client/ObjectStore.java | 51 +
.../ozone/client/io/BlockOutputStreamEntry.java | 8 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 382 +++---
.../hadoop/ozone/client/io/KeyOutputStream.java | 25 +-
.../ozone/client/protocol/ClientProtocol.java | 41 +
.../hadoop/ozone/client/rest/RestClient.java | 65 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 112 +-
hadoop-ozone/common/pom.xml | 2 +-
hadoop-ozone/common/src/main/bin/ozone | 10 +-
hadoop-ozone/common/src/main/bin/ozone-config.sh | 5 +
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 109 ++
.../java/org/apache/hadoop/ozone/OzoneAcl.java | 255 ++++
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 33 +-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 7 +
.../apache/hadoop/ozone/om/OzoneManagerLock.java | 25 +
.../hadoop/ozone/om/codec/OmPrefixInfoCodec.java | 53 +
.../hadoop/ozone/om/exceptions/OMException.java | 9 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 4 +-
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 18 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 26 +-
.../hadoop/ozone/om/helpers/OmOzoneAclMap.java | 147 ++-
.../hadoop/ozone/om/helpers/OmPrefixInfo.java | 170 +++
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 40 +-
.../ozone/om/protocol/OzoneManagerHAProtocol.java | 52 -
.../ozone/om/protocol/OzoneManagerProtocol.java | 41 +
...OzoneManagerProtocolClientSideTranslatorPB.java | 127 +-
.../apache/hadoop/ozone/protocolPB/OMPBHelper.java | 63 +-
.../ozone/security/acl/IAccessAuthorizer.java | 30 +-
.../ozone/security/acl/OzoneAccessAuthorizer.java | 4 +-
.../hadoop/ozone/security/acl/OzoneAclConfig.java | 71 ++
.../ozone/security/acl/OzoneAclException.java | 71 --
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 27 +-
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 111 +-
.../org/apache/hadoop/ozone/util/RadixNode.java | 59 +
.../org/apache/hadoop/ozone/util/RadixTree.java | 220 ++++
.../apache/hadoop/ozone/web/utils/OzoneUtils.java | 29 +
.../src/main/proto/OzoneManagerProtocol.proto | 111 +-
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 247 ++++
.../ozone/om/codec/TestOmPrefixInfoCodec.java | 98 ++
.../ozone/security/acl/TestOzoneObjInfo.java | 69 +-
.../apache/hadoop/ozone/util/TestRadixTree.java | 127 ++
.../org/apache/hadoop/ozone/util/package-info.java | 21 +
.../csi/dev-support/findbugsExcludeFile.xml | 22 +
hadoop-ozone/csi/pom.xml | 188 +++
.../apache/hadoop/ozone/csi/ControllerService.java | 123 ++
.../org/apache/hadoop/ozone/csi/CsiServer.java | 160 +++
.../apache/hadoop/ozone/csi/IdentitiyService.java | 72 ++
.../org/apache/hadoop/ozone/csi/NodeService.java | 142 +++
.../org/apache/hadoop/ozone/csi/package-info.java | 22 +
hadoop-ozone/csi/src/main/proto/csi.proto | 1323 ++++++++++++++++++++
hadoop-ozone/datanode/pom.xml | 2 +-
hadoop-ozone/dev-support/checks/acceptance.sh | 3 +-
.../intellij/runConfigurations/OzoneManager.xml | 4 +-
.../runConfigurations/OzoneManagerInit.xml | 4 +-
.../runConfigurations/StorageContainerManager.xml | 2 +-
.../StorageContainerManagerInit.xml | 2 +-
.../dist/dev-support/bin/dist-layout-stitching | 6 +-
hadoop-ozone/dist/pom.xml | 103 +-
hadoop-ozone/dist/src/main/Dockerfile | 6 +-
hadoop-ozone/dist/src/main/blockade/README.md | 54 -
.../src/main/blockade/blockadeUtils/blockade.py | 98 --
.../src/main/blockade/clusterUtils/__init__.py | 14 -
.../main/blockade/clusterUtils/cluster_utils.py | 328 -----
.../main/blockade/test_blockade_client_failure.py | 121 --
.../blockade/test_blockade_datanode_isolation.py | 147 ---
.../dist/src/main/blockade/test_blockade_flaky.py | 61 -
.../main/blockade/test_blockade_mixed_failure.py | 149 ---
...t_blockade_mixed_failure_three_nodes_isolate.py | 225 ----
.../test_blockade_mixed_failure_two_nodes.py | 173 ---
.../main/blockade/test_blockade_scm_isolation.py | 157 ---
hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env | 3 +-
.../main/compose/ozone-hdfs/docker-compose.yaml | 6 +-
hadoop-ozone/dist/src/main/compose/ozone-mr/.env | 19 +
.../src/main/compose/ozone-mr/docker-compose.yaml | 95 ++
.../dist/src/main/compose/ozone-mr/docker-config | 130 ++
.../dist/src/main/compose/ozone-mr/test.sh | 36 +
.../dist/src/main/compose/ozone-net-topology/.env | 17 +
.../compose/ozone-net-topology/docker-compose.yaml | 110 ++
.../main/compose/ozone-net-topology/docker-config | 88 ++
.../main/compose/ozone-net-topology/network-config | 22 +
.../src/main/compose/ozone-net-topology/test.sh | 35 +
.../dist/src/main/compose/ozone-om-ha/.env | 3 +-
.../main/compose/ozone-om-ha/docker-compose.yaml | 10 +-
.../dist/src/main/compose/ozone-recon/.env | 3 +-
.../main/compose/ozone-recon/docker-compose.yaml | 8 +-
hadoop-ozone/dist/src/main/compose/ozone/.env | 3 +-
.../src/main/compose/ozone/docker-compose.yaml | 6 +-
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 2 +-
.../dist/src/main/compose/ozoneblockade/.env | 18 +
.../main/compose/ozoneblockade/docker-compose.yaml | 8 +-
.../src/main/compose/ozoneblockade/docker-config | 6 +
hadoop-ozone/dist/src/main/compose/ozonefs/.env | 18 +
.../src/main/compose/ozonefs/docker-compose.yaml | 6 +-
hadoop-ozone/dist/src/main/compose/ozoneperf/.env | 1 +
.../src/main/compose/ozoneperf/docker-compose.yaml | 8 +-
hadoop-ozone/dist/src/main/compose/ozones3/.env | 18 +
.../src/main/compose/ozones3/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozonescripts/.env | 18 +
.../dist/src/main/compose/ozonesecure-mr/.env | 1 +
.../compose/ozonesecure-mr/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozonesecure/.env | 1 +
.../main/compose/ozonesecure/docker-compose.yaml | 8 +-
hadoop-ozone/dist/src/main/compose/ozonetrace/.env | 19 +
.../main/compose/ozonetrace/docker-compose.yaml | 8 +-
hadoop-ozone/dist/src/main/compose/test-all.sh | 2 +-
hadoop-ozone/dist/src/main/dockerbin/entrypoint.sh | 149 +++
hadoop-ozone/dist/src/main/dockerbin/envtoconf.py | 117 ++
.../dist/src/main/dockerbin/transformation.py | 150 +++
.../k8s/definitions/ozone-csi/csi-controller.yaml | 53 +
.../main/k8s/definitions/ozone-csi/csi-crd.yaml | 21 +
.../main/k8s/definitions/ozone-csi/csi-node.yaml | 95 ++
.../main/k8s/definitions/ozone-csi/csi-rbac.yaml | 66 +
.../definitions/ozone-csi/csi-storageclass.yaml | 20 +
.../k8s/definitions/ozone-csi/definitions/csi.yaml | 28 +
.../dist/src/main/k8s/definitions/ozone/om-ss.yaml | 2 +-
.../src/main/k8s/definitions/ozone/scm-ss.yaml | 2 +-
.../main/k8s/definitions/pv-test/flekszible.yaml | 16 +
.../definitions/pv-test/nginx-conf-configmap.yaml | 37 +
.../k8s/definitions/pv-test/nginx-deployment.yaml | 49 +
.../definitions/pv-test/nginx-service-service.yaml | 28 +
.../nginx-storage-persistentvolumeclaim.yaml | 28 +
.../src/main/k8s/examples/ozone-csi/Flekszible | 25 +
.../examples/{ozone => ozone-csi}/LICENSE.header | 0
.../k8s/examples/ozone-csi/config-configmap.yaml | 37 +
.../k8s/examples/ozone-csi/csi-node-daemonset.yaml | 97 ++
.../examples/ozone-csi/csi-ozone-clusterrole.yaml | 98 ++
.../ozone-csi/csi-ozone-clusterrolebinding.yaml | 28 +
.../ozone-csi/csi-ozone-serviceaccount.yaml | 21 +
.../ozone-csi/csi-provisioner-deployment.yaml | 54 +
.../{ozone => ozone-csi}/datanode-daemonset.yaml | 0
.../examples/{ozone => ozone-csi}/om-service.yaml | 0
.../{ozone => ozone-csi}/om-statefulset.yaml | 0
.../org.apache.hadoop.ozone-csidriver.yaml | 22 +
.../k8s/examples/ozone-csi/ozone-storageclass.yaml | 21 +
.../ozone-csi/pv-test/nginx-conf-configmap.yaml | 38 +
.../ozone-csi/pv-test/nginx-deployment.yaml | 50 +
.../ozone-csi/pv-test/nginx-service-service.yaml | 29 +
.../nginx-storage-persistentvolumeclaim.yaml | 29 +
.../examples/{ozone => ozone-csi}/s3g-service.yaml | 0
.../{ozone => ozone-csi}/s3g-statefulset.yaml | 0
.../examples/{ozone => ozone-csi}/scm-service.yaml | 0
.../{ozone => ozone-csi}/scm-statefulset.yaml | 0
.../main/smoketest/auditparser/auditparser.robot | 18 +-
.../src/main/smoketest/basic/ozone-shell.robot | 62 +-
.../dist/src/main/smoketest/createmrenv.robot | 48 +
.../dist/src/main/smoketest/mapreduce.robot | 37 +
.../main/smoketest/security/ozone-secure-fs.robot | 50 +-
hadoop-ozone/dist/src/main/smoketest/test.sh | 3 +-
.../fault-injection-test/network-tests/pom.xml | 103 ++
.../network-tests/src/test/blockade/README.md | 60 +
.../src/test}/blockade/blockadeUtils/__init__.py | 0
.../src/test/blockade/blockadeUtils/blockade.py | 92 ++
.../src/test/blockade/clusterUtils}/__init__.py | 0
.../test/blockade/clusterUtils/cluster_utils.py | 335 +++++
.../network-tests/src/test}/blockade/conftest.py | 0
.../src/test/blockade/ozone}/__init__.py | 0
.../src/test/blockade/ozone/cluster.py | 303 +++++
.../test/blockade/test_blockade_client_failure.py | 131 ++
.../blockade/test_blockade_datanode_isolation.py | 137 ++
.../src/test/blockade/test_blockade_flaky.py | 77 ++
.../test/blockade/test_blockade_mixed_failure.py | 159 +++
...t_blockade_mixed_failure_three_nodes_isolate.py | 235 ++++
.../test_blockade_mixed_failure_two_nodes.py | 183 +++
.../test/blockade/test_blockade_scm_isolation.py | 167 +++
.../network-tests/src/test/blockade/util.py | 52 +
.../src/test/compose/docker-compose.yaml | 50 +
.../network-tests/src/test/compose/docker-config | 77 ++
hadoop-ozone/fault-injection-test/pom.xml | 35 +
hadoop-ozone/integration-test/pom.xml | 8 +-
.../TestContainerStateManagerIntegration.java | 24 +
.../TestRatisPipelineCreateAndDestory.java | 131 ++
.../hdds/scm/pipeline/TestRatisPipelineUtils.java | 130 --
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 6 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 18 +-
.../TestContainerStateMachineIdempotency.java | 11 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 2 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 54 +-
.../hadoop/ozone/TestStorageContainerManager.java | 28 +-
.../ozone/TestStorageContainerManagerHelper.java | 14 +-
.../ozone/client/CertificateClientTestImpl.java | 4 +
.../rpc/TestBlockOutputStreamWithFailures.java | 42 +-
.../rpc/TestContainerStateMachineFailures.java | 85 ++
.../client/rpc/TestFailureHandlingByClient.java | 67 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 438 +++++--
.../hadoop/ozone/client/rpc/TestReadRetries.java | 5 +-
.../ozone/container/TestContainerReplication.java | 12 +-
.../container/common/TestBlockDeletingService.java | 165 +--
.../common/impl/TestContainerPersistence.java | 4 +-
.../commandhandler/TestBlockDeletion.java | 28 +-
.../TestCloseContainerByPipeline.java | 10 +-
.../transport/server/ratis/TestCSMMetrics.java | 21 +-
.../ozone/om/TestMultipleContainerReadWrite.java | 2 +
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 10 +-
.../hadoop/ozone/om/TestOmBlockVersioning.java | 3 +
.../org/apache/hadoop/ozone/om/TestOmInit.java | 6 +-
.../org/apache/hadoop/ozone/om/TestOmMetrics.java | 6 +-
.../apache/hadoop/ozone/om/TestOzoneManager.java | 51 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 4 +-
.../apache/hadoop/ozone/om/TestScmSafeMode.java | 4 +
.../snapshot/TestOzoneManagerSnapshotProvider.java | 126 ++
.../hadoop/ozone/ozShell/TestOzoneShell.java | 16 +-
.../hadoop/ozone/scm/TestContainerSmallFile.java | 34 +-
.../scm/TestGetCommittedBlockLengthAndPutKey.java | 18 +-
.../hadoop/ozone/scm/TestXceiverClientManager.java | 9 +-
hadoop-ozone/objectstore-service/pom.xml | 2 +-
.../web/storage/DistributedStorageHandler.java | 31 +-
hadoop-ozone/ozone-manager/pom.xml | 19 +-
.../org/apache/hadoop/ozone/om/BucketManager.java | 30 +-
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 315 ++++-
.../java/org/apache/hadoop/ozone/om/IOzoneAcl.java | 67 +
.../org/apache/hadoop/ozone/om/KeyManager.java | 1 -
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 466 +++++--
.../hadoop/ozone/om/OMDBCheckpointServlet.java | 26 +-
.../org/apache/hadoop/ozone/om/OMNodeDetails.java | 41 +-
.../apache/hadoop/ozone/om/OMStarterInterface.java | 33 +
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 65 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 532 ++++----
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 9 +-
.../hadoop/ozone/om/OzoneManagerStarter.java | 130 ++
.../org/apache/hadoop/ozone/om/PrefixManager.java | 45 +
.../apache/hadoop/ozone/om/PrefixManagerImpl.java | 316 +++++
.../hadoop/ozone/om/S3BucketManagerImpl.java | 14 +-
.../org/apache/hadoop/ozone/om/VolumeManager.java | 3 +-
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 191 ++-
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 3 +-
.../ozone/om/ratis/OzoneManagerDoubleBuffer.java | 273 ++++
.../ozone/om/ratis/OzoneManagerRatisServer.java | 141 ++-
.../ozone/om/ratis/OzoneManagerRatisSnapshot.java | 32 +
.../ozone/om/ratis/OzoneManagerStateMachine.java | 78 +-
.../ozone/om/ratis/helpers/DoubleBufferEntry.java | 44 +
.../ozone/om/ratis/helpers/package-info.java | 20 +
.../metrics/OzoneManagerDoubleBufferMetrics.java | 89 ++
.../ozone/om/ratis/metrics/package-info.java | 21 +
.../om/ratis/utils/OzoneManagerRatisUtils.java | 96 ++
.../hadoop/ozone/om/ratis/utils/package-info.java | 21 +
.../hadoop/ozone/om/request/OMClientRequest.java | 215 ++++
.../hadoop/ozone/om/request/RequestAuditor.java | 50 +
.../om/request/bucket/OMBucketCreateRequest.java | 234 ++++
.../om/request/bucket/OMBucketDeleteRequest.java | 156 +++
.../request/bucket/OMBucketSetPropertyRequest.java | 228 ++++
.../ozone/om/request/bucket/package-info.java | 23 +
.../hadoop/ozone/om/request/package-info.java | 21 +
.../om/request/volume/OMVolumeCreateRequest.java | 196 +++
.../om/request/volume/OMVolumeDeleteRequest.java | 207 +++
.../ozone/om/request/volume/OMVolumeRequest.java | 98 ++
.../om/request/volume/OMVolumeSetOwnerRequest.java | 212 ++++
.../om/request/volume/OMVolumeSetQuotaRequest.java | 166 +++
.../ozone/om/request/volume/package-info.java | 22 +
.../hadoop/ozone/om/response/OMClientResponse.java | 59 +
.../om/response/bucket/OMBucketCreateResponse.java | 64 +
.../om/response/bucket/OMBucketDeleteResponse.java | 67 +
.../bucket/OMBucketSetPropertyResponse.java | 57 +
.../ozone/om/response/bucket/package-info.java | 23 +
.../hadoop/ozone/om/response/package-info.java | 24 +
.../om/response/volume/OMVolumeCreateResponse.java | 74 ++
.../om/response/volume/OMVolumeDeleteResponse.java | 70 ++
.../response/volume/OMVolumeSetOwnerResponse.java | 79 ++
.../response/volume/OMVolumeSetQuotaResponse.java | 55 +
.../ozone/om/response/volume/package-info.java | 22 +
.../om/snapshot/OzoneManagerSnapshotProvider.java | 210 ++++
.../hadoop/ozone/om/snapshot/package-info.java | 23 +
.../protocolPB/OzoneManagerHARequestHandler.java | 4 +-
.../OzoneManagerHARequestHandlerImpl.java | 210 +---
...OzoneManagerProtocolServerSideTranslatorPB.java | 64 +-
.../protocolPB/OzoneManagerRequestHandler.java | 119 +-
.../web/ozShell/bucket/AddAclBucketHandler.java | 101 ++
.../ozone/web/ozShell/bucket/BucketCommands.java | 6 +-
.../web/ozShell/bucket/GetAclBucketHandler.java | 84 ++
.../web/ozShell/bucket/RemoveAclBucketHandler.java | 101 ++
.../web/ozShell/bucket/SetAclBucketHandler.java | 101 ++
.../ozone/web/ozShell/keys/AddAclKeyHandler.java | 104 ++
.../ozone/web/ozShell/keys/GetAclKeyHandler.java | 87 ++
.../hadoop/ozone/web/ozShell/keys/KeyCommands.java | 6 +-
.../web/ozShell/keys/RemoveAclKeyHandler.java | 104 ++
.../ozone/web/ozShell/keys/SetAclKeyHandler.java | 103 ++
.../web/ozShell/volume/AddAclVolumeHandler.java | 98 ++
.../web/ozShell/volume/GetAclVolumeHandler.java | 78 ++
.../web/ozShell/volume/RemoveAclVolumeHandler.java | 98 ++
.../web/ozShell/volume/SetAclVolumeHandler.java | 101 ++
.../ozone/web/ozShell/volume/VolumeCommands.java | 6 +-
.../hadoop/ozone/om/TestBucketManagerImpl.java | 18 +-
.../apache/hadoop/ozone/om/TestChunkStreams.java | 11 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 2 +
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 250 +++-
.../hadoop/ozone/om/TestOzoneManagerStarter.java | 154 +++
...tOzoneManagerDoubleBufferWithDummyResponse.java | 171 +++
...TestOzoneManagerDoubleBufferWithOMResponse.java | 450 +++++++
.../om/ratis/TestOzoneManagerRatisServer.java | 28 +-
.../om/ratis/TestOzoneManagerStateMachine.java | 13 +-
.../request/TestOMClientRequestWithUserInfo.java | 119 ++
.../ozone/om/request/TestOMRequestUtils.java | 176 +++
.../request/bucket/TestOMBucketCreateRequest.java | 248 ++++
.../request/bucket/TestOMBucketDeleteRequest.java | 158 +++
.../bucket/TestOMBucketSetPropertyRequest.java | 169 +++
.../ozone/om/request/bucket/package-info.java | 23 +
.../request/volume/TestOMVolumeCreateRequest.java | 265 ++++
.../request/volume/TestOMVolumeDeleteRequest.java | 222 ++++
.../volume/TestOMVolumeSetOwnerRequest.java | 204 +++
.../volume/TestOMVolumeSetQuotaRequest.java | 195 +++
.../ozone/om/request/volume/package-info.java | 21 +
.../ozone/om/response/TestOMResponseUtils.java | 40 +
.../bucket/TestOMBucketCreateResponse.java | 87 ++
.../bucket/TestOMBucketDeleteResponse.java | 97 ++
.../bucket/TestOMBucketSetPropertyResponse.java | 88 ++
.../ozone/om/response/bucket/package-info.java | 23 +
.../volume/TestOMVolumeCreateResponse.java | 125 ++
.../volume/TestOMVolumeDeleteResponse.java | 130 ++
.../volume/TestOMVolumeSetOwnerResponse.java | 142 +++
.../volume/TestOMVolumeSetQuotaResponse.java | 117 ++
.../org.mockito.plugins.MockMaker | 16 +
hadoop-ozone/ozone-recon-codegen/pom.xml | 4 +-
.../recon/codegen/ReconSchemaGenerationModule.java | 3 +-
.../schema/ReconInternalSchemaDefinition.java | 65 +
hadoop-ozone/ozone-recon/pom.xml | 12 +-
.../hadoop/ozone/recon/ReconControllerModule.java | 5 +
.../org/apache/hadoop/ozone/recon/ReconServer.java | 25 +-
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 4 +
.../ozone/recon/api/ContainerKeyService.java | 19 +-
.../recon/recovery/ReconOmMetadataManagerImpl.java | 1 -
.../recon/spi/ContainerDBServiceProvider.java | 28 +-
.../spi/impl/ContainerDBServiceProviderImpl.java | 38 +-
.../spi/impl/OzoneManagerServiceProviderImpl.java | 6 +-
.../ozone/recon/tasks/ContainerKeyMapperTask.java | 154 ++-
.../hadoop/ozone/recon/tasks/OMDBUpdateEvent.java | 150 +++
.../ozone/recon/tasks/OMDBUpdatesHandler.java | 220 ++++
.../ozone/recon/tasks/OMUpdateEventBatch.java | 69 +
.../ozone/recon/tasks/ReconDBUpdateTask.java | 66 +
.../ozone/recon/tasks/ReconTaskController.java | 46 +
.../ozone/recon/tasks/ReconTaskControllerImpl.java | 198 +++
.../ozone/recon/api/TestContainerKeyService.java | 58 +-
.../recon/persistence/AbstractSqlDatabaseTest.java | 5 +-
.../TestReconInternalSchemaDefinition.java | 143 +++
.../recovery/TestReconOmMetadataManagerImpl.java | 17 -
.../impl/TestContainerDBServiceProviderImpl.java | 25 +
.../hadoop/ozone/recon/tasks/DummyReconDBTask.java | 77 ++
.../recon/tasks/TestContainerKeyMapperTask.java | 155 ++-
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 207 +++
.../recon/tasks/TestReconTaskControllerImpl.java | 171 +++
hadoop-ozone/ozonefs-lib-current/pom.xml | 2 +-
hadoop-ozone/ozonefs-lib-legacy/pom.xml | 2 +-
hadoop-ozone/ozonefs/pom.xml | 2 +-
.../hadoop/fs/ozone/FilteredClassLoader.java | 6 +
.../hadoop/fs/ozone/TestFilteredClassLoader.java | 63 +
hadoop-ozone/pom.xml | 45 +-
hadoop-ozone/s3gateway/pom.xml | 2 +-
hadoop-ozone/tools/pom.xml | 2 +-
.../hadoop/ozone/freon/RandomKeyGenerator.java | 357 +++---
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 5 +-
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 2 +-
.../org/apache/hadoop/ozone/scm/cli/SQLCLI.java | 2 +-
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 35 +-
hadoop-ozone/upgrade/pom.xml | 2 +-
hadoop-project-dist/pom.xml | 4 +-
hadoop-project/pom.xml | 6 +-
hadoop-project/src/site/site.xml | 1 +
hadoop-submarine/dev-support/checks/rat.sh | 24 +
hadoop-submarine/hadoop-submarine-all/pom.xml | 6 +-
hadoop-submarine/hadoop-submarine-core/README.md | 7 +-
hadoop-submarine/hadoop-submarine-core/pom.xml | 6 +-
.../src/site/markdown/DeveloperGuide.md | 24 -
.../src/site/markdown/Examples.md | 6 +-
.../src/site/markdown/HowToInstall.md | 24 +-
.../src/site/markdown/Index.md | 19 +-
.../src/site/markdown/InstallationGuide.md | 79 +-
.../src/site/markdown/QuickStart.md | 48 +-
.../markdown/RunningDistributedCifar10TFJobs.md | 16 +-
.../src/site/markdown/RunningZeppelinOnYARN.md | 37 -
.../src/site/markdown/TestAndTroubleshooting.md | 8 +-
.../src/site/markdown/TonYRuntimeGuide.md | 309 +++++
hadoop-submarine/hadoop-submarine-dist/pom.xml | 6 +-
.../hadoop-submarine-tony-runtime/pom.xml | 15 +-
.../src/site/markdown/QuickStart.md | 301 -----
.../src/site/resources/css/site.css | 29 -
.../src/site/site.xml | 28 -
.../hadoop-submarine-yarnservice-runtime/pom.xml | 10 +-
hadoop-submarine/pom.xml | 4 +-
hadoop-tools/hadoop-aliyun/pom.xml | 2 +-
hadoop-tools/hadoop-archive-logs/pom.xml | 2 +-
hadoop-tools/hadoop-archives/pom.xml | 2 +-
hadoop-tools/hadoop-aws/pom.xml | 13 +-
.../java/org/apache/hadoop/fs/s3a/Constants.java | 95 +-
.../java/org/apache/hadoop/fs/s3a/Listing.java | 2 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 704 +++++++----
.../org/apache/hadoop/fs/s3a/S3AInputStream.java | 19 +-
.../apache/hadoop/fs/s3a/S3AInstrumentation.java | 116 +-
.../java/org/apache/hadoop/fs/s3a/S3AUtils.java | 83 +-
.../apache/hadoop/fs/s3a/S3ObjectAttributes.java | 52 +-
.../java/org/apache/hadoop/fs/s3a/Statistic.java | 11 +
.../apache/hadoop/fs/s3a/WriteOperationHelper.java | 62 +-
.../hadoop/fs/s3a/commit/AbstractS3ACommitter.java | 42 +-
.../hadoop/fs/s3a/commit/CommitOperations.java | 155 ++-
.../fs/s3a/commit/magic/MagicS3GuardCommitter.java | 7 +-
.../fs/s3a/commit/staging/StagingCommitter.java | 12 +-
.../hadoop/fs/s3a/impl/AbstractStoreOperation.java | 49 +
.../hadoop/fs/s3a/impl/CallableSupplier.java | 126 ++
.../apache/hadoop/fs/s3a/impl/ChangeTracker.java | 3 +-
.../hadoop/fs/s3a/impl/ContextAccessors.java | 74 ++
.../hadoop/fs/s3a/impl/InternalConstants.java | 60 +
.../fs/s3a/impl/MultiObjectDeleteSupport.java | 244 ++++
.../apache/hadoop/fs/s3a/impl/RenameOperation.java | 634 ++++++++++
.../apache/hadoop/fs/s3a/impl/StoreContext.java | 335 +++++
.../hadoop/fs/s3a/s3guard/BulkOperationState.java | 82 ++
.../fs/s3a/s3guard/DelayedUpdateRenameTracker.java | 192 +++
.../fs/s3a/s3guard/DynamoDBMetadataStore.java | 834 ++++++++++--
.../hadoop/fs/s3a/s3guard/ITtlTimeProvider.java | 34 +
.../hadoop/fs/s3a/s3guard/LocalMetadataStore.java | 186 ++-
.../hadoop/fs/s3a/s3guard/MetadataStore.java | 172 ++-
.../hadoop/fs/s3a/s3guard/NullMetadataStore.java | 68 +-
.../s3guard/PathMetadataDynamoDBTranslation.java | 22 +-
.../fs/s3a/s3guard/PathOrderComparators.java | 133 ++
.../fs/s3a/s3guard/ProgressiveRenameTracker.java | 252 ++++
.../hadoop/fs/s3a/s3guard/RenameTracker.java | 275 ++++
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 306 ++++-
.../apache/hadoop/fs/s3a/s3guard/S3GuardTool.java | 44 +-
.../s3a/s3guard/TableDeleteTimeoutException.java | 34 +
.../src/site/markdown/tools/hadoop-aws/s3guard.md | 171 ++-
.../src/site/markdown/tools/hadoop-aws/testing.md | 4 +
.../fs/contract/s3a/ITestS3AContractRename.java | 71 +-
.../hadoop/fs/s3a/ITestS3AFailureHandling.java | 102 +-
.../s3a/ITestS3AMetadataPersistenceException.java | 10 +-
.../java/org/apache/hadoop/fs/s3a/ITestS3ASSL.java | 72 --
.../org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java | 79 +-
.../hadoop/fs/s3a/ITestS3GuardListConsistency.java | 13 +-
.../fs/s3a/ITestS3GuardOutOfBandOperations.java | 349 +++++-
.../org/apache/hadoop/fs/s3a/ITestS3GuardTtl.java | 194 ++-
.../apache/hadoop/fs/s3a/MockS3AFileSystem.java | 4 +-
.../org/apache/hadoop/fs/s3a/S3ATestConstants.java | 10 +
.../hadoop/fs/s3a/TestStreamChangeTracker.java | 10 +-
.../apache/hadoop/fs/s3a/auth/ITestAssumeRole.java | 219 +---
.../apache/hadoop/fs/s3a/auth/RoleTestUtils.java | 46 +-
.../hadoop/fs/s3a/commit/AbstractCommitITest.java | 2 +
.../fs/s3a/commit/AbstractITCommitMRJob.java | 4 +
.../fs/s3a/commit/AbstractYarnClusterITest.java | 8 +-
.../fs/s3a/commit/ITestCommitOperations.java | 127 +-
.../fs/s3a/commit/magic/ITestMagicCommitMRJob.java | 26 +-
.../s3a/commit/magic/ITestS3AHugeMagicCommits.java | 9 +-
.../integration/ITestDirectoryCommitMRJob.java | 2 +-
.../integration/ITestPartitionCommitMRJob.java | 2 +-
.../integration/ITestStagingCommitMRJob.java | 2 +-
.../ITestStagingCommitMRJobBadDest.java | 2 +-
.../commit/terasort/AbstractCommitTerasortIT.java | 5 +
.../fs/s3a/impl/ITestPartialRenamesDeletes.java | 871 +++++++++++++
.../fs/s3a/impl/TestPartialDeleteFailures.java | 393 ++++++
.../s3a/s3guard/AbstractS3GuardToolTestBase.java | 54 +-
.../hadoop/fs/s3a/s3guard/DDBCapacities.java | 27 +-
.../fs/s3a/s3guard/ITestDynamoDBMetadataStore.java | 757 ++++++++---
.../s3guard/ITestDynamoDBMetadataStoreScale.java | 230 ++--
.../fs/s3a/s3guard/ITestS3GuardToolDynamoDB.java | 46 +-
.../fs/s3a/s3guard/ITestS3GuardToolLocal.java | 3 +-
.../fs/s3a/s3guard/MetadataStoreTestBase.java | 372 ++++--
.../fs/s3a/s3guard/TestDynamoDBMiscOperations.java | 69 +
.../fs/s3a/s3guard/TestLocalMetadataStore.java | 7 +-
.../fs/s3a/s3guard/TestNullMetadataStore.java | 5 +
.../fs/s3a/s3guard/TestPathOrderComparators.java | 197 +++
.../apache/hadoop/fs/s3a/s3guard/TestS3Guard.java | 195 ++-
.../hadoop/fs/s3a/s3guard/ThrottleTracker.java | 134 ++
.../scale/AbstractITestS3AMetadataStoreScale.java | 31 +-
.../apache/hadoop/fs/s3a/test/ExtraAssertions.java | 138 ++
hadoop-tools/hadoop-azure-datalake/pom.xml | 4 +-
hadoop-tools/hadoop-azure/pom.xml | 6 +-
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 4 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 23 +-
.../constants/FileSystemConfigurations.java | 6 +-
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 4 +-
.../hadoop/fs/azurebfs/services/AbfsClient.java | 8 +-
.../fs/azurebfs/services/AbfsHttpOperation.java | 4 +-
.../fs/azurebfs/utils/SSLSocketFactoryEx.java | 246 ++++
.../TestAbfsConfigurationFieldsValidation.java | 16 +-
.../fs/azurebfs/services/TestAbfsClient.java | 6 +-
hadoop-tools/hadoop-datajoin/pom.xml | 2 +-
hadoop-tools/hadoop-distcp/pom.xml | 2 +-
.../hadoop-distcp/src/site/markdown/DistCp.md.vm | 4 +-
hadoop-tools/hadoop-extras/pom.xml | 2 +-
hadoop-tools/hadoop-gridmix/pom.xml | 2 +-
hadoop-tools/hadoop-kafka/pom.xml | 2 +-
hadoop-tools/hadoop-openstack/pom.xml | 2 +-
hadoop-tools/hadoop-pipes/pom.xml | 2 +-
hadoop-tools/hadoop-resourceestimator/pom.xml | 2 +-
hadoop-tools/hadoop-rumen/pom.xml | 2 +-
hadoop-tools/hadoop-sls/pom.xml | 2 +-
.../src/main/data/2jobs2min-rumen-jh.json | 28 -
hadoop-tools/hadoop-streaming/pom.xml | 2 +-
hadoop-tools/hadoop-tools-dist/pom.xml | 2 +-
hadoop-tools/pom.xml | 2 +-
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 2 +-
.../yarn/api/records/timeline/TimelineHealth.java | 82 ++
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 2 +-
.../pom.xml | 4 +-
.../pom.xml | 19 +-
.../hadoop-yarn-applications-catalog/pom.xml | 4 +-
.../pom.xml | 2 +-
.../distributedshell/ApplicationMaster.java | 29 +-
.../yarn/applications/distributedshell/Client.java | 48 +-
.../distributedshell/TestDistributedShell.java | 27 +
.../hadoop-yarn-applications-mawo-core/pom.xml | 8 +-
.../hadoop-yarn-applications-mawo/pom.xml | 6 +-
.../pom.xml | 2 +-
.../hadoop-yarn-services-api/pom.xml | 2 +-
.../hadoop-yarn-services-core/pom.xml | 2 +-
.../hadoop-yarn-services/pom.xml | 2 +-
.../hadoop-yarn/hadoop-yarn-applications/pom.xml | 2 +-
.../org/apache/hadoop/yarn/client/cli/LogsCLI.java | 44 +-
.../hadoop/yarn/client/cli/SchedConfCLI.java | 26 +-
.../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 95 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 2 +-
.../AggregatedLogDeletionService.java | 10 +-
.../yarn/logaggregation/LogAggregationUtils.java | 49 +-
.../hadoop/yarn/logaggregation/LogCLIHelpers.java | 156 ++-
.../tfile/LogAggregationTFileController.java | 8 +-
.../tfile/TFileAggregatedLogsBlock.java | 9 +-
.../org/apache/hadoop/yarn/webapp/Dispatcher.java | 9 +
.../hadoop/yarn/webapp/util/WebAppUtils.java | 39 +-
.../yarn/webapp/util/YarnWebServiceUtils.java | 29 +-
.../src/main/resources/yarn-default.xml | 10 -
.../hadoop/yarn/conf/TestYarnConfiguration.java | 6 +
.../TestAggregatedLogDeletionService.java | 19 +-
.../logaggregation/TestAggregatedLogsBlock.java | 2 +-
.../logaggregation/TestContainerLogsUtils.java | 12 +-
.../TestLogAggregationIndexFileController.java | 445 -------
.../TestLogAggregationIndexedFileController.java | 447 +++++++
.../hadoop-yarn/hadoop-yarn-csi/pom.xml | 2 +-
.../hadoop-yarn/hadoop-yarn-registry/pom.xml | 2 +-
.../pom.xml | 2 +-
.../webapp/TestAHSWebServices.java | 5 +
.../hadoop-yarn-server-common/pom.xml | 2 +-
.../server/util/timeline/TimelineServerUtils.java | 10 +-
.../hadoop/yarn/server/webapp/AppAttemptBlock.java | 3 +-
.../hadoop/yarn/server/webapp/dao/AppInfo.java | 16 +
.../hadoop-yarn-server-nodemanager/pom.xml | 2 +-
.../yarn/server/nodemanager/ContainerExecutor.java | 14 +-
.../nodemanager/DefaultContainerExecutor.java | 62 +-
.../server/nodemanager/LinuxContainerExecutor.java | 54 +-
.../yarn/server/nodemanager/NodeManager.java | 8 +-
.../localizer/ContainerLocalizer.java | 11 +-
.../monitor/ContainersMonitorImpl.java | 113 +-
.../resourceplugin/fpga/FpgaDiscoverer.java | 5 +-
.../scheduler/ContainerScheduler.java | 26 +-
.../timelineservice/NMTimelinePublisher.java | 12 -
.../localizer/TestContainerLocalizer.java | 29 +
.../logaggregation/TestLogAggregationService.java | 14 +-
.../resourceplugin/fpga/TestFpgaDiscoverer.java | 33 +
.../scheduler/TestContainerSchedulerQueuing.java | 15 +-
.../hadoop-yarn-server-resourcemanager/pom.xml | 2 +-
.../DecommissioningNodesWatcher.java | 47 +-
.../server/resourcemanager/rmapp/RMAppImpl.java | 1 +
.../scheduler/AppSchedulingInfo.java | 14 +
.../scheduler/SchedulerApplicationAttempt.java | 7 +-
.../scheduler/activities/ActivitiesLogger.java | 6 +-
.../scheduler/activities/ActivitiesManager.java | 71 +-
.../scheduler/activities/ActivitiesUtils.java | 61 +
.../scheduler/activities/ActivityNode.java | 16 +-
.../scheduler/activities/AppAllocation.java | 11 +-
.../capacity/AbstractManagedParentQueue.java | 8 +
.../scheduler/capacity/CapacityScheduler.java | 6 +-
.../scheduler/capacity/LeafQueue.java | 3 +
.../scheduler/capacity/ParentQueue.java | 4 +
.../allocator/AbstractContainerAllocator.java | 11 +-
.../allocator/RegularContainerAllocator.java | 8 +-
.../SingleConstraintAppPlacementAllocator.java | 5 +-
.../server/resourcemanager/webapp/RMWSConsts.java | 30 +-
.../resourcemanager/webapp/RMWebAppUtil.java | 4 +
.../webapp/RMWebServiceProtocol.java | 27 +-
.../resourcemanager/webapp/RMWebServices.java | 142 ++-
.../resourcemanager/webapp/dao/ActivitiesInfo.java | 6 +-
.../webapp/dao/ActivityNodeInfo.java | 50 +-
.../webapp/dao/AppActivitiesInfo.java | 6 +-
.../webapp/dao/AppAllocationInfo.java | 15 +-
.../webapp/dao/AppRequestAllocationInfo.java | 15 +-
.../webapp/dao/NodeAllocationInfo.java | 6 +-
.../TestDecommissioningNodesWatcher.java | 101 +-
.../rmapp/TestRMAppTransitions.java | 5 +
.../scheduler/TestSchedulerApplicationAttempt.java | 2 +-
.../scheduler/TestSchedulerOvercommit.java | 6 +-
.../activities/TestActivitiesManager.java | 112 +-
.../TestCapacitySchedulerAutoCreatedQueueBase.java | 27 +-
.../TestCapacitySchedulerAutoQueueCreation.java | 11 +-
.../capacity/TestCapacitySchedulerMultiNodes.java | 57 +
.../scheduler/capacity/TestLeafQueue.java | 10 +
.../TestSingleConstraintAppPlacementAllocator.java | 7 +-
.../webapp/ActivitiesTestUtils.java | 15 +
.../webapp/TestRMWebServicesContainers.java | 158 +++
.../TestRMWebServicesSchedulerActivities.java | 411 +++---
...esSchedulerActivitiesWithMultiNodesEnabled.java | 223 +++-
.../hadoop-yarn-server-router/pom.xml | 2 +-
.../webapp/DefaultRequestInterceptorREST.java | 15 +-
.../router/webapp/FederationInterceptorREST.java | 12 +-
.../server/router/webapp/RouterWebServices.java | 32 +-
.../router/webapp/BaseRouterWebServicesTest.java | 6 +-
.../router/webapp/MockRESTRequestInterceptor.java | 12 +-
.../webapp/PassThroughRESTRequestInterceptor.java | 17 +-
.../hadoop-yarn-server-tests/pom.xml | 2 +-
.../apache/hadoop/yarn/server/MiniYARNCluster.java | 25 +
.../security/TestTimelineAuthFilterForV2.java | 177 +--
.../pom.xml | 2 +-
.../timeline/EntityGroupFSTimelineStore.java | 70 +-
.../server/timeline/LevelDBCacheTimelineStore.java | 8 +
.../timeline/TestEntityGroupFSTimelineStore.java | 64 +-
.../timeline/TestLevelDBCacheTimelineStore.java | 13 +
.../pom.xml | 2 +-
.../DocumentStoreTimelineReaderImpl.java | 13 +
.../pom.xml | 2 +-
.../storage/TestTimelineReaderHBaseDown.java | 4 +-
.../pom.xml | 15 +-
.../storage/HBaseStorageMonitor.java | 90 ++
.../storage/HBaseTimelineReaderImpl.java | 99 +-
.../storage/common/HBaseTimelineStorageUtils.java | 32 +-
.../common/TestHBaseTimelineStorageUtils.java | 74 ++
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../hadoop-yarn-server-timelineservice/pom.xml | 2 +-
.../reader/TimelineReaderManager.java | 10 +
.../reader/TimelineReaderServer.java | 11 +-
.../reader/TimelineReaderWebServices.java | 33 +
.../storage/FileSystemTimelineReaderImpl.java | 15 +
.../storage/NoOpTimelineReaderImpl.java | 7 +
.../timelineservice/storage/TimelineReader.java | 8 +
.../storage/TimelineStorageMonitor.java | 106 ++
.../reader/TestTimelineReaderWebServices.java | 19 +
.../hadoop-yarn-server-web-proxy/pom.xml | 2 +-
.../webproxy/amfilter/TestSecureAmFilter.java | 10 +-
.../hadoop-yarn/hadoop-yarn-server/pom.xml | 2 +-
.../hadoop-yarn/hadoop-yarn-site/pom.xml | 2 +-
.../src/site/markdown/ResourceManagerRest.md | 54 +
.../src/site/markdown/ResourceModel.md | 2 +-
.../hadoop-yarn-site/src/site/markdown/YarnUI2.md | 2 +-
.../site/markdown/yarn-service/Configurations.md | 1 +
.../hadoop-yarn/hadoop-yarn-ui/pom.xml | 2 +-
.../hadoop-yarn-ui/src/main/webapp/.gitignore | 4 +
.../main/webapp/app/adapters/timeline-health.js | 30 +
.../main/webapp/app/components/nodes-heatmap.js | 6 +-
.../src/main/webapp/app/controllers/application.js | 7 +
.../src/main/webapp/app/models/timeline-health.js | 27 +
.../src/main/webapp/app/models/yarn-user.js | 11 +-
.../src/main/webapp/app/routes/application.js | 4 +
.../src/main/webapp/app/routes/timeline-error.js | 3 +
.../main/webapp/app/serializers/timeline-health.js | 31 +
.../app/serializers/yarn-queue/capacity-queue.js | 11 +
.../src/main/webapp/app/styles/app.scss | 18 +
.../src/main/webapp/app/templates/application.hbs | 4 +-
.../app/templates/components/nodes-heatmap.hbs | 1 -
.../main/webapp/app/templates/yarn-queue/apps.hbs | 47 +-
hadoop-yarn-project/hadoop-yarn/pom.xml | 4 +-
hadoop-yarn-project/pom.xml | 4 +-
pom.ozone.xml | 2 +-
pom.xml | 65 +-
1046 files changed, 49534 insertions(+), 12123 deletions(-)
copy hadoop-assemblies/src/main/resources/assemblies/{hadoop-src-with-hdds.xml => hadoop-src-submarine.xml} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FunctionsRaisingIOE.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/WebServlet.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProcessingDetails.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilter.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilterInitializer.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/package-info.java
delete mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/OpenSSLSocketFactory.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/pmdk_load.c
create mode 100644 hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/pmdk_load.h
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpnego.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestProcessingDetails.java
delete mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithSecurityOn.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authentication/server/TestProxyUserAuthenticationFilter.java
delete mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestOpenSSLSocketFactory.java
delete mode 100644 hadoop-common-project/hadoop-common/src/test/resources/kdc/killKdc.sh
delete mode 100644 hadoop-common-project/hadoop-common/src/test/resources/kdc/ldif/users.ldif
delete mode 100644 hadoop-common-project/hadoop-common/src/test/resources/kdc/server.xml
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkInputStream.java
create mode 100644 hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/storage/TestChunkInputStream.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/ContainerPlacementPolicy.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/CheckedBiFunction.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneAcl.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/UniqueId.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/TestHddsUtils.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/TestOzoneAcls.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/ReferenceCountedDB.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestContainerCache.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/ContainerPlacementPolicy.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/ContainerPlacementPolicyFactory.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRackAware.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/SafeModeMetrics.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStarterInterface.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManagerStarter.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/TestContainerPlacementFactory.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/TestSCMContainerPlacementRackAware.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestStorageContainerManagerStarter.java
create mode 100644 hadoop-hdds/server-scm/src/test/resources/nodegroup-mapping
create mode 100644 hadoop-hdds/server-scm/src/test/resources/rack-mapping
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/TopologySubcommand.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-signature.secret
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerWebServerWithRandomSecret.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HostRestrictingAuthorizationFilter.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/NativePmemMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/NativePmemMappedBlock.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/HostRestrictingAuthorizationFilterHandler.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/package-info.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilterInitializer.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRedundancyMonitor.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestHostRestrictingAuthorizationFilter.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/TestHostRestrictingAuthorizationFilterHandler.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestRefreshNamenodeReplicationConfig.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestAdminHelper.java
create mode 100644 hadoop-ozone/.gitignore
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OzoneAcl.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmPrefixInfoCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmPrefixInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneAclConfig.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneAclException.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/RadixNode.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/RadixTree.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/TestOzoneAcls.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmPrefixInfoCodec.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/util/TestRadixTree.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/util/package-info.java
create mode 100644 hadoop-ozone/csi/dev-support/findbugsExcludeFile.xml
create mode 100644 hadoop-ozone/csi/pom.xml
create mode 100644 hadoop-ozone/csi/src/main/java/org/apache/hadoop/ozone/csi/ControllerService.java
create mode 100644 hadoop-ozone/csi/src/main/java/org/apache/hadoop/ozone/csi/CsiServer.java
create mode 100644 hadoop-ozone/csi/src/main/java/org/apache/hadoop/ozone/csi/IdentitiyService.java
create mode 100644 hadoop-ozone/csi/src/main/java/org/apache/hadoop/ozone/csi/NodeService.java
create mode 100644 hadoop-ozone/csi/src/main/java/org/apache/hadoop/ozone/csi/package-info.java
create mode 100644 hadoop-ozone/csi/src/main/proto/csi.proto
delete mode 100644 hadoop-ozone/dist/src/main/blockade/README.md
delete mode 100644 hadoop-ozone/dist/src/main/blockade/blockadeUtils/blockade.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/clusterUtils/__init__.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/clusterUtils/cluster_utils.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_client_failure.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_datanode_isolation.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_flaky.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure_three_nodes_isolate.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure_two_nodes.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_scm_isolation.py
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-mr/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-mr/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-mr/docker-config
create mode 100755 hadoop-ozone/dist/src/main/compose/ozone-mr/test.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-net-topology/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-net-topology/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-net-topology/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-net-topology/network-config
create mode 100755 hadoop-ozone/dist/src/main/compose/ozone-net-topology/test.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/ozoneblockade/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonefs/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozones3/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonetrace/.env
create mode 100755 hadoop-ozone/dist/src/main/dockerbin/entrypoint.sh
create mode 100755 hadoop-ozone/dist/src/main/dockerbin/envtoconf.py
create mode 100755 hadoop-ozone/dist/src/main/dockerbin/transformation.py
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/ozone-csi/csi-controller.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/ozone-csi/csi-crd.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/ozone-csi/csi-node.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/ozone-csi/csi-rbac.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/ozone-csi/csi-storageclass.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/ozone-csi/definitions/csi.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/pv-test/flekszible.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/pv-test/nginx-conf-configmap.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/pv-test/nginx-deployment.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/pv-test/nginx-service-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/definitions/pv-test/nginx-storage-persistentvolumeclaim.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/Flekszible
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/LICENSE.header (100%)
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/config-configmap.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/csi-node-daemonset.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/csi-ozone-clusterrole.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/csi-ozone-clusterrolebinding.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/csi-ozone-serviceaccount.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/csi-provisioner-deployment.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/datanode-daemonset.yaml (100%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/om-service.yaml (100%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/om-statefulset.yaml (100%)
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/org.apache.hadoop.ozone-csidriver.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/ozone-storageclass.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/pv-test/nginx-conf-configmap.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/pv-test/nginx-deployment.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/pv-test/nginx-service-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/examples/ozone-csi/pv-test/nginx-storage-persistentvolumeclaim.yaml
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/s3g-service.yaml (100%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/s3g-statefulset.yaml (100%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/scm-service.yaml (100%)
copy hadoop-ozone/dist/src/main/k8s/examples/{ozone => ozone-csi}/scm-statefulset.yaml (100%)
create mode 100644 hadoop-ozone/dist/src/main/smoketest/createmrenv.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/mapreduce.robot
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/pom.xml
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/README.md
copy hadoop-ozone/{dist/src/main => fault-injection-test/network-tests/src/test}/blockade/blockadeUtils/__init__.py (100%)
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/blockadeUtils/blockade.py
copy hadoop-ozone/{dist/src/main/blockade/blockadeUtils => fault-injection-test/network-tests/src/test/blockade/clusterUtils}/__init__.py (100%)
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/clusterUtils/cluster_utils.py
rename hadoop-ozone/{dist/src/main => fault-injection-test/network-tests/src/test}/blockade/conftest.py (100%)
rename hadoop-ozone/{dist/src/main/blockade/blockadeUtils => fault-injection-test/network-tests/src/test/blockade/ozone}/__init__.py (100%)
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/ozone/cluster.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_client_failure.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_datanode_isolation.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_flaky.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_mixed_failure.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_mixed_failure_three_nodes_isolate.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_mixed_failure_two_nodes.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/test_blockade_scm_isolation.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/blockade/util.py
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/compose/docker-compose.yaml
create mode 100644 hadoop-ozone/fault-injection-test/network-tests/src/test/compose/docker-config
create mode 100644 hadoop-ozone/fault-injection-test/pom.xml
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineCreateAndDestory.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineUtils.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestOzoneManagerSnapshotProvider.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/IOzoneAcl.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMStarterInterface.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerStarter.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/PrefixManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/PrefixManagerImpl.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerRatisSnapshot.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/helpers/DoubleBufferEntry.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/helpers/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/metrics/OzoneManagerDoubleBufferMetrics.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/metrics/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/utils/OzoneManagerRatisUtils.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/utils/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/OMClientRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/RequestAuditor.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketDeleteRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketSetPropertyRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeCreateRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeDeleteRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeSetOwnerRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeSetQuotaRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/OMClientResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/bucket/OMBucketCreateResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/bucket/OMBucketDeleteResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/bucket/OMBucketSetPropertyResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/bucket/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/volume/OMVolumeCreateResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/volume/OMVolumeDeleteResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/volume/OMVolumeSetOwnerResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/volume/OMVolumeSetQuotaResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/volume/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OzoneManagerSnapshotProvider.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/AddAclBucketHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/GetAclBucketHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/RemoveAclBucketHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/SetAclBucketHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/AddAclKeyHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/GetAclKeyHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/RemoveAclKeyHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/SetAclKeyHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/AddAclVolumeHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/GetAclVolumeHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/RemoveAclVolumeHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/SetAclVolumeHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerStarter.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerDoubleBufferWithDummyResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerDoubleBufferWithOMResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/TestOMClientRequestWithUserInfo.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/TestOMRequestUtils.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/TestOMBucketCreateRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/TestOMBucketDeleteRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/TestOMBucketSetPropertyRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/volume/TestOMVolumeCreateRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/volume/TestOMVolumeDeleteRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/volume/TestOMVolumeSetOwnerRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/volume/TestOMVolumeSetQuotaRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/volume/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/TestOMResponseUtils.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/bucket/TestOMBucketCreateResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/bucket/TestOMBucketDeleteResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/bucket/TestOMBucketSetPropertyResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/bucket/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/volume/TestOMVolumeCreateResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/volume/TestOMVolumeDeleteResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/volume/TestOMVolumeSetOwnerResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/volume/TestOMVolumeSetQuotaResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker
create mode 100644 hadoop-ozone/ozone-recon-codegen/src/main/java/org/hadoop/ozone/recon/schema/ReconInternalSchemaDefinition.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OMDBUpdateEvent.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OMDBUpdatesHandler.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OMUpdateEventBatch.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconDBUpdateTask.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskController.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/persistence/TestReconInternalSchemaDefinition.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/DummyReconDBTask.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestOMDBUpdatesHandler.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestReconTaskControllerImpl.java
create mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestFilteredClassLoader.java
create mode 100755 hadoop-submarine/dev-support/checks/rat.sh
delete mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/DeveloperGuide.md
delete mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/RunningZeppelinOnYARN.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/TonYRuntimeGuide.md
delete mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/site/markdown/QuickStart.md
delete mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/site/resources/css/site.css
delete mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/site/site.xml
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AbstractStoreOperation.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CallableSupplier.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ContextAccessors.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/InternalConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/RenameOperation.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/StoreContext.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/BulkOperationState.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DelayedUpdateRenameTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ITtlTimeProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/PathOrderComparators.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ProgressiveRenameTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/RenameTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/TableDeleteTimeoutException.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ASSL.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestPartialDeleteFailures.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/TestPathOrderComparators.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ThrottleTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/test/ExtraAssertions.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/SSLSocketFactoryEx.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelineHealth.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/TestLogAggregationIndexFileController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/TestLogAggregationIndexedFileController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/activities/ActivitiesUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesContainers.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseStorageMonitor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineStorageMonitor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/.gitignore
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/timeline-health.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/timeline-health.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/timeline-health.js
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org