You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by eh...@apache.org on 2019/12/02 17:11:57 UTC
[hadoop] branch HDFS-12090 updated (64da704 -> afdb20b)
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a change to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
omit 64da704 HDFS-13794. [PROVIDED Phase 2] Teach BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
omit f482c67 HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode. Contributed by Ewan Higgs.
omit ed8d57b HDFS-13310. The DatanodeProtocol should have a DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
add de934ba HDDS-1076. TestSCMNodeManager crashed the jvm. Contributed by Lokesh Jain.
add 7c1b561 YARN-8927. Added support for top level Dockerhub images to trusted registry using library keyword. Contributed by Zhankun Tang
add d10444e HDDS-1092. Use Java 11 JRE to run Ozone in containers.
add 217bdbd HDDS-1116.Add java profiler servlet to the Ozone web servers. Contributed by Elek, Marton.
add afe126d HDDS-1114. Fix findbugs/checkstyle/accepteance errors in Ozone. Contributed by Marton Elek.
add dde0ab5 HDFS-14258. Introduce Java Concurrent Package To DataXceiverServer Class. Contributed by BELUGA BEHR.
add 7ea9149 HDDS-1041. Support TDE(Transparent Data Encryption) for Ozone. Contributed by Xiaoyu Yao.
add 9057aa9 SUBMARINE-1. Move code base of submarine from yarn-applications to top directory. Contributed by Wangda Tan.
add ba56bc2 YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth.
add 0f2b65c HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
add db4d1a1 YARN-9060. [YARN-8851] Phase 1 - Support device isolation and use the Nvidia GPU plugin as an example. Contributed by Zhankun Tang.
add f2fb653 HDDS-1106. Introduce queryMap in PipelineManager. Contributed by Lokesh Jain.
add 920a896 Revert "HADOOP-15843. s3guard bucket-info command to not print a stack trace on bucket-not-found."
add 235e3da HDFS-14287. DataXceiverServer May Double-Close PeerServer. Contributed by BELUGA BEHR.
add 1de25d1 HDFS-9596. Remove Shuffle Method From DFSUtil. Contributed by BELUGA BEHR.
add 7587f97 HDFS-14296. Prefer ArrayList over LinkedList in VolumeScanner. Contributed by BELUGA BEHR.
add 67af509 HDDS-1122. Fix TestOzoneManagerRatisServer#testSubmitRatisRequest unit test failure. Contributed by Yiqun Lin.
add 588b4c4 HDDS-1085. Create an OM API to serve snapshots to Recon server. Contributed by Aravindan Vijayan.
add 1e0ae6e HADOOP-15843. s3guard bucket-info command to not print a stack trace on bucket-not-found.
add cf1a66d HDDS-1101. SCM CA: Write Certificate information to SCM Metadata. Contributed by Anu Engineer.
add 779dae4 YARN-9309. Improve graph text in SLS to avoid overlapping. Contributed by Bilwa S T.
add 02d04bd HDDS-1121. Key read failure when data is written parallel in to Ozone. Contributed by Bharat Viswanadham.
add b8de78c YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T.
add 14282e3 HDFS-14188. Make hdfs ec -verifyClusterSetup command accept an erasure coding policy as a parameter. Contributed by Kitti Nanasi.
add 0525d85 HADOOP-15967. KMS Benchmark Tool. Contributed by George Huang.
add e8d7e3b HDDS-1139 : Fix findbugs issues caused by HDDS-1085. Contributed by Aravindan Vijayan.
add 51950f1 Logging stale datanode information. Contributed by Karthik Palanisamy.
add 1d30fd9 HDDS-1130. Make BenchMarkBlockManager multi-threaded. Contributed by Lokesh Jain.
add 642fe6a HDDS-1135. Ozone jars are missing in the Ozone Snapshot tar. Contributed by Dinesh Chitlangia.
add 41e18fe HDFS-14235. Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon. Contributed by Ranith Sardar.
add aa3ad36 HADOOP-16104. Wasb tests to downgrade to skip when test a/c is namespace enabled. Contributed by Masatake Iwasaki.
add 1374f8f HDDS-1060. Add API to get OM certificate from SCM CA. Contributed by Ajay Kumar.
add a30059b HDFS-14267. Add test_libhdfs_ops to libhdfs tests, mark libhdfs_read/write.c as examples. Contributed by Sahil Takiar.
add 676a9cb HDDS-1053. Generate RaftGroupId from OMServiceID. Contributed by Aravindan Vijayan.
add f5b4e0f HDFS-14302. Refactor NameNodeWebHdfsMethods#generateDelegationToken() to allow better extensibility. Contributed by CR Hota.
add 1bea785 HDFS-14081. hdfs dfsadmin -metasave metasave_test results NPE. Contributed by Shweta Yakkali.
add b1397ff HDDS-1109. Setup Failover Proxy Provider for OM client.
add 371a6db HDFS-14273. Fix checkstyle issues in BlockLocation's method javadoc (Contributed by Shweta Yakkali via Daniel Templeton)
add a87e458 HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp.
add a63c358 YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph.
add eedcc8e YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
add 7c802c4 HDDS-1129. Fix findbug/checkstyle errors hdds projects. Contributed by Elek, Marton.
add a868f59 HADOOP-16105. WASB in secure mode does not set connectingUsingSAS.
add 92b53c4 HDFS-14216. NullPointerException happens in NamenodeWebHdfs. Contributed by lujie.
add b9b182e HADOOP-16035. Jenkinsfile for Hadoop
add 2bc3cfe HDDS-1126. Datanode is trying to qausi-close a container which is already closed.
add 6c4ab03 YARN-9258. Support to specify allocation tags without constraint in distributed shell CLI. Contributed by Prabhu Joseph.
add b17a260 YARN-9319. Fix compilation issue of handling typedef an existing name by gcc compiler. (Zhankun Tang via wangda)
add d33f066 HDDS-1141. Update DBCheckpointSnapshot to DBCheckpoint.
add 28d0bf9 HDDS-1161. Disable failing test which are tracked by a separated jira. Contributed by Elek, Marton.
add 632d5e8 YARN-9156. Improve debug message in device plugin method compatibility check of ResourcePluginManager. Contributed by Zhankun Tang.
add 9c88695 YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
add 9636fe4 YARN-8891. Documentation of the pluggable device framework. Contributed by Zhankun Tang.
add 95fbbfe YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth.
add ed13cf8 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
add 7d3b567 HDFS-14298. Improve log messages of ECTopologyVerifier. Contributed by Kitti Nanasi.
add 7057980 HDDS-1148. After allocating container, we are not adding to container DB.
add bad3ffd HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen.
add 05bce33 YARN-9244. Document docker registry deployment with direct S3 driver. Contributed by Suma Shivaprasad
add 9daf43c HADOOP-16129. Misc. bug fixes for KMS Benchmark. Contributed by George Huang.
add f19c844 HDFS-14285. libhdfs hdfsRead copies entire array even if its only partially filled. Contributed by Sahil Takiar.
add f7a27cd HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li.
add d0e21ac HDDS-1158. TestOzoneManagerHA.testTwoOMNodesDown is failing with ratis error.
add 1b87668 HDDS-1165. Document generation in maven should be configured on execution level. Contributed by Anu Engineer.
add 014e17a HDDS-1166. Fix checkstyle line length issues. Contributed by Nandakumar.
add fb546d1 HDDS-1167. Error in hadoop-ozone/dev-support/checks/checkstyle.sh. Contributed by Nandakumar.
add c1e5b19 HDDS-1147. Fix failing unit tests in TestOzoneManager. Contributed by Nandakumar.
add 490206e HDDS-1155.Fix failing unit test methods of TestDeadNodeHandler. Contributed by Nandakumar.
add 106bdc6 HDDS-1120. Add a config to disable checksum verification during read.
add dddcfa4 YARN-8821. [YARN-8851] GPU hierarchy/topology scheduling support based on pluggable device framework. Contributed by Zhankun Tang.
add 50094d7 YARN-9300. Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler. Contributed by Tao Yang.
add 9cd5c54 YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
add 021f8de HDDS-1168.Use random ports in TestBlockManager and TestDeletedBlockLog. Contributed by Nandakumar.
add 8387bbd HDDS-1154. Add tracing to the client side of StorageContainerLocationProtocol and OzoneManagerProtocol. Contributed by Elek, Marton.
add d82014b HDDS-1149. Change the default ozone.client.checksum.type. Contributed by Bharat Viswanadham.
add 236b105 HDDS-1089. Disable OzoneFSStorageStatistics for hadoop versions older than 2.8. Contributed by Elek, Marton.
add fc5c83b HDDS-1090. Add a recipe to the ozone docs about using Spark, OzoneFS, and K8s. Contributed by Elek, Marton.
add 416b730 HDDS-1115. Provide ozone specific top-level pom.xml. Contributed by Elek, Marton.
add 5c1f946 HDDS-1145. Add optional web server to the Ozone freon test tool. Contributed by Elek, Marton.
add 3aa0a57 HDDS-1152. Add trace information for the client side of the datanode writes Contributed by Elek, Marton.
add 92b1fdc HDDS-1070. Adding Node and Pipeline related metrics in SCM. Contributed by Nandakumar.
add 5e91ebd YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth.
add a5eb7ce HDDS-1170. Fix the FindBugs issue in OzoneClientAdapterFactory. Contributed by Anu Engineer.
add 6cec906 YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang.
add 3e1739d YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
add f4ae00c HDFS-7133. Support clearing namespace quota on '/'. Contributed by Ayush Saxena."
add 9537265 YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena.
add 0edb0c5 HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections.
add ba4e7bd HADOOP-16125. Support multiple bind users in LdapGroupsMapping. Contributed by Lukas Majercak.
add 9de34d2 HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol. Contributed by Elek, Marton.
add a6ab371 HDFS-14130. [SBN read] Make ZKFC ObserverNode aware. Contributed by xiangheng and Konstantin Shvachko.
add c6ea28c YARN-9331. [YARN-8851] Fix a bug that lacking cgroup initialization when bootstrap DeviceResourceHandlerImpl. Contributed by Zhankun Tang.
add 59ba355 HDFS-14299. ViewFs: Correct error message for read only operations. Contributed by hu xiaodong.
add 9e0f3d1 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar.
add 585cebf HDDS-1104. Use picocli with Ozone genesis tool. Contributed by Lokesh Jain.
add 3988e75 HADOOP-16136. ABFS: Should only transform username to short name
add 52b2eab HADOOP-16093. Move DurationInfo from hadoop-aws to hadoop-common org.apache.hadoop.util.
add a106d2d YARN-8783. Improve the documentation for the docker.trusted.registries configuration. Contributed by Eric Yang.
add a5a751b HDDS-1174. Freon tests are failing with null pointer exception. Contributed by Shashikant Banerjee.
add 9192f71 HADOOP-16127. In ipc.Client, put a new connection could happen after stop.
add 625e937 HDDS-1177. Add validation to AuthorizationHeaderV4. Contributed by Ajay Kumar
add 8eae260 YARN-9311. Fix TestRMRestart hangs. Contributed by Prabhu Joseph.
add 6c96f5e YARN-8378. ApplicationHistoryManagerImpl#getApplications doesn't honor filters. Contributed by Lantao Jin.
add 8c30114 YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
add 0e45020 HDDS-1019. Use apache/hadoop-runner image to test ozone secure cluster. Contributed by Xiaoyu Yao.
add 6c8c422 HDDS-1178. Healthy pipeline Chill Mode Rule.
add 29e27fa YARN-9318. Resources#multiplyAndRoundUp does not consider Resource Types (Contributed by Szilard Nemeth via Daniel Templeton)
add fef2e2f HDDS-1176. Allow persisting X509CertImpl to SCM certificate table. Contributed by Xiaoyu Yao.
add fbc7bb3 YARN-9245. Added query docker image command ability to node manager. Contributed by Chandni Singh
add 5b43e42 HADOOP-16149 hadoop-mapreduce-client-app build not converging due to transient dependencies
add feccd28 HADOOP-16107. Update ChecksumFileSystem createFile/openFile API to generate checksum. Contributed by Steve Loughran
add 0ec962a YARN-5336 Limit the flow name size & consider cleanup for hex chars. Contributed by Sushil Ks
add ea3cdc6 YARN-3841 [atsv2 Storage implementation] Adding retry semantics to HDFS backing storage. Contributed by Abhishek Modi.
add 04b228e YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple. Contributed by Billie Rinaldi
add cbf82fa HDDS-1061. DelegationToken: Add certificate serial id to Ozone Delegation Token Identifier. Contributed by Ajay Kumar.
add 1779fc5 YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
add 7b928f1 YARN-9322. Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo (Contributed by Szilard Nemeth via Daniel Templeton)
add 538bb48 YARN-9323. FSLeafQueue#computeMaxAMResource does not override zero values for custom resources (Contributed by Szilard Nemeth via Daniel Templeton)
add 84c4966 HADOOP-16067. Incorrect Format Debug Statement KMSACLs. Contributed by Charan Hebri.
add 65f60e5 HADOOP-16068. ABFS Authentication and Delegation Token plugins to optionally be bound to specific URI of the store.
add 3a8118b HDDS-1190. Fix jdk 11 issue for ozonesecure base image and docker-compose . Contributed by Xiaoyu Yao.
add 0feba43 HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao.
add 05e4dde HDFS-14324. Fix TestDataNodeVolumeFailure. Contributed by Ayush Saxena.
add 0d61fac HADOOP-15889. Add hadoop.token configuration parameter to load tokens. Contributed by Íñigo Goiri
add eae8819 HDDS-1187. Healthy pipeline Chill Mode rule to consider only pipelines with replication factor three.
add eae3db9 HDDS-1191. Replace Ozone Rest client with S3 client in smoketests and docs. Contributed Elek, Marton.
add 77b23c8 HDDS-1182. Pipeline Rule where at least one datanode is reported in the pipeline.
add 7a0db2f HDFS-13997. Secondary NN Web UI displays nothing, and the console log shows moment is not defined. Contributed by Ayush Saxena
add 8e12259 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client.
add dcaca19 YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth.
add 80b77de HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao.
add de1dae6 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee.
add cab8529 YARN-7904. Privileged, trusted containers should be supported only in ENTRYPOINT mode. Contributed by Eric Yang.
add eea40cb HDDS-1204. Fix ClassNotFound issue with javax.xml.bind.DatatypeConverter used by DefaultProfile. Contributed by Xiaoyu Yao.
add 8b72aea Merge pull request #542 from xiaoyuyao/HDDS-1204
add 4a3cdde HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar.
add 6fef6fc HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao.
add 78c8a33 Fix checkstyle issue
add 9e53088 Revert "HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contr…" (#544)
add fc17ba1 HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen.
add b18c1c2 Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client."
add e20b5ef YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin.
add 6c4d566 Revert "HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng."
add bd8d299 YARN-7477. Moving logging APIs over to slf4j in hadoop-yarn-common. Contributed by Prabhu Joseph.
add 18ea0c1 HDFS-14182. Datanode usage histogram is clicked to show ip list. Contributed by fengchuang.
add 15098df Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
add eed8b79 HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao. (#545)
add 387dbe5 HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star.
add 10b802b HDDS-1136 : Add metric counters to capture the RocksDB checkpointing statistics. Contributed by Aravindan Vijayan.
add cb0fa0c HDFS-14321. Fix -Xcheck:jni issues in libhdfs, run ctest with -Xcheck:jni enabled. Contributed by Sahil Takiar.
add 90c37ac HDDS-623. On SCM UI, Node Manager info is empty (#523)
add 9fcd89a HADOOP-16148. Cleanup LineReader Unit Test.
add fe7551f HADOOP-16162. Remove unused Job Summary Appender configurations from log4j.properties
add e40e2d6 YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
add 4b7313e HDFS-14336. Fix checkstyle for NameNodeMXBean. Contributed by Danny Becker.
add 0aefe28 YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
add f940ab2 HDFS-7663. Erasure Coding: Append on striped file. Contributed by Ayush Saxena.
add 05df151 HADOOP-16163. NPE in setup/teardown of ITestAbfsDelegationTokens.
add 686c014 HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately.
add 2576401 YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
add 353b90d HDDS-1156. testDelegationToken is failing in TestSecureOzoneCluster. Contributed by Ajay Kumar.
add 5fcea54 HDDS-935. Avoid creating an already created container on a datanode in case of disk removal followed by datanode restart. Contributed by Shashikant Banerjee.
add 7b42e0e YARN-7266. Fixed deadlock in Timeline Server thread initialization. Contributed by Prabhu Joseph
add 12402b7 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client.
add 02da06d HDDS-1222. Remove TestContainerSQLCli unit test stub. Contributed by Elek, Marton.
add 549d54f Merge pull request #556 from elek/HDDS-1222
add 296259e HDDS-1219. TestContainerActionsHandler.testCloseContainerAction has an intermittent failure. Contributed by Elek, Marton.
add df99137 Merge pull request #555 from elek/HDDS-1219
add 3ef1235 HDDS-1218. Do the dist-layout-stitching for Ozone after the test-compile phase. Contributed by Marton Elek.
add 3c5b713 HADOOP-16150. Added concat method to ChecksumFS as unsupported operation. Contributed by Steve Loughran
add 7fd8901 HDDS-1171. Add benchmark for OM and OM client in Genesis. Contributed by Lokesh Jain.
add 313e8b9 HDDS-1193. Refactor ContainerChillModeRule and DatanodeChillMode rule. (#534)
add 7f636b4 HDDS-919. Enable prometheus endpoints for Ozone datanodes (#502)
add 1ad5bfc HDDS-1188. Implement a skeleton patch for Recon server with initial set of interfaces. Contributed by Siddharth Wagle.
add 945b504 HDFS-14326. Add CorruptFilesCount to JMX. Contributed by Danny Becker.
add 62e89dc HDDS-1184. Parallelization of write chunks in datanodes is broken. Contributed by Shashikant Banerjee.
add 46045c5 YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth.
add c79f139 YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB.
add 129fd5d HDDS-1208. ContainerStateMachine should set chunk data as state machine data for ratis. Contributed by Lokesh Jain.
add 9d87247 HDDS-1216. Change name of ozoneManager service in docker compose files to om.
add 2c3ec37 HDDS-1210. Ratis pipeline creation doesn't check raft client reply status during initialization. Contributed by Mukul Kumar Singh.
add 45f976f HDDS-1113. Remove default dependencies from hadoop-ozone project. Contributed by Elek, Marton.
add 01ada40 YARN-9348. Application catalog build system bug fixes. Contributed by Eric Yang
add f5a4b43 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
add 6192c1f Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
add 618e009 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
add a55fc36 HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values.
add bb12e81 HDDS-1175. Serve read requests directly from RocksDB. (#557)
add 09a9938 SUBMARINE-41. Fix ASF warnings related to submarine target files. Contributed by Dinesh Chitlangia. This closes #559.
add 491313a YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
add 0eba407 HDDS-1225. Provide docker-compose for OM HA.
add 2b94e51 HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host.
add 475011b HDDS-1226. ozone-filesystem jar missing in hadoop classpath
add 1bc282e HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam.
add 39b4a37 YARN-9341. Fixed enentrant lock usage in YARN project. Contributed by Prabhu Joseph
add 064f38b HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547)
add 373705f YARN-9239. Document docker registry deployment with Ozone CSI driver. Contributed by Eric Yang.
add fb851c9 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan.
add de15a66 YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko.
add e026041 HADOOP-16169. ABFS: Bug fix for getPathProperties.
add 341c076 HDDS-1196. Add a ReplicationStartTimer class. (#567)
add c072458 HDDS-1213. Support plain text S3 MPU initialization request.
add 0cbe9ad HADOOP-16109. Parquet reading S3AFileSystem causes EOF
add 8458ced HDDS-1238. Fix Ratis Snapshot creation error if the snapshot file already exists in ContainerStateMachine. Contributed by Shashikant Banerjee.
add 11db469 HDDS-1235. BaseHttpServer NPE is HTTP policy is HTTPS_ONLY. Contributed by Xiaoyu Yao.
add bb8ad09 HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore (Contributed by Shweta Yakkali via Daniel Templeton)
add ddede7a HADOOP-16174. Disable wildfly logs to the console.
add 73e1801 HDDS-1242. In S3 when bucket already exists, it should just return location.
add 68aa2cc HDDS-1240. Fix check style issues caused by HDDS-1196.
add f060514 MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang.
add ebb5fa1 HDDS-1220. KeyManager#openKey should release the bucket lock before doing an allocateBlock. Contributed by Lokesh Jain.
add 4fc493e HDDS-1236. Fix incorrect Ozone ClientProtocol KerberosInfo annotation. Contributed by Xiaoyu Yao. (#573)
add b4aa24d HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee.
add fa7a0b2 YARN-8805. Automatically convert the launch command to the exec form when using entrypoint support
add 4ad295a HDFS-14270.[SBN Read] Add trace level logging for stateId in RPC Server. Contributed by Shweta Yakkali.
add 599e926 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar.
add 2a54fea HDFS-14347. [SBN Read] Restore a comment line mistakenly removed in ProtobufRpcEngine. Contributed by Fengnan Li.
add c730786 HDDS-1245. OM delegation expiration time should use Time.now instead … (#587)
add d17e31e HDDS-1214. Enable tracing for the datanode read/write path. Contributed by Elek, Marton.
add daf4660 HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh.
add c24af4b YARN-9336. JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController. Contributed by Tarun Parimi.
add 2778259 HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593)
add aab7b77 YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
add 34b1406 HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell.
add dcb0de8 HDDS-1043. Enable token based authentication for S3 api
add c449cde HDDS-807. Period should be an invalid character in bucket names. Contributed by Siddharth Wagle.
add 024b3ba Revert "YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang"
add 55069bf Revert "YARN-9348. Application catalog build system bug fixes."
add ab574ff Revert "YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang"
add 24793d2 HDDS-1163. Basic framework for Ozone Data Scrubber. Contributed by Supratim Deka.
add 8e1539e YARN-9266. General improvements in IntelFpgaOpenclPlugin. Contributed by Peter Bacsko.
add 67cc24a YARN-9381. The yarn-default.xml has two identical property named yarn.timeline-service.http-cross-origin.enabled. Contributed by Abhishek Modi.
add 17a3e14 YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi.
add 159a715 MAPREDUCE-7192. JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable. Contributed by Jiandan Yang.
add d602d46 HDDS-1256. Fix failure in TestKeyValueContainerCheck. Contributed by Supratim Deka.
add 0d62753 HDDS-1128. Create stateful manager class for the pipeline creation scheduling.
add c1649bf HDDS-1226. Addendum. ozone-filesystem jar missing in hadoop classpath
add 1f47fb7 HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline.
add f74159c HADOOP-16166. TestRawLocalFileSystemContract fails with build Docker container running on Mac. Also provided similar fix for Windows.
add 86d508c HDDS-1087. Fix TestDefaultCertificateClient#testSignDataStream. Contributed by Xiaoyu Yao. (#596)
add 4fa0099 HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598)
add 6635757 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.
add 6fa2298 HADOOP-15625. S3A input stream to use etags/version number to detect changed source files.
add f235a94 YARN-9016 DocumentStore as a backend for ATSv2. Contributed by Sushil Ks.
add ea9fb07 HDDS-1241. Update ozone to latest ratis snapshot build (0.4.0-5680cf5-SNAPSHOT). Contributed by Mukul Kumar Singh.
add 3d140a5 HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton.
add 983b78a HDDS-1237. Fix test TestSecureContainerServer.testClientServerRatisGrpc. Contributed by Mukul Kumar Singh.
add d60673c HDDS-1257. Incorrect object because of mismatch in block lengths. Contributed by Shashikant Banerjee.
add 688b177 YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
add 091a664 HDDS-917. Expose NodeManagerMXBean as a MetricsSource. Contributed by Siddharth Wagle.
add 2627dad HDDS-1265. ozone sh s3 getsecret throws Null Pointer Exception for unsecured clusters.
add 9001508 HDDS-761. Create S3 subcommand to run S3 related operations.
add ba50a36 HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He.
add 155ab6d HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee.
add a7f5e74 HDDS-1088. Add blockade Tests to test Replica Manager. Contributed by Nilotpal Nandi.
add dc21655 HDDS-1259. OzoneFS classpath separation is broken by the token validation. Contributed by Elek Marton.
add d1afa03 HDDS-1138. Ozone Client should avoid talking to SCM directly. Contributed by Xiaoyu Yao and Mukul Kumar Singh.
add ff06ef0 HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun.
add 16b7862 HDDS-1283. Fix the dynamic documentation of basic s3 client usage. Contributed by Elek, Marton. (#605)
add 44b8451 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)
add 03f3c8a YARN-4404. Corrected typo in javadoc. Contributed by Yesha Vora
add af2dfc9 HDDS-1263. SCM CLI does not list container with id 1.
add 5cfb88a HDDS-1246. Add ozone delegation token utility subcmd for Ozone CLI. Contributed by Xiaoyu Yao. (#594)
add 2064ca0 YARN-9349. Changed logging to use slf4j api. Contributed by Prabhu Joseph
add 926d548 HDDS-1281. Fix the findbug issue caused by HDDS-1163. Contributed by Aravindan Vijayan.
add 4d2a116 HDDS-699. Detect Ozone Network topology. Contributed by Sammi Chen.
add cb4d911 MAPREDUCE-7188. [Clean-up] Remove NULL check before instanceof and fix checkstyle issue in TaskResult (Contributed by Shweta Yakkali via Daniel Templeton)
add 1edf191 HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file.
add 0e7e901 YARN-9340. [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey (Contributed by Shweta Yakkali via Daniel Templeton)
add 2db38ab HDFS-14328. [Clean-up] Remove NULL check before instanceof in TestGSet (Contributed by Shweta Yakkali via Daniel Templeton)
add 8c95cb9 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed.
add 66a104b HDDS-1296. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#616)
add 5446e3c HADOOP-16167. Fixed Hadoop shell script for Ubuntu 18. Contributed by Daniel Templeton
add 19b22c4 YARN-9385. Fixed ApiServiceClient to use current UGI. Contributed by Eric Yang
add 5f6e225 YARN-9363. Replaced debug logging with SLF4J parameterized log message. Contributed by Prabhu Joseph
add c0427c8 HADOOP-16124. Extend documentation in testing.md about S3 endpoint constants.
add 93db5da HDDS-1250. In OM HA AllocateBlock call where connecting to SCM from OM should not happen on Ratis.
add ae3a2c3 Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)"
add 09eabda YARN-9364. Remove commons-logging dependency from YARN. Contributed by Prabhu Joseph
add 568d3ab HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu.
add f10d493 HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601)
add c9e50c4 YARN-9365. Fix table creation command in TimelineServiceV2.md
add 55b3a71 HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen.
add 992489c HDDS-1215. Change hadoop-runner and apache/hadoop base image to use Java8. Contributed by Xiaoyu Yao.
add 5d8bd0e YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
add 1639071 HDDS-1307. Test ScmChillMode testChillModeOperations failed. (#622)
add 310ebf5 HDDS-1308. Fix asf license errors. (#623)
add 072750c HDDS-1306. TestContainerStateManagerIntegration fails in Ratis shutdown. Contributed by Lokesh Jain.
add c1a4eeb YARN-9389. FlowActivity and FlowRun table prefix is wrong. Contributed by Prabhu Joseph.
add f6e60e9 YARN-9387. Update document for ATS HBase Custom tablenames (-entityTableName). Contributed by Prabhu Joseph.
add 0d24684 YARN-9357. Modify HBase Liveness monitor log to debug. Contributed by Prabhu Joseph.
add b3b0e33 YARN-9299. TestTimelineReaderWhitelistAuthorizationFilter ignores Http Errors. Contributed by Prabhu Joseph.
add 399563f HDFS-14176. Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.
add f2b862c YARN-9398. Fixed javadoc errors for FPGA related java files. Contributed by Peter Bacsko
add 506502b YARN-9370. Added logging for recovering assigned GPU devices. Contributed by Yesha Vora
add 60cdd4c HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan.
add 9f1c017 HADOOP-16058. S3A tests to include Terasort.
add a99eb80 YARN-9267. General improvements in FpgaResourceHandlerImpl. Contributed by Peter Bacsko.
add 548997d YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
add 2828f8c HDDS-1321. TestOzoneManagerHttpServer depends on hard-coded port numbers. Contributed by Arpit Agarwal. (#633)
add 90afc9a HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh.
add 246ab77 HADOOP-16196. Path Parameterize Comparable.
add d18d085 HADOOP-16181. HadoopExecutors shutdown Cleanup.
add faba359 HADOOP-16147. Allow CopyListing sequence file keys and values to be more easily customized.
add ce5eb9c YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322 (Contributed by Zoltan Siegl via Daniel Templeton)
add ae2eb2d HADOOP-16201: S3AFileSystem#innerMkdirs builds needless lists (#636)
add 1d389ec HDDS-1323. Ignore unit test TestFailureHandlingByClient. Contributed by Supratim Deka.
add 509b20b YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
add f854a89 HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED containers. Contributed by Nanda kumar. (#620)
add dea6f2a HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading from stale pipeline. Contributed by Xiaoyu Yao. (#639)
add 73f7b04 HDDS-1302. Fix SCM CLI does not list container with id 1.
add efad571 HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri.
add 771ea6b Revert "HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri."
add 43e421a HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth Wagle.
add 6a34c9b HDDS-1299. Support TokenIssuer interface for running jobs with OzoneFileSystem. Contributed by Xiaoyu Yao.
add 128dd91 HDDS-1331. In DatanodeStateMachine join check for not null.
add d4e4a7d HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee.
add 67dd45f HDDS-1185. Optimize GetFileStatus in OzoneFileSystem by reducing the number of rpc call to OM. Contributed by Mukul Kumar Singh.
add e5d72f5 HDDS-1234. Iterate the OM DB snapshot and populate the recon container DB. Contributed by Aravindan Vijayan.
add 3c45762 YARN-9391. Fixed node manager environment leaks into Docker containers. Contributed by Jim Brennan
add 8739693 HDDS-1217. Refactor ChillMode rules and chillmode manager. (#558)
add eeda689 YARN-9268. General improvements in FpgaDevice. Contributed by Peter Bacsko.
add 710cbc9 YARN-9405. Fixed flaky tests in TestYarnNativeServices. Contributed by Prabhu Joseph
add 3f6d6d2 HDFS-14359. Inherited ACL permissions masked when parent directory does not exist (mkdir -p) (Contributed by Stephen O'Donnell via Daniel Templeton)
add c99b107 HDDS-1334. Fix asf license errors in newly added files by HDDS-1234. Contributed by Aravindan Vijayan.
add 5257f50 YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
add 5c0a81a HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails.
add 82d4772 HDDS-939. Add S3 access check to Ozone manager. Contributed by Ajay Kumar. (#634)
add 55fb3c3 HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory.
add ce4bafd HADOOP-16037. DistCp: Document usage of Sync (-diff option) in detail.
add fe29b39 HDFS-14348: Fix JNI exception handling issues in libhdfs
add f426b7c HDDS-139. Output of createVolume can be improved. Contributed by Shweta.
add 18c57cf HDFS-14304: High lock contention on hdfsHashMutex in libhdfs
add eef8cae HDDS-1262. In OM HA OpenKey call Should happen only leader OM. (#626)
add b226958 HDDS-1264. Remove Parametrized in TestOzoneShell. (#614)
add b4ed81c HDDS-1332. Add some logging for flaky test testStartStopDatanodeStateMachine. Contributed by Arpit Agarwal. (#649)
add a4cd75e YARN-9269. Minor cleanup in FpgaResourceAllocator. Contributed by Peter Bacsko.
add 9cd6619 YARN-9411. TestYarnNativeServices fails sporadically with bind address in use. Contributed by Prabhu Joseph.
add 8a59efe HDDS-1346. Remove hard-coded version ozone-0.5.0 from ReadMe of ozonesecure-mr docker-compose. Contributed by Xiaoyu Yao.
add 15d38b1 HDFS-14295. Add Threadpool for DataTransfers. Contributed by David Mollitor.
add cfb0186 HADOOP-16186. S3Guard: NPE in DynamoDBMetadataStore.lambda$listChildren.
add b5db238 HADOOP-15999. S3Guard: Better support for out-of-band operations.
add df578c0 HADOOP-16195 MarshalledCredentials toString
add 49b02d4 HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor.
add f3f5128 HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R
add ca5e4ce HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao.
add ac4010b HDDS-1293. ExcludeList#getProtoBuf throws ArrayIndexOutOfBoundsException. Contributed by Shashikant Banerjee.
add 4cceeb2 HDDS-1309 . change logging from warn to debug in XceiverClient. Contributed by Nilotpal Nandi. (#638)
add d7a2f94 HDDS-1350. Fix checkstyle issue in TestDatanodeStateMachine. Contributed by Xiaoyu Yao. (#656)
add f41f938 HADOOP-16199. KMSLoadBlanceClientProvider does not select token correctly. Contributed by Xiaoyu Yao.
add 82769ba HDDS-1352. Remove unused call in TestStorageContainerManagerHttpServer.
add 6186ed9 HDDS-1300. Optimize non-recursive ozone filesystem apis. Contributed by Lokesh Jain.
add 7dc0ecc [HDDS-1351] NoClassDefFoundError when running ozone genconf (). Contributed by Doroszlai Attila.
add 56f1e13 YARN-9270. Minor cleanup in TestFpgaDiscoverer. Contributed by Peter Bacsko.
add 7ee32a2 YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
add d9e9e56 YARN-9348. Application catalog build system bug fixes. Contributed by Eric Yang
add ec82e4c HDDS-1357. ozone s3 shell command has confusing subcommands (#663)
add bf3b7fd HDDS-1288. SCM - Failing test on trunk that waits for HB report processing. Contributed by Siddharth Wagle.
add 35ff31d HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He.
add 53a86e2 HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project
add 509f31b HDDS-1295. Add MiniOzoneChaosCluster to mimic long running workload in a unit test environment. Contributed by Mukul Kumar Singh.
add ef5de29 HDDS-1312. Add more unit tests to verify BlockOutputStream functionalities. Contributed by Shashikant Banerjee.
add 8bfef21 HDDS-1360. Invalid metric type due to fully qualified class name. Contributed by Doroszlai, Attila.
add 0df9492 HDDS-1067 . freon run on client gets hung when two of the datanodes are down in 3 datanode cluster. Contributed by Nilotpal Nandi.
add 5f951ea HDDS-1255. Refactor ozone acceptance test to allow run in secure mode
add 3675670 HDDS-1322. Hugo errors when building Ozone (#671)
add 35b0a38 YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
add 04f1db8 HDDS-1260. Create Recon Server lifecyle integration with Ozone. (#643)
add 856cbf6 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal.
add b0d24ef YARN-9227. DistributedShell RelativePath is not removed at end. Contributed by Prabhu Joseph.
add 332cab5 YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
add da7f8c2 YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
add ab2bda5 YARN-9428. Add metrics for paused containers in NodeManager. Contributed by Abhishek Modi.
add ebd0d21 HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji.
add 2f75283 YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
add 61d1911 HADOOP-16218. Findbugs warning of null param to non-nullable method in Configuration with Guava update. (#655)
add d31c868 HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee.
add aeb43df HDDS-1153. Make tracing instrumentation configurable
add bfc90bd HDDS-1355. Only FQDN is accepted for OM rpc address in secure environment. Contributed by Ajay Kumar.
add cf26811 HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain.
add aaaf856 HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str
add f96fb05 HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka.
add 8b6deeb HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji.
add 002dcc4 YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
add 7cd7045 HDDS-1164. Add New blockade Tests to test Replica Manager. Contributed by Nilotpal Nandi.
add 3b0c501 HDFS-14394: Add -std=c99 / -std=gnu99 to libhdfs compile flags
add be488b6 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang.
add d6c233f HDDS-1377. OM failed to start with incorrect hostname set as ip address in CSR. (#683)
add d797907 HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
add 59022b2 HDDS-1330 : Add a docker compose for Ozone deployment with Recon. (#669)
add 366186d HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685)
add e62cbcb HADOOP-16011. OsSecureRandom very slow compared to other SecureRandom implementations. Contributed by Siyao Meng.
add 8ff41d6 HDDS-1358 : Recon Server REST API not working as expected. (#668)
add 358f7f9 HDDS-1211. Test SCMChillMode failing randomly in Jenkins run (#543)
add 4e64f8d HDDS-1324. TestOzoneManagerHA tests are flaky (#676)
add 7b5b783 HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li.
add f09a78f HDDS-1339. Implement ratis snapshots on OM (#651)
add eb03f7c YARN-9303 Username splits won't help timelineservice.app_flow table. Contributed by Prabhu Joseph.
add a92806d HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle.
add 945e8c6 YARN-9394. Use new API of RackResolver to get better performance. Contributed by Lantao Jin.
add 48a58bc HDDS-1207. Refactor Container Report Processing logic and plugin new Replication Manager. (#662)
add 33e159b HDDS-1349. Remove watchClient from XceiverClientRatis. Contributed by Shashikant Banerjee.
add d2637cb HDFS-14389. getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured. Contributed by Stephen O'Donnell.
add 993f36e HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681)
add c907363 HADOOP-16208. Do Not Log InterruptedException in Client.
add 8d15006 YARN-9396. Fixed duplicated RM Container created event to ATS. Contributed by Prabhu Joseph
add 2bf82ae YARN-9441. Updated YARN app catalog name for consistency. Contributed by Weiwei Yang
add 215ffc7 HADOOP-16197 S3AUtils.translateException to map CredentialInitializationException to AccessDeniedException
add d2141ef HDDS-1189. Build failing due to rat check failure introduced by HDDS-1189. Contributed by Siddharth Wagle.
add 67020f0 HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji.
add 77fe51e HDDS-1379. Convert all OM Volume related operations to HA model. (#689)
add 260d843 HDFS-14416. Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN. Contributed by Fengnan Li.
add 0356b36 HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653)
add 89f9c86 HDDS-1385. Make the ozonesecure-mr environment definition version independent. Contributed by Elek, Marton. (#695)
add 5750bb9 HDDS-1389. Fix testSCMChillModeRestrictedOp. (#696)
add 27039a2 YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi.
add 22362c8 YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi.
add 26dc940 HDDS-1390. Fix OzoneS3 Gateway server due to exclusion of hk2-api. (#698)
add 274ab4e HDDS-1332. Attempt to fix flaky test testStartStopDatanodeStateMachine (#697)
add dc013f7 HDDS-1329. Update documentation for Ozone-0.4.0 release. Contributed By Ajay Kumar.
add e9b859f HDFS-14371. Improve Logging in FSNamesystem by adding parameterized logging. Contributed by Shweta.
add 1b9ba0e SUBMARINE-47. Provide an implementation to parse configuration values from a YAML file for submarine run CLI. Contributed by Szilard Nemeth.
add ec143cb YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
add 0d47d28 HADOOP-10848. Cleanup calling of sun.security.krb5.Config.
add 72f4b9c HADOOP-15242. Fix typos in hadoop-functions.sh. Contributed by Ray Chiang.
add fc05b0e YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang.
add ab645b3 HADOOP-14635. Javadoc correction for AccessControlList#buildACL. Contributed by Yeliang Cang.
add bb8dda2 HADOOP-12890. Fix typo in AbstractService. Contributed by Gabor Liptak.
add 69e3745 HDDS-1340. Add List Containers API for Recon (#648)
add 2d4f6b6 HDDS-372. There are three buffer copies in BlockOutputStream. Contributed by Shashikant Banerjee.
add 73f43ac HDDS-1372. getContainerWithPipeline for a standalone pipeline fails with ConcurrentModificationException. (#682)
add 8ef3bc8 HDDS-1383. Create project skeleton with CLI interface for In-place upgrade.
add a78bdd1 HDDS-1405. ITestOzoneContractCreate is failing. (#712)
add cfec455 YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri.
add 312d4d4 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709)
add 358e928 YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Addendum. Contributed by Inigo Goiri.
add 25c421b HDDS-1393. Convert all OM Bucket related operations to HA model. (#704)
add 32722d2 YARN-9433. Remove unused constants in YARN resource manager
add df01469 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708)
add 8c1bba3 YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
add d8239f3 HADOOP-16240. start-build-env.sh can consume all disk space during image creation. Contributed by Craig Condit.
add 65deb1a HADOOP-16179. hadoop-common pom should not depend on kerb-simplekdc
add dfb518b HDDS-1397. Avoid the usage of signal handlers in datanodes of the MiniOzoneClusters.
add 0e770a6 HDDS-1370. Command Execution in Datanode fails because of NPE (#715)
add 813cee1 HDFS-14420. Fix typo in KeyShell console. Contributed by Hu Xiaodong.
add feaab24 HDDS-1418. Move bang line to the start of the start-chaos.sh script. Contributed by Arpit Agarwal. (#720)
add 8740755 HADOOP-16243. Change Log Level to trace in NetUtils.java. Contributed by chencan.
add e9c4109 HDDS-1417. After successfully importing a container, datanode should delete the container tar.gz file from working directory. (#721)
add 586826f HDFS-12245. Fix INodeId javadoc
add bdbca0e HADOOP-16052. Remove Subversion and Forrest from Dockerfile
add a0468c5 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee.
add ed3747c YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi.
add fcc7f9b HDDS-1410. TestSCMNodeMetrics is flaky. Contributed by Siddharth Wagle.
add cf4efca HADOOP-16118. S3Guard to support on-demand DDB tables.
add bbdbc7a HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki.
add c298415 HDDS-1404. Fix typos in HDDS. Contributed by bianqi.
add 4003849 HDDS-1416. MiniOzoneCluster should set custom value for hdds.datanode.replication.work.dir. Contributed by chencan.
add 3b08ac4 HDDS-1363. ozone.metadata.dirs doesn't pick multiple dirs (#691)
add fd676e1 HDDS-1426. Minor logging improvements for MiniOzoneChaosCluster
add a9b46c5 HDDS-1421. Avoid unnecessary object allocations in TracingUtil
add 75ba719 HDDS-1419. Fix shellcheck errors in start-chaos.sh
add abace70 HDDS-1420. Tracing exception in DataNode HddsDispatcher
add a458c5b HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612)
add 0c1fec3 HDDS-1425. Ozone compose files are not compatible with the latest docker-compose. (#727)
add 62f4808 HDDS-1428. Remove benign warning in handleCreateContainer. Contributed by Siddharth Wagle.
add 5379d85 HDDS-1387. ConcurrentModificationException in TestMiniChaosOzoneCluster. Contributed by Marton Elek. (#732)
add 626fec6 HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang.
add 2382f63 HADOOP-14747. S3AInputStream to implement CanUnbuffer.
add 1943db5 HADOOP-16237. Fix new findbugs issues after updating guava to 27.0-jre.
add ebbda18 HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737)
add b2cdf80 YARN-9281. Add express upgrade button to Appcatalog UI. Contributed by Eric Yang
add 4b4fef2 HADOOP-16227. Upgrade checkstyle to 8.19
add 7a68e7a YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi.
add c4c16ca YARN-9481. [JDK 11] Build fails due to hard-coded target version in hadoop-yarn-applications-catalog-webapp. Contributed by Kei Kori.
add 7fa73fa YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang.
add 254efc9 YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang
add 62e38ea HDDS-1396 : Recon start fails due to changes in Aggregate Schema definition. (#700)
add 5583e1b YARN-7848 Force removal of docker containers that do not get removed on first try. Contributed by Eric Yang
add b1120d2 HADOOP-16249. Make CallerContext LimitedPrivate scope to Public.
add c65218e YARN-8943. Upgrade JUnit from 4 to 5 in hadoop-yarn-api.
add a5ceed2 HADOOP-16253. Update AssertJ to 3.12.2.
add 008766c HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy.
add bfcb653 HDDS-1380. Add functonality to write from multiple clients in MiniOzoneChaosCluster. Contributed by Shashikant Banerjee.
add ad86588 YARN-8530. Add SPNEGO filter to application catalog. Contributed by Eric Yang
add 2364c7d YARN-9466. Fixed application catalog navigation bar height in Safari. Contributed by Eric Yang
add be6c801 HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena.
add b8086ae YARN-9123. Clean up and split testcases in TestNMWebServices for GPU support. Contributed by Szilard Nemeth.
add f2ab279 HDDS-1432. Ozone client list command truncates response without any indication. Contributed by Siddharth Wagle.
add 3ac3e50 HDDS-1374. ContainerStateMap cannot find container while allocating blocks. (#735)
add e543c3b HDDS-1376. Datanode exits while executing client command when scmId is null (#724)
add 9cf7401 YARN-9349. Improved log level practices for InvalidStateTransitionException. Contributed by Anuhan Torgonshar
add 0a87fd1 HDDS-1434. TestDatanodeStateMachine is flaky (#740)
add 04c0437 HDDS-1402. Remove unused ScmBlockLocationProtocol from ObjectStoreHandler (#707)
add 22d0468 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee
add 4ff4314 Merge pull request #729 from bshashikant/HDDS-1373
add e5a918c Revert "Merge pull request #729 from bshashikant/HDDS-1373"
add df2ae27f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)
add 082f1e0 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)"
add d608be6 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729)
add 75e94b5 HADOOP-15881. Remove JUnit from LICENSE.txt
add 13907d8 HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml
add 685cb83 HDFS-14433. Remove the extra empty space in the DataStreamer logging. Contributed by Yishuang Lu. (#747)
add 6e4399e YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng.
add aa4c744 YARN-9470. Fix order of actual and expected expression in assert statements
add 8aea5e4 HDDS-1433. Rename GetScmInfoRespsonseProto to GetScmInfoResponseProto due to typos. Contributed by Wanqiang Ji.
add b979fdd HDDS-1447. Fix CheckStyle warnings. Contributed by Wanqiang Ji.
add df76cdc YARN-6695. Fixed NPE in publishing appFinished events to ATSv2. Contributed by Prabhu Joseph
add ef97a20 YARN-8622. Fixed container-executor compilation on MacOSX. Contributed by Siyao Meng
add 518f47b HDDS-976: Parse network topology from yaml file. Contributed by Junjie Chen. (#661)
add aeadb94 YARN-9448. Fix Opportunistic Scheduling for node local allocations. Contributed by Abhishek Modi.
add 317fcba YARN-9254. Add support for storing application catalog data to HDFS. Contributed by Eric Yang
add adefd37 YARN-9495. Fix findbugs warnings in hadoop-yarn-server-resourcemanager module. Contributed by Tao Yang.
add 1ddb488 HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin.
add 96e3027 YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi.
add 5321235 HDFS-14445. TestTrySendErrorReportWhenNNThrowsIOException fails in trunk. Contributed by Ayush Saxena.
add 174b7d3 HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen.
add fb1c549 HDFS-14374. Expose total number of delegation tokens in AbstractDelegationTokenSecretManager. Contributed by CR Hota.
add a54c1e3 YARN-8551. Project setup for MaWo application. Contributed by Yesha Vora
add f4ab937 HADOOP-16026:Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.
add 1c8046d YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
add e79a9c1 SUBMARINE-40. Add TonY runtime to Submarine. Contributed by Keqiu Hu.
add 8a95ea6 YARN-9475. [YARN-9473] Create basic VE plugin. Contributed by Peter Bacsko.
add 7e1f8d3 HDDS-1368. Cleanup old ReplicationManager code from SCM.
add 59ded76 HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755)
add c504eee YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi.
add 4a0ba24 YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph.
add fec9bf4 YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph.
add 9d40062 YARN-9081. Update jackson from 1.9.13 to 2.x in hadoop-yarn-services-core.
add 3f2f418 YARN-9424. Change getDeclaredMethods to getMethods in FederationClientInterceptor#invokeConcurrent. Contributed by Shen Yinjie.
add 64f30da HDDS-1450. Fix nightly run failures after HDDS-976. Contributed by Xiaoyu Yao. (#757)
add 3f787cd HDDS-1192. Support -conf command line argument in GenericCli
add e95c055 HDDS-1457. Sequence of configuring Ozone File System instruction needs some work. Contributed by Eric Yang.
add e1c5ddf HADOOP-16252. Add prefix to dynamo tables in tests.
add a703dae HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
add afe6613 MAPREDUCE-7200. Remove stale eclipse templates. Contributed by Wanqiang Ji.
add 0b3d41b SUBMARINE-54. Add test coverage for YarnServiceJobSubmitter and make it ready for extension for PyTorch. Contributed by Szilard Nemeth.
add b5dcf64 SUBMARINE-44. Upgrade Docker image's tensorflow version to 1.13.1. Contributed by Zac Zhou.
add 79d3d35 YARN-9486. Docker container exited with failure does not get clean up correctly. Contributed by Eric Yang
add c35abcd HDDS-999. Make the DNS resolution in OzoneManager more resilient
add 556eafd HDDS-1460: Add the optmizations of HDDS-1300 to BasicOzoneFileSystem (#765)
add 3758270 HDDS-1403. KeyOutputStream writes fails after max retries while writing to a closed container (#753)
add 43b2a4b HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769)
add b434f55 HDDS-1471. Update ratis dependency to 0.3.0. Contributed by Ajay Kumar. (#777)
add 1cef194 HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream.
add dbce729 HDDS-1472. Add retry to kinit command in smoketests. Contributed by Ajay Kumar. (#778)
add a96d16a HDDS-1455. Inconsistent naming convention with Ozone Kerberos configuration. Contributed by Xiaoyu Yao. (#762)
add 5bca062 HDDS-1476. Fix logIfNeeded logic in EndPointStateMachine. (#779)
add 59816df HDDS-1462. Fix content and format of Ozone documentation. Contributed by Elek, Marton. (#767)
add 95790bb HDDS-1430. NPE if secure ozone if KMS uri is not defined. Contributed by Ajay Kumar. (#752)
add 4b4200f HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq.
add 382a962 Revert "HDDS-973. HDDS/Ozone fail to build on Windows."
add 24f218a SUBMARINE-64. Improve TonY runtime's document. Contributed by Keqiu Hu.
add 7fbaa7d YARN-9476. [YARN-9473] Create unit tests for VE plugin. Contributed by Peter Bacsko.
add 0af4011 HADOOP-16221. S3Guard: add option to fail operation on metadata write failure.
add dead9b4 HDDS-1384. TestBlockOutputStreamWithFailures is failing
add accb811 YARN-6929. Improved partition algorithm for yarn remote-app-log-dir. Contributed by Prabhu Joseph
add 4877f0a HDFS-3246: pRead equivalent for direct read path (#597)
add 7cb46f0 HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng.
add f682a17 HDDS-1483. Fix getMultipartKey javadoc. (#790)
add 2ab318b HDDS-1412. Provide example k8s deployment files as part of the release package
add f1673b0 HDDS-1478. Provide k8s resources files for prometheus and performance tests
add a2887f5 HDDS-1468. Inject configuration values to Java objects
add 3cb1d09 HDDS-1479. Update S3.md documentation
add e2f0f72 HDDS-1469. Generate default configuration fragments based on annotations
add 4605db3 HDDS-1482. Use strongly typed codec implementations for the S3Table
add 6a42745 HADOOP-16183. Use latest Yetus to support ozone specific build process
add 865c328 HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota.
add b094b94 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein.
add 7a3188d HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.
add 1df6799 HDDS-1473. DataNode ID file should be human readable. (#781)
add d6b7609 HDFS-14453. Improve Bad Sequence Number Error Message. Contributed by Shweta.
add f1875b2 HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena.
add 1d59cc4 HADOOP-16292. Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner. Contributed by Sherwood Zheng.
add f194540 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786)
add d331a2a YARN-9528. Federation RMs starting up at the same time can give duplicate application IDs. Contributed by Young Chen.
add e424392 HDFS-14438. Fix typo in OfflineEditsVisitorFactory. Contributed by bianqi.
add 69b903b HDFS-14372. NPE while DN is shutting down. Contributed by lujie.
add 1d70c8c HDDS-1464. Client should have different retry policies for different exceptions. (#785)
add 12b7059 YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang.
add fb7c1ca Revert "HDDS-1384. TestBlockOutputStreamWithFailures is failing"
add 597fa47 YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung)
add 93f2283 HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng.
add 49e1292 YARN-9524. Fixed TestAHSWebService and TestLogsCLI unit tests. Contributed by Prabhu Joseph
add 1a696cc HDDS-1424. Support multi-container robot test execution
add 8ecbf61 YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal.
add d525633 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)
add 01451a5 Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)"
add 7f0e2c6 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee
add eb9c890 HDDS-1475 : Fix OzoneContainer start method. (#788)
add c4be3ea HDDS-1489. Unnecessary log messages on console with Ozone shell. (#797)
add 713e8a2 HADOOP-16238. Add the possbility to set SO_REUSEADDR in IPC Server Listener. Contributed by Peter Bacsko.
add 66c2a4e HADOOP-16270. [JDK 11] Remove unintentional override of the version of Maven Dependency Plugin. Contributed by Xieming Li.
add c336af3 YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang.
add 3172f6c YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola.
add 96dc5ce HADOOP-16293. AuthenticationFilterInitializer doc has speudo instead of pseudo.
add 9b0aace HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He.
add 3418bbb HADOOP-16269. ABFS: add listFileStatus with StartFrom.
add 0c5fa2e YARN-9535. Fixed spell check for container-executor in Docker documentation. Contributed by Charan Hebri
add f257497 HADOOP-16299. [JDK 11] Build fails without specifying -Djavac.version=11
add 2595125 HDDS-1500. Allocate block failures in client should print exception trace. Contributed by Aravindan Vijayan (#801).
add 90add05 YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang.
add 18cca58 HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792)
add ec36126 YARN-9483. DistributedShell does not release container when failed to localize at launch. Contributed by Prabhu Joseph.
add 2d31ccc Revert "HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792)" (#808)
add 591ca69 HADOOP-16278. With S3A Filesystem, Long Running services End up Doing lot of GC and eventually die.
add 1b48100 YARN-9522. AppBlock ignores full qualified class name of PseudoAuthenticationHandler. Contributed by Prabhu Joseph.
add b832e17 HDDS-1474. "ozone.scm.datanode.id" config should take path for a dir and not a file (#809)
add 64c7f36 YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl.
add 36267b6 SUBMARINE-52. [SUBMARINE-14] Generate Service spec + launch script for single-node PyTorch learning job. Contributed by Szilard Nemeth.
add 7c2d7c0 HADOOP-16251. ABFS: add FSMainOperationsBaseTest
add 5f1a346 Revert "HADOOP-16251. ABFS: add FSMainOperationsBaseTest"
add ff27e8e HADOOP-16251. ABFS: add FSMainOperationsBaseTest. Re-commit to fix git metadata.
add 1a47c2b YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang.
add 5847e00 HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen.
add 29ff7fb YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji.
add 206e633 YARN-9453. Clean up code long if-else chain in ApplicationCLI#run. Contributed by Wanqiang Ji.
add 389e640 HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao.
add 02c9efc HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795)
add 6bcc1dc HDDS-1503. Reduce garbage generated by non-netty threads in datanode ratis server (#813)
add 7d831ec YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal.
add e2dfdae HDFS-14482: Crash when using libhdfs with bad classpath
add 2d8282b HADOOP-16306. AliyunOSS: Remove temporary files when upload small files to OSS. Contributed by wujinhu.
add 2de1e30 YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T.
add 570fa2d YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
add 9569015 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812)
add 55bd359 YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.
add 77170e7 HDFS-14390. Provide kerberos support for AliasMap service used by Provided storage. Contributed by Ashvin Agrawal
add d4c8858 HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy.
add de01422 SUBMARINE-56. Update documentation to describe single-node PyTorch integration. Contributed by Szilard Nemeth.
add e66ecc5 HDDS-1531. Disable the sync flag by default during chunk writes in Datanode (#820). Contributed by Shashikant Banerjee.
add 2713dcf HADOOP-16307. Intern User Name and Group Name in FileStatus.
add c15b3bc HADOOP-16294: Enable access to input options by DistCp subclasses.
add b7de735 HDDS-1522. Provide intellij runConfiguration for Ozone components. Contributed by Elek, Marton.
add b067f8a HADOOP-16050: s3a SSL connections should use OpenSSL
add 9248b7d HDDS-1284. Adjust default values of pipline recovery for more resilient service restart.
add 03ea8ea HDDS-1297. Fix IllegalArgumentException thrown with MiniOzoneCluster Initialization. Contributed by Yiqun Lin.
add fab5b80 YARN-9554. Fixed TimelineEntity DAO serialization handling. Contributed by Prabhu Joseph
add c183bd8 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle.
add 4cb3da6 HADOOP-16248. MutableQuantiles leak memory under heavy load.
add 3e5e5b0 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.
add 12c8161 YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi.
add 732133c HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725)
add 729ccb2 SUBMARINE-58. Submarine client needs to generate fat jar. Contributed by Zac Zhou.
add a36274d HADOOP-16085. S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite.
add 0d1d7c8 HDDS-1499. OzoneManager Cache. (#798)
add 24c53e0 YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak.
add f633002 HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799)
add 05db2a5 HDDS-1487. Bootstrap React framework for Recon UI (#831)
add 1cb2eb0 HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong.
add c1d7d68 HDDS-1461. Optimize listStatus api in OzoneFileSystem (#782)
add ff5691c HDDS-1442. add spark container to ozonesecure-mr compose files
add 5906268 HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors
add a85451c YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph.
add ef1cc72 HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828)
add 77c49f2 HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714)
add 9dff6ef HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles
add 67f9a7b MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event.
add 2fc6f85 HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825)
add a315913 HDDS-1517. AllocateBlock call fails with ContainerNotFoundException (#826). Contributed by Shashikant Banerjee.
add 9c61494 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834)
add 5565f2c MAPREDUCE-7198. mapreduce.task.timeout=0 configuration used to disable timeout doesn't work.
add 03aa70f SUBMARINE-83. Refine the documents of submarine targeting 0.2.0 release. Contributed by Zhankun Tang.
add a771e2a HADOOP-12948. Remove the defunct startKdc profile from hadoop-common. Contributed by Wei-Chiu Chuang.
add ea0b1d8 HADOOP-16287. Implement ProxyUserAuthenticationFilter for web protocol impersonation. Contributed by Prabhu Joseph
add 7b03072 YARN-9080. Added clean up of bucket directories. Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
add e2b7991 YARN-9469. Fix typo in YarnConfiguration.
add f96a2df HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.
add 20a4ec3 HDDS-700. Support rack awared node placement policy based on network topology. Contributed by Sammi Chen.
add 869a1ab HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka
add 64c3985 Fixed checkstyle issues.
add 456bb8a verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion
add 9da62f3 Merge branch 'trunk' into HDDS-1535
add 72bef0f fixed merge error. adapted to new signature of BlockUtils getDB
add ca93760 fixed checkstyle issue post merge
add b4b9120 HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka (#832)
add 6a0e7dd YARN-9482. DistributedShell job with localization fails in unsecure cluster. Contributed by Prabhu Joseph.
add 4b099b8 HDDS-1501 : Create a Recon task interface to update internal DB on updates from OM. (#819)
add 460ba7f YARN-9558. Fixed LogAggregation test cases. Contributed by Prabhu Joseph
add 93d7630 HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta.
add c9393dd Revert "HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong."
add 6d0e79c HDDS-1512. Implement DoubleBuffer in OzoneManager. (#810)
add c31b7b8 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko.
add 55e0c13 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.
add 37900c5 HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
add 9f056d9 YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
add 9f933e6 HADOOP-16323. https everywhere in Maven settings.
add a3745c5 HDDS-1588. Update Ozone dist image to use jdk11. Contributed by Eric Yang.
add f0e44b3 HDDS-1584. Fix TestFailureHandlingByClient tests. Contributed by Shashikant Banerjee (#845).
add 83549db HDDS-1509. TestBlockOutputStreamWithFailures#test2DatanodesFailure fails intermittently. Contributed by Shashikant Banerjee (#805).
add ec92ca6 HDDS-1598. Fix Ozone checkstyle issues on trunk. Contributed by Elek, Marton. (#854)
add b70d1be HDDS-1559. Fix TestReplicationManager. Contributed by Bharat Viswanadham. (#856)
add 72dd790 HDDS-1534. freon should return non-zero exit code on failure. Contributed by Nilotpal Nandi.
add 9078e28 YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
add 4a692bc YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang.
add 9f0d341 HDDS-1533. JVM exit on TestHddsDatanodeService
add 06fa4d2 HDDS-1565. Rename k8s-dev and k8s-dev-push profiles to docker and docker-push.
add d1ec1c5 HDDS-1580. Obtain Handler reference in ContainerScrubber (#842)
add d8b18e8 HDDS-1602. Fix TestContainerPersistence#testDeleteBlockTwice. (#858)
add fb0b39f HDDS-1536. testSCMSafeModeRestrictedOp is failing consistently. Contributed by Xiaoyu Yao. (#865)
add d78854b HDFS-14434. Ignore user.name query parameter in secure WebHDFS. Contributed by KWON BYUNGCHANG
add 79d14d0 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862)
add 0c73dba HADOOP-16332. Remove S3A dependency on http core.
add 7f2e87a HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866)
add 346c2b7 HDDS-1559. Include committedBytes to determine Out of Space in VolumeChoosingPolicy. Contributed by Supratim Deka (#841)
add afd8440 HADOOP-16331. Fix ASF License check in pom.xml
add 544876f YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
add 3c63551 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph.
add abf76ac YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
add 751f0df HDDS-1231. Add ChillMode metrics. Contributed by Bharat Viswanadham.
add 0ead209 HDDS-1542. Create Radix tree to support ozone prefix ACLs. Contributed by Xiaoyu Yao.
add 9ad7cad HDDS-1530. Freon support big files larger than 2GB and add --bufferSize and --validateWrites options. Contributed by Xudong Cao. (#830)
add c1caab4 HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena.
add d23e8e9 HADOOP-16334. Fix yetus-wrapper not working when HADOOP_YETUS_VERSION >= 0.9.0
add 12be6ff HDDS-1539. Implement addAcl,removeAcl,setAcl,getAcl for Volume. Contributed Ajay Kumar. (#847)
add 2b303e9 SUBMARINE-86. Bump TonY version to pick up a docker related fix & Update documentation. Contributed by Keqiu Hu.
add 18c1eeb HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833).
add 8c8cb2d HDDS-1502. Fix the compilation issue with commit 18c1eebc08f93055ffdef1812247b439c8404163.
add 1b041d4 HDDS-1497. Refactor blockade Tests. Contributed by Nilotpal Nandi.
add 0b907bc0 HDDS-1581. Atleast one of the metadata dir config property must be tagged as REQUIRED.
add 33419a9 HDDS-1568 : Add RocksDB metrics to OM. Contributed by Aravindan Vijayan
add baee715 HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)
add 30c6dd9 YARN-9452. Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures. Contributed by Prabhu Joseph.
add f1552f6 YARN-9553. Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines. Contributed by Prabhu Joseph.
add 6f5a36c HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta.
add 33c62f8 HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao.
add c965f7f HDDS-1551. Implement Bucket Write Requests to use Cache and DoubleBuffer. (#850)
add 9122b9b HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee
add 219e286 HDDS-1608. Support Ozone Prefix ACLs in OM metadata table. Contributed by Xiaoyu Yao. (#875)
add 35f1014 HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default.
add 7861a5e YARN-9033. ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled. Contributed by Zhankun Tang.
add e49162f YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
add 52128e3 YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B.
add 1ae062c HDDS-1382. Create customized CSI server for Ozone
add 4cb559e YARN-9027. Fixed LevelDBCacheTimelineStore initialization. Contributed by Prabhu Joseph
add c1d2d92 HDDS-1540. Implement addAcl,removeAcl,setAcl,getAcl for Bucket. Contributed by Ajay Kumar. (#874)
add 2210897 YARN-9592. Use Logger format in ContainersMonitorImpl. Contributed by Inigo Goiri.
add 08363db SUBMARINE-87. Add an apache rat check script for submarine. Contributed by Zhankun Tang.
add 2185249 Preparing for submarine-0.3.0 development
add 4530f45 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T.
add 2a97a37 Revert "HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)"
add 7f46dda YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB.
add 59719dc YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB.
add bd2590d YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
add f327112 HDDS-1558. IllegalArgumentException while processing container Reports.
add 21de9af HDDS-1625 : ConcurrentModificationException when SCM has containers of different owners. (#883)
add 277e9a8 Opening of rocksDB in datanode fails with "No locks available"
add 606061a YARN-9595. FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource(). Contributed by Peter Bacsko.
add 1fc359f HDDS-1607. Create smoketest for non-secure mapreduce example (#869)
add e140a45 HDDS-1629. Tar file creation can be optional for non-dist builds. Contributed by Elek, Marton. (#887)
add 5d5081e HDDS-1631. Fix auditparser smoketests (#892)
add 7991159 SUBMARINE-82. Fix english grammar mistakes in documentation. Contributed by Szilard Nemeth.
add 50909a7 HDDS-1641. Csi server fails because transitive Netty dependencies (#906)
add 827a847 HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer -follow-on patch.
add ea3b0a1 HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
add e4b4fec HDDS-1224. Restructure code to validate the response from server in the Read path (#806)
add 97607f3 HDDS-1510. Classpath files are deployed to the maven repository as pom/jar files. Contributed by Elek, Marton. (#900)
add 580b639 HDDS-1600. Add userName and IPAddress as part of OMRequest. (#857)
add 1a78794 HDDS-1624 : Refactor operations inside the bucket lock in OM key write. (#882)
add d45669c YARN-7537. Add ability to load hbase config from distributed file system. Contributed by Prabhu Joseph
add e7e30a5 MAPREDUCE-7210. Replace `mapreduce.job.counters.limit` with `mapreduce.job.counters.max` in mapred-default.xml
add 96d818a SUBMARINE-88. rat.sh regex pattern not working issue while using lower version of grep in ubuntu16.04. Contributed by Zhankun Tang.
add cd17cc2 SUBMARINE-89. Add submarine-src profile to generate source package. Contributed by Sunil Govindan.
add ef2da40 HDDS-1640. Reduce the size of recon jar file
add 433e97c YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang.
add 42cd861 HDDS-1628. Fix the execution and return code of smoketest executor shell script
add 7724d80 Revert "HADOOP-16321: ITestS3ASSL+TestOpenSSLSocketFactory failing with java.lang.UnsatisfiedLinkErrors"
add 309501c Revert "HADOOP-16050: s3a SSL connections should use OpenSSL"
add d1aad44 HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
add 0b1e288 HDDS-1637. Fix random test failure TestSCMContainerPlacementRackAware. Contributed by Sammi Chen. (#904)
add 3b1c257 HDDS-1541. Implement addAcl,removeAcl,setAcl,getAcl for Key. Contributed by Ajay Kumat. (#885)
add 294695d HADOOP-16314. Make sure all web end points are covered by the same authentication filter. Contributed by Prabhu Joseph
add 73954c1 HDDS-1612. Add 'scmcli printTopology' shell command to print datanode topology. Contributed by Sammi Chen.(#910)
add ec26c43 HADOOP-16117. Update AWS SDK to 1.11.563.
add 649666e YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal.
add f7c77b3 HDDS-1458. Create a maven profile to run fault injection tests. Contributed by Eric Yang.
add 829848b HADOOP-16344. Make DurationInfo public unstable.
add 9fded67 HDDS-1621. writeData in ChunkUtils should not use AsynchronousFileChannel. Contributed by Supratim Deka (#917)
add 0976392 YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
add 36757ad HDDS-1645. Change the version of Pico CLI to the latest 3.x release - 3.9.6
add 1f244b4 Revert "HADOOP-16344. Make DurationInfo public unstable."
add 0b8a260 HDDS-1605. Implement AuditLogging for OM HA Bucket write requests. (#867)
add 944adc6 HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao.
add c8276f3 HDDS-1201. Reporting Corruptions in Containers to SCM (#912)
add fe06957 HDDS-1647 : Recon config tag does not show up on Ozone UI. (#914)
add 76c0183 HDDS-1652. HddsDispatcher should not shutdown volumeSet. Contributed by Xiaoyu Yao. (#916)
add c7e6f07 HDDS-1650. Fix Ozone tests leaking volume checker thread. Contributed by Xiaoyu Yao. (#915)
add e1dfc06 HDFS-14486. The exception classes in some throw statements do not accurately describe why they are thrown. Contributed by Ayush Saxena.
add 8ca58ef HDDS-1490. Support configurable container placement policy through 'o… (#903)
add cb9bc6e YARN-9581. Fixed yarn logs cli to access RM2. Contributed by Prabhu Joseph
add 3ea4f41 MAPREDUCE-6794. Remove unused properties from TTConfig.java
add 0b115b6 HDDS-1371. OMSnapshotProvider to download DB checkpoint from leader OM. (#703)
add a91d24f HDDS-1496. Support partial chunk reads and checksum verification (#804)
add 14552d1 HADOOP-16344. Make DurationInfo public unstable.
add 8547957 HDDS-1622. Use picocli for StorageContainerManager
add 4e38daf HADOOP-15563. S3Guard to support creating on-demand DDB tables.
add 76b94c2 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen.
add 46b23c1 HDDS-1636. Tracing id is not propagated via async datanode grpc call (#895)
add 9deac3b HDDS-1657. Fix parallelStream usage in volume and key native acl. Contributed by Ajay Kumar. (#926)
add fcfe7a3 HDDS-1655. Redundant toString() call for metaDataPath in KeyValueContainerCheck (#932)
add d6d95d2 HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
add e94e643 YARN-9471. Cleanup in TestLogAggregationIndexFileController. Contributed by Adam Antal.
add 7217494 HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru.
add 0d160a0 HDFS-10210. Remove the defunct startKdc profile from hdfs. Contributed by Wei-Chiu Chuang, Akira Ajisaka.
add 9191e08 YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
add bd46bdf HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao.
add 101d5b5 HDFS-14234. Limit WebHDFS to specifc user, host, directory triples. Contributed by Clay B.
add b417a4c MAPREDUCE-7214. Remove unused pieces related to `mapreduce.job.userlog.retain.hours`
add b057479 HDDS-1662. Missing test resources of integrataion-test project in target directory after compile
add f7df55f YARN-9602. Use logger format in Container Executor. Contributed by Abhishek Modi.
add 6d80b9b YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie.
add f918e3f HDDS-1669. SCM startup is failing if network-topology-default.xml is part of a jar
add 60c95e9 YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
add 2263ead YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T.
add 3c9a5e7 HDDS-1635. Maintain docker entrypoint and envtoconf inside ozone project (#894)
add 96e1e41 HDDS-1627. Make the version of the used hadoop-runner configurable (#886)
add e997f2a HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)
add 5740eea HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.
add 4fecc2a HADOOP-16263. Update BUILDING.txt with macOS native build instructions. Contributed by Siyao Meng.
add 4ea6c2f HADOOP-16354. Enable AuthFilter as default for WebHDFS. Contributed by Prabhu Joseph
add 23c0379 Revert "HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)"
add 3b31694 HDDS-1545. Cli to add,remove,get and delete acls for Ozone objects. Contributed by Ajay Kumar. (#920)
add 50de087 HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell.
add cf84881 HADOOP-16365. Upgrade jackson-databind to 2.9.9. Contributed by Shweta Yakkali.
add 1732312 HDDS-1663. Add datanode to network topology cluster during node regis… (#937)
add 205dd2d HADOOP-16367. Fixed MiniYarnCluster AuthenticationFilter initialization. Contributed by Prabhu Joseph
add a43f444 HDDS-1543. Implement addAcl,removeAcl,setAcl,getAcl for Prefix. Contr… (#927)
add 88c53d5 HDDS-1620. Implement Volume Write Requests to use Cache and DoubleBuffer. (#884)
add 970b0b0 YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
add 940bcf0 HDDS-1587. Support dynamically adding delegated classes from to isolated class loader. Contributed by Junjie Chen. (#942)
add bcfd228 YARN-9599. TestContainerSchedulerQueuing#testQueueShedding fails intermittently. Contributed by Abhishek Modi.
add 89acbbc HDDS-1677. Auditparser robot test shold use a world writable working directory (#958)
add e094b3b HDDS-1680. Create missing parent directories during the creation of HddsVolume dirs (#961)
add 54f9f75 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.
add 4f45529 HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell.
add 9ebbda3 HADOOP-16372. Fix typo in DFSUtil getHttpPolicy method
add 3ba090f HADOOP-16366. Fixed ProxyUserAuthenticationFilterInitializer for timeline server. Contributed by Prabhu Joseph
add ae4143a HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
add cdc5de6 HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
add b24efa1 HDFS-14549. EditLogTailer shouldn't output full stack trace when interrupted. Contributed by Inigo Goiri.
add c7554ff HADOOP-16373. Fix typo in FileSystemShell#test documentation (#968)
add cda9f33 YARN-8499 ATSv2 Generalize TimelineStorageMonitor. Contributed by Prabhu Joseph
add 0766186 HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm.
add 8370a0a HDFS-14203. Refactor OIV Delimited output entry building mechanism. Contributed by Adam Antal.
add e70aeb4 HDDS-1601. Implement updating lastAppliedIndex after buffer flush to OM DB. (#972)
add f9cc9e1 HADOOP-16279. S3Guard: Implement time-based (TTL) expiry for entries (and tombstones).
add 1da09bd HADOOP-16376. ABFS: Override access() to no-op.
add cc16302 HDFS-14556: Spelling Mistake "gloablly" (#938)
add ba681bb HDDS-1692. RDBTable#iterator should disabled caching of the keys during iterator. (#975)
add 304a47e YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
add 22b36dd HDDS-1660. Use Picocli for Ozone Manager
add 3d020e9 HDDS-1532. Improve the concurrent testing framework of Freon. (#957)
add f9a7b44 HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai.
add a50c35b Revert "HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
add 7314185 Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
add 6822193 HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
add a95e87d HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.
add 1e92db5 HDFS-11949. Add testcase for ensuring that FsShell cann't move file to the target directory that file exists. Contributed by legend.
add 10311c3 HADOOP-14807. should prevent the possibility of NPE about ReconfigurableBase.java. Contributed by hu xiaodong.
add 62ad988 HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.
add 67414a1 YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.
add e317222 HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang.
add f1c239c HADOOP-9157. Better option for curl in hadoop-auth-examples. Contributed by Andras Bokor.
add 098c325 HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar.
add eb6be46 YARN-9630. [UI2] Add a link in docs's top page
add 56c621c HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal.
add d41310a HDFS-12315. Use Path instead of String to check closedFiles set. Contributed by Oleg Danilov.
add 54cdde3 HDFS-12314. Typo in the TestDataNodeHotSwapVolumes.testAddOneNewVolume(). Contributed by Oleg Danilov.
add dd4a763 HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali.
add e188bb1 HDDS-1694. TestNodeReportHandler is failing with NPE
add 335c1c9 HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti.
add fb1ce0d HDDS-1670. Add limit support to /api/containers and /api/containers/{id} endpoints (#954)
add 3ab77d9 HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.
add 3c1a1ce HDFS-14487. Missing Space in Client Error Message (Contributed by Shweta Yakkali via Daniel Templeton)
add b14f056 YARN-9574. Update hadoop-yarn-applications-mawo artifactId to match directory name. Contributed by Wanqiang Ji
add 71edfce HDDS-1699. Update RocksDB version to 6.0.1 (#980)
add 81ec909 HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton.
add 37bd5bb HDDS-1702. Optimize Ozone Recon build time (#982)
add 2dfa932 HDDS-1684. OM should create Ratis related dirs only if ratis is enabled (#965)
add f9ee97d HDFS-14398. Update HAState.java to fix typos. Contributed by Nikhil Navadiya.
add a1c3868 HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti.
add 48e564f HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.
add d3ac516 Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti."
add 9d68425 HDDS-1454. GC other system pause events can trigger pipeline destroy for all the nodes in the cluster. Contributed by Supratim Deka (#852)
add 450c070 HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations.
add f5ecc0b HDFS-14103. Review Logging of BlockPlacementPolicyDefault. Contributed by David Mollitor.
add 71ecd2e HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
add 5bfdf62 YARN-9631. Added ability to select JavaScript test or skip JavaScript tests for YARN application catalog. Contributed by Eric Yang
add 28291a9 HADOOP-16379: S3AInputStream.unbuffer should merge input stream stats into fs-wide stats
add e02eb24 HADOOP-15183. S3Guard store becomes inconsistent after partial failure of rename.
add 9c4b15d HDDS-1508. Provide example k8s deployment files for the new CSI server
add 5962a51 HDFS-14581. Appending to EC files crashes NameNode. Contributed by Surendra Singh Lilhore.
add d9a9e99 HDDS-1579. Create OMDoubleBuffer metrics. (#871)
add 840d02c HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang.
add 98d2065 HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma.
add 272b96d HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain.
add b95a58e HDDS-1674. Make ScmBlockLocationProtocol message type based.
add 83dcb9d YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
add 256bdd6 HDDS-1678. Default image name for kubernetes examples should be ozone and not hadoop
add cba13c7 HDDS-1715. Update the Intellij runner definitition of SCM to use the new class name
add 8194a11 HDDS-1690. ContainerController should provide a way to retrieve containers per volume (#986)
add 371452e HADOOP-15989. Synchronized at CompositeService#removeService is not required. Contributed by Prabhu Joseph.
add 1524e2e HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi.
add 645d67b HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu.
add b28ddb2 HDDS-1706. Replication Manager thread running too frequently. Contributed by Nilotpal Nandi.
add 0514540 HDFS-14339. Inconsistent log level practices in RpcProgramNfs3.java. Contributed by Anuhan Torgonshar.
add 81d2f1b HDDS-1719 : Increase ratis log segment size to 1MB. (#1005)
add b220ec6 YARN-9374. Improve Timeline service resilience when HBase is unavailable. Contributed by Prabhu Joseph and Szilard Nemeth
add 41c94a6 HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
add b3fee1d HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
add c5065bf HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
add 7b0bc49 HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
add 6f2c871 HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
add ebfd2d8 HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
add 04caaba HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
add fa55eac HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
add f4bd111 HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
add f2355c7 HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
add 19088e1 HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
add b320cae HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
add 6aa7aab HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
add 0ca7142 Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
add 94a8dec HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
add 01b4126 HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
add bbe8591 HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
add 3d97142 HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
add 8f6f9d9 HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
add 1dc01e5 HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
add f3cbf0e HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
add 4244653 HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
add b8bcbd0 HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
add f4e2bfc HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
add 221f24c HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
add f40e10b HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
add 7b61cbf HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
add c012b09 HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
add 235406d HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
add 020f83f HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
add 8b9b58b HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
add acdf911 HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
add 9eed3a4 HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
add 559cb11 HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.
add 9c4e556 HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
add 912b90f HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
add 7e63e37 HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
add 75f8b6c HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
add e2a3c44 HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
add 50aee18 HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
add 8b8ff5c HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
add de719b0 HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
add c4b1fa9 HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
add 1c7ab59 HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
add 7bbe01a HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
add 1ce25e7 HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
add 6cdf8db5 HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
add fcabc8f HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
add 2a2d5eb HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
add f539e2a HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
add 9a9fbbe HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
add 0dbd878 HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
add 6c42d40 HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
add dd8c2b9 HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.
add 0f9b8d7 HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka.
add de7da9b HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri.
add 021a43b HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota.
add b522b52 HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao.
add 203664e HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri.
add 3284117 HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li.
add 62fa53a HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena.
add 3deb5d3 HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena.
add 393f151 HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.
add 2636a54 HDFS-13995. RBF: Security documentation. Contributed by CR Hota.
add 8e42676 HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena.
add ffbb6b6 HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka.
add 9b197c2 HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena.
add d240eec HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma.
add b6fff8c HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri.
add b062dd4 HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota.
add ddbe08d HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota.
add 6915d7e HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena.
add ade8d3b HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma.
add ba221dc HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
add b52fd05 HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma.
add 8627f65 HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao.
add 719d57b HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena.
add 0042079 HDDS-1597. Remove hdds-server-scm dependency from ozone-common (#969)
add 95c94dc HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.
add d023f1f HDDS-1646. Support real persistence in the k8s example files (#945)
add 129576f HDFS-14403. Cost-based extension to the RPC Fair Call Queue. Contributed by Christopher Gregorian.
add daa1e14 when evictableMmapped or evictable size is zero, do not throw NoSuchElementException
add 38a560c Merge pull request #977 from leosunli/trunk
add b76b843 HDFS-13371. NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X. Contributed by Sherwood Zheng.
add 041e7a7 HDFS-14598. Findbugs warning caused by HDFS-12487. Contributed by He Xiaoqiao.
add 366f3de HADOOP-16390. escape javadoc in S3AUtils public methods
add 48d7f00 HDFS-14590. [SBN Read] Add the document link to the top page. Contributed by Takanobu Asanuma.
add ab0b180 HDFS-12345 Add Dynamometer to hadoop-tools, a tool for scale testing the HDFS NameNode with real metadata and workloads. Contributed by Erik Krogen.
add 049bb42 HDDS-1723. Create new OzoneManagerLock class. (#1006)
add 4b50981 HDDS-1709. TestScmSafeNode is flaky. Contributed by Elek, Marton.
add a79bdf7 HDDS-1727. Use generation of resourceName for locks in OzoneManagerLock. (#1014)
add 062eb60 YARN-9477. Implement VE discovery using libudev. Contributed by Peter Bacsko.
add 0a37ccf HDDS-1691 : RDBTable#isExist should use Rocksdb#keyMayExist (#1013)
add 1ac967a YARN-6055. ContainersMonitorImpl need be adjusted when NM resource changed. Contributed by Inigo Goiri.
add 4848280 HDDS-1638. Implement Key Write Requests to use Cache and DoubleBuffer. (#956)
add be80334 YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
add 041e0c0 HDFS-14036. RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default. Contributed by Takanobu Asanuma.
add 4a21224 HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao.
add cbae241 YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang.
add 5e7caf1 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023
add f09c31a Revert "YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023"
add 9fd3c70 HDDS-1639. Restructure documentation pages for better understanding
add f02b0e1 YARN-9581. Add support for get multiple RM webapp URLs. Contributed by Prabhu Joseph
add 49c5e8a HDDS-1672. Improve locking in OzoneManager. (#1016)
add 29465bf YARN-9560. Restructure DockerLinuxContainerRuntime to extend OCIContainerRuntime. Contributed by Eric Badger, Jim Brennan, Craig Condit
add da56899 HDDS-1733. Fix Ozone documentation (#1031)
add d203045 HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward.
add 1e727cf HDDS-1730. Implement File CreateDirectory Request to use Cache and Do… (#1026)
add 9df6275 HDDS-1555. Disable install snapshot for ContainerStateMachine. Contributed by Siddharth Wagle. (#846)
add 1f75660 HDDS-373. Genconf tool must generate ozone-site.xml with sample values (#1025)
add b1dafc3 YARN-9661:Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java (#1042)
add f8d62a9 HDDS-1258. Fix error propagation for SCM protocol
add db674a0 HDDS-1685. Recon: Add support for "start" query param to containers and containers/{id} endpoints.
add d8bac50 HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. (#1015)
add 2f4b37b HDDS-1667. Docker compose file may referring to incorrect docker image name. Contributed by Eric Yang.
add 570eee3 YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.
add 0c8813f YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt.
add e966edd YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
add 564758a HDDS-1698. Switch to use apache/ozone-runner in the compose/Dockerfile (#979)
add aa9f085 HADOOP-16385. Namenode crashes with 'RedundancyMonitor thread received Runtime exception'. Contributed by Ayush Saxena.
add 75b1e45 HDDS-1716. Smoketest results are generated with an internal user
add 8b0d1ad HDDS-1616. ManagedChannel references are being leaked in while removing RaftGroup. Contributed by Mukul Kumar Singh. (#1039)
add 91cc197 HDDS-1757. Use ExecutorService in OzoneManagerStateMachine. (#1048)
add 15d82fc YARN-9658. Fix UT failures in TestLeafQueue. Contributed by Tao Yang.
add a2a8be1 YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal.
add 34747c3 HADOOP-16396. Allow authoritative mode on a subdirectory. (#1043)
add 8965ddc HDDS-1731. Implement File CreateFile Request to use Cache and DoubleBuffer. (#1044)
add acd2d52 HDDS-1741. Fix prometheus configuration in ozoneperf example cluster
add 729cb3a HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
add 1c254a8 HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina.
add c3ca348 HDFS-14620. RBF: Fix 'not a super user' error when disabling a namespace in kerberos with superuser principal. Contributed by luhuachao.
add 928edb2 SUBMARINE-94. Support different docker images for
add c9c3429 HDDS-1758. Add replication and key deletion tests to MiniOzoneChaosCluster. Contributed by Mukul Kumar Singh. (#1049)
add 9c90729 HDFS-13694. Making md5 computing being in parallel with image loading.
add ce91d35 HDDS-1728. Add metrics for leader's latency in ContainerStateMachine. Contributed by Mukul Kumar Singh. (#1022)
add ec851e4 HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL.
add de6b7bc HADOOP-16409. Allow authoritative mode on non-qualified paths. Contributed by Sean Mackrory
add 4632708 HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.
add 738c093 HADOOP-16411. Fix javadoc warnings in hadoop-dynamometer.
add b5d30e4 HDDS-1750. Add block allocation metrics for pipelines in SCM. Contributed by Lokesh Jain. (#1047)
add e5ffb88 HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh. (#1050)
add 82d88a8 HDDS-1705. Recon: Add estimatedTotalCount to the response of containers and containers/{id} endpoints. Contributed by Vivek Ratnavel Subramanian.
add 8861573 HDDS-1717. MR Job fails as OMFailoverProxyProvider has dependency hadoop-3.2
add 1dc58c5 HDDS-1742. Merge ozone-perf and ozonetrace example clusters
add 96d0555 HDDS-1718. Increase Ratis Leader election timeout default. Contributed by Aravindan Vijayan & Siddharth Wagle. (#1062)
add 0303072 HDDS-1586. Allow Ozone RPC client to read with topology awareness. Contributed by Sammi Chen.
add ac7a8ac HDDS-1603. Handle Ratis Append Failure in Container State Machine. Contributed by Supratim Deka (#1019)
add 993dc87 HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.
add e0fa104 HDDS-1748. Error message for 3 way commit failure is not verbose. Contributed by Supratim Deka (#1051)
add 9382488 HDDS-1764. Fix hidden errors in acceptance tests
add bbf5844 HDDS-1525. Mapreduce failure when using Hadoop 2.7.5
add efb9164 HDDS-1778. Fix existing blockade tests. (#1068)
add 90b10a0 HDFS-14622. [Dynamometer] Update XML FsImage parsing logic to ignore non-INodeSection entries to fix an issue caused by the presence of Centralized Cache Management functionality. Contributed by Erik Krogen.
add cdb20ad HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973)
add eccc9a4 HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.
add 6872efc HDDS-1611.[Addendum] Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973)
add c7b5f85 HADOOP-16393. S3Guard init command uses global settings, not those of target bucket.
add 555dabf YARN-9660. Update support documentation for Docker on YARN. Contributed by Peter Bacsko
add ccaa99c HADOOP-16381. The JSON License is included in binary tarball via azure-documentdb:1.16.2. Contributed by Sushil Ks.
add acef5e0 HDDS-1201. Reporting corrupted containers info to SCM (#1032)
add 9cec023 YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl
add fc0656d HADOOP-16418. [Dynamometer] Fix checkstyle and findbugs warnings. Contributed by Erik Krogen.
add 32925d0 HDFS-14640. [Dynamometer] Fix TestDynamometerInfra failure. Contributed by Erik Krogen.
add 5043840 HDFS-14410. Make Dynamometer documentation properly compile onto the Hadoop site. Contributed by Erik Krogen.
add 5747f6c HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell.
add 6a3433b HADOOP-16357. TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists.
add 8fb5ca3 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
add 9b54dd7 YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung)
add 0f399b0 HDDS-1784. Missing HostName and IpAddress in the response of register command.
add cdcc205 HDDS-1784. Missing HostName and IpAddress in the response of register command. (#1078)
add 00dd843 HDFS-14466. Add a regression test for HDFS-14323.
add 738fab3 HDDS-1754. getContainerWithPipeline fails with PipelineNotFoundException. Contributed by Supratim Deka (#1081)
add 14c43f8 HDDS-1752. ConcurrentModificationException while handling DeadNodeHandler event. (#1080)
add 9119ed0 HDDS-1384. TestBlockOutputStreamWithFailures is failing
add f9fab9f HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee.
add b15ef7d HADOOP-16384: S3A: Avoid inconsistencies between DDB and S3.
add 7b8177b HDDS-1791. Update network-tests/src/test/blockade/README.md file
add 190e434 HDDS-1790. Fix checkstyle issues in TestDataScrubber
add c416284 YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal
add 8b3c679 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko
add 61b0c2b YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal
add 557056e YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl
add 0bae9e8 HDDS-1735. Create separate unit and integration test executor dev-support script
add 6f25420 fixing ALL the checkstule issues under hadoop-ozone/dev-support/checks
add 987bf3f fix print0 consequences
add 317f2c8 fix rat problem
add ca7b3e8 reenable unit test execution
add 51e7dc3 make author check yetus safe
add 4a70a0d HDDS-1735. Create separate unit and integration test executor dev-support script. Contributed by Elek, Marton.
add 0976f6f HDDS-1766. ContainerStateMachine is unable to increment lastAppliedTermIndex. Contributed by Mukul Kumar Singh. (#1072)
add e98adb0 HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
add 30a8f84 HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.
add 91ce09e YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko
add be784de SUBMARINE-62. PS_LAUNCH_CMD CLI description is wrong in RunJobCli. Contributed by Adam Antal
add 18ee109 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko
add 5446308 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal
add 61bbdee HDDS-1800. Result of author check is inverted
add 64d4abf HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma.
add 68f53f9 HDDS-1763. Use vendor neutral s3 logo in ozone doc. Contributed by Elek, Marton.
add 47345e5 Merge pull request #1058 from elek/HDDS-1763
add d3a5abe HDDS-1668. Add liveness probe to the example k8s resources files
add 1411513 Merge pull request #944 from elek/HDDS-1668
add 585f4d5 HDDS-1761. Fix class hierarchy for KeyRequest and FileRequest classes. (#1052)
add ef66e49 HDDS-1666. Issue in openKey when allocating block. (#943)
add f77d54c HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell.
add 395cb3c HDDS-1736. Cleanup 2phase old HA code for Key requests. (#1038)
add 7a93be0 YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
add 1502e6a HDDS-1756. DeleteContainerCommandHandler fails with NPE. Contributed by Nanda kumar(#1095).
add d21eccf HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084)
add c5e3ab5 HDDS-1793. Acceptance test of ozone-topology cluster is failing
add 493b0b5 HDDS-1787. NPE thrown while trying to find DN closest to client. Contributed by Sammi Chen. (#1094)
add d429d74 HDDS-1544. Support default Acls for volume, bucket, keys and prefix. Contributed by Ajay Kumar, Xiaoyu Yao.
add 71c21f0 HDDS-1813. Fix false warning from ozones3 acceptance test. Contributed by Xiaoyu Yao. (#1107)
add 7adc4a3 HDDS-1775. Make OM KeyDeletingService compatible with HA model (#1063)
add 5672efa HADOOP-15729. [s3a] Allow core threads to time out. (#1075)
add 5915c90 YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang)
add 85d9111 YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam.
add ee3115f HDFS-9913. DistCp to add -useTrash to move deleted files to Trash.
add 19a0018 Revert "HDFS-9913. DistCp to add -useTrash to move deleted files to Trash."
add b4466a3 HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679
add 256fcc1 HDDS-1785. OOM error in Freon due to the concurrency handling
add c58e11b HADOOP-16383. Pass ITtlTimeProvider instance in initialize method in MetadataStore interface. Contributed by Gabor Bota. (#1009)
add eb0849b HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao.
add af0665c HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.
add 303a7f8 HDDS-1721 : Client Metrics are not being pushed to the configured sink while running a hadoop command to write to Ozone. (#1034)
add 0541322 HDDS-1675. Cleanup Volume Request 2 phase old code. (#964)
add 5e6cc6f HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.
add 9792f58 HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena.
add 73e6ffc HDDS-1802. Add Eviction policy for table cache. (#1100)
add 3dc256e HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. (#1088)
add 53a4c22 HDDS-1481: Cleanup BasicOzoneFileSystem#mkdir (#1114)
add 23e9beb HDDS-1767: ContainerStateMachine should have its own executors for executing applyTransaction calls (#1087)
add ccceedb HDDS-1780. TestFailureHandlingByClient tests are flaky. Contributed by Shashikant Banerjee. (#1073)
add 20cf50c HDDS-1654. Ensure container state on datanode gets synced to disk whennever state change happens. Cotributed by Shashikant Banerjee. (#923)
add 79f6118 MAPREDUCE-6521. MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test.
add 8ef2365 HDDS-1779. TestWatchForCommit tests are flaky.Contributed by Shashikant Banerjee. (#1071)
add c34ceb5 YARN-9568. Fixed NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover. Contributed by Steve Loughran
add f22818d HDDS-1820. Fix numKeys metrics in OM HA. (#1116)
add 9838a47 YARN-6046. Fixed documentation error in YarnApplicationSecurity. Contributed by Yousef Abu-Salah
add d5ef38b HDDS-1822. NPE in SCMCommonPolicy.chooseDatanodes (#1120)
add d545f9c HADOOP-16437 documentation typo fix: fs.s3a.experimental.input.fadvise
add 4e66cb9 HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao. (#1067)
add 6282c02 HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota.
add cd967c7 HADOOP-15847. S3Guard testConcurrentTableCreations to set R/W capacity == 0
add 7f1b76c HADOOP-13868. [s3a] New default for S3A multi-part configuration (#1125)
add 69a46a9 HDDS-1713. ReplicationManager fail to find proper node topology based… (#1112)
add f5b2f75 HDDS-1795. Implement S3 Delete Bucket request to use Cache and DoubleBuffer. (#1097)
add 0b45293 HADOOP-16404. ABFS default blocksize change(256MB from 512MB)
add e60f5e2 HADOOP-16440. Distcp can not preserve timestamp with -delete option. Contributed by ludun.
add 4aa76e3 HDDS-1805. Implement S3 Initiate MPU request to use Cache and DoubleBuffer. (#1108)
add acdb0a1 HDDS-1841. Fix TestSecureContainerServer. (#1136)
add 340bbaf HDDS-1840. Fix TestSecureOzoneContainer. (#1135)
add cdc36fe HDDS-1649. On installSnapshot notification from OM leader, download checkpoint and reload OM state (#948)
add c958edd HDDS-1811. Prometheus metrics are broken.
add 2ea71d9 HDDS-1686. Remove check to get from openKeyTable in acl implementatio… (#966)
add f3f2f34 HDDS-1585. Add LICENSE.txt and NOTICE.txt to Ozone Recon Web
add d70ec4b HDDS-1799. Add goofyfs to the ozone-runner docker image
add d59f271 HDDS-1803. shellcheck.sh does not work on Mac
add c533b79 HDDS-1710. Publish JVM metrics via Hadoop metrics Signed-off-by: Anu Engineer <ae...@apache.org>
add ee87e9a MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build
add 377f95b HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
add eb36b09 HADOOP-16443. Improve help text for setfacl --set option.
add 4317d33 HADOOP-16380. S3Guard to determine empty directory status for all non-root directories.
add 278390f HDDS-1710. Fix checkstyle errors
add fbe2fc6 HDDS-1846. Default value for checksum bytes is different in ozone-site.xml and code.
add ecb9f81 HDDS-1845. Fix OMVolumeSetQuota|OwnerRequest#validateAndUpdateCache return response. (#1141)
add a1251ad HADOOP-16431. Remove useless log in IOUtils.java and ExceptionDiags.java.
add cf9ff08 HDDS-1848. Fix TestOzoneManagerHA and TestOzoneManagerSnapShotProvider. (#1148)
add 7b21977 HADOOP-16433. S3Guard: Filter expired entries and tombstones when listing with MetadataStore.listChildren().
add cb69700 HDDS-1853. Fix failing blockade test-cases. (#1151)
add 1d98a21 HDDS-1819. Implement S3 Commit MPU request to use Cache and DoubleBuffer. (#1140)
add 2546e6e HDDS-1817. GetKey fails with IllegalArgumentException.
add 9b8b3ac HADOOP-16451. Update jackson-databind to 2.9.9.1. Contributed by Siyao Meng.
add 62deab1 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li.
add ba43233 HDDS-1749 : Ozone Client should randomize the list of nodes in pipeline for reads. (#1124)
add 3c4159f HDDS-1842. Implement S3 Abort MPU request to use Cache and DoubleBuffer. (#1155)
add b41ef61 HDDS-1858. mTLS support for Ozone is not correct. Contributed by Siddharth Wagle.
add 2b1d8ae HDDS-1850. ReplicationManager should consider inflight replication and deletion while picking datanode for re-replication.
add b7fba78 HDDS-1830 OzoneManagerDoubleBuffer#stop should wait for daemon thread to die (#1156)
add 6b8107a HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena.
add a2cc961 HDDS-1855. TestStorageContainerManager#testScmProcessDatanodeHeartbeat is failing. (#1153)
add ce99cc3 HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
add 3426777 HDDS-1861. Fix TableCacheImpl cleanup logic. (#1165)
add 98653ec HADOOP-16447 Upgrade JUnit5 from 5.3.1 to 5.5.1 to support global timeout
add 0753031 HADOOP-9844. NPE when trying to create an error message response of SASL RPC
add aebac6d HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock.
add c0a0c35 MAPREDUCE-6973. Fix comments on creating _SUCCESS file.
add c7c7a88 HDDS-1839: Change topology sorting related logs in Pipeline from INFO to DEBUG. Contributed by Junjie Chen.
add c01e137 HDDS-1864. Turn on topology aware read in TestFailureHandlingByClient. (#1168)
add ecc8acf HDFS-14673. The console log is noisy when using DNSDomainNameResolver to resolve NameNode.
add 62efb63 HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen.
add 2fe450c HDFS-14425. Native build fails on macos due to jlong in hdfs.c (#741)
add 02bd02b HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun.
add 902ff4a HDDS-1867. Invalid Prometheus metric name from JvmMetrics
add 61ec03c HDDS-1852. Fix typo in TestOmAcls
add b039f75 HDDS-1682. TestEventWatcher.testMetrics is flaky
add 90afb7b HDDS-1725. pv-test example to test csi is not working
add 8a59cd1 HDFS-12967. NNBench should support multi-cluster access. Contributed by Chen Zhang.
add d023663 HDDS-1833. Moved RefCountedDB stacktrace to log level trace. Contributed by Siddharth Wagle
add 60325c9 HDDS-1391 : Add ability in OM to serve delta updates through an API. (#1033)
add 611718f HDFS-14670: RBF: Create secret manager instance using FederationUtil#newInstance.
add 72900de HDFS-14639. [Dynamometer] Remove unnecessary duplicate directory from the distribution. Contributed by Erik Krogen.
add 8053085 HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.
add 4d07134 HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated (#1164)
add 68f9f2d Revert HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. (#1183)
add cbfa3f3 HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich.
add 204a977 HADOOP-15910. Fix Javadoc for LdapAuthenticationHandler#ENABLE_START_TLS
add bca86bd HADOOP-16469. Update committers.md
add 88daa73 HDDS-1872. Fix entry clean up from openKeyTable during complete MPU. (#1182)
add 8deced0 HDFS-14677. TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk. Contributed by Chen Zhang.
add 12a526c HADOOP-16460: ABFS: fix for Sever Name Indication (SNI)
add c75f16d HADOOP-16452. Increase ipc.maximum.data.length default from 64MB to 128MB. Contributed by Siyao Meng.
add 42683ae YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
add e68d844 HDDS-1834. parent directories not found in secure setup due to ACL check. Contributed by Doroszlai, Attila.
add 7849bdc HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li.
add 1f26cc8 HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang.
add 3ae775d HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
add 99f88c3 HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao.
add 0f2dad6 HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina.
add e5c4131 HDDS-1856. Make required changes for Non-HA to use new HA code in OM. (#1174)
add d4ab9ae HDDS-1816: ContainerStateMachine should limit number of pending apply transactions. Adds a config, uses snapshot threshold default value. (#1150)
add ac8ed7b HDDS-1877. hadoop31-mapreduce fails due to wrong HADOOP_VERSION
add 90e5eb0 HDDS-1849. Implement S3 Complete MPU request to use Cache and DoubleBuffer. (#1181)
add a6f47b5 HDDS-1875. Fix failures in TestS3MultipartUploadAbortResponse. (#1188)
add 8bda91d HADOOP-16398. Exports Hadoop metrics to Prometheus (#1170)
add b008072 HDFS-14681. RBF: TestDisableRouterQuota failed because port 8888 was occupied.
add c1f7440 HDDS-1885. Fix bug in checkAcls in OzoneManager. (#1199)
add 89b102f HDFS-14661. RBF: updateMountTableEntry shouldn't update mountTableEntry if targetPath not exist. Contributed by xuzq.
add a7371a7 MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko.
add e111789 HDDS-1882. TestReplicationManager failed with NPE. (#1197)
add f86de6f HDFS-13529. Fix default trash policy emptier trigger time correctly. Contributed by He Xiaoqiao.
add 32607db HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.
add b94eba9 HADOOP-12282. Connection thread's name should be updated after address changing is detected. Contributed by Lisheng Sun.
add d086d05 HDFS-14652. HealthMonitor connection retry times should be configurable. Contributed by Chen Zhang.
add 99bf1dc HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response (#1189) Contributed by Siyao Meng.
add 17e8cf5 HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null (#1192) Contributed by Siyao Meng.
add e20b195 HADOOP-15681. AuthenticationFilter should generate valid date format for Set-Cookie header regardless of default Locale. Contributed by Cao Manh Dat.
add c2d00c8 HDFS-13131. Modifying testcase testEnableAndDisableErasureCodingPolicy. Contributed by chencan.
add e872ceb HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko.
add 95fc38f YARN-9375. Use Configured in GpuDiscoverer and FpgaDiscoverer (#1131)
add cbcada80 YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently (#1121)
add 1930a7b YARN-9093. Remove commented code block from the beginning of Tes… (#444)
add 54ac801 Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph
add 797d14e Revert "HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm."
add b5c74d4 HDDS-1782. Add an option to MiniOzoneChaosCluster to read files multiple times. Contributed by Mukul Kumar Singh. (#1076)
add 946e301 HDDS-1878. checkstyle error in ContainerStateMachine (#1195)
add 6e8c841 HDDS-1832 : Improve logging for PipelineActions handling in SCM and datanode. (#1200)
add eaf350c HDDS-1887. Enable all the blockade test-cases. Contibuted by NandaKumar(#1206).
add b964b81 HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq.
add e7a0b8a HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.
add b01efe5 HADOOP-16472. findbugs warning on LocalMetadataStore.ttlTimeProvider sync
add 61180f4 HADOOP-15942. Change the logging level form DEBUG to ERROR for RuntimeErrorException in JMXJsonServlet. Contributed by Anuhan Torgonshar.
add e503db5 HDFS-12826. Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. Contributed by usharani.
add 8f40856 HDFS-14669. TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk. Contributed by qiang Liu.
add ec1d453 HDDS-1788. Fix kerberos principal error in Ozone Recon. (#1201)
add f4df97f HDDS-1870. ConcurrentModification at PrometheusMetricsSink (#1179)
add 065cbc6 HDDS-1896. Suppress WARN log from NetworkTopology#getDistanceCost. Contributed by Xiaoyu Yao. (#1214)
add 9680a8b HADOOP-16487. Update jackson-databind to 2.9.9.2. Contributed by Siyao Meng.
add f8ea6e1 HDDS-1798. Propagate failure in writeStateMachineData to Ratis. Contributed by Supratim Deka (#1113)
add 71aad60 HDFS-14691. Wrong usage hint for hadoop fs command "test".
add c589983 HDDS-1893. Fix bug in removeAcl in Bucket. (#1216)
add d6697da YARN-9667. Use setbuf with line buffer to reduce fflush complexity in container-executor. Contributed by Peter Bacsko
add 1127215 HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou.
add 2662e20 HDDS-1905. PipelineActionHandler is not closing the pipeline when close action is received. (#1227)
add 1c53ce0 YARN-9410. Typo in documentation: Using FPGA On YARN (#1220) Contributed by kevin su.
add 35e0a01 HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell.
add b8bf09b YARN-9678. TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
add 7c2042a YARN-9678. Addendum: TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
add a63023f HDDS-1901. Fix Ozone HTTP WebConsole Authentication. Contributed by Xiaoyu Yao. (#1228)
add b77761b HADOOP-16275. Upgrade Mockito to the latest version. Contributed by kevin su.
add f51702d YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)
add 22430c1 HADOOP-16457. Fixed Kerberos activation in ServiceAuthorizationManager. Contributed by Prabhu Joseph
add 8cef9f8 HDFS-14652. Addendum: HealthMonitor connection retry times should be configurable. Contributed by Chen Zhang.
add 954ff36 HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain.
add c4d97ae HDDS-1918. hadoop-ozone-tools has integration tests run as unit (#1236)
add 0b0ba70 HADOOP-15908. hadoop-build-tools jar is downloaded from remote repository instead of using from local. Contributed by Oleksandr Shevchenko.
add 38e6968 HDDS-1919. Fix Javadoc in TestAuditParser (#1240)
add a5bb1e8 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.
add 9cd211a HDDS-1921. TestOzoneManagerDoubleBufferWithOMResponse is flaky (#1238)
add 70f4674 HDDS-1907. TestOzoneRpcClientWithRatis is failing with ACL errors.
add 0520f5c HDDS-1924. ozone sh bucket path command does not exist
add 827dbb1 HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.
add de64253 HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri.
add 11f750e HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying.
add 3cc0ace HADOOP-16491. Upgrade jetty version to 9.3.27. Contributed by Hrishikesh Gadre.
add 8f9245b HDDS-1865. Use "ozone.network.topology.aware.read" to control both RPC client and server side logic (#1184)
add ab6a5c9 HDDS-1925. ozonesecure acceptance test broken by HTTP auth requirement (#1248)
add 70b4617 HDDS-1900. Remove UpdateBucket handler which supports add/remove Acl. (#1219)
add b0131bc HADOOP-15014. Addendum: KMS should log the IP address of the clients. Contributed by Zsombor Gegesy.
add 22d7d1f YARN-9601.Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations (#908) Contributed by hunshenshi.
add 00b5a27 SUBMARINE-72. Kill and destroy the job through the submarine client (#1090) Contributed by kevin su.
add 397a563 HDDS-1888. Add containers to node2container map in SCM as part of ICR processing.
add 3b38f20 YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
add 9e6519a YARN-9711. Missing spaces in NMClientImpl (#1177) Contributed by Charles Xu.
add 63161cf HDDS-1926. The new caching layer is used for old OM requests but not updated
add 3ac0f3a HDDS-1619. Support volume acl operations for OM HA. Contributed by… (#1147)
add 5840df8 HADOOP-16479. ABFS FileStatus.getModificationTime returns localized time instead of UTC.
add b079914 HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell.
add 14a4ce3 HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. Contributed by Siyao Meng.
add 23f91f6 HDFS-14662. Document the usage of the new Balancer "asService" parameter. Contributed by Chen Zhang.
add 28a8484 HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by Lisheng Sun.
add 2265872 HDFS-14705. Remove unused configuration dfs.min.replication. Contributed by CR Hota.
add 6ad9a11 HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao.
add aa5f445 HDDS-1863. Freon RandomKeyGenerator even if keySize is set to 0, it returns some random data to key. (#1167)
add 91f41b7 HDDS-1884. Support Bucket ACL operations for OM HA. (#1202)
add 88ed1e0 HDDS-1934. TestSecureOzoneCluster may fail due to port conflict (#1254)
add a92b7a5 YARN-9694. UI always show default-rack for all the nodes while running SLS.
add df30d8e YARN-9727: Allowed Origin pattern is discouraged if regex contains *. Contributed by Zoltan Siegl
add 72d7e57 YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak
add 742e30b YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak
add e0c21c6 YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak
add f6fa865 HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
add e5f4cd0 SUBMARINE-57. Add more elaborate message if submarine command is not recognized. Contributed by Adam Antal
add acffec7 YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB.
add 43a91f8 HADOOP-16315. ABFS: transform full UPN for named user in AclStatus
add e25a5c2 HADOOP-16499. S3A retry policy to be exponential (#1246). Contributed by Steve Loughran.
add 189dc10 HADOOP-16481. ITestS3GuardDDBRootOperations.test_300_MetastorePrune needs to set region. (#1209). Contributed by Steve Loughran.
add a79564f YARN-9732. yarn.system-metrics-publisher.enabled=false is not honored by RM. Contributed by KWON BYUNGCHANG.
add 6ff0453 YARN-9527. Prevent rogue Localizer Runner from downloading same file repeatly. Contributed by Jim Brennan
add 98dd7c4 HDDS-1906. TestScmSafeMode#testSCMSafeModeRestrictedOp is failing. (#1260)
add 865021b HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong.
add ce3c5a3 HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina.
add e02ffed HDFS-12125. Document the missing EC removePolicy command (#1258) Contributed by Wenxin He, Siyao Meng.
add 8a77a224 HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin.
add bd4be6e HDDS-1895. Support Key ACL operations for OM HA. (#1230)
add fba222a HDDS-1943. TestKeyManagerImpl.testLookupKeyWithLocation is failing. Contributed by Xiaoyu Yao. (#1262)
add 50cec5a YARN-9729. [UI2] Fix error message for logs without ATSv2. Contributed by Zoltan Siegl.
add e69db45 Revert "YARN-9729. [UI2] Fix error message for logs without ATSv2. Contributed by Zoltan Siegl."
add d29007f HDDS-1366. Add ability in Recon to track the number of small files in an Ozone Cluster (#1146)
add 1c5b286 YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl.
add cf5d895 YARN-9657. AbstractLivelinessMonitor add serviceName to PingChecker thread. Contributed by Bilwa S T.
add 6b4564f HADOOP-16453. Update how exceptions are handled in NetUtils. Contributed by Lisheng Sun.
add 8fbf8b2 YARN-9722. PlacementRule logs object ID in place of queue name. Contributed by Prabhu Joseph.
add 13a5803 YARN-9464. Support pending resource metrics in RM's RESTful API. Contributed by Prabhu Joseph.
add dfe772d HDDS-1912. start-ozone.sh fail due to ozone-config.sh not found. Contributed by kevin su.
add ac6c4f0 MAPREDUCE-7197. Fix order of actual and expected expression in assert statements. Contributed by Adam Antal
add e0517fe YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko
add b91099e YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph
add e4b538b YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
add 201dc66 YARN-9719. Fixed YARN service restart bug when application ID no longer exist in RM. Contributed by kyungwan nam
add c92b498 HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory (#1274) Contributed by Siyao Meng.
add 454420e HDFS-14708. TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk. Contributed by Lisheng Sun.
add 0b507d2 HDDS-1908. TestMultiBlockWritesWithDnFailures is failing (#1282)
add b4097b9 YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph.
add 3dc22d6 HDDS-1952. Disable TestMiniChaosOzoneCluster in integration.sh. (#1284)
add e9b6b81 HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng.
add 274966e HDFS-14717. [Dynamometer] Remove explicit search for JUnit dependency JAR from Dynamometer Client as it is packaged in the primary JAR. Contributed by Kevin Su.
add 2ac029b YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.
add 689a80d HDDS-1886. Use ArrayList#clear to address audit failure scenario
add 69b74e9 HDDS-1488. Scm cli command to start/stop replication manager.
add 68c8184 HDDS-1891. Ozone fs shell command should work with default port when port number is not specified
add cb390df HDDS-1961. TestStorageContainerManager#testScmProcessDatanodeHeartbeat is flaky.
add 3cff73a HDDS-1917. TestOzoneRpcClientAbstract is failing.
add 6ae8bc3 HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields (#1267) Contributed by Siyao Meng.
add da0006f HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
add 633b7c1 HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina.
add 9691117 HDDS-1916. Only contract tests are run in ozonefs module
add 50a22b6 HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#950)
add c13ec7a HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng.
add e6d240d HDDS-1928. Cannot run ozone-recon compose due to syntax error
add 0f8add8 HADOOP-16495. Fix invalid metric types in PrometheusMetricsSink (#1244)
add bf45779 HDDS-1920. Place ozone.om.address config key default value in ozone-site.xml
add 78b714a HDDS-1956. Aged IO Thread exits on first read
add 15545c8 HDDS-1915. Remove hadoop script from ozone distribution
add fc229b6 HDDS-1832 : Improve logging for PipelineActions handling in SCM and datanode. (Change to Error logging)
add 5a248de HDDS-1947. fix naming issue for ScmBlockLocationTestingClient. Contributed by star.
add 48dc441 HDDS-1929. OM started on recon host in ozonesecure compose
add 846848a HDDS-1914. Ozonescript example docker-compose cluster can't be started
add 89a53c7 YARN-9747. Reduce additional namenode call by EntityGroupFSTimelineStore#cleanLogs. Contributed by Prabhu Joseph.
add 0e4b757 HADOOP-16500 S3ADelegationTokens to only log at debug on startup (#1269). Contributed by Steve Loughran.
add 83e452e HDDS-1965. Compile error due to leftover ScmBlockLocationTestIngClient file (#1293)
add 3c0382f HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng.
add e5e6093 YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko
add 2432356 HDDS-1955. TestBlockOutputStreamWithFailures#test2DatanodesFailure failing because of assertion error.
add 3e04104 YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko
add b06c234 HDFS-14713. RBF: RouterAdmin supports refreshRouterArgs command but not on display. Contributed by wangzhaohui.
add c89bdfa YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… (#1261)
add 06d8ac9 HDDS-1966. Wrong expected key ACL in acceptance test
add 8242085 HDDS-1964. TestOzoneClientProducer fails with ConnectException
add c720441 SUBMARINE-107. Reduce the scope of mockito-core in submarine to test. Contributed by kevin su.
add 167acd8 YARN-9683. Remove reapDockerContainerNoPid left behind by YARN-9074 (#1212) Contributed by Kevin Su.
add 0a85af9 HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.
add 85a00fd SUBMARINE-45. Can't specify queue by using the parameter --queue. Contributed by Ayush Saxena, Zac Zhou.
add 3468164 HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina.
add 3f4f097 MAPREDUCE-7230. TestHSWebApp.testLogsViewSingle fails. Contributed by Prabhu Joseph
add 1845a83 YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
add 22c4f38 YARN-9679. Regular code cleanup in TestResourcePluginManager (#1122)
add c801f7a HDDS-1768. Audit xxxAcl methods in OzoneManager (#1204)
add 46d6191 HDDS-1954. StackOverflowError in OzoneClientInvocationHandler
add 77d102c HDDS-1910. Cannot build hadoop-hdds-config from scratch in IDEA
add 5882cf9 HADOOP-16504. Increase ipc.server.listen.queue.size default from 128 to 256. Contributed by Lisheng Sun.
add bf37515 HDDS-1894. Add filter to scmcli listPipelines. (#1286)
add 2a05e0f YARN-9749. TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk. Contributed by Adam Antal
add 2216ec5 YARN-9100. Add tests for GpuResourceAllocator and do minor code cleanup. Contributed by Peter Bacsko
add 4456ea6 YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko
add 9b8359b YARN-9461. TestRMWebServicesDelegationTokenAuthentication.testCancelledDelegationToken fails with HTTP 400. Contributed by Peter Bacsko
add 9a1d8cf HDFS-14678. Allow triggerBlockReport to a specific namenode. (#1252). Contributed by Leon Gao.
add e356e4f HADOOP-16391 Add a prefix to the metric names for MutableRatesWithAggregation used for deferred RPC metrics to avoid collision with non-deferred metrics. Contributed by Bilwa S T.
add 8943e13 HDDS-1969. Implement OM GetDelegationToken request to use Cache and DoubleBuffer. (#1296)
add a38b9e1 HDFS-14456:HAState#prepareToEnterState neednt a lock (#770) Contributed by hunshenshi.
add c8675ec HDDS-1911. Support Prefix ACL operations for OM HA. (#1275)
add a46ba03 HDDS-1913. Fix OzoneBucket and RpcClient APIS for acl. (#1257)
add 971a4c8 HDFS-14523. Remove excess read lock for NetworkToplogy. Contributed by Wu Weiwei.
add 8d754c2 HADOOP-16351. Change ":" to ApplicationConstants.CLASS_PATH_SEPARATOR. Contributed by kevin su.
add 0e0ddfa YARN-2599. Standby RM should expose jmx endpoint. Contributed by Rohith Sharma K S.
add 02015e3 HDDS-1959. Decrement purge interval for Ratis logs in datanode (#1301)
add 52837878 HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan.
add d873ddd HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng.
add e618256 HDDS-1979. Fix checkstyle errors (#1312)
add 3bba808 HDDS-1938. Change omPort parameter type from String to int in BasicOzoneFileSystem#createAdapter (#1305)
add b83eae7 HDDS-1974. Implement OM CancelDelegationToken request to use Cache and DoubleBuffer. (#1308)
add e32f52c HDDS-1903 : Use dynamic ports for SCM in TestSCMClientProtocolServer … (#1303)
add b58a35f HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
add b8db5b9 HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.
add 12c7084 HDDS-1971. Update document for HDDS-1891: Ozone fs shell command should work with default port when port number is not specified (#1306)
add a707bb7 HADOOP-15246. SpanReceiverInfo - Prefer ArrayList over LinkedList. Contributed by David Mollitor.
add c765584 HADOOP-16158. DistCp to support checksum validation when copy blocks in parallel (#919)
add d69a1a0 HDDS-1105 : Add mechanism in Recon to obtain DB snapshot 'delta' updates from Ozone Manager (#1259)
add 51b6537 HADOOP-14784. [KMS] Improve KeyAuthorizationKeyProvider#toString(). Contributed by Yeliang Cang.
add abae6ff HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su.
add 360a96f HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang.
add 4f925af HDDS-1972. Provide example ha proxy with multiple s3 servers back end. (#1304)
add 6244502 HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.
add bd92462 HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297
add 094d736 YARN-9758. Upgrade JQuery to latest version for YARN UI. Contributed by Akhil PB.
add aa6995f HDFS-13201. Fix prompt message in testPolicyAndStateCantBeNull. Contributed by chencan.
add 4cb22cd HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.
add 269b543 HADOOP-16523. Minor spell mistake in comment : Line number 101 (#388)
add 3a145e2 HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore.
add 7653ebd Revert "HDDS-1965. Compile error due to leftover ScmBlockLocationTestIngClient file (#1293)"
add 10b4997 HDDS-1965. Compile error due to leftover ScmBlockLocationTestIngClient file. (fix build)
add 30ce854 HADOOP-16496. Apply HDDS-1870 (ConcurrentModification at PrometheusMetricsSink) to Hadoop common.
add 8aaf5e1 HDFS-14759. HDFS cat logs an info message. Contributed by Eric Badger.
add 10ec31d YARN-9765. SLS runner crashes when run with metrics turned off. Contributed by Abhishek Modi.
add e684b17 YARN-5857. TestLogAggregationService.testFixedSizeThreadPool fails intermittently on trunk. Contributed by Bilwa S T.
add e8fa192 YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko
add 7ab88db YARN-7291. Better input parsing for resource in allocation file. Contributed by Zoltan Siegl
add 3ad1fcf YARN-9752. Add support for allocation id in SLS. Contributed by Abhishek Modi
add 57f7370 Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."
add 5eeb6da HDFS-14714. RBF: implement getReplicatedBlockStats interface. Contributed by Chen Zhang.
add 93595fe HDFS-14756. RBF: getQuotaUsage may ignore some folders. Contributed by Chen Zhang.
add 217e748 HDDS-1973. Implement OM RenewDelegationToken request to use Cache and DoubleBuffer. (#1316)
add 2ae7f44 HDDS-1999. Basic acceptance test and SCM/OM web UI broken by Bootstrap upgrade (#1327)
add 8fc6567 HDDS-1871. Remove anti-affinity rules from k8s minkube example
add 0f598ae HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang.
add f9029c4 HDFS-14744. RBF: Non secured routers should not log in error mode when UGI is default. Contributed by CR Hota.
add d58eba8 HDDS-1927. Consolidate add/remove Acl into OzoneAclUtil class. Contributed by Xiaoyu Yao.
add 34dd9ee HADOOP-16494. Add SHA-512 checksum to release artifact to comply with the release distribution policy.
add 52c77bc HDFS-14741. RBF: RecoverLease should be return false when the file is open in multiple destination. Contributed by xuzq
add 5e156b9 HADOOP-16061. Upgrade Yetus to 0.10.0
add 76790a1 HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina.
add ee7c261 HDFS-14763. Fix package name of audit log class in Dynamometer document (#1335)
add 69ddb36 HADOOP-16193. Add extra S3A MPU test to see what happens if a file is created during the MPU. Contributed by Steve Loughran
add 61b2df2 HADOOP-16470. Make last AWS credential provider in default auth chain EC2ContainerCredentialsProviderWrapper.
add 63c295e HDFS-14755. [Dynamometer] Enhance compatibility of Dynamometer with branch-2 builds. Contributed by Takanobu Asanuma.
add e04dcfd HDFS-14583. FileStatus#toString() will throw IllegalArgumentException. Contributed by xuzq.
add 28fb4b5 HDDS-2008 : Wrong package for RatisHelper class in hadoop-hdds/common module. (#1333)
add 4028cac HDDS-1347. In OM HA getS3Secret call Should happen only leader OM. (#670)
add 93daf69 HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell.
add b67812e HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index (#1028). Contributed by Stephen O'Donnell.
add f6af7d0 HDDS-1808. TestRatisPipelineCreateAndDestory times out (#1338)
add 84b1982 YARN-9774. Fix order of arguments for assertEquals in TestSLSUtils. Contributed by Nikhil Navadiya.
add abc8fde HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.
add edd7085 HDDS-1948. S3 MPU can't be created with octet-stream content-type (#1266)
add bd7baea HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.
add b4a95a2 HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trunk snapshot
add e2a5548 HDDS-2023. Fix rat check failures in trunk
add d3fe993 HDDS-2023. Fix rat check failures in trunk (addendum)
add 75bf090 HDDS-2024. rat.sh: grep: warning: recursive search of stdin
add c92de82 HDFS-14747. RBF: IsFileClosed should be return false when the file is open in multiple destination. Contributed by xuzq.
add 20064b6 HDDS-1978. Create helper script to run blockade tests. (#1310)
add 894e230 HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly (#1334)
add d699022 HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.
add ebef99d HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.
add 3f887f3 HDDS-1827. Load Snapshot info when OM Ratis server starts. (#1130)
add 0b79675 HDFS-14766. RBF: MountTableStoreImpl#getMountTableEntries returns extra entry. Contributed by Chen Zhang.
add b661dcf HDDS-2002. Update documentation for 0.4.1 release.
add d2225c8 HDFS-14722. RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException. Contributed by xuzq.
add 23e532d Revert "HADOOP-16193. Add extra S3A MPU test to see what happens if a file is created during the MPU. Contributed by Steve Loughran"
add 689d2e6 YARN-8917. Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource. Contributed by Tao Yang.
add 6d7f01c YARN-9771. Add GPU in the container-executor.cfg example. Contributed by Julia Kinga Marton
add d3ce53e YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt.
add d1aa859 HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. (#1315)
add 8660e48 YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph
add 07e3cf9 HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.
add d70f523 YARN-9640. Slow event processing could cause too many attempt unregister events. Contributed by Bibin A Chundatt.
add 567091a HADOOP-15958. Revisiting LICENSE and NOTICE files.
add 4379370 HDDS-1981: Datanode should sync db when container is moved to CLOSED or QUASI_CLOSED state (#1319)
add b69ac57 HDFS-14772. RBF: hdfs-rbf-site.xml can't be loaded automatically. Contributed by Yuxuan Wang
add 3329257 HDDS-1998. TestSecureContainerServer#testClientServerRatisGrpc is failing (#1328)
add 717c853 YARN-9755. Fixed RM failing to start when FileSystemBasedConfigurationProvider is configured. Contributed by Prabhu Joseph
add 8ab7020 HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns
add ce8eb12 Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
add 66cfa48 HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee(#1226).
add dde9399 HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
add 0883ce1 HDDS-2026. Overlapping chunk region cannot be read concurrently
add 8ef4659 YARN-9438. launchTime not written to state store for running applications
add d85d68f HADOOP-16533. Upgrade jackson-databind to 2.9.9.3. (#1354). Contributed by Akira Ajisaka.
add 2b9cc7e HDDS-2037. Fix hadoop version in pom.ozone.xml.
add 6f068cf HADOOP-16416. mark DynamoDBMetadataStore.deleteTrackingValueMap as final. Contributed by kevin su.
add 6e37d65 HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota.
add b1eee8b HDDS-1946. CertificateClient should not persist keys/certs to ozone.m… (#1311)
add 5d31a4e HDDS-1753. Datanode unable to find chunk while replication data using ratis. (#1318)
add 55cc115 HADOOP-16527. Add a whitelist of endpoints to skip Kerberos authentication (#1336) Contributed by Akira Ajisaka.
add c0499bd HDDS-1596. Create service endpoint to download configuration from SCM.
add dc72782 HDDS-738. Removing REST protocol support from OzoneClient. Contributed by Elek, Marton.
add 1407414 HDDS-1094. Performance test infrastructure : skip writing user data on Datanode. Contributed by Supratim Deka (#1323)
add c7d426d HDDS-1881. Design doc: decommissioning in Ozone (#1196)
add 48cb583 HDFS-14710. RBF: Improve some RPC performance by using previous block. Contributed by xuzq.
add 29bd6f3 HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.
add addfb7f HDDS-1937. Acceptance tests fail if scm webui shows invalid json
add 2fcd0da HDDS-1942. Support copy during S3 multipart upload part creation
add aef6a4f HDDS-1950. S3 MPU part-list call fails if there are no parts
add 3e6a016 HDDS-2051. Rat check failure in decommissioning.md (#1372)
add 6f2226a YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).
add 872cdf4 HDDS-1941. Unused executor in SimpleContainerDownloader (#1367)
add 16576fd YARN-9754. Add support for arbitrary DAG AM Simulator. Contributed by Abhishek Modi.
add 371c9eb Revert "HDDS-1596. Create service endpoint to download configuration from SCM."
add c749f62 HDDS-2045. Partially started compose cluster left running
add 8c0759d YARN-9664. Improve response of scheduler/app activities for better understanding. Contributed by Tao Yang.
add 8e779a1 HDFS-14721. RBF: ProxyOpComplete is not accurate in FederationRPCPerformanceMonitor. Contributed by xuzq.
add f600fbb HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla.
add ffca734 HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.
add 3b22fcd HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.
add fcb7884 HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth.
add 7b3fa4f HDDS-2050. Error while compiling ozone-recon-web (#1374)
add 4f5f46e HDDS-1935. Improve the visibility with Ozone Insight tool (#1255)
add 7bebad6 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
add dc0acce YARN-9783. Remove low-level zookeeper test to be able to build Hadoop against zookeeper 3.5.5. Contributed by Mate Szalay-Beko.
add 9152712 Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314). Contributed by Lisheng Sun.
add e220dac HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.
add d59fc59 HADOOP-15998. Ensure jar validation works on Windows.
add 5b557de YARN-9714. ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby. Contributed by Tao Yang.
add 22a5861 HDDS-2058. Remove hadoop dependencies in ozone build
add 96f7dc1 HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.
add c929b38 YARN-9540. TestRMAppTransitions fails intermittently. Contributed by Tao Yang.
add ec34cee HDDS-2047. Datanodes fail to come up after 10 retries in a secure env… (#1379)
add 46696bd HDDS-2014. Create Symmetric Key for GDPR (#1362)
add 472a26d HDDS-2063. Integration tests create untracked file audit.log (#1384)
add 7085512 HDDS-2061. Add hdds.container.chunk.persistdata as exception to TestOzoneConfigurationFields (#1382)
add c4411f7 HDDS-2042. Avoid log on console with Ozone shell
add a2d083f HDDS-1413. Attempt to fix TestCloseContainerCommandHandler by adjusting timeouts
add eeccd2f YARN-9798. ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently. Contributed by Tao Yang.
add 7d998cb YARN-9800. TestRMDelegationTokens can fail in testRemoveExpiredMasterKeyInRMStateStore. Contributed by Adam Antal.
add c187d2c HDDS-2060. Create Ozone specific LICENSE file for the Ozone source package
add d2d963f YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam.
add c7ef4fb YARN-8678. Queue Management API - rephrase error messages. Contributed by Prabhu Joseph.
add fef65b4 YARN-9793. Remove duplicate sentence from TimelineServiceV2.md. Contributed by Julia Kinga Marton.
add 18d74fe HDFS-14711. RBF: RBFMetrics throws NullPointerException if stateStore disabled. Contributed by Chen Zhang.
add 751b5a1 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph.
add c3abfce HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena.
add bc27f7f YARN-7982. Do ACLs check while retrieving entity-types per application. Contribued by Prabhu Joseph.
add b939ae9 YARN-8174. Add containerId to ResourceLocalizationService fetch failure log statement. Contributed by Prabhu Joseph.
add f4d6e82 YARN-9400. Remove unnecessary if at EntityGroupFSTimelineStore#parseApplicationId. Contributed by Prabhu Joseph.
add 040f6e9 HDFS-14654. RBF: TestRouterRpc#testNamenodeMetrics is flaky. Contributed by Chen Zhang.
add d207aba Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell."
add 915cbc9 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
add 0348912 YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T.
add bb26514 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T.
add 5ff76cb HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
add b53d19a HDDS-1783 : Latency metric for applyTransaction in ContainerStateMachine (#1363).
add 0b9704f HDDS-1810. SCM command to Activate and Deactivate pipelines. (#1224)
add ac5a0ae HADOOP-16534. Exclude submarine from hadoop source build. (#1356)
add 0f549ec HDDS-2069. Default values of properties hdds.datanode.storage.utilization.{critical | warning}.threshold are not reasonable. (#1393)
add 91b01a1 HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
add 3c11716 HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
add f25fe92 HDDS-1909. Use new HA code for Non-HA in OM. (#1225)
add cfa41a49 HDDS-2018. Handle Set DtService of token for OM HA. (#1371)
add 0570475 HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar.
add 1ae7759 HDDS-2077. Add maven-gpg-plugin.version to pom.ozone.xml. (#1396)
add 337e9b7 HADOOP-16268. Allow StandbyException to be thrown as CallQueueOverflowException when RPC call queue is filled. Contributed by CR Hota.
add ae28747 HDDS-2079. Fix TestSecureOzoneManager. Contributed by Xiaoyu Yao. (#1400)
add 2f70b52 HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq.
add 3db7184 YARN-9804. Update ATSv2 document for latest feature supports.
add f347c34 Add support for checksum verification in data scrubber
add 172bcd8 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi.
add 2b16d53 HDDS-1898. GrpcReplicationService#download cannot replicate the container. (#1326)
add 511df1e HADOOP-16430. S3AFilesystem.delete to incrementally update s3guard with deletions
add 72d8b92 HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
add 2e2e540 YARN-9718. Fixed yarn.service.am.java.opts shell injection. Contributed by Eric Yang
add 0ccf4b0 YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta
add acbea8d HDDS-1708. Add container scrubber metrics. Contributed by Hrishikesh Gadre.
add 494d75e HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.
add 6e4cdf8 HDDS-1561: Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove (#1401)
add fa7f03f HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
add d98c548 HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
add ae42c8c HDFS-4819. [Dynamometer] Fix parsing of audit logs which contain = in path names. Contributed by Soya Miyoshi.
add e7d44e4 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar
add c92a3e9 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.
add 9637097 HDFS-14817. [Dynamometer] Fix start script options parsing which incorrectly interpret options starting with h as a help argument. Contributed by Soya Miyoshi.
add a234175 HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang.
add b71a7f1 YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar
add b15c116 HDDS-2015. Encrypt/decrypt key using symmetric key while writing/reading
add bb0b922 HADOOP-16255. Add ChecksumFs.rename(path, path, boolean)
add 5d497ab YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph.
add 7357570 YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li.
add c46d43a HDDS-1553. Add metrics in rack aware container placement policy. (#1361)
add 34b82e6 YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda)
add e8ca4fa YARN-9812. mvn javadoc:javadoc fails in hadoop-sls. Contributed by Abhishek Modi.
add 162af6f YARN-9784. org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue is flaky. Contributed by Julia Kinga Marton.
add ca32917 HDDS-2100. Ozone TokenRenewer provider is incorrectly configured. Contributed by Jitendra Nath Pandey.
add 43e389b YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar
add 3b9584d HDDS-2087. Remove the hard coded config key in ChunkManager (#1409)
add 387c332 YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph
add 60af879 YARN-9821. NM hangs at serviceStop when ATSV2 Backend Hbase is Down. Contributed by Prabhu Joseph.
add 147f986 HADOOP-16438. ADLS Gen1 OpenSSL config control.
add 469165e HDDS-1843. Undetectable corruption after restart of a datanode. Contributed by Shashikant Banerjee(#1364).
add d69b811 HDDS-2102. HddsVolumeChecker should use java optional in place of Guava optional. Contributed by Mukul Kumar Singh. (#1416)
add 650c4ce HDDS-2053. Fix TestOzoneManagerRatisServer failure. Contributed by Xiaoyu Yao. (#1373)
add bc2d3a7 HADOOP-16549. Remove Unsupported SSL/TLS Versions from Docs/Properties. Contributed by Daisuke Kobayashi.
add c3beeb7 HDDS-2048: State check during container state transition in datanode should be lock protected (#1375)
add 38c1a10 HADOOP-16542. Update commons-beanutils version to 1.9.4. Contributed by kevin su.
add dc9abd2 HADOOP-16554. mvn javadoc:javadoc fails in hadoop-aws.
add 10144a5 YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API. Contributed by Prabhu Joseph
add f8f8598 YARN-9824. Fall back to configured queue ordering policy class name
add 524b553 HDFS-14835. RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager. (#1414)
add dacc448 HADOOP-15184. Add GitHub pull request template. (#1419)
add c255333 HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li
add 5a381f7 HDDS-2103. TestContainerReplication fails due to unhealthy container (#1421)
add 9221704 HADOOP-16490. Avoid/handle cached 404s during S3A file creation.
add 64ed6b1 HDDS-2075. Tracing in OzoneManager call is propagated with wrong parent (#1415)
add 56b7571 YARN-9815 ReservationACLsTestBase fails with NPE. Contributed by Ahmed Hussein
add f537410 HDDS-2106. Avoid usage of hadoop projects as parent of hdds/ozone
add 3b06f0b YARN-9819. Make TestOpportunisticContainerAllocatorAMService more resilient. Contribued by Abhishek Modi
add 68612a0 HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
add 44850f6 YARN-9816. EntityGroupFSTimelineStore#scanActiveLogs fails when undesired files are present under /ats/active. Contribued by Prabhu Joseph.
add 4e273a3 HADOOP-16423. S3Guard fsck: Check metadata consistency between S3 and metadatastore (log) (#1208). Contributed by Gabor Bota.
add f4f9f0f HADOOP-16562. [pb-upgrade] Update docker image to have 3.7.1 protoc executable (#1429). Contributed by Vinayakumar B.
add d1c303a HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
add 2ff2a7f HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
add fe8cdf0 HDDS-2076. Read fails because the block cannot be located in the container (#1410)
add 1505d3f HADOOP-16566. S3Guard fsck: Use org.apache.hadoop.util.StopWatch instead of com.google.common.base.Stopwatch (#1433). Contributed by Gabor Bota.
add 4852a90 HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
add 4a9a6a2 HDDS-2122. Broken logo image on category sub-pages (#1437)
add 39e82ac HADOOP-16562. [pb-upgrade] Update docker image to have 3.7.1 protoc executable (#1429).
add 95010a4 HDDS-2057. Incorrect Default OM Port in Ozone FS URI Error Message.
add 4239007 YARN-9787. Typo in analysesErrorMsg. Contributed by kevin su.
add 06ad654 HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
add 8024818 HDDS-1786 : Datanodes takeSnapshot should delete previously created snapshots (#1163)
add 1843c46 HADOOP-16555. Update commons-compress to 1.19. (#1425) Contributed by YiSheng Lien.
add e22a324 HDDS-2007. Make ozone fs shell command work with OM HA service ids (#1360)
add 6a9f7ca Revert "HDDS-2057. Incorrect Default OM Port in Ozone FS URI Error Message."
add 9a931b8 HDDS-2125. maven-javadoc-plugin.version is missing in pom.ozone.xml
add e04b8a4 HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
add da2b4fe YARN-9794. RM crashes due to runtime errors in TimelineServiceV2Publisher. Contributed by Tarun Parimi.
add 85b1c72 HDDS-2129. Using dist profile fails with pom.ozone.xml as parent pom (#1449)
add 1e13fe6 HDDS-2044.Remove 'ozone' from the recon module names.
add b633438 HDDS-2096. Ozone ACL document missing AddAcl API
add e952ecf HDDS-2109. Refactor scm.container.client config
add 363373e HDDS-2124. Random next links
add 56f042c HDDS-2078. Get/Renew DelegationToken NPE after HDDS-1909
add c8d61ff HDDS-2030. Generate simplifed reports by the dev-support/checks/*.sh scripts
add 4222b62 HDDS-1879. Support multiple excluded scopes when choosing datanodes in NetworkTopology (#1194)
add 66bd168 HDDS-2107. Datanodes should retry forever to connect to SCM in an unsecure environment (#1424)
add 2358e53 HDDS-2111. XSS fragments can be injected to the S3g landing page
add 292bce7 HDDS-2114. Rename does not preserve non-explicitly created interim directories. Contributed by Lokesh Jain & Istvan Fajth.
add 4f67081 HDDS-2098 : Ozone shell command prints out ERROR when the log4j file … (#1411)
add e54977f HDDS-2132. TestKeyValueContainer is failing (#1457).
add 3a549ce HDDS-2120. Remove hadoop classes from ozonefs-current jar
add 55ce454 HADOOP-16371: Option to disable GCM for SSL connections when running on Java 8.
add f3de141 HDDS-2135. OM Metric mismatch (MultipartUpload failures)
add 7f90731 HDDS-2117. ContainerStateMachine#writeStateMachineData times out. (#1430)
add c474e24 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko.
add eefe9bc YARN-9837. Fixed reading YARN Service JSON spec file larger than 128k. Contributed by Tarun Parimi
add f580a87 HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
add 3cf6e42 HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs
add a9ba2b6 HDDS-2142. OM metrics mismatch (abort multipart request) (#1461)
add b88b682 HDDS-2136. OM block allocation metric not paired with its failures (#1460)
add 0dbfc4d HDDS-2139. Update BeanUtils and Jackson Databind dependency versions (#1456)
add f8c1432 YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu.
add 01d7924 YARN-9814. JobHistoryServer can't delete aggregated files, if remote app root directory is created by NodeManager. Contributed by Adam Antal.
add e97f0f1 HADOOP-16565. Region must be provided when requesting session credentials or SdkClientException will be thrown (#1454). Contributed by Gabor Bota.
add 15fded2 HDDS-2022. Add additional freon tests
add 419dd0f HDDS-2134. OM Metrics graphs include empty request type
add 087ed86 HDDS-2138. OM bucket operations do not add up
add 285ed0a HDDS-2137. HddsClientUtils and OzoneUtils have duplicate verifyResourceName()
add 6d4b20c HDDS-2143. Rename classes under package org.apache.hadoop.utils
add 111b08a HDDS-2143. Rename classes under package org.apache.hadoop.utils
add c28e731 Wrong commit message: Revert second "HDDS-2143. Rename classes under package org.apache.hadoop.utils"
add 8d9e9ec HDDS-2065. Implement OMNodeDetails#toString
add 5dd859a HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
add 2c52d00 HDDS-2032. Ozone client should retry writes in case of any ratis/stateMachine exceptions. Contributed by Shashikant Banerjee (#1420).
add 5db32b8 HADOOP-16547. make sure that s3guard prune sets up the FS (#1402). Contributed by Steve Loughran.
add 53b57fd HDDS-2121. Create a shaded ozone filesystem (client) jar (#1452)
add f16eb09 HDDS-2144. MR job failing on secure Ozone cluster. (#1466)
add 4ed0aef HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
add ef478fe HDDS-730. ozone fs cli prints hadoop fs in usage
add 1029060 HDDS-2147. Include dumpstream in test report
add 5c963a7 HDDS-2016. Add option to enforce GDPR in Bucket Create command
add e78848f HDDS-2119. Use checkstyle.xml and suppressions.xml in hdds/ozone projects for checkstyle validation
add 28913f7 HDDS-2148. Remove redundant code in CreateBucketHandler.java
add c9900a0 HDDS-2141. Missing total number of operations
add d4205dc HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee
add 56248f9 HADOOP-16556. Fix some alerts raised by LGTM.
add f6d884c HDDS-2110. Arbitrary file can be downloaded with the help of ProfilerServlet
add f260b5a HDDS-2127. Detailed Tools doc not reachable
add 1ada99b HDDS-2151. Ozone client logs the entire request payload at DEBUG level (#1477)
add a79f286 HDFS-14609. RBF: Security should use common AuthenticationFilter. Contributed by Chen Zhang.
add da1c67e HDDS-1054. List Multipart uploads in a bucket (#1277)
add 126ef77 HDDS-2154. Fix Checkstyle issues (#1475)
add b7ae8a9 HDDS-2101. Ozone filesystem provider doesn't exist (#1473)
add 298f07a HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao.
add 9be448b HDDS-2156. Fix alignment issues in HDDS doc pages
add d072d33 HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao.
add 1654497 HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432)
add dd08346 HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463)
add 5553887 HDDS-1949. Missing or error-prone test cleanup. Contributed by Doroszlai, Attila.
add b3173e1 HDDS-2001. Update Ratis version to 0.4.0.
add 3f223be HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
add 5363730 HDDS-2157. checkstyle: print filenames relative to project root (#1485)
add d7d6ec8 HDDS-2128. Make ozone sh command work with OM HA service ids (#1445)
add aa93866 HDFS-14833. RBF: Router Update Doesn't Sync Quota. Contributed by Ayush Saxena.
add efed445 HADOOP-16589. [pb-upgrade] Update docker image to make 3.7.1 protoc as default (#1482). Contributed by Vinayakumar B.
add dbdc612 HDDS-2163. Add 'Replication factor' to the output of list keys (#1493)
add e02b102 HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB (#1332)
add a94aa1f HDDS-2150. Update dependency versions to avoid security vulnerabilities. (#1472)
add 659c888 HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
add 4c0a7a9 Make upstream aware of 3.2.1 release.
add 07c81e9 HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494). Contributed by Vinayakumar B.
add aa664d7 HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302). Contributed by Gabor Bota.
add 2b5fc95 HADOOP-16591 Fix S3A ITest*MRjob failures.
add c30e495 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.
add 3d78b12 YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar
add 3fd3d74 HDDS-2161. Create RepeatedKeyInfo structure to be saved in deletedTable
add 6cbe5d3 HDDS-2160. Add acceptance test for ozonesecure-mr compose. Contributed by Xiaoyu Yao. (#1490)
add 0a716bd HDDS-2159. Fix Race condition in ProfileServlet#pid.
add bfe1dac HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496)
add e8e7d7b HADOOP-16561. [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes (#1500)
add 8f1a135 HDDS-2081. Fix TestRatisPipelineProvider#testCreatePipelinesDnExclude. Contributed by Aravindan Vijayan. (#1506)
add 51c64b3 HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running
add 91f50b9 HDDS-2167. Hadoop31-mr acceptance test is failing due to the shading
add 43203b4 HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun.
add 816d3cb HDFS-14837. Review of Block.java. Contributed by David Mollitor.
add afa1006 HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.
add f16cf87 HDDS-2170. Add Object IDs and Update ID to Volume Object (#1510)
add eb96a30 HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.
add 66400c1 HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.
add c2731d4 YARN-9730. Support forcing configured partitions to be exclusive based on app node label
add 6917754 HDDS-2172.Ozone shell should remove description about REST protocol support. Contributed by Siddharth Wagle.
add a346381 HDDS-2168. TestOzoneManagerDoubleBufferWithOMResponse sometimes fails with out of memory error (#1509)
add 3f89084 HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs.
add bec0864 YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal
add c724577 YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko
add 8baebb5 HDDS-2171. Dangling links in test report due to incompatible realpath
add e6fb6ee HDDS-1738. Add nullable annotation for OMResponse classes
add e346e36 HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.
add 16f626f HDDS-2165. Freon fails if bucket does not exists
add c89d22d HADOOP-16602. mvn package fails in hadoop-aws.
add bdaaa3b HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina
add f647185 HDDS-2067. Create generic service facade with tracing/metrics/logging support
add 606e341 Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label
add 587a8ee HDFS-14874. Fix TestHDFSCLI and TestDFSShell test break because of logging change in mkdir (#1522). Contributed by Gabor Bota.
add 7b6219a HDDS-2182. Fix checkstyle violations introduced by HDDS-1738
add a3f6893 HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
add 1a2a352 HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya.
add 18a8c24 YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein
add 06998a1 HDDS-2180. Add Object ID and update ID on VolumeList Object. (#1526)
add b1e55cf HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He.
add 2adcc3c HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang.
add c55ac6a HDDS-2174. Delete GDPR Encryption Key from metadata when a Key is deleted
add b6ef8cc HDD-2193. Adding container related metrics in SCM.
add 0371e95 HDDS-2179. ConfigFileGenerator fails with Java 10 or newer
add 9bf7a6e HDDS-2149. Replace findbugs with spotbugs
add 2870668 Make upstream aware of 3.1.3 release.
add 8a9ede5 HADOOP-15616. Incorporate Tencent Cloud COS File System Implementation. Contributed by Yang Yu.
add a93a139 HDDS-2185. createmrenv failure not reflected in acceptance test result
add ce58c05 HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
add 13b427f HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable (#963) Contributed by Sahil Takiar.
add 14b4fbc HDDS-1146. Adding container related metrics in SCM. (#1541)
add 0d5d0b9 Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
add c4c8d5f HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
add 2200871 HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
add d8313b2 HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun.
add c0edc84 HADOOP-16548 : Disable Flush() over config
add 760b523 Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao."
add d6b0a8d HDDS-2183. Container and pipline subcommands of scmcli should be grouped
add a530ac3 HDDS-2153. Add a config to tune max pending requests in Ratis leader
add b46d823 HDDS-2202. Remove unused import in OmUtils
add 98ca07e HDDS-2207. Update Ratis to latest snapshot. Contributed by Shashikant Bannerjee. (#1550)
add 4d3c580 YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi.
add e5bba59 HDDS-2205. checkstyle.sh reports wrong failure count
add b3275ab HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko.
add 137546a YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph
add 8efd25b HDDS-1615. ManagedChannel references are being leaked in ReplicationSupervisor.java. Contributed by Mukul Kumar Singh. (#1547)
add 425a6c8 YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.
add 99cd757 YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph
add 6ef6594 HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. (#1370)
add 918b470 HDDS-2166. Some RPC metrics are missing from SCM prometheus endpoint
add 1921e94 HADOOP-16458. LocatedFileStatusFetcher.getFileStatuses failing intermittently with S3
add d947ded HADOOP-16619. Upgrade jackson and jackson-databind to 2.9.10 (#1554). Contributed by Siyao Meng.
add 770adc5 HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists
add d1ddba6 YARN-9862. Increase yarn-services-core test timeout value. Contributed by Prabhu Joseph
add 3df733c HDFS-14885. UI: Fix a typo on WebUI of DataNode. Contributed by Xieming Li.
add 41440ec HDDS-2210. ContainerStateMachine should not be marked unhealthy if applyTransaction fails with closed container exception(#1552).
add f1ba9bf HDDS-2187. ozone-mr test fails with No FileSystem for scheme "o3fs"
add 61a8436 YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi.
add 2e1fd44 HDDS-2201. Rename VolumeList to UserVolumeInfo. (#1566)
add 0d2d6f9 YARN-9792. Document examples of SchedulerConf with Node Labels. Contributed by Prabhu Joseph
add e8ae632 HDDS-2068. Make StorageContainerDatanodeProtocolService message based
add ffd4e52 HDDS-2073. Make SCMSecurityProtocol message based. Contributed by Elek, Marton.
add 685918e HDDS-2227. GDPR key generation could benefit from secureRandom. (#1574)
add 169cef7 HDDS-2162. Make OM Generic related configuration support HA style config. (#1511)
add 53ed78b HDDS-2224. Fix loadup cache for cache cleanup policy NEVER. (#1567)
add b09d389 HDDS-2019. Handle Set DtService of token in S3Gateway for OM HA. (#1489)
add 559ee27 HADOOP-16599. Allow a SignerInitializer to be specified along with a Custom Signer
add 1303255 HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang.
add 4c24f24 HDDS-2072. Make StorageContainerLocationProtocolService message based Contributed by Elek, Marton.
add c5665b2 HDDS-2228. Fix NPE in OzoneDelegationTokenManager#addPersistedDelegat… (#1571)
add 0e026cb HADOOP-16620. [pb-upgrade] Remove protocol buffers 3.7.1 from requirements in BUILDING.txt (#1558)
add c19fa3d HADOOP-16605. Fix testcase testSSLChannelModeConfig
add 5a7483c HDFS-14888. RBF: Enable Parallel Test Profile for builds. Contributed by Ayush Saxena.
add d59bcbf HDDS-2226. S3 Secrets should use a strong RNG. (#1572)
add a3fe404 HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C.
add 51eaeca HDDS-2211. Collect docker logs if env fails to start (#1553)
add 47d721d HDDS-2234. rat.sh fails due to ozone-recon-web/build files (#1580)
add 9446686 HDDS-2231. test-single.sh cannot copy results (#1575)
add 76605f1 HDDS-1720 : Add ability to configure RocksDB logs for Ozone Manager.
add b7cb8fe HDDS-2200 : Recon does not handle the NULL snapshot from OM DB cleanly.
add 1dde3ef HADOOP-16624. Upgrade hugo to the latest version in Dockerfile
add cdaa480 HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. (#1540)
add 9700e20 HDDS-2223. Support ReadWrite lock in LockManager. (#1564)
add 844b766 HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. (#1573)"
add c99a121 HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell.
add ec8f691 HDDS-2225. SCM fails to start in most unsecure environments due to leftover secure config
add b23bdaf HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina
add 2478cba YARN-9782. Avoid DNS resolution while running SLS. Contributed by Abhishek Modi.
add 4cf0b36 HDDS-2222 (#1578)
add a9849f6 Revert "HDDS-2222 (#1578)" (#1594)
add bffcd33 HDDS-2230. Invalid entries in ozonesecure-mr config
add d061c84 HDDS-2140. Add robot test for GDPR feature
add 6171a41 HDDS-2199. In SCMNodeManager dnsToUuidMap cannot track multiple DNs on the same host
add bca014b HDDS-2216. Rename HADOOP_RUNNER_VERSION to OZONE_RUNNER_VERSION in compose .env files.
add f44abc3 HADOOP-16207 Improved S3A MR tests.
add 531cc93 HDDS-2222. Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C. (#1595)
add f826420 HDDS-2230. Invalid entries in ozonesecure-mr config. (Addendum)
add 4510970 YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph
add 3f16651 HDDS-2237. KeyDeletingService throws NPE if it's started too early (#1584)
add aa24add HDFS-14890. Fixed namenode and journalnode startup on Windows. Contributed by Siddharth Wagle
add 6574f27 HADOOP-16570. S3A committers encounter scale issues.
add 10bdc59 HADOOP-16579. Upgrade to Apache Curator 4.2.0 excluding ZK (#1531). Contributed by Norbert Kalmár.
add f3eaa84 HDDS-2164 : om.db.checkpoints is getting filling up fast. (#1536)
add 8de4374 HDDS-2158. Fixing Json Injection Issue in JsonUtils. (#1486)
add a3cf54c HDDS-2250. Generated configs missing from ozone-filesystem-lib jars
add f209722 HDDS-2257. Fix checkstyle issues in ChecksumByteBuffer (#1603)
add fb1ecff Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph"
add 579dc87 HDDS-2251. Add an option to customize unit.sh and integration.sh parameters
add b8086bf HADOOP-16626. S3A ITestRestrictedReadAccess fails without S3Guard.
add 55c5436 Revert "HADOOP-16579. Upgrade to Apache Curator 4.2.0 excluding ZK (#1531). Contributed by Norbert Kalmár."
add 022fe5f HDDS-2169. Avoid buffer copies while submitting client requests in Ratis. Contributed by Tsz-wo Sze(#1517).
add 14cd969 HADOOP-16512. [hadoop-tools] Fix order of actual and expected expression in assert statements
add 7f332eb HDDS-2252. Enable gdpr robot test in daily build
add 1a77a15 HADOOP-16587. Make ABFS AAD endpoints configurable.
add 9685a6c HDDS-2239. Fix TestOzoneFsHAUrls (#1600)
add 382967b HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
add 1877312 HDDS-2238. Container Data Scrubber spams log in empty cluster
add cfba6ac HDDS-2264. Improve output of TestOzoneContainer
add aaa94c3 HDDS-2259. Container Data Scrubber computes wrong checksum
add 012d897 HDDS-2262. SLEEP_SECONDS: command not found
add 4fdf016 HDDS-2245. Use dynamic ports for SCM in TestSecureOzoneCluster Contributed by kevin su.
add 7615945 HDFS-14814. RBF: RouterQuotaUpdateService supports inherited rule. Contributed by Jinglun.
add 91320b4 HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.
add 15a9bee HDDS-2260. Avoid evaluation of LOG.trace and LOG.debug statement in the read/write path (HDDS). (#1612)
add 5462d21 YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena
add 72ae371 HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.
add 87d9f36 HDDS-2244. Use new ReadWrite lock in OzoneManager. (#1589)
add 1d27930 HDDS-2233 - Remove ByteStringHelper and refactor the code to the place where it used (#1596)
add be901f4 YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph
add 35f093f YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl
add 6f1ab95 YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl
add 1f954e6 HDDS-2217. Remove log4j and audit configuration from the docker-config files
add 4b0a5bc HDDS-2217. Remove log4j and audit configuration from the docker-config files
add b034350 Squashed commit of the following:
add 2d81abc HDDS-2265. integration.sh may report false negative
add d76e265 HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore.
add eeb58a0 HDFS-14898. Use Relative URLS in Hadoop HDFS HTTP FS. Contributed by David Mollitor.
add a031388 HDDS-2266. Avoid evaluation of LOG.trace and LOG.debug statement in the read/write path. (#1633)
add 104ccca HDFS-14900. Fix build failure of hadoop-hdfs-native-client. Contributed by Masatake Iwasaki.
add effe608 HADOOP-16650. ITestS3AClosedFS failing.
add 4850b3a HDDS-2269. Provide config for fair/non-fair for OM RW Lock. (#1623)
add 957253f HDDS-1984. Fix listBucket API. (#1555)
add 7a4b3d4 HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos.
add 31e0122 YARN-9860. Enable service mode for Docker containers on YARN Contributed by Prabhu Joseph and Shane Kumpf
add 9c72bf4 HDDS-1986. Fix listkeys API. (#1588)
add f267917 HDDS-2282. scmcli pipeline list command throws NullPointerException. Contributed by Xiaoyu Yao. (#1642)
add 4a700c2 HADOOP-16520. Race condition in DDB table init and waiting threads. (#1576). Contributed by Gabor Bota.
add 62b5cef YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal
add ec86f42 YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal
add c561a70 HDDS-2213.Reduce key provider loading log level in OzoneFileSystem#getAdditionalTokenIssuers (#1556)
add 44bc095 HADOOP-13907. Fix TestWebDelegationToken#testKerberosDelegationTokenAuthenticator on Windows. Contributed by Kitti Nanasi.
add 6e5cd52 HDFS-14899. Use Relative URLS in Hadoop HDFS RBF. Contributed by David Mollitor.
add 5f4641a HDFS-14238. A log in NNThroughputBenchmark should change log level to INFO instead of ERROR. Contributed by Shen Yinjie.
add 5cc7873 YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko
add dee9e97 Revert "HADOOP-15870. S3AInputStream.remainingInFile should use nextReadPos."
add 74e5018 HADOOP-16635. S3A "directories only" scan still does a HEAD.
add fabd41f HDFS-14856. Fetch file ACLs while mounting external store. (#1478)
add 336abbd HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina.
add a8ef03e YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R
add 85af77c HADOOP-16643. Update netty4 to the latest 4.1.42. Contributed by Lisheng Sun.
add c39e9fc HADOOP-15169. "hadoop.ssl.enabled.protocols" should be considered in httpserver2. Contributed by Brahma Reddy Battula, Wei-Chiu Chuang.
add bbcf0b9 HADOOP-16478. S3Guard bucket-info fails if the caller lacks s3:GetBucketLocation.
add c79a5f2 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal
add 090f73a YARN-9900. Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format. Contributed by Prabhu Joseph.
add 375224e HDFS-14739. RBF: LS command for mount point shows wrong owner and permission information. Contributed by Jinglun.
add 9a8edb0 HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI
add a5034c7 YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R.
add 3990ffa YARN-9909. Offline format of YarnConfigurationStore. Contributed by Prabhu Joseph.
add 5527d79 HDFS-14810. Review FSNameSystem editlog sync. Contributed by Xiaoqiao He.
add 54dc6b7 HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore.
add f0699a7 YARN-9841. Capacity scheduler: add support for combined %user + %primary_group mapping. Contributed by Manikandan R
add 3d41f33 HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng.
add 6d92aa7 HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 (#1656). Contributed by Norbert Kalmár, Mate Szalay-Beko
add 72b1bed YARN-9884. Make container-executor mount logic modular Contributed by Eric Badger
add 155864d YARN-9875. Improve fair scheduler configuration store on HDFS. Contributed by Prabhu Joseph
add 2ae4b33 HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun.
add e6f95eb HDFS-14887. RBF: In Router Web UI, Observer Namenode Information displaying as Unavailable. Contributed by hemanthboyina.
add 447f46d HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui.
add ff6a492 HDFS-14916. RBF: line breaks are missing from the output of 'hdfs dfsrouteradmin -ls'. Contributed by Xieming Li.
add 74c2329 HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena.
add 3f7756d YARN-9922. Fixed YARN javadoc errors from YARN-9699. Contributed by Peter Bacsko
add 83d1480 YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph.
add c4e27ef HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.
add 72003b1 HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.
add 19f35cf HDFS-14918. Remove useless getRedundancyThread from BlockManagerTestUtil. Contributed by Fei Hui.
add 6020505 HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena.
add a901405 HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh.
add 1d5d7d0 HADOOP-16658. S3A connector does not support including the token renewer in the token identifier.
add fd84ca5 YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi
add ee699dc HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena.
add ac6b6a6 HADOOP-16614. Add aarch64 support for dependent leveldbjni. Contributed by liusheng close #1546
add 2eba2624 HADOOP-16615. Add password check for credential provider,
add a1b4eeb HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. (#1672)
add b41394e HDFS-14638. [Dynamometer] Fix scripts to refer to current build structure. (#1673)
add 0db0f1e HDFS-14917. Change the ICON of "Decommissioned & dead" datanode on "dfshealth.html". Contributed by Xieming Li
add 8625265 HDFS-14933. Fixing a typo in documentation of Observer NameNode. Contributed by Xieming Li
add eef34f2 YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan
add 30db895 HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() (#1667)
add 7be5508 HDFS-14923. Remove dead code from HealthMonitor. Contributed by Fei Hui.
add d5e9971 HADOOP-16653. S3Guard DDB overreacts to no tag access (#1660). Contributed by Gabor Bota.
add 9ef6ed9 HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger
add 30ed24a HDFS-14730. Removed unused configuration dfs.web.authentication.filter. Contributed by Chen Zhang
add fa4904c HDFS-14935. Refactor DFSNetworkTopology#isNodeInScope. Contributed by Lisheng Sun.
add ed45c13 YARN-2442. ResourceManager JMX UI does not give HA State. Contributed by Rohith Sharma and Cyrus Jackson.
add e32ab5e HDFS-14907. [Dynamometer] Add JUnit JAR to classpath for SimulatedDataNodes (#1680)
add bd498ba HADOOP-16669. TestRawLocalFileSystemContract.testPermission fails if no native library.
add 012756a HADOOP-16675. Upgrade jackson-databind to 2.9.10.1. Contributed by Lisheng Sun.
add e3e7daa YARN-9889. [UI] Add Application Tag column to RM All Applications table. Contributed by Kinga Marton
add 9a2e43e YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph.
add e6137d0 Make upstream aware of 2.10.0 release
add 8c9173c HADOOP-16510. [hadoop-common] Fix order of actual and expected expression in assert statements. Contributed by Adam Antal
add d9fbedc HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun.
add 9d25ae7 HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.
add f9b99d2 HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng.
add ef9d12d Add 2.10.0 release notes for HDFS-12943
add 477505c HDFS-14824. [Dynamometer] Dynamometer in org.apache.hadoop.tools does not output the benchmark results. (#1685)
add 02009c3 HDFS-14768. EC : Busy DN replica should be consider in live replica check. Contributed by guojh.
add f18bbdd HDFS-14927. RBF: Add metrics for async callers thread pool. Contributed by Leon Gao.
add 7d7acb0 HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.
add de6b8b0 HDFS-14925. Rename operation should check nest snapshot (#1670)
add d462308 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph.
add dca19fc HADOOP-16484. S3A to warn or fail if S3Guard is disabled (#1661). Contributed by Gabor Bota.
add eb73ba6 HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.
add 51e7d1b YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph.
add 2ffec34 HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.
add 6f0190d HADOOP-16678: Review of ArrayWritable (#1692)
add 7d0addd YARN-9689: Support proxy user for Router to support kerberos (#1654)
add b643a1c HDFS-14938. Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Lisheng Sun.
add 7901062 HDFS-14891. RBF: namenode links in NameFederation Health page (federationhealth.html) cannot use https scheme. Contributed by Xieming Li
add d17ba85 HADOOP-16681. mvn javadoc:javadoc fails in hadoop-aws. Contributed by Xieming Li
add bfb8f28 HDFS-14775. Add Timestamp for longest FSN write/read lock held log. Contributed by Chen Zhang.
add ed302f1 MAPREDUCE-7208. Tuning TaskRuntimeEstimator. (Ahmed Hussein via jeagles)
add ee8addb Revert "HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng."
add c360141 HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
add dcf5583 HDFS-14880. Correct the sequence of statistics & exit message in balencer. Contributed by Renukaprasad C.
add 9e28705 HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
add 9d0d580 HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.
add dd90025 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.
add 247584e HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.
add 42fc888 HDFS-14815. RBF: Update the quota in MountTable when calling setQuota on a MountTable src. Contributed by Jinglun.
add 31f172f YARN-9677. Make FpgaDevice and GpuDevice classes more similar to each other. Contributed by kevin su
add ceb9c61 YARN-9890. [UI2] Add Application tag to the app table and app detail page. Contributed by Julia Kinga Marton
add 3d24930 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal
add 320008b HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.
add 6663d6a HDFS-14928. UI: unifying the WebUI across different components. Contributed by Xieming Li
add b25a37c HDFS-14962. RBF: ConnectionPool#newConnection() error log wrong protocol class (#1699). Contributed by Yuxuan Wang.
add 77934bc HDFS-14975. Add CR for SetECPolicyCommand usage. Contributed by Fei Hui.
add 516377b YARN-9965. Fix NodeManager failing to start when Hdfs Auxillary Jar is set. Contributed by Prabhu Joseph.
add 30b93f9 YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R
add b988487 HADOOP-16701. Fix broken links in site index.
add a98c4df HADOOP-16702. Move documentation of hadoop-cos to under src directory.
add fb512f5 YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi.
add f6697aa HADOOP-16477. S3A delegation token tests fail if fs.s3a.encryption.key set.
add b83b9ab YARN-9537. Add configuration to disable AM preemption. Contributed by ZhouKang
add 990063d HADOOP-16665. Filesystems to be closed if they failed during initialize().
add 40150da HDFS-14922. Prevent snapshot modification time got change on startup. Contributed by hemanthboyina.
add 3ab98ee HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng.
add 97ec34e HDFS-14959: [SBNN read] access time should be turned off (#1706) Contributed by Chao Sun.
add df6b316 HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. (#1687)
add 586defe HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.
add 73a386a HADOOP-16585. [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write. Contributed by Ranith Sardar.
add a139a87 HADOOP-16708. HadoopExecutors cleanup to only log at debug.
add c73334a HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log.
add d0302d3 HADOOP-16691. Unify Logging in UserGroupInformation.
add 92c28c1 HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li.
add b2cc8b6 HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
add 9f0610f HADOOP-16654:Delete hadoop-ozone and hadoop-hdds subprojects from apache trunk
add c892a87 HDFS-14882. Consider DataNode load when #getBlockLocation. Contributed by Xiaoqiao He.
add 67f2c49 HDFS-14802. The feature of protect directories should be used in RenameOp (#1669)
add b3119b9 HDFS-14648. Implement DeadNodeDetector basic model. Contributed by Lisheng Sun.
add 96c4520 HADOOP-16710. Testing_azure.md documentation is misleading.
add bd45434 YARN-9982. Fix Container API example link in NodeManager REST API doc. Contributed by Charan Hebri.
add 3b5a0e8 HDFS-14974. RBF: Make tests use free ports. Contributed by Inigo Goiri.
add 12617fa HDFS-14955. RBF: getQuotaUsage() on mount point should return global quota. Contributed by Jinglun.
add cad5408 HADOOP-16484. S3A to warn or fail if S3Guard is disabled - addendum: silent for S3GuardTool (#1714). Contributed by Gabor Bota.
add 34cb595 HDFS-14967. TestWebHDFS fails in Windows. Contributed by Renukaprasad C.
add 7f81172 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li.
add 2764236 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko
add 289bbca YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger
add 0e22e9a YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger
add 4050471 HADOOP-16657. Move remaining log4j APIs over to slf4j in hadoop-common. Contributed by Minni Mittal.
add dc3f4fc YARN-9965. Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set (addendum). Contributed by Prabhu Joseph.
add 215f205 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg.
add ea68756 YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg.
add 3cecb2a HADOOP-16110 Upgrade to yetus 0.11.1 and use emoji vote on github pre commit (#1527). Contributed by Duo Zhang.
add 30b2096 YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam.
add 9fbfe6c HADOOP-16632 Speculating & Partitioned S3A magic committers can leave pending files under __magic (#1599)
add b033c68 HADOOP-16612. Track Azure Blob File System client-perceived latency
add c870514 HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina.
add ea8ffac YARN-9983. Fixed typo in YARN Service overview. Contributed by Denes Gerencser
add dfdc6d6 YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton
add 0b50aa2 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
add dfc61d8 HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724)
add 1415ad3 HADOOP-16707. NPE in UGI.getCurrentUser in ITestAbfsIdentityTransformer setup.
add be77231 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq.
add fd264b8 HDFS-14995. Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime(). Contributed by Lisheng Sun.
add 3037762 HDFS-14949. Add getServerDefaults() support to HttpFS. Contributed by hemanthboyina.
add 9664b9c HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang.
add 6f899e9 HADOOP-15852. Refactor QuotaUsage. Contributed by David Mollitor.
add 1a0c0e4 HADOOP-16276. Fix jsvc startup command in hadoop-functions.sh due to jsvc >= 1.0.11 changed default current working directory (#1272)
add de38045 HADOOP-16687. ABFS: Fix testcase added for HADOOP-16138 for namespace enabled account. (#1701)
add 98d249d HDFS-14996. RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations. Contributed by Ayush Saxena.
add 2627019 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina.
add b89fd4d HDFS-15002. RBF: Fix annotation in RouterAdmin. Contributed by Jinglun.
add 9b6906f HDFS-14651. DeadNodeDetector checks dead node periodically. Contributed by Lisheng Sun.
add b25e94c HDFS-14924. RenameSnapshot not updating new modification time. Contributed by hemanthboyina
add c422e36 HDFS-13842. RBF: Exceptions are conflicting when creating the same mount entry twice. Contributed by Ranith Sardar.
add 8afabe4 YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu.
add 4c1a128 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi
add 049940e HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar.
add da94351 HADOOP-16717. Remove GenericsUtil isLog4jLogger dependency on Log4jLoggerAdapter. Contributed by Xieming Li.
add f8e36e0 YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709)
add 6c20512 Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger"
add 448ffb1 YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
add ea25f4d HADOOP-16709. S3Guard: Make authoritative mode exclusive for metadata - don't check for expiry for authoritative paths (#1721). Contributed by Gabor Bota.
add 52e9ee3 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak
add 3161813 HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists (#1695)
add 8c9018d YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko
add aa7ab27 YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth.
add 828ab40 YARN-9362. Code cleanup in TestNMLeveldbStateStoreService. Contributed by Denes Gerencser
add ef950b0 YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph
add c8bef4d HDFS-14649. Add suspect probe for DeadNodeDetector. Contributed by Lisheng Sun.
add 7f2ea2a MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER.
add 9e69628 HADOOP-16455. ABFS: Implement FileSystem.access() method.
add 82ad9b5 HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.
add 2b452b4 HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago.
add 013c6e5 MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure.
add c3659f8 HDFS-15019. Refactor the unit test of TestDeadNodeDetection. Contributed by Lisheng Sun.
add 46166bd HDFS-14961. [SBN read] Prevent ZKFC changing Observer Namenode state. Contributed by Ayush Saxena.
add 0384687 HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore.
add 44f7b91 HDFS-15013. Reduce NameNode overview tab response time. Contributed by HuangTao.
add a2dadac YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph.
add 6b2d6d4 HDFS-15009. FSCK -list-corruptfileblocks return Invalid Entries. Contributed by hemanthboyina.
new 4f7bd41 HDFS-13310. The DatanodeProtocol should have a DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
new 1abb8f3 HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode. Contributed by Ewan Higgs.
new 30607ec HDFS-13794. [PROVIDED Phase 2] Teach BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
new afdb20b HDFS-12090. Fixup TestBlockReportLease and TestBPOfferService in branch rebase.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (64da704)
\
N -- N -- N refs/heads/HDFS-12090 (afdb20b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/pull_request_template.md | 6 +
.gitignore | 2 +
BUILDING.txt | 99 +-
Jenkinsfile | 212 +
LICENSE-binary | 522 +
LICENSE.txt | 2698 +-
NOTICE-binary | 840 +
NOTICE.txt | 671 +-
README.txt | 28 +-
dev-support/bin/create-release | 2 +-
dev-support/bin/dist-copynativelibs | 8 +
dev-support/bin/dist-layout-stitching | 5 +
dev-support/bin/yetus-wrapper | 26 +-
dev-support/docker/Dockerfile | 25 +-
hadoop-assemblies/pom.xml | 2 +-
.../src/main/resources/assemblies/hadoop-dist.xml | 13 +
.../assemblies/hadoop-dynamometer-blockgen.xml | 35 +
.../assemblies/hadoop-dynamometer-infra.xml | 35 +
.../assemblies/hadoop-dynamometer-workload.xml | 35 +
.../resources/assemblies/hadoop-dynamometer.xml | 73 +
...-src-with-hdds.xml => hadoop-src-submarine.xml} | 0
.../src/main/resources/assemblies/hadoop-src.xml | 9 +-
.../src/main/resources/assemblies/hadoop-tools.xml | 2 +-
.../main/resources/assemblies/hadoop-yarn-dist.xml | 5 +
hadoop-build-tools/pom.xml | 2 +-
.../src/main/resources/checkstyle/checkstyle.xml | 4 +-
.../src/main/resources/checkstyle/suppressions.xml | 4 +-
hadoop-client-modules/hadoop-client-api/pom.xml | 2 +-
.../hadoop-client-check-invariants/pom.xml | 5 +-
.../resources/ensure-jars-have-correct-contents.sh | 31 +-
.../hadoop-client-check-test-invariants/pom.xml | 5 +-
.../resources/ensure-jars-have-correct-contents.sh | 33 +-
.../hadoop-client-integration-tests/pom.xml | 2 +-
.../hadoop-client-minicluster/pom.xml | 22 +-
.../hadoop-client-runtime/pom.xml | 12 +-
hadoop-client-modules/hadoop-client/pom.xml | 2 +-
hadoop-client-modules/pom.xml | 2 +-
.../hadoop-cloud-storage/pom.xml | 2 +-
.../hadoop-cos/dev-support/findbugs-exclude.xml | 0
hadoop-cloud-storage-project/hadoop-cos/pom.xml | 140 +
.../java/org/apache/hadoop/fs/cosn/BufferPool.java | 245 +
.../hadoop/fs/cosn/ByteBufferInputStream.java | 89 +
.../hadoop/fs/cosn/ByteBufferOutputStream.java | 74 +
.../apache/hadoop/fs/cosn/ByteBufferWrapper.java | 103 +
.../java/org/apache/hadoop/fs/cosn/Constants.java | 43 +
.../main/java/org/apache/hadoop/fs/cosn/CosN.java | 41 +
.../org/apache/hadoop/fs/cosn/CosNConfigKeys.java | 86 +
.../apache/hadoop/fs/cosn/CosNCopyFileContext.java | 66 +
.../apache/hadoop/fs/cosn/CosNCopyFileTask.java | 68 +
.../apache/hadoop/fs/cosn/CosNFileReadTask.java | 125 +
.../org/apache/hadoop/fs/cosn/CosNFileSystem.java | 814 +
.../org/apache/hadoop/fs/cosn/CosNInputStream.java | 365 +
.../apache/hadoop/fs/cosn/CosNOutputStream.java | 284 +
.../java/org/apache/hadoop/fs/cosn/CosNUtils.java | 167 +
.../hadoop/fs/cosn/CosNativeFileSystemStore.java | 768 +
.../org/apache/hadoop/fs/cosn/FileMetadata.java | 68 +
.../hadoop/fs/cosn/NativeFileSystemStore.java | 99 +
.../org/apache/hadoop/fs/cosn/PartialListing.java | 64 +
.../main/java/org/apache/hadoop/fs/cosn/Unit.java | 32 +
.../fs/cosn/auth/COSCredentialProviderList.java | 139 +
.../EnvironmentVariableCredentialProvider.java | 55 +
.../fs/cosn/auth/NoAuthWithCOSException.java | 37 +
.../fs/cosn/auth/SimpleCredentialProvider.java | 54 +
.../apache/hadoop/fs/cosn/auth/package-info.java | 18 +
.../org/apache/hadoop/fs/cosn/package-info.java | 18 +
.../src/site/markdown/cloud-storage/index.md | 367 +
.../hadoop-cos}/src/site/resources/css/site.css | 0
.../apache/hadoop/fs/cosn/CosNTestConfigKey.java | 33 +
.../org/apache/hadoop/fs/cosn/CosNTestUtils.java | 78 +
.../apache/hadoop/fs/cosn/TestCosNInputStream.java | 167 +
.../hadoop/fs/cosn/TestCosNOutputStream.java | 87 +
.../hadoop/fs/cosn/contract/CosNContract.java | 46 +
.../fs/cosn/contract/TestCosNContractCreate.java | 32 +
.../fs/cosn/contract/TestCosNContractDelete.java | 32 +
.../fs/cosn/contract/TestCosNContractDistCp.java | 54 +
.../contract/TestCosNContractGetFileStatus.java | 33 +
.../fs/cosn/contract/TestCosNContractMkdir.java | 32 +
.../fs/cosn/contract/TestCosNContractOpen.java | 32 +
.../fs/cosn/contract/TestCosNContractRename.java | 32 +
.../fs/cosn/contract/TestCosNContractRootDir.java | 33 +
.../fs/cosn/contract/TestCosNContractSeek.java | 32 +
.../hadoop/fs/cosn/contract/package-info.java | 18 +
.../src/test/resources/contract/cosn.xml | 120 +
.../hadoop-cos/src/test/resources/core-site.xml | 107 +
.../hadoop-cos/src/test/resources/log4j.properties | 18 +
hadoop-cloud-storage-project/pom.xml | 3 +-
hadoop-common-project/hadoop-annotations/pom.xml | 2 +-
hadoop-common-project/hadoop-auth-examples/pom.xml | 2 +-
hadoop-common-project/hadoop-auth/pom.xml | 2 +-
.../server/AuthenticationFilter.java | 2 +-
.../server/KerberosAuthenticationHandler.java | 37 +
.../server/LdapAuthenticationHandler.java | 7 +-
.../security/authentication/util/KerberosName.java | 2 +-
.../security/authentication/util/KerberosUtil.java | 49 +-
.../util/ZKSignerSecretProvider.java | 5 +-
.../hadoop-auth/src/site/markdown/Examples.md | 6 +-
.../server/TestKerberosAuthenticationHandler.java | 24 +
.../hadoop-common/HadoopCommon.cmake | 4 +-
.../dev-support/findbugsExcludeFile.xml | 7 +
.../jdiff/Apache_Hadoop_Common_2.10.0.xml | 40847 +++++++++++++++++++
.../jdiff/Apache_Hadoop_Common_2.6.0.xml | 2 +-
hadoop-common-project/hadoop-common/pom.xml | 181 +-
.../hadoop-common/src/CMakeLists.txt | 21 +
.../hadoop-common/src/config.h.cmake | 1 +
.../hadoop-common/src/main/bin/hadoop | 4 +
.../hadoop-common/src/main/bin/hadoop-functions.sh | 34 +-
.../hadoop-common/src/main/bin/hadoop.cmd | 5 +
.../hadoop-common/src/main/conf/hadoop-policy.xml | 10 +
.../hadoop-common/src/main/conf/log4j.properties | 49 +-
.../java/org/apache/hadoop/conf/Configuration.java | 75 +-
.../org/apache/hadoop/conf/ReconfigurableBase.java | 3 +-
.../apache/hadoop/crypto/CryptoInputStream.java | 340 +-
.../hadoop/crypto/OpensslAesCtrCryptoCodec.java | 4 +-
.../org/apache/hadoop/crypto/key/KeyShell.java | 2 +-
.../key/kms/LoadBalancingKMSClientProvider.java | 3 +
.../apache/hadoop/crypto/key/kms/ValueQueue.java | 14 +-
.../hadoop/crypto/random/OsSecureRandom.java | 9 +-
.../org/apache/hadoop/fs/AbstractFileSystem.java | 29 +-
.../java/org/apache/hadoop/fs/BlockLocation.java | 36 +-
.../hadoop/fs/ByteBufferPositionedReadable.java | 90 +
.../org/apache/hadoop/fs/ByteBufferReadable.java | 17 +-
.../org/apache/hadoop/fs/CachingGetSpaceUsed.java | 34 +-
.../org/apache/hadoop/fs/ChecksumFileSystem.java | 114 +
.../main/java/org/apache/hadoop/fs/ChecksumFs.java | 26 +
.../apache/hadoop/fs/CommonConfigurationKeys.java | 41 +-
.../hadoop/fs/CommonConfigurationKeysPublic.java | 40 +-
.../apache/hadoop/fs/CommonPathCapabilities.java | 126 +
.../org/apache/hadoop/fs/DelegateToFileSystem.java | 7 +
.../apache/hadoop/fs/DelegationTokenRenewer.java | 4 +-
.../org/apache/hadoop/fs/FSDataInputStream.java | 34 +-
.../java/org/apache/hadoop/fs/FileContext.java | 23 +-
.../main/java/org/apache/hadoop/fs/FileSystem.java | 192 +-
.../main/java/org/apache/hadoop/fs/FileUtil.java | 17 +-
.../org/apache/hadoop/fs/FilterFileSystem.java | 11 +
.../main/java/org/apache/hadoop/fs/FilterFs.java | 5 +
.../main/java/org/apache/hadoop/fs/FsShell.java | 4 +-
.../java/org/apache/hadoop/fs/FsUrlConnection.java | 15 +-
.../java/org/apache/hadoop/fs/GetSpaceUsed.java | 31 +-
.../main/java/org/apache/hadoop/fs/Globber.java | 208 +-
.../java/org/apache/hadoop/fs/HarFileSystem.java | 19 +-
.../src/main/java/org/apache/hadoop/fs/Path.java | 17 +-
.../org/apache/hadoop/fs/PathCapabilities.java | 61 +
.../main/java/org/apache/hadoop/fs/QuotaUsage.java | 126 +-
.../org/apache/hadoop/fs/RawLocalFileSystem.java | 19 +
.../org/apache/hadoop/fs/StreamCapabilities.java | 12 +
.../src/main/java/org/apache/hadoop/fs/Trash.java | 6 +
.../java/org/apache/hadoop/fs/TrashPolicy.java | 5 +
.../org/apache/hadoop/fs/TrashPolicyDefault.java | 22 +-
.../hadoop/fs/http/AbstractHttpFileSystem.java | 18 +
.../apache/hadoop/fs/impl/FsLinkResolution.java | 98 +
.../apache/hadoop/fs/impl/FunctionsRaisingIOE.java | 69 +
.../org/apache/hadoop/fs/impl/FutureIOSupport.java | 52 +-
.../hadoop/fs/impl/PathCapabilitiesSupport.java | 50 +
.../org/apache/hadoop/fs/protocolPB/PBHelper.java | 5 +-
.../org/apache/hadoop/fs/shell/AclCommands.java | 6 +-
.../java/org/apache/hadoop/fs/shell/Command.java | 2 +-
.../hadoop/fs/shell/CommandWithDestination.java | 74 +-
.../org/apache/hadoop/fs/shell/CopyCommands.java | 7 +-
.../java/org/apache/hadoop/fs/shell/Delete.java | 37 +-
.../java/org/apache/hadoop/fs/shell/Display.java | 27 +-
.../java/org/apache/hadoop/fs/shell/Mkdir.java | 7 +
.../main/java/org/apache/hadoop/fs/shell/Test.java | 2 +-
.../hadoop/fs/viewfs/ChRootedFileSystem.java | 45 +-
.../org/apache/hadoop/fs/viewfs/Constants.java | 7 +
.../org/apache/hadoop/fs/viewfs/NflyFSystem.java | 6 +-
.../apache/hadoop/fs/viewfs/ViewFileSystem.java | 153 +-
.../java/org/apache/hadoop/fs/viewfs/ViewFs.java | 6 +-
.../java/org/apache/hadoop/ha/HAServiceTarget.java | 17 +-
.../java/org/apache/hadoop/ha/HealthMonitor.java | 19 +-
.../org/apache/hadoop/ha/ZKFailoverController.java | 31 +-
.../org/apache/hadoop/http/HttpRequestLog.java | 12 +-
.../java/org/apache/hadoop/http/HttpServer2.java | 163 +-
.../org/apache/hadoop/http/PrometheusServlet.java | 47 +
.../java/org/apache/hadoop/http/WebServlet.java | 59 +
.../java/org/apache/hadoop/io/ArrayWritable.java | 32 +-
.../main/java/org/apache/hadoop/io/IOUtils.java | 2 -
.../org/apache/hadoop/io/nativeio/NativeIO.java | 151 +-
.../org/apache/hadoop/io/retry/RetryPolicies.java | 14 +
.../org/apache/hadoop/ipc/CallQueueManager.java | 34 +-
.../java/org/apache/hadoop/ipc/CallerContext.java | 3 +-
.../main/java/org/apache/hadoop/ipc/Client.java | 133 +-
.../java/org/apache/hadoop/ipc/ClientCache.java | 7 +-
.../java/org/apache/hadoop/ipc/CostProvider.java | 46 +
.../org/apache/hadoop/ipc/DecayRpcScheduler.java | 261 +-
.../org/apache/hadoop/ipc/DefaultCostProvider.java | 43 +
.../org/apache/hadoop/ipc/DefaultRpcScheduler.java | 4 +-
.../java/org/apache/hadoop/ipc/ExternalCall.java | 5 +
.../java/org/apache/hadoop/ipc/FairCallQueue.java | 39 +-
.../java/org/apache/hadoop/ipc/IpcException.java | 5 +-
.../org/apache/hadoop/ipc/ProcessingDetails.java | 96 +
.../org/apache/hadoop/ipc/ProtobufRpcEngine.java | 32 +-
.../org/apache/hadoop/ipc/RemoteException.java | 2 +-
.../java/org/apache/hadoop/ipc/RpcScheduler.java | 41 +-
.../java/org/apache/hadoop/ipc/RpcWritable.java | 2 +-
.../main/java/org/apache/hadoop/ipc/Server.java | 203 +-
.../hadoop/ipc/WeightedTimeCostProvider.java | 110 +
.../org/apache/hadoop/ipc/WritableRpcEngine.java | 20 +-
.../hadoop/ipc/metrics/RpcDetailedMetrics.java | 12 +-
.../org/apache/hadoop/ipc/metrics/RpcMetrics.java | 67 +-
.../java/org/apache/hadoop/jmx/JMXJsonServlet.java | 3 +-
.../hadoop/metrics2/impl/MBeanInfoBuilder.java | 2 +-
.../hadoop/metrics2/lib/MutableQuantiles.java | 2 +-
.../metrics2/lib/MutableRatesWithAggregation.java | 9 +-
.../org/apache/hadoop/metrics2/sink/FileSink.java | 6 +-
.../metrics2/sink/PrometheusMetricsSink.java | 124 +
.../apache/hadoop/net/DNSDomainNameResolver.java | 63 +
.../org/apache/hadoop/net/DomainNameResolver.java | 62 +
.../hadoop/net/DomainNameResolverFactory.java | 74 +
.../main/java/org/apache/hadoop/net/InnerNode.java | 3 +
.../java/org/apache/hadoop/net/InnerNodeImpl.java | 47 +-
.../main/java/org/apache/hadoop/net/NetUtils.java | 14 +-
.../org/apache/hadoop/net/NetworkTopology.java | 105 +-
.../org/apache/hadoop/net/SocketIOWithTimeout.java | 24 +-
.../java/org/apache/hadoop/net/TableMapping.java | 6 +-
.../java/org/apache/hadoop/net/package-info.java | 23 +
.../main/java/org/apache/hadoop/net/package.html | 23 -
.../security/AuthenticationFilterInitializer.java | 2 +-
.../org/apache/hadoop/security/Credentials.java | 4 +-
.../hadoop/security/FastSaslClientFactory.java | 80 +
.../hadoop/security/FastSaslServerFactory.java | 78 +
.../java/org/apache/hadoop/security/Groups.java | 2 +-
.../java/org/apache/hadoop/security/KDiag.java | 4 +-
.../apache/hadoop/security/LdapGroupsMapping.java | 338 +-
.../org/apache/hadoop/security/ProviderUtils.java | 5 +-
.../org/apache/hadoop/security/SaslRpcClient.java | 12 +-
.../org/apache/hadoop/security/SaslRpcServer.java | 58 +-
.../hadoop/security/ShellBasedIdMapping.java | 4 +-
.../hadoop/security/UserGroupInformation.java | 201 +-
.../hadoop/security/alias/CredentialShell.java | 94 +-
.../security/alias/LocalJavaKeyStoreProvider.java | 6 +-
.../server/ProxyUserAuthenticationFilter.java | 199 +
.../ProxyUserAuthenticationFilterInitializer.java | 60 +
.../authentication/server/package-info.java | 22 +
.../security/authorize/AccessControlList.java | 6 +-
.../authorize/ServiceAuthorizationManager.java | 32 +-
.../hadoop/security/http/CrossOriginFilter.java | 2 +-
.../security/ssl/DelegatingSSLSocketFactory.java | 287 +
.../security/ssl/FileBasedKeyStoresFactory.java | 5 +-
.../security/ssl/ReloadingX509TrustManager.java | 5 +-
.../org/apache/hadoop/security/ssl/SSLFactory.java | 11 +-
.../hadoop/security/token/SecretManager.java | 2 +-
.../org/apache/hadoop/security/token/Token.java | 29 +-
.../AbstractDelegationTokenSecretManager.java | 9 +-
.../delegation/ZKDelegationTokenSecretManager.java | 8 +-
.../org/apache/hadoop/service/AbstractService.java | 2 +-
.../apache/hadoop/service/CompositeService.java | 3 +-
.../service/launcher/ServiceLaunchException.java | 14 +
.../hadoop/service/launcher/ServiceLauncher.java | 70 +-
.../java/org/apache/hadoop/tools/TableListing.java | 2 +-
.../apache/hadoop/tracing/SpanReceiverInfo.java | 4 +-
.../main/java/org/apache/hadoop/util/ConfTest.java | 4 +-
.../java/org/apache/hadoop/util/DataChecksum.java | 28 +-
.../java/org/apache/hadoop/util/DiskChecker.java | 2 +-
.../java/org/apache/hadoop/util/DurationInfo.java | 92 +
.../org/apache/hadoop/util/FileBasedIPList.java | 4 +-
.../java/org/apache/hadoop/util/GenericsUtil.java | 8 +-
.../org/apache/hadoop/util/HostsFileReader.java | 40 +-
.../org/apache/hadoop/util/HttpExceptionUtils.java | 12 +-
.../org/apache/hadoop/util/JsonSerialization.java | 4 +-
.../apache/hadoop/util/NativeLibraryChecker.java | 10 +
.../apache/hadoop/util/NodeHealthScriptRunner.java | 3 +-
.../org/apache/hadoop/util/OperationDuration.java | 65 +
.../main/java/org/apache/hadoop/util/RunJar.java | 8 +-
.../hadoop/util/SemaphoredDelegatingExecutor.java | 6 +-
.../org/apache/hadoop/util/SequentialNumber.java | 13 +
.../apache/hadoop/util/ShutdownHookManager.java | 21 +-
.../java/org/apache/hadoop/util/StringUtils.java | 26 +
.../java/org/apache/hadoop/util/SysInfoLinux.java | 35 +-
.../main/java/org/apache/hadoop/util/ZKUtil.java | 2 +-
.../hadoop/util/concurrent/HadoopExecutors.java | 58 +-
.../hadoop/util/curator/ZKCuratorManager.java | 38 +-
.../org/apache/hadoop/util/hash/JenkinsHash.java | 6 +-
.../hadoop/io/compress/zstd/ZStandardCompressor.c | 2 +-
.../src/org/apache/hadoop/io/nativeio/NativeIO.c | 264 +
.../src/org/apache/hadoop/io/nativeio/pmdk_load.c | 106 +
.../src/org/apache/hadoop/io/nativeio/pmdk_load.h | 90 +
.../hadoop/security/JniBasedUnixGroupsMapping.c | 3 -
.../hadoop-common/src/main/proto/FSProtos.proto | 2 +-
.../src/main/proto/GenericRefreshProtocol.proto | 2 +-
.../src/main/proto/GetUserMappingsProtocol.proto | 2 +-
.../src/main/proto/HAServiceProtocol.proto | 2 +-
.../src/main/proto/IpcConnectionContext.proto | 2 +-
.../src/main/proto/ProtobufRpcEngine.proto | 2 +-
.../src/main/proto/ProtocolInfo.proto | 2 +-
.../proto/RefreshAuthorizationPolicyProtocol.proto | 2 +-
.../src/main/proto/RefreshCallQueueProtocol.proto | 2 +-
.../main/proto/RefreshUserMappingsProtocol.proto | 2 +-
.../hadoop-common/src/main/proto/RpcHeader.proto | 2 +-
.../hadoop-common/src/main/proto/Security.proto | 3 +-
.../hadoop-common/src/main/proto/TraceAdmin.proto | 2 +-
.../src/main/proto/ZKFCProtocol.proto | 2 +-
.../src/main/resources/core-default.xml | 495 +-
.../src/site/markdown/CommandsManual.md | 3 +
.../src/site/markdown/DeprecatedProperties.md | 25 +-
.../src/site/markdown/DownstreamDev.md | 9 +-
.../src/site/markdown/FairCallQueue.md | 19 +
.../src/site/markdown/FileSystemShell.md | 28 +-
.../src/site/markdown/GroupsMapping.md | 26 +-
.../src/site/markdown/HttpAuthentication.md | 10 +
.../hadoop-common/src/site/markdown/Metrics.md | 43 +
.../src/site/markdown/filesystem/filesystem.md | 73 +-
.../site/markdown/filesystem/fsdatainputstream.md | 37 +
.../filesystem/fsdatainputstreambuilder.md | 2 +-
.../src/site/markdown/filesystem/index.md | 1 +
.../site/markdown/filesystem/pathcapabilities.md | 158 +
.../site/markdown/release/2.10.0/CHANGES.2.10.0.md | 788 +
.../markdown/release/2.10.0/RELEASENOTES.2.10.0.md | 116 +-
.../site/markdown/release/3.1.3/CHANGES.3.1.3.md | 336 +
.../markdown/release/3.1.3/RELEASENOTES.3.1.3.md | 59 +
.../site/markdown/release/3.2.1/CHANGELOG.3.2.1.md | 553 +
.../markdown/release/3.2.1/RELEASENOTES.3.2.1.md | 80 +
.../hadoop/conf/TestCommonConfigurationFields.java | 24 +
.../org/apache/hadoop/conf/TestConfiguration.java | 13 +-
.../hadoop/crypto/CryptoStreamsTestBase.java | 308 +-
.../apache/hadoop/crypto/TestCryptoStreams.java | 59 +-
.../hadoop/crypto/TestCryptoStreamsForLocalFS.java | 15 +
.../hadoop/crypto/TestCryptoStreamsNormal.java | 15 +
.../TestCryptoStreamsWithJceAesCtrCryptoCodec.java | 9 +-
.../apache/hadoop/crypto/key/TestValueQueue.java | 159 +-
.../kms/TestLoadBalancingKMSClientProvider.java | 35 +-
.../org/apache/hadoop/fs/FileContextURIBase.java | 19 +-
.../java/org/apache/hadoop/fs/TestChecksumFs.java | 135 +
.../hadoop/fs/TestFileSystemInitialization.java | 125 +
.../java/org/apache/hadoop/fs/TestFileUtil.java | 6 +
.../org/apache/hadoop/fs/TestFilterFileSystem.java | 3 +-
.../java/org/apache/hadoop/fs/TestFsShellCopy.java | 47 +-
.../apache/hadoop/fs/TestFsUrlConnectionPath.java | 107 +
.../org/apache/hadoop/fs/TestHarFileSystem.java | 60 +-
.../org/apache/hadoop/fs/TestLocalFileSystem.java | 270 +-
.../test/java/org/apache/hadoop/fs/TestPath.java | 2 +
.../hadoop/fs/TestRawLocalFileSystemContract.java | 46 +-
.../test/java/org/apache/hadoop/fs/TestTrash.java | 375 +-
.../fs/contract/AbstractContractAppendTest.java | 16 +
.../fs/contract/AbstractContractConcatTest.java | 7 +
.../fs/contract/AbstractContractDeleteTest.java | 2 +-
.../AbstractContractGetFileStatusTest.java | 24 +-
.../fs/contract/AbstractContractOpenTest.java | 12 +-
.../fs/contract/AbstractContractRenameTest.java | 2 +-
.../AbstractContractRootDirectoryTest.java | 73 +-
.../fs/contract/AbstractContractSeekTest.java | 4 +-
.../fs/contract/AbstractContractUnbufferTest.java | 125 +
.../apache/hadoop/fs/contract/ContractOptions.java | 5 +
.../hadoop/fs/contract/ContractTestUtils.java | 105 +-
.../TestLocalFSContractMultipartUploader.java | 8 +
.../org/apache/hadoop/fs/impl/TestFutureIO.java | 76 +
.../hadoop/fs/loadGenerator/LoadGenerator.java | 4 +-
.../java/org/apache/hadoop/fs/shell/TestCopy.java | 52 +-
.../hadoop/fs/viewfs/TestChRootedFileSystem.java | 28 +-
.../fs/viewfs/TestViewFileSystemDelegation.java | 40 +-
.../TestViewFileSystemDelegationTokenSupport.java | 12 +-
.../apache/hadoop/fs/viewfs/TestViewFsTrash.java | 2 +-
.../hadoop/fs/viewfs/ViewFileSystemBaseTest.java | 106 +-
.../apache/hadoop/fs/viewfs/ViewFsBaseTest.java | 7 +-
.../org/apache/hadoop/ha/ClientBaseWithFixes.java | 9 +
.../java/org/apache/hadoop/ha/DummyHAService.java | 13 +-
.../apache/hadoop/ha/TestZKFailoverController.java | 41 +
.../org/apache/hadoop/http/TestGlobalFilter.java | 4 +-
.../org/apache/hadoop/http/TestHttpRequestLog.java | 5 +-
.../hadoop/http/TestHttpServerWithSpnego.java | 238 +
.../org/apache/hadoop/http/TestPathFilter.java | 2 -
.../org/apache/hadoop/http/TestSSLHttpServer.java | 106 +
.../org/apache/hadoop/http/TestServletFilter.java | 1 -
.../java/org/apache/hadoop/io/TestArrayFile.java | 11 +-
.../org/apache/hadoop/io/TestArrayWritable.java | 23 +-
.../org/apache/hadoop/io/TestBloomMapFile.java | 16 +-
.../java/org/apache/hadoop/io/TestMapFile.java | 38 +-
.../org/apache/hadoop/io/TestSequenceFileSync.java | 14 +-
.../test/java/org/apache/hadoop/io/TestText.java | 10 +-
.../io/file/tfile/TestTFileUnsortedByteArrays.java | 193 +-
.../org/apache/hadoop/io/file/tfile/TestVLong.java | 10 +-
.../apache/hadoop/io/nativeio/TestNativeIO.java | 153 +
.../org/apache/hadoop/io/retry/TestRetryProxy.java | 48 +
.../hadoop/io/retry/UnreliableImplementation.java | 23 +
.../hadoop/io/retry/UnreliableInterface.java | 21 +
.../apache/hadoop/ipc/TestCallQueueManager.java | 28 +-
.../apache/hadoop/ipc/TestDecayRpcScheduler.java | 174 +-
.../org/apache/hadoop/ipc/TestFairCallQueue.java | 102 +-
.../test/java/org/apache/hadoop/ipc/TestIPC.java | 14 +-
.../apache/hadoop/ipc/TestProcessingDetails.java | 61 +
.../org/apache/hadoop/ipc/TestProtoBufRpc.java | 48 +-
.../test/java/org/apache/hadoop/ipc/TestRPC.java | 130 +-
.../java/org/apache/hadoop/ipc/TestRpcBase.java | 28 +
.../org/apache/hadoop/ipc/TestSocketFactory.java | 25 +-
.../ipc/TestWeightedRoundRobinMultiplexer.java | 85 +-
.../hadoop/ipc/TestWeightedTimeCostProvider.java | 86 +
.../apache/hadoop/ipc/metrics/TestRpcMetrics.java | 60 +
.../java/org/apache/hadoop/log/TestLogLevel.java | 9 +
.../hadoop/metrics2/lib/TestMutableMetrics.java | 17 +
.../metrics2/sink/TestPrometheusMetricsSink.java | 133 +
.../hadoop/metrics2/util/TestSampleQuantiles.java | 28 +-
.../apache/hadoop/net/MockDomainNameResolver.java | 98 +
.../hadoop/net/TestMockDomainNameResolver.java | 71 +
.../java/org/apache/hadoop/net/TestNetUtils.java | 15 +-
.../apache/hadoop/net/TestSocketIOWithTimeout.java | 38 +
.../org/apache/hadoop/net/TestTableMapping.java | 16 +-
.../apache/hadoop/security/TestGroupsCaching.java | 23 +-
.../hadoop/security/TestLdapGroupsMappingBase.java | 39 +-
.../TestLdapGroupsMappingWithBindUserSwitch.java | 247 +
.../apache/hadoop/security/TestSecurityUtil.java | 3 +-
.../hadoop/security/TestShellBasedIdMapping.java | 9 +-
.../hadoop/security/TestUGIWithSecurityOn.java | 117 -
.../hadoop/security/TestUserGroupInformation.java | 93 +
.../hadoop/security/alias/TestCredShell.java | 31 +-
.../server/TestProxyUserAuthenticationFilter.java | 125 +
.../security/authorize/TestAccessControlList.java | 123 +-
.../authorize/TestServiceAuthorization.java | 52 +
.../security/http/TestXFrameOptionsFilter.java | 16 +-
.../ssl/TestDelegatingSSLSocketFactory.java | 57 +
.../token/delegation/TestDelegationToken.java | 26 +-
.../delegation/web/TestWebDelegationToken.java | 11 +-
.../apache/hadoop/test/AbstractHadoopTestBase.java | 110 +
.../org/apache/hadoop/test/GenericTestUtils.java | 9 +
.../org/apache/hadoop/test/HadoopTestBase.java | 4 +-
.../org/apache/hadoop/test/LambdaTestUtils.java | 3 +
.../org/apache/hadoop/util/TestDurationInfo.java | 54 +
.../test/java/org/apache/hadoop/util/TestGSet.java | 3 +-
.../apache/hadoop/util/TestHostsFileReader.java | 71 +-
.../hadoop/util/TestLightWeightResizableGSet.java | 55 +-
.../org/apache/hadoop/util/TestLineReader.java | 222 +-
.../hadoop/util/TestNodeHealthScriptRunner.java | 9 +
.../java/org/apache/hadoop/util/TestProtoUtil.java | 2 +-
.../hadoop/util/TestShutdownHookManager.java | 29 +-
.../java/org/apache/hadoop/util/TestZKUtil.java | 2 +-
.../hadoop/util/curator/TestChildReaper.java | 19 +-
.../hadoop-common/src/test/proto/test.proto | 2 +-
.../src/test/proto/test_rpc_service.proto | 2 +
.../src/test/resources/kdc/killKdc.sh | 19 -
.../src/test/resources/kdc/ldif/users.ldif | 78 -
.../src/test/resources/kdc/server.xml | 258 -
.../hadoop-common/src/test/resources/testConf.xml | 6 +-
.../hadoop-kms/dev-support/findbugsExcludeFile.xml | 8 +
hadoop-common-project/hadoop-kms/pom.xml | 2 +-
.../hadoop/crypto/key/kms/server/KMSACLs.java | 2 +-
.../key/kms/server/KMSExceptionsProvider.java | 5 +-
.../hadoop/crypto/key/kms/server/KMSMDCFilter.java | 58 +-
.../hadoop/crypto/key/kms/server/KMSWebApp.java | 6 +
.../kms/server/KeyAuthorizationKeyProvider.java | 2 +-
.../hadoop/crypto/key/kms/server/KMSBenchmark.java | 617 +
.../hadoop/crypto/key/kms/server/TestKMS.java | 13 +-
.../crypto/key/kms/server/TestKMSMDCFilter.java | 88 +
.../hadoop-kms/src/test/resources/log4j.properties | 1 +
hadoop-common-project/hadoop-minikdc/pom.xml | 2 +-
hadoop-common-project/hadoop-nfs/pom.xml | 7 +-
.../org/apache/hadoop/nfs/nfs3/TestFileHandle.java | 11 +-
hadoop-common-project/hadoop-registry/pom.xml | 2 +-
.../registry/client/impl/zk/RegistrySecurity.java | 40 +-
.../client/impl/zk/ZookeeperConfigOptions.java | 6 +-
.../server/services/MicroZookeeperService.java | 25 +-
.../hadoop/registry/secure/TestSecureRegistry.java | 34 -
hadoop-common-project/pom.xml | 2 +-
hadoop-dist/pom.xml | 2 +-
hadoop-hdds/client/pom.xml | 44 -
.../hdds/scm/ClientCredentialInterceptor.java | 65 -
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 398 -
.../hadoop/hdds/scm/XceiverClientManager.java | 243 -
.../hadoop/hdds/scm/XceiverClientMetrics.java | 92 -
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 311 -
.../hdds/scm/client/ContainerOperationClient.java | 469 -
.../hadoop/hdds/scm/client/HddsClientUtils.java | 255 -
.../hadoop/hdds/scm/client/package-info.java | 23 -
.../org/apache/hadoop/hdds/scm/package-info.java | 23 -
.../hadoop/hdds/scm/storage/BlockInputStream.java | 407 -
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 629 -
.../hadoop/hdds/scm/storage/package-info.java | 23 -
.../common/dev-support/findbugsExcludeFile.xml | 28 -
hadoop-hdds/common/pom.xml | 278 -
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 183 -
.../java/org/apache/hadoop/hdds/HddsIdFactory.java | 53 -
.../java/org/apache/hadoop/hdds/HddsUtils.java | 413 -
.../org/apache/hadoop/hdds/cli/GenericCli.java | 104 -
.../hadoop/hdds/cli/GenericParentCommand.java | 29 -
.../hadoop/hdds/cli/HddsVersionProvider.java | 35 -
.../hdds/cli/MissingSubcommandException.java | 35 -
.../org/apache/hadoop/hdds/cli/package-info.java | 22 -
.../org/apache/hadoop/hdds/client/BlockID.java | 127 -
.../hadoop/hdds/client/ContainerBlockID.java | 79 -
.../org/apache/hadoop/hdds/client/OzoneQuota.java | 203 -
.../hadoop/hdds/client/ReplicationFactor.java | 63 -
.../apache/hadoop/hdds/client/ReplicationType.java | 28 -
.../apache/hadoop/hdds/client/package-info.java | 23 -
.../apache/hadoop/hdds/conf/HddsConfServlet.java | 188 -
.../hadoop/hdds/conf/OzoneConfiguration.java | 164 -
.../org/apache/hadoop/hdds/conf/package-info.java | 18 -
.../java/org/apache/hadoop/hdds/package-info.java | 23 -
.../hadoop/hdds/protocol/DatanodeDetails.java | 401 -
.../hadoop/hdds/protocol/SCMSecurityProtocol.java | 55 -
.../apache/hadoop/hdds/protocol/package-info.java | 22 -
.../SCMSecurityProtocolClientSideTranslatorPB.java | 124 -
.../hdds/protocolPB/SCMSecurityProtocolPB.java | 35 -
.../SCMSecurityProtocolServerSideTranslatorPB.java | 94 -
.../hadoop/hdds/protocolPB/package-info.java | 22 -
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 363 -
.../java/org/apache/hadoop/hdds/scm/ScmInfo.java | 81 -
.../apache/hadoop/hdds/scm/XceiverClientReply.java | 73 -
.../apache/hadoop/hdds/scm/XceiverClientSpi.java | 177 -
.../apache/hadoop/hdds/scm/client/ScmClient.java | 206 -
.../hadoop/hdds/scm/client/package-info.java | 24 -
.../hdds/scm/container/ContainerException.java | 46 -
.../hadoop/hdds/scm/container/ContainerID.java | 107 -
.../hadoop/hdds/scm/container/ContainerInfo.java | 477 -
.../scm/container/ContainerNotFoundException.java | 44 -
.../ContainerReplicaNotFoundException.java | 45 -
.../container/common/helpers/AllocatedBlock.java | 66 -
.../common/helpers/BlockNotCommittedException.java | 36 -
.../common/helpers/ContainerNotOpenException.java | 36 -
.../common/helpers/ContainerWithPipeline.java | 137 -
.../common/helpers/DeleteBlockResult.java | 53 -
.../helpers/InvalidContainerStateException.java | 35 -
.../common/helpers/StorageContainerException.java | 104 -
.../scm/container/common/helpers/package-info.java | 22 -
.../hadoop/hdds/scm/container/package-info.java | 18 -
.../org/apache/hadoop/hdds/scm/package-info.java | 24 -
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 310 -
.../hadoop/hdds/scm/pipeline/PipelineID.java | 80 -
.../scm/pipeline/PipelineNotFoundException.java | 46 -
.../pipeline/UnknownPipelineStateException.java | 46 -
.../hadoop/hdds/scm/pipeline/package-info.java | 24 -
.../hadoop/hdds/scm/protocol/LocatedContainer.java | 127 -
.../scm/protocol/ScmBlockLocationProtocol.java | 63 -
.../hadoop/hdds/scm/protocol/ScmLocatedBlock.java | 100 -
.../protocol/StorageContainerLocationProtocol.java | 172 -
.../hadoop/hdds/scm/protocol/package-info.java | 19 -
...lockLocationProtocolClientSideTranslatorPB.java | 192 -
.../scm/protocolPB/ScmBlockLocationProtocolPB.java | 39 -
...inerLocationProtocolClientSideTranslatorPB.java | 419 -
.../StorageContainerLocationProtocolPB.java | 40 -
.../hadoop/hdds/scm/protocolPB/package-info.java | 24 -
.../hdds/scm/storage/ContainerProtocolCalls.java | 596 -
.../hadoop/hdds/scm/storage/package-info.java | 23 -
.../security/exception/SCMSecurityException.java | 79 -
.../hdds/security/exception/package-info.java | 23 -
.../hdds/security/token/BlockTokenException.java | 53 -
.../hdds/security/token/BlockTokenVerifier.java | 131 -
.../security/token/OzoneBlockTokenIdentifier.java | 199 -
.../security/token/OzoneBlockTokenSelector.java | 75 -
.../hadoop/hdds/security/token/TokenVerifier.java | 38 -
.../hadoop/hdds/security/token/package-info.java | 22 -
.../hadoop/hdds/security/x509/SecurityConfig.java | 462 -
.../x509/certificate/authority/BaseApprover.java | 249 -
.../certificate/authority/CertificateApprover.java | 86 -
.../certificate/authority/CertificateServer.java | 113 -
.../certificate/authority/DefaultApprover.java | 128 -
.../certificate/authority/DefaultCAServer.java | 439 -
.../authority/PKIProfiles/DefaultCAProfile.java | 46 -
.../authority/PKIProfiles/DefaultProfile.java | 333 -
.../authority/PKIProfiles/PKIProfile.java | 140 -
.../authority/PKIProfiles/package-info.java | 33 -
.../x509/certificate/authority/package-info.java | 22 -
.../x509/certificate/client/CertificateClient.java | 173 -
.../certificate/client/DNCertificateClient.java | 40 -
.../client/DefaultCertificateClient.java | 632 -
.../certificate/client/OMCertificateClient.java | 102 -
.../x509/certificate/client/package-info.java | 22 -
.../x509/certificate/utils/CertificateCodec.java | 280 -
.../x509/certificate/utils/package-info.java | 22 -
.../certificates/utils/CertificateSignRequest.java | 276 -
.../certificates/utils/SelfSignedCertificate.java | 238 -
.../x509/certificates/utils/package-info.java | 22 -
.../x509/exceptions/CertificateException.java | 87 -
.../security/x509/exceptions/package-info.java | 23 -
.../hdds/security/x509/keys/HDDSKeyGenerator.java | 118 -
.../hadoop/hdds/security/x509/keys/KeyCodec.java | 411 -
.../hdds/security/x509/keys/SecurityUtil.java | 138 -
.../hdds/security/x509/keys/package-info.java | 23 -
.../hadoop/hdds/security/x509/package-info.java | 99 -
.../hadoop/hdds/tracing/GrpcClientInterceptor.java | 57 -
.../hadoop/hdds/tracing/GrpcServerInterceptor.java | 51 -
.../apache/hadoop/hdds/tracing/StringCodec.java | 89 -
.../apache/hadoop/hdds/tracing/TraceAllMethod.java | 86 -
.../apache/hadoop/hdds/tracing/TracingUtil.java | 112 -
.../apache/hadoop/hdds/tracing/package-info.java | 23 -
.../java/org/apache/hadoop/ozone/OzoneAcl.java | 233 -
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 386 -
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 274 -
.../org/apache/hadoop/ozone/OzoneSecurityUtil.java | 60 -
.../org/apache/hadoop/ozone/audit/AuditAction.java | 30 -
.../hadoop/ozone/audit/AuditEventStatus.java | 36 -
.../org/apache/hadoop/ozone/audit/AuditLogger.java | 76 -
.../apache/hadoop/ozone/audit/AuditLoggerType.java | 37 -
.../org/apache/hadoop/ozone/audit/AuditMarker.java | 38 -
.../apache/hadoop/ozone/audit/AuditMessage.java | 131 -
.../org/apache/hadoop/ozone/audit/Auditable.java | 32 -
.../org/apache/hadoop/ozone/audit/Auditor.java | 33 -
.../org/apache/hadoop/ozone/audit/DNAction.java | 48 -
.../org/apache/hadoop/ozone/audit/SCMAction.java | 45 -
.../apache/hadoop/ozone/audit/package-info.java | 138 -
.../org/apache/hadoop/ozone/common/BlockGroup.java | 106 -
.../org/apache/hadoop/ozone/common/Checksum.java | 249 -
.../apache/hadoop/ozone/common/ChecksumData.java | 190 -
.../ozone/common/DeleteBlockGroupResult.java | 97 -
.../common/InconsistentStorageStateException.java | 51 -
.../ozone/common/OzoneChecksumException.java | 66 -
.../org/apache/hadoop/ozone/common/Storage.java | 249 -
.../apache/hadoop/ozone/common/StorageInfo.java | 182 -
.../apache/hadoop/ozone/common/package-info.java | 18 -
.../InvalidStateTransitionException.java | 42 -
.../ozone/common/statemachine/StateMachine.java | 68 -
.../ozone/common/statemachine/package-info.java | 21 -
.../ozone/container/common/helpers/BlockData.java | 273 -
.../ozone/container/common/helpers/ChunkInfo.java | 185 -
.../helpers/ContainerCommandRequestPBHelper.java | 192 -
.../container/common/helpers/package-info.java | 23 -
.../java/org/apache/hadoop/ozone/lease/Lease.java | 189 -
.../ozone/lease/LeaseAlreadyExistException.java | 46 -
.../hadoop/ozone/lease/LeaseCallbackExecutor.java | 65 -
.../apache/hadoop/ozone/lease/LeaseException.java | 45 -
.../hadoop/ozone/lease/LeaseExpiredException.java | 45 -
.../apache/hadoop/ozone/lease/LeaseManager.java | 251 -
.../lease/LeaseManagerNotRunningException.java | 45 -
.../hadoop/ozone/lease/LeaseNotFoundException.java | 46 -
.../apache/hadoop/ozone/lease/package-info.java | 26 -
.../org/apache/hadoop/ozone/lock/ActiveLock.java | 101 -
.../org/apache/hadoop/ozone/lock/LockManager.java | 101 -
.../hadoop/ozone/lock/PooledLockFactory.java | 43 -
.../org/apache/hadoop/ozone/lock/package-info.java | 21 -
.../java/org/apache/hadoop/ozone/package-info.java | 35 -
...lockLocationProtocolServerSideTranslatorPB.java | 133 -
...inerLocationProtocolServerSideTranslatorPB.java | 309 -
.../hadoop/ozone/protocolPB/package-info.java | 24 -
.../apache/hadoop/ozone/web/utils/JsonUtils.java | 71 -
.../hadoop/ozone/web/utils/package-info.java | 19 -
.../org/apache/hadoop/utils/BackgroundService.java | 161 -
.../org/apache/hadoop/utils/BackgroundTask.java | 28 -
.../apache/hadoop/utils/BackgroundTaskQueue.java | 64 -
.../apache/hadoop/utils/BackgroundTaskResult.java | 44 -
.../org/apache/hadoop/utils/BatchOperation.java | 90 -
.../org/apache/hadoop/utils/EntryConsumer.java | 38 -
.../org/apache/hadoop/utils/HddsVersionInfo.java | 56 -
.../java/org/apache/hadoop/utils/LevelDBStore.java | 387 -
.../apache/hadoop/utils/LevelDBStoreIterator.java | 64 -
.../org/apache/hadoop/utils/MetaStoreIterator.java | 39 -
.../apache/hadoop/utils/MetadataKeyFilters.java | 206 -
.../org/apache/hadoop/utils/MetadataStore.java | 227 -
.../apache/hadoop/utils/MetadataStoreBuilder.java | 146 -
.../org/apache/hadoop/utils/RetriableTask.java | 78 -
.../java/org/apache/hadoop/utils/RocksDBStore.java | 391 -
.../apache/hadoop/utils/RocksDBStoreIterator.java | 66 -
.../org/apache/hadoop/utils/RocksDBStoreMBean.java | 144 -
.../java/org/apache/hadoop/utils/VersionInfo.java | 97 -
.../org/apache/hadoop/utils/db/BatchOperation.java | 27 -
.../apache/hadoop/utils/db/ByteArrayKeyValue.java | 67 -
.../java/org/apache/hadoop/utils/db/Codec.java | 41 -
.../org/apache/hadoop/utils/db/CodecRegistry.java | 89 -
.../apache/hadoop/utils/db/DBConfigFromFile.java | 146 -
.../java/org/apache/hadoop/utils/db/DBProfile.java | 120 -
.../java/org/apache/hadoop/utils/db/DBStore.java | 141 -
.../org/apache/hadoop/utils/db/DBStoreBuilder.java | 208 -
.../apache/hadoop/utils/db/RDBBatchOperation.java | 69 -
.../java/org/apache/hadoop/utils/db/RDBStore.java | 249 -
.../apache/hadoop/utils/db/RDBStoreIterator.java | 107 -
.../java/org/apache/hadoop/utils/db/RDBTable.java | 167 -
.../org/apache/hadoop/utils/db/StringCodec.java | 45 -
.../java/org/apache/hadoop/utils/db/Table.java | 109 -
.../org/apache/hadoop/utils/db/TableConfig.java | 93 -
.../org/apache/hadoop/utils/db/TableIterator.java | 62 -
.../org/apache/hadoop/utils/db/TypedTable.java | 209 -
.../org/apache/hadoop/utils/db/package-info.java | 22 -
.../java/org/apache/hadoop/utils/package-info.java | 18 -
.../main/java/org/apache/ratis/RatisHelper.java | 225 -
.../main/java/org/apache/ratis/package-info.java | 22 -
.../src/main/proto/DatanodeContainerProtocol.proto | 461 -
.../src/main/proto/SCMSecurityProtocol.proto | 82 -
.../src/main/proto/ScmBlockLocationProtocol.proto | 134 -
.../proto/StorageContainerLocationProtocol.proto | 278 -
hadoop-hdds/common/src/main/proto/hdds.proto | 237 -
.../main/resources/hdds-version-info.properties | 26 -
.../common/src/main/resources/ozone-default.xml | 1876 -
.../org/apache/hadoop/hdds/scm/package-info.java | 21 -
.../token/TestOzoneBlockTokenIdentifier.java | 313 -
.../hadoop/hdds/security/token/package-info.java | 22 -
.../x509/certificate/authority/MockApprover.java | 57 -
.../certificate/authority/TestDefaultCAServer.java | 169 -
.../certificate/authority/TestDefaultProfile.java | 364 -
.../x509/certificate/authority/package-info.java | 22 -
.../client/TestCertificateClientInit.java | 206 -
.../client/TestDefaultCertificateClient.java | 336 -
.../certificate/utils/TestCertificateCodec.java | 218 -
.../x509/certificate/utils/package-info.java | 23 -
.../certificates/TestCertificateSignRequest.java | 285 -
.../x509/certificates/TestRootCertificate.java | 258 -
.../security/x509/certificates/package-info.java | 22 -
.../security/x509/keys/TestHDDSKeyGenerator.java | 87 -
.../hdds/security/x509/keys/TestKeyCodec.java | 231 -
.../hdds/security/x509/keys/package-info.java | 22 -
.../hadoop/hdds/security/x509/package-info.java | 22 -
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 141 -
.../org/apache/hadoop/ozone/audit/DummyAction.java | 45 -
.../org/apache/hadoop/ozone/audit/DummyEntity.java | 57 -
.../hadoop/ozone/audit/TestOzoneAuditLogger.java | 166 -
.../apache/hadoop/ozone/audit/package-info.java | 23 -
.../apache/hadoop/ozone/common/TestChecksum.java | 101 -
.../hadoop/ozone/common/TestStateMachine.java | 106 -
.../hadoop/ozone/lease/TestLeaseManager.java | 388 -
.../apache/hadoop/ozone/lease/package-info.java | 21 -
.../apache/hadoop/ozone/lock/TestLockManager.java | 64 -
.../org/apache/hadoop/ozone/lock/package-info.java | 21 -
.../java/org/apache/hadoop/ozone/package-info.java | 21 -
.../org/apache/hadoop/utils/TestHddsIdFactory.java | 77 -
.../org/apache/hadoop/utils/TestMetadataStore.java | 588 -
.../org/apache/hadoop/utils/TestRetriableTask.java | 76 -
.../apache/hadoop/utils/TestRocksDBStoreMBean.java | 96 -
.../hadoop/utils/db/TestDBConfigFromFile.java | 116 -
.../apache/hadoop/utils/db/TestDBStoreBuilder.java | 173 -
.../org/apache/hadoop/utils/db/TestRDBStore.java | 258 -
.../apache/hadoop/utils/db/TestRDBTableStore.java | 226 -
.../hadoop/utils/db/TestTypedRDBTableStore.java | 235 -
.../org/apache/hadoop/utils/db/package-info.java | 22 -
.../java/org/apache/hadoop/utils/package-info.java | 22 -
.../common/src/test/resources/log4j2.properties | 76 -
hadoop-hdds/common/src/test/resources/test.db.ini | 145 -
.../dev-support/findbugsExcludeFile.xml | 33 -
hadoop-hdds/container-service/pom.xml | 106 -
.../org/apache/hadoop/hdds/scm/HddsServerUtil.java | 375 -
.../org/apache/hadoop/hdds/scm/VersionInfo.java | 81 -
.../org/apache/hadoop/hdds/scm/package-info.java | 19 -
.../apache/hadoop/ozone/HddsDatanodeService.java | 311 -
.../container/common/DataNodeLayoutVersion.java | 80 -
.../container/common/helpers/ContainerMetrics.java | 121 -
.../container/common/helpers/ContainerUtils.java | 305 -
.../common/helpers/DatanodeVersionFile.java | 95 -
.../helpers/DeletedContainerBlocksSummary.java | 104 -
.../container/common/helpers/package-info.java | 22 -
.../container/common/impl/ChunkLayOutVersion.java | 98 -
.../ozone/container/common/impl/ContainerData.java | 484 -
.../container/common/impl/ContainerDataYaml.java | 318 -
.../ozone/container/common/impl/ContainerSet.java | 221 -
.../container/common/impl/HddsDispatcher.java | 486 -
.../common/impl/OpenContainerBlockMap.java | 151 -
.../RandomContainerDeletionChoosingPolicy.java | 70 -
.../common/impl/StorageLocationReport.java | 300 -
...TopNOrderedContainerDeletionChoosingPolicy.java | 90 -
.../ozone/container/common/impl/package-info.java | 22 -
.../container/common/interfaces/BlockIterator.java | 57 -
.../container/common/interfaces/Container.java | 154 -
.../ContainerDeletionChoosingPolicy.java | 58 -
.../common/interfaces/ContainerDispatcher.java | 78 -
.../interfaces/ContainerLocationManager.java | 58 -
.../interfaces/ContainerLocationManagerMXBean.java | 34 -
.../common/interfaces/ContainerPacker.java | 58 -
.../ozone/container/common/interfaces/Handler.java | 165 -
.../interfaces/StorageLocationReportMXBean.java | 40 -
.../common/interfaces/VolumeChoosingPolicy.java | 46 -
.../container/common/interfaces/package-info.java | 20 -
.../ozone/container/common/package-info.java | 28 -
.../report/CommandStatusReportPublisher.java | 87 -
.../common/report/ContainerReportPublisher.java | 86 -
.../common/report/NodeReportPublisher.java | 66 -
.../common/report/PipelineReportPublisher.java | 73 -
.../container/common/report/ReportManager.java | 148 -
.../container/common/report/ReportPublisher.java | 115 -
.../common/report/ReportPublisherFactory.java | 79 -
.../container/common/report/package-info.java | 80 -
.../common/statemachine/DatanodeStateMachine.java | 465 -
.../common/statemachine/EndpointStateMachine.java | 295 -
.../statemachine/EndpointStateMachineMBean.java | 34 -
.../common/statemachine/SCMConnectionManager.java | 211 -
.../statemachine/SCMConnectionManagerMXBean.java | 27 -
.../common/statemachine/StateContext.java | 485 -
.../CloseContainerCommandHandler.java | 170 -
.../commandhandler/CommandDispatcher.java | 182 -
.../commandhandler/CommandHandler.java | 75 -
.../commandhandler/DeleteBlocksCommandHandler.java | 266 -
.../DeleteContainerCommandHandler.java | 85 -
.../ReplicateContainerCommandHandler.java | 102 -
.../statemachine/commandhandler/package-info.java | 18 -
.../common/statemachine/package-info.java | 28 -
.../container/common/states/DatanodeState.java | 58 -
.../common/states/datanode/InitDatanodeState.java | 182 -
.../states/datanode/RunningDatanodeState.java | 178 -
.../common/states/datanode/package-info.java | 21 -
.../states/endpoint/HeartbeatEndpointTask.java | 398 -
.../states/endpoint/RegisterEndpointTask.java | 257 -
.../states/endpoint/VersionEndpointTask.java | 128 -
.../common/states/endpoint/package-info.java | 20 -
.../container/common/states/package-info.java | 18 -
.../transport/server/GrpcXceiverService.java | 97 -
.../server/ServerCredentialInterceptor.java | 74 -
.../common/transport/server/XceiverServer.java | 87 -
.../common/transport/server/XceiverServerGrpc.java | 192 -
.../common/transport/server/XceiverServerSpi.java | 68 -
.../common/transport/server/package-info.java | 24 -
.../common/transport/server/ratis/CSMMetrics.java | 116 -
.../server/ratis/ContainerStateMachine.java | 644 -
.../transport/server/ratis/DispatcherContext.java | 133 -
.../transport/server/ratis/XceiverServerRatis.java | 592 -
.../transport/server/ratis/package-info.java | 23 -
.../container/common/utils/ContainerCache.java | 174 -
.../container/common/utils/HddsVolumeUtil.java | 219 -
.../ozone/container/common/utils/package-info.java | 18 -
.../container/common/volume/AbstractFuture.java | 1298 -
.../ozone/container/common/volume/HddsVolume.java | 433 -
.../container/common/volume/HddsVolumeChecker.java | 421 -
.../volume/RoundRobinVolumeChoosingPolicy.java | 83 -
.../common/volume/ThrottledAsyncChecker.java | 247 -
.../container/common/volume/TimeoutFuture.java | 161 -
.../container/common/volume/VolumeIOStats.java | 139 -
.../ozone/container/common/volume/VolumeInfo.java | 155 -
.../ozone/container/common/volume/VolumeSet.java | 517 -
.../ozone/container/common/volume/VolumeUsage.java | 181 -
.../container/common/volume/package-info.java | 21 -
.../container/keyvalue/KeyValueBlockIterator.java | 148 -
.../container/keyvalue/KeyValueContainer.java | 648 -
.../container/keyvalue/KeyValueContainerData.java | 276 -
.../ozone/container/keyvalue/KeyValueHandler.java | 957 -
.../container/keyvalue/TarContainerPacker.java | 249 -
.../container/keyvalue/helpers/BlockUtils.java | 201 -
.../container/keyvalue/helpers/ChunkUtils.java | 326 -
.../helpers/KeyValueContainerLocationUtil.java | 112 -
.../keyvalue/helpers/KeyValueContainerUtil.java | 225 -
.../container/keyvalue/helpers/SmallFileUtils.java | 91 -
.../container/keyvalue/helpers/package-info.java | 21 -
.../container/keyvalue/impl/BlockManagerImpl.java | 282 -
.../container/keyvalue/impl/ChunkManagerImpl.java | 314 -
.../container/keyvalue/impl/package-info.java | 21 -
.../keyvalue/interfaces/BlockManager.java | 85 -
.../keyvalue/interfaces/ChunkManager.java | 83 -
.../keyvalue/interfaces/package-info.java | 21 -
.../ozone/container/keyvalue/package-info.java | 21 -
.../background/BlockDeletingService.java | 258 -
.../statemachine/background/package-info.java | 18 -
.../container/ozoneimpl/ContainerController.java | 138 -
.../ozone/container/ozoneimpl/ContainerReader.java | 218 -
.../ozone/container/ozoneimpl/OzoneContainer.java | 211 -
.../ozone/container/ozoneimpl/package-info.java | 21 -
.../container/replication/ContainerDownloader.java | 40 -
.../replication/ContainerReplicationSource.java | 49 -
.../container/replication/ContainerReplicator.java | 27 -
.../replication/ContainerStreamingOutput.java | 45 -
.../replication/DownloadAndImportReplicator.java | 136 -
.../replication/GrpcReplicationClient.java | 169 -
.../replication/GrpcReplicationService.java | 129 -
.../OnDemandContainerReplicationSource.java | 76 -
.../replication/ReplicationSupervisor.java | 129 -
.../container/replication/ReplicationTask.java | 102 -
.../replication/SimpleContainerDownloader.java | 120 -
.../ozone/container/replication/package-info.java | 21 -
.../java/org/apache/hadoop/ozone/package-info.java | 23 -
.../protocol/StorageContainerDatanodeProtocol.java | 79 -
.../protocol/StorageContainerNodeProtocol.java | 78 -
.../hadoop/ozone/protocol/VersionResponse.java | 154 -
.../protocol/commands/CloseContainerCommand.java | 82 -
.../protocol/commands/CommandForDatanode.java | 52 -
.../ozone/protocol/commands/CommandStatus.java | 164 -
.../commands/DeleteBlockCommandStatus.java | 100 -
.../protocol/commands/DeleteBlocksCommand.java | 71 -
.../protocol/commands/DeleteContainerCommand.java | 86 -
.../ozone/protocol/commands/RegisteredCommand.java | 215 -
.../commands/ReplicateContainerCommand.java | 98 -
.../ozone/protocol/commands/ReregisterCommand.java | 58 -
.../hadoop/ozone/protocol/commands/SCMCommand.java | 62 -
.../ozone/protocol/commands/package-info.java | 21 -
.../apache/hadoop/ozone/protocol/package-info.java | 23 -
...inerDatanodeProtocolClientSideTranslatorPB.java | 167 -
.../StorageContainerDatanodeProtocolPB.java | 40 -
...inerDatanodeProtocolServerSideTranslatorPB.java | 94 -
.../hadoop/ozone/protocolPB/package-info.java | 19 -
.../proto/StorageContainerDatanodeProtocol.proto | 401 -
.../com.sun.jersey.spi.container.ContainerProvider | 16 -
.../hadoop/ozone/TestHddsDatanodeService.java | 96 -
.../ozone/container/common/ContainerTestUtils.java | 68 -
.../ozone/container/common/SCMTestUtils.java | 126 -
.../hadoop/ozone/container/common/ScmTestMock.java | 355 -
.../container/common/TestChunkLayOutVersion.java | 42 -
.../common/TestDatanodeLayOutVersion.java | 38 -
.../container/common/TestDatanodeStateMachine.java | 425 -
.../common/TestKeyValueContainerData.java | 93 -
.../common/helpers/TestDatanodeVersionFile.java | 134 -
.../common/impl/TestContainerDataYaml.java | 220 -
.../container/common/impl/TestContainerSet.java | 185 -
.../container/common/impl/TestHddsDispatcher.java | 298 -
.../ozone/container/common/impl/package-info.java | 22 -
.../container/common/interfaces/TestHandler.java | 101 -
.../ozone/container/common/package-info.java | 22 -
.../container/common/report/TestReportManager.java | 52 -
.../common/report/TestReportPublisher.java | 191 -
.../common/report/TestReportPublisherFactory.java | 68 -
.../container/common/report/package-info.java | 22 -
.../TestCloseContainerCommandHandler.java | 296 -
.../statemachine/commandhandler/package-info.java | 22 -
.../states/endpoint/TestHeartbeatEndpointTask.java | 295 -
.../common/states/endpoint/package-info.java | 18 -
.../container/common/volume/TestHddsVolume.java | 148 -
.../common/volume/TestHddsVolumeChecker.java | 212 -
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 126 -
.../container/common/volume/TestVolumeSet.java | 254 -
.../common/volume/TestVolumeSetDiskChecks.java | 182 -
.../container/common/volume/package-info.java | 22 -
.../container/keyvalue/TestBlockManagerImpl.java | 197 -
.../container/keyvalue/TestChunkManagerImpl.java | 291 -
.../keyvalue/TestKeyValueBlockIterator.java | 278 -
.../container/keyvalue/TestKeyValueContainer.java | 377 -
.../TestKeyValueContainerMarkUnhealthy.java | 172 -
.../container/keyvalue/TestKeyValueHandler.java | 312 -
.../TestKeyValueHandlerWithUnhealthyContainer.java | 231 -
.../container/keyvalue/TestTarContainerPacker.java | 234 -
.../ozone/container/keyvalue/package-info.java | 22 -
.../container/ozoneimpl/TestOzoneContainer.java | 130 -
.../replication/TestReplicationSupervisor.java | 136 -
.../ozone/container/replication/package-info.java | 22 -
.../testutils/BlockDeletingServiceTestImpl.java | 104 -
.../ozone/container/testutils/package-info.java | 18 -
.../src/test/resources/additionalfields.container | 14 -
.../test/resources/incorrect.checksum.container | 13 -
.../src/test/resources/incorrect.container | 13 -
.../src/test/resources/log4j.properties | 23 -
hadoop-hdds/docs/README.md | 55 -
hadoop-hdds/docs/archetypes/default.md | 20 -
hadoop-hdds/docs/config.yaml | 46 -
hadoop-hdds/docs/content/AuditParser.md | 72 -
hadoop-hdds/docs/content/BucketCommands.md | 131 -
hadoop-hdds/docs/content/BuildingSources.md | 54 -
hadoop-hdds/docs/content/CommandShell.md | 108 -
hadoop-hdds/docs/content/Concepts.md | 108 -
hadoop-hdds/docs/content/Dozone.md | 109 -
hadoop-hdds/docs/content/Freon.md | 64 -
hadoop-hdds/docs/content/Hdds.md | 65 -
hadoop-hdds/docs/content/JavaApi.md | 172 -
hadoop-hdds/docs/content/KeyCommands.md | 144 -
hadoop-hdds/docs/content/OzoneFS.md | 95 -
hadoop-hdds/docs/content/OzoneManager.md | 77 -
hadoop-hdds/docs/content/Prometheus.md | 94 -
hadoop-hdds/docs/content/RealCluster.md | 74 -
hadoop-hdds/docs/content/Rest.md | 544 -
hadoop-hdds/docs/content/RunningViaDocker.md | 73 -
hadoop-hdds/docs/content/RunningWithHDFS.md | 77 -
hadoop-hdds/docs/content/S3.md | 130 -
hadoop-hdds/docs/content/S3Commands.md | 41 -
hadoop-hdds/docs/content/SCMCLI.md | 29 -
hadoop-hdds/docs/content/Settings.md | 142 -
hadoop-hdds/docs/content/VolumeCommands.md | 116 -
hadoop-hdds/docs/content/_index.md | 38 -
hadoop-hdds/docs/content/genconf.md | 28 -
hadoop-hdds/docs/dev-support/bin/generate-site.sh | 29 -
hadoop-hdds/docs/pom.xml | 81 -
hadoop-hdds/docs/static/NOTES.md | 20 -
hadoop-hdds/docs/static/OzoneOverview.png | Bin 41729 -> 0 bytes
hadoop-hdds/docs/static/OzoneOverview.svg | 238 -
hadoop-hdds/docs/static/SCMBlockDiagram.png | Bin 14714 -> 0 bytes
hadoop-hdds/docs/static/ozone-logo.png | Bin 40237 -> 0 bytes
.../themes/ozonedoc/layouts/_default/single.html | 38 -
.../docs/themes/ozonedoc/layouts/index.html | 37 -
.../themes/ozonedoc/layouts/partials/footer.html | 22 -
.../themes/ozonedoc/layouts/partials/header.html | 34 -
.../themes/ozonedoc/layouts/partials/navbar.html | 37 -
.../themes/ozonedoc/layouts/partials/sidebar.html | 58 -
.../ozonedoc/static/css/bootstrap-theme.min.css | 6 -
.../static/css/bootstrap-theme.min.css.map | 1 -
.../themes/ozonedoc/static/css/bootstrap.min.css | 6 -
.../ozonedoc/static/css/bootstrap.min.css.map | 1 -
.../docs/themes/ozonedoc/static/css/ozonedoc.css | 138 -
.../static/fonts/glyphicons-halflings-regular.eot | Bin 20127 -> 0 bytes
.../static/fonts/glyphicons-halflings-regular.svg | 288 -
.../static/fonts/glyphicons-halflings-regular.ttf | Bin 45404 -> 0 bytes
.../static/fonts/glyphicons-halflings-regular.woff | Bin 23424 -> 0 bytes
.../fonts/glyphicons-halflings-regular.woff2 | Bin 18028 -> 0 bytes
.../themes/ozonedoc/static/js/bootstrap.min.js | 7 -
.../docs/themes/ozonedoc/static/js/jquery.min.js | 5 -
.../docs/themes/ozonedoc/static/js/ozonedoc.js | 23 -
hadoop-hdds/docs/themes/ozonedoc/theme.toml | 2 -
hadoop-hdds/framework/README.md | 24 -
hadoop-hdds/framework/pom.xml | 69 -
.../apache/hadoop/hdds/server/BaseHttpServer.java | 242 -
.../hadoop/hdds/server/PrometheusMetricsSink.java | 133 -
.../hadoop/hdds/server/PrometheusServlet.java | 46 -
.../org/apache/hadoop/hdds/server/ServerUtils.java | 181 -
.../hadoop/hdds/server/ServiceRuntimeInfo.java | 64 -
.../hadoop/hdds/server/ServiceRuntimeInfoImpl.java | 60 -
.../apache/hadoop/hdds/server/events/Event.java | 42 -
.../hadoop/hdds/server/events/EventExecutor.java | 68 -
.../hadoop/hdds/server/events/EventHandler.java | 33 -
.../hadoop/hdds/server/events/EventPublisher.java | 28 -
.../hadoop/hdds/server/events/EventQueue.java | 254 -
.../hadoop/hdds/server/events/EventWatcher.java | 217 -
.../hdds/server/events/EventWatcherMetrics.java | 79 -
.../server/events/IdentifiableEventPayload.java | 28 -
.../hdds/server/events/SingleThreadExecutor.java | 118 -
.../hadoop/hdds/server/events/TypedEvent.java | 58 -
.../hadoop/hdds/server/events/package-info.java | 23 -
.../apache/hadoop/hdds/server/package-info.java | 23 -
.../src/main/resources/webapps/datanode/dn.js | 92 -
.../resources/webapps/static/angular-1.6.4.min.js | 332 -
.../webapps/static/angular-nvd3-1.0.9.min.js | 1 -
.../webapps/static/angular-route-1.6.4.min.js | 17 -
.../main/resources/webapps/static/d3-3.5.17.min.js | 5 -
.../src/main/resources/webapps/static/dfs-dust.js | 133 -
.../resources/webapps/static/nvd3-1.8.5.min.css | 2 -
.../webapps/static/nvd3-1.8.5.min.css.map | 1 -
.../resources/webapps/static/nvd3-1.8.5.min.js | 11 -
.../resources/webapps/static/nvd3-1.8.5.min.js.map | 1 -
.../src/main/resources/webapps/static/ozone.css | 60 -
.../src/main/resources/webapps/static/ozone.js | 387 -
.../resources/webapps/static/templates/config.html | 91 -
.../resources/webapps/static/templates/jvm.html | 26 -
.../resources/webapps/static/templates/menu.html | 60 -
.../webapps/static/templates/overview.html | 39 -
.../webapps/static/templates/rpc-metrics.html | 87 -
.../hadoop/hdds/server/TestBaseHttpServer.java | 98 -
.../hdds/server/TestPrometheusMetricsSink.java | 96 -
.../hdds/server/events/EventHandlerStub.java | 38 -
.../hadoop/hdds/server/events/TestEventQueue.java | 83 -
.../hdds/server/events/TestEventQueueChain.java | 79 -
.../hdds/server/events/TestEventWatcher.java | 295 -
.../hadoop/hdds/server/events/package-info.java | 22 -
.../framework/src/test/resources/ozone-site.xml | 24 -
hadoop-hdds/pom.xml | 379 -
hadoop-hdds/server-scm/pom.xml | 140 -
.../java/org/apache/hadoop/hdds/scm/ScmUtils.java | 82 -
.../apache/hadoop/hdds/scm/block/BlockManager.java | 78 -
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 380 -
.../hadoop/hdds/scm/block/BlockmanagerMXBean.java | 30 -
.../block/DatanodeDeletedBlockTransactions.java | 146 -
.../hadoop/hdds/scm/block/DeletedBlockLog.java | 118 -
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 357 -
.../hdds/scm/block/PendingDeleteHandler.java | 41 -
.../hdds/scm/block/PendingDeleteStatusList.java | 85 -
.../hdds/scm/block/SCMBlockDeletingService.java | 197 -
.../apache/hadoop/hdds/scm/block/package-info.java | 22 -
.../hdds/scm/chillmode/ChillModeExitRule.java | 32 -
.../hdds/scm/chillmode/ChillModePrecheck.java | 69 -
.../hdds/scm/chillmode/ChillModeRestrictedOps.java | 41 -
.../hdds/scm/chillmode/ContainerChillModeRule.java | 111 -
.../hdds/scm/chillmode/DataNodeChillModeRule.java | 83 -
.../hdds/scm/chillmode/PipelineChillModeRule.java | 108 -
.../apache/hadoop/hdds/scm/chillmode/Precheck.java | 29 -
.../hdds/scm/chillmode/SCMChillModeManager.java | 175 -
.../hadoop/hdds/scm/chillmode/package-info.java | 18 -
.../scm/command/CommandStatusReportHandler.java | 139 -
.../hadoop/hdds/scm/command/package-info.java | 26 -
.../scm/container/CloseContainerEventHandler.java | 113 -
.../scm/container/ContainerActionsHandler.java | 60 -
.../hdds/scm/container/ContainerManager.java | 155 -
.../hdds/scm/container/ContainerReplica.java | 231 -
.../hdds/scm/container/ContainerReportHandler.java | 213 -
.../hdds/scm/container/ContainerStateManager.java | 586 -
.../container/DeleteContainerCommandWatcher.java | 56 -
.../IncrementalContainerReportHandler.java | 80 -
.../hdds/scm/container/ReportHandlerHelper.java | 365 -
.../hdds/scm/container/SCMContainerManager.java | 432 -
.../hdds/scm/container/closer/package-info.java | 23 -
.../hadoop/hdds/scm/container/package-info.java | 22 -
.../algorithms/ContainerPlacementPolicy.java | 44 -
.../placement/algorithms/SCMCommonPolicy.java | 201 -
.../algorithms/SCMContainerPlacementCapacity.java | 137 -
.../algorithms/SCMContainerPlacementRandom.java | 93 -
.../placement/algorithms/package-info.java | 18 -
.../container/placement/metrics/ContainerStat.java | 165 -
.../placement/metrics/DatanodeMetric.java | 90 -
.../container/placement/metrics/LongMetric.java | 161 -
.../scm/container/placement/metrics/NodeStat.java | 67 -
.../container/placement/metrics/SCMMetrics.java | 154 -
.../container/placement/metrics/SCMNodeMetric.java | 224 -
.../container/placement/metrics/SCMNodeStat.java | 141 -
.../container/placement/metrics/package-info.java | 20 -
.../hdds/scm/container/placement/package-info.java | 19 -
.../replication/ReplicationActivityStatus.java | 118 -
.../ReplicationActivityStatusMXBean.java | 28 -
.../replication/ReplicationCommandWatcher.java | 56 -
.../container/replication/ReplicationManager.java | 384 -
.../container/replication/ReplicationQueue.java | 73 -
.../container/replication/ReplicationRequest.java | 123 -
.../scm/container/replication/package-info.java | 23 -
.../scm/container/states/ContainerAttribute.java | 245 -
.../scm/container/states/ContainerQueryKey.java | 110 -
.../hdds/scm/container/states/ContainerState.java | 80 -
.../scm/container/states/ContainerStateMap.java | 547 -
.../hdds/scm/container/states/package-info.java | 22 -
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 257 -
.../hadoop/hdds/scm/events/package-info.java | 23 -
.../hadoop/hdds/scm/exceptions/SCMException.java | 124 -
.../hadoop/hdds/scm/exceptions/package-info.java | 19 -
.../metadata/DeletedBlocksTransactionCodec.java | 46 -
.../apache/hadoop/hdds/scm/metadata/LongCodec.java | 39 -
.../hadoop/hdds/scm/metadata/SCMMetadataStore.java | 73 -
.../hdds/scm/metadata/SCMMetadataStoreRDBImpl.java | 148 -
.../hadoop/hdds/scm/metadata/package-info.java | 21 -
.../apache/hadoop/hdds/scm/node/CommandQueue.java | 190 -
.../apache/hadoop/hdds/scm/node/DatanodeInfo.java | 129 -
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 138 -
.../hadoop/hdds/scm/node/NewNodeHandler.java | 47 -
.../apache/hadoop/hdds/scm/node/NodeManager.java | 174 -
.../hadoop/hdds/scm/node/NodeManagerMXBean.java | 45 -
.../hadoop/hdds/scm/node/NodeReportHandler.java | 55 -
.../hadoop/hdds/scm/node/NodeStateManager.java | 643 -
.../scm/node/NonHealthyToHealthyNodeHandler.java | 48 -
.../hadoop/hdds/scm/node/SCMNodeManager.java | 516 -
.../hdds/scm/node/SCMNodeStorageStatMXBean.java | 77 -
.../hdds/scm/node/SCMNodeStorageStatMap.java | 368 -
.../hadoop/hdds/scm/node/StaleNodeHandler.java | 72 -
.../hadoop/hdds/scm/node/StorageReportResult.java | 87 -
.../apache/hadoop/hdds/scm/node/package-info.java | 31 -
.../hdds/scm/node/states/Node2ContainerMap.java | 92 -
.../hdds/scm/node/states/Node2ObjectsMap.java | 164 -
.../hdds/scm/node/states/Node2PipelineMap.java | 77 -
.../node/states/NodeAlreadyExistsException.java | 45 -
.../hadoop/hdds/scm/node/states/NodeException.java | 44 -
.../scm/node/states/NodeNotFoundException.java | 47 -
.../hadoop/hdds/scm/node/states/NodeStateMap.java | 272 -
.../hadoop/hdds/scm/node/states/ReportResult.java | 110 -
.../hadoop/hdds/scm/node/states/package-info.java | 22 -
.../org/apache/hadoop/hdds/scm/package-info.java | 22 -
.../hdds/scm/pipeline/PipelineActionHandler.java | 74 -
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 57 -
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 70 -
.../hadoop/hdds/scm/pipeline/PipelineProvider.java | 35 -
.../hdds/scm/pipeline/PipelineReportHandler.java | 107 -
.../hdds/scm/pipeline/PipelineStateManager.java | 131 -
.../hadoop/hdds/scm/pipeline/PipelineStateMap.java | 299 -
.../hdds/scm/pipeline/RatisPipelineProvider.java | 138 -
.../hdds/scm/pipeline/RatisPipelineUtils.java | 275 -
.../hdds/scm/pipeline/SCMPipelineManager.java | 274 -
.../hdds/scm/pipeline/SimplePipelineProvider.java | 75 -
.../hadoop/hdds/scm/pipeline/package-info.java | 24 -
.../apache/hadoop/hdds/scm/ratis/package-info.java | 22 -
.../hdds/scm/server/SCMBlockProtocolServer.java | 298 -
.../hdds/scm/server/SCMClientProtocolServer.java | 555 -
.../hadoop/hdds/scm/server/SCMConfigurator.java | 202 -
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 275 -
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 424 -
.../apache/hadoop/hdds/scm/server/SCMMXBean.java | 67 -
.../hdds/scm/server/SCMSecurityProtocolServer.java | 168 -
.../hadoop/hdds/scm/server/SCMStorageConfig.java | 73 -
.../hdds/scm/server/StorageContainerManager.java | 1219 -
.../server/StorageContainerManagerHttpServer.java | 76 -
.../hadoop/hdds/scm/server/package-info.java | 22 -
.../commands/RetriableDatanodeEventWatcher.java | 58 -
.../ozone/protocol/commands/package-info.java | 18 -
.../src/main/resources/webapps/scm/index.html | 76 -
.../src/main/resources/webapps/scm/main.html | 20 -
.../main/resources/webapps/scm/scm-overview.html | 60 -
.../src/main/resources/webapps/scm/scm.js | 58 -
.../apache/hadoop/hdds/scm/HddsServerUtilTest.java | 308 -
.../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 110 -
.../hadoop/hdds/scm/HddsWhiteboxTestUtils.java | 103 -
.../hadoop/hdds/scm/TestHddsServerUtils.java | 206 -
.../scm/TestStorageContainerManagerHttpServer.java | 143 -
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 487 -
.../hadoop/hdds/scm/block/TestBlockManager.java | 302 -
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 437 -
.../apache/hadoop/hdds/scm/block/package-info.java | 23 -
.../scm/chillmode/TestSCMChillModeManager.java | 278 -
.../command/TestCommandStatusReportHandler.java | 123 -
.../hadoop/hdds/scm/command/package-info.java | 22 -
.../hadoop/hdds/scm/container/MockNodeManager.java | 514 -
.../container/TestCloseContainerEventHandler.java | 165 -
.../scm/container/TestContainerActionsHandler.java | 68 -
.../scm/container/TestContainerReportHandler.java | 669 -
.../scm/container/TestContainerReportHelper.java | 113 -
.../scm/container/TestContainerStateManager.java | 128 -
.../TestIncrementalContainerReportHandler.java | 218 -
.../scm/container/TestSCMContainerManager.java | 231 -
.../hdds/scm/container/closer/package-info.java | 22 -
.../hadoop/hdds/scm/container/package-info.java | 22 -
.../TestSCMContainerPlacementCapacity.java | 109 -
.../TestSCMContainerPlacementRandom.java | 89 -
.../replication/TestReplicationActivityStatus.java | 63 -
.../replication/TestReplicationManager.java | 290 -
.../replication/TestReplicationQueue.java | 134 -
.../scm/container/replication/package-info.java | 23 -
.../container/states/TestContainerAttribute.java | 143 -
.../hdds/scm/container/states/package-info.java | 22 -
.../hdds/scm/node/TestContainerPlacement.java | 171 -
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 332 -
.../hdds/scm/node/TestNodeReportHandler.java | 96 -
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 980 -
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 262 -
.../apache/hadoop/hdds/scm/node/package-info.java | 22 -
.../scm/node/states/TestNode2ContainerMap.java | 327 -
.../hadoop/hdds/scm/node/states/package-info.java | 23 -
.../org/apache/hadoop/hdds/scm/package-info.java | 21 -
.../scm/server/TestSCMClientProtocolServer.java | 60 -
.../server/TestSCMDatanodeHeartbeatDispatcher.java | 175 -
.../scm/server/TestSCMSecurityProtocolServer.java | 60 -
.../ozone/container/common/TestEndPoint.java | 543 -
.../ozone/container/common/package-info.java | 22 -
.../placement/TestContainerPlacement.java | 137 -
.../container/placement/TestDatanodeMetrics.java | 56 -
.../ozone/container/placement/package-info.java | 22 -
.../ozone/container/replication/package-info.java | 18 -
.../testutils/ReplicationNodeManagerMock.java | 312 -
.../ozone/container/testutils/package-info.java | 18 -
hadoop-hdds/tools/pom.xml | 57 -
.../hdds/scm/cli/ChillModeCheckSubcommand.java | 61 -
.../hadoop/hdds/scm/cli/ChillModeCommands.java | 60 -
.../hdds/scm/cli/ChillModeExitSubcommand.java | 57 -
.../org/apache/hadoop/hdds/scm/cli/SCMCLI.java | 155 -
.../hdds/scm/cli/container/CloseSubcommand.java | 54 -
.../hdds/scm/cli/container/CreateSubcommand.java | 65 -
.../hdds/scm/cli/container/DeleteSubcommand.java | 60 -
.../hdds/scm/cli/container/InfoSubcommand.java | 89 -
.../hdds/scm/cli/container/ListSubcommand.java | 83 -
.../hdds/scm/cli/container/package-info.java | 22 -
.../apache/hadoop/hdds/scm/cli/package-info.java | 23 -
.../scm/cli/pipeline/ClosePipelineSubcommand.java | 53 -
.../scm/cli/pipeline/ListPipelinesSubcommand.java | 48 -
.../hadoop/hdds/scm/cli/pipeline/package-info.java | 22 -
hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 38 +-
.../java/org/apache/hadoop/hdfs/ClientContext.java | 49 +
.../java/org/apache/hadoop/hdfs/DFSClient.java | 98 +
.../org/apache/hadoop/hdfs/DFSInputStream.java | 145 +-
.../org/apache/hadoop/hdfs/DFSOutputStream.java | 18 +-
.../apache/hadoop/hdfs/DFSStripedInputStream.java | 24 +-
.../apache/hadoop/hdfs/DFSStripedOutputStream.java | 20 +-
.../java/org/apache/hadoop/hdfs/DFSUtilClient.java | 20 +
.../java/org/apache/hadoop/hdfs/DataStreamer.java | 31 +-
.../org/apache/hadoop/hdfs/DeadNodeDetector.java | 590 +
.../apache/hadoop/hdfs/DistributedFileSystem.java | 38 +-
.../org/apache/hadoop/hdfs/HdfsConfiguration.java | 2 +
.../java/org/apache/hadoop/hdfs/HdfsKMSUtil.java | 18 +-
.../java/org/apache/hadoop/hdfs/StripeReader.java | 4 +
.../hadoop/hdfs/client/BlockReportOptions.java | 20 +-
.../hadoop/hdfs/client/DfsPathCapabilities.java | 62 +
.../org/apache/hadoop/hdfs/client/HdfsAdmin.java | 6 +
.../hadoop/hdfs/client/HdfsClientConfigKeys.java | 50 +
.../hdfs/client/impl/BlockReaderFactory.java | 5 +-
.../hadoop/hdfs/client/impl/BlockReaderRemote.java | 11 +-
.../hadoop/hdfs/client/impl/DfsClientConf.java | 33 +-
.../org/apache/hadoop/hdfs/net/BasicInetPeer.java | 4 +-
.../org/apache/hadoop/hdfs/net/NioInetPeer.java | 4 +-
.../org/apache/hadoop/hdfs/protocol/Block.java | 141 +-
.../hadoop/hdfs/protocol/ClientProtocol.java | 6 +-
.../hadoop/hdfs/protocol/ECBlockGroupStats.java | 71 +
.../apache/hadoop/hdfs/protocol/HdfsConstants.java | 5 +
.../hdfs/protocol/HdfsLocatedFileStatus.java | 2 +-
.../hadoop/hdfs/protocol/HdfsNamedFileStatus.java | 2 +-
.../hadoop/hdfs/protocol/ReplicatedBlockStats.java | 42 +
.../hdfs/protocol/datatransfer/PipelineAck.java | 2 +-
.../datatransfer/sasl/DataTransferSaslUtil.java | 72 +
.../datatransfer/sasl/SaslDataTransferClient.java | 131 +-
.../datatransfer/sasl/SaslParticipant.java | 26 +-
.../ClientDatanodeProtocolTranslatorPB.java | 10 +-
.../hadoop/hdfs/protocolPB/PBHelperClient.java | 39 +-
.../security/token/block/BlockTokenIdentifier.java | 43 +
.../hdfs/server/datanode/BlockMetadataHeader.java | 32 +-
.../datanode/CorruptMetaHeaderException.java | 36 +
.../ha/AbstractNNFailoverProxyProvider.java | 60 +-
.../namenode/ha/ObserverReadProxyProvider.java | 216 +-
.../hdfs/shortcircuit/ShortCircuitCache.java | 49 +-
.../hadoop/hdfs/util/CombinedHostsFileReader.java | 9 +-
.../hadoop/hdfs/util/CombinedHostsFileWriter.java | 7 +-
.../apache/hadoop/hdfs/util/StripedBlockUtil.java | 20 +-
.../org/apache/hadoop/hdfs/web/JsonUtilClient.java | 61 +-
.../hadoop/hdfs/web/SSLConnectionConfigurator.java | 72 +
.../hadoop/hdfs/web/URLConnectionFactory.java | 43 +-
.../apache/hadoop/hdfs/web/WebHdfsFileSystem.java | 173 +-
.../hadoop/hdfs/web/resources/GetOpParam.java | 1 +
.../hdfs/web/resources/NameSpaceQuotaParam.java | 44 +
.../hadoop/hdfs/web/resources/PutOpParam.java | 3 +
.../hdfs/web/resources/StorageSpaceQuotaParam.java | 45 +
.../hdfs/web/resources/StorageTypeParam.java | 37 +
.../src/main/proto/ClientDatanodeProtocol.proto | 3 +-
.../src/main/proto/ClientNamenodeProtocol.proto | 2 +-
.../src/main/proto/ReconfigurationProtocol.proto | 2 +-
.../hadoop-hdfs-client/src/main/proto/acl.proto | 2 +-
.../src/main/proto/datatransfer.proto | 8 +-
.../src/main/proto/encryption.proto | 2 +-
.../src/main/proto/erasurecoding.proto | 2 +-
.../hadoop-hdfs-client/src/main/proto/hdfs.proto | 5 +-
.../src/main/proto/inotify.proto | 2 +-
.../hadoop-hdfs-client/src/main/proto/xattr.proto | 2 +-
.../hadoop/hdfs/TestDefaultNameNodePort.java | 38 +-
.../hdfs/protocol/TestErasureCodingPolicyInfo.java | 2 +-
.../ha/TestConfiguredFailoverProxyProvider.java | 150 +-
.../hadoop/hdfs/web/TestURLConnectionFactory.java | 53 +
hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 3 +-
.../src/main/conf/httpfs-signature.secret | 1 -
.../hadoop/fs/http/client/HttpFSFileSystem.java | 141 +-
.../apache/hadoop/fs/http/server/FSOperations.java | 132 +-
.../fs/http/server/HttpFSAuthenticationFilter.java | 65 +-
.../fs/http/server/HttpFSExceptionProvider.java | 2 +-
.../fs/http/server/HttpFSParametersProvider.java | 2 +
.../apache/hadoop/fs/http/server/HttpFSServer.java | 16 +
.../fs/http/server/HttpFSServerWebServer.java | 22 +
.../java/org/apache/hadoop/lib/server/Server.java | 4 +-
.../service/hadoop/FileSystemAccessService.java | 21 +-
.../src/main/resources/httpfs-default.xml | 23 +-
.../src/main/resources/webapps/static/index.html | 16 +-
.../src/site/markdown/ServerSetup.md.vm | 3 +-
.../hadoop/fs/http/client/BaseTestHttpFSWith.java | 126 +-
.../hadoop/fs/http/server/TestHttpFSServer.java | 31 +
.../TestHttpFSServerWebServerWithRandomSecret.java | 58 +
.../org/apache/hadoop/test/TestJettyHelper.java | 3 +-
.../hadoop-hdfs-native-client/pom.xml | 3 +-
.../hadoop-hdfs-native-client/src/CMakeLists.txt | 1 +
.../main/native/libhdfs-examples/CMakeLists.txt | 34 +
.../src/main/native/libhdfs-examples/README.md | 24 +
.../main/native/libhdfs-examples/libhdfs_read.c | 77 +
.../main/native/libhdfs-examples/libhdfs_write.c | 104 +
.../main/native/libhdfs-examples/test-libhdfs.sh | 152 +
.../src/main/native/libhdfs-tests/CMakeLists.txt | 5 +-
.../src/main/native/libhdfs-tests/hdfs_test.h | 18 +
.../main/native/libhdfs-tests/native_mini_dfs.c | 44 +-
.../src/main/native/libhdfs-tests/test_htable.c | 100 -
.../main/native/libhdfs-tests/test_libhdfs_ops.c | 300 +-
.../main/native/libhdfs-tests/test_libhdfs_read.c | 72 -
.../main/native/libhdfs-tests/test_libhdfs_write.c | 99 -
.../src/main/native/libhdfs/CMakeLists.txt | 12 +-
.../src/main/native/libhdfs/common/htable.c | 287 -
.../src/main/native/libhdfs/common/htable.h | 161 -
.../src/main/native/libhdfs/exception.c | 6 +-
.../src/main/native/libhdfs/hdfs.c | 842 +-
.../src/main/native/libhdfs/include/hdfs/hdfs.h | 21 +-
.../src/main/native/libhdfs/jclasses.c | 136 +
.../src/main/native/libhdfs/jclasses.h | 112 +
.../src/main/native/libhdfs/jni_helper.c | 231 +-
.../src/main/native/libhdfs/jni_helper.h | 37 +-
.../src/main/native/libhdfs/os/mutexes.h | 6 +-
.../src/main/native/libhdfs/os/posix/mutexes.c | 2 +-
.../native/libhdfs/os/posix/thread_local_storage.c | 94 +-
.../src/main/native/libhdfs/os/windows/mutexes.c | 4 +-
.../main/native/libhdfspp/tests/hdfs_ext_test.cc | 5 +-
.../src/main/native/libhdfspp/tests/hdfs_shim.c | 6 +
.../libhdfspp/tests/libhdfs_wrapper_defines.h | 1 +
.../libhdfspp/tests/libhdfs_wrapper_undefs.h | 1 +
.../libhdfspp/tests/libhdfspp_wrapper_defines.h | 1 +
hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 2 +-
.../hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java | 6 +-
hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 94 +-
.../src/main/conf/hdfs-rbf-site.xml | 20 +
.../hdfs/protocolPB/RouterAdminProtocol.java | 34 +
.../hdfs/protocolPB/RouterAdminProtocolPB.java | 6 +-
.../RouterAdminProtocolServerSideTranslatorPB.java | 45 +
.../RouterAdminProtocolTranslatorPB.java | 42 +
.../hdfs/protocolPB/RouterPolicyProvider.java | 52 +
.../server/federation/metrics/FederationMBean.java | 66 +-
.../federation/metrics/FederationMetrics.java | 766 -
.../federation/metrics/FederationRPCMBean.java | 8 +
.../federation/metrics/FederationRPCMetrics.java | 16 +
.../metrics/FederationRPCPerformanceMonitor.java | 47 +-
.../federation/metrics/NamenodeBeanMetrics.java | 220 +-
.../federation/metrics/NullStateStoreMetrics.java | 56 +
.../hdfs/server/federation/metrics/RBFMetrics.java | 878 +
.../server/federation/metrics/RouterMBean.java | 104 +
.../federation/metrics/StateStoreMetrics.java | 4 +-
.../resolver/FederationNamenodeContext.java | 11 +-
.../resolver/FederationNamenodeServiceState.java | 6 +-
.../resolver/FileSubclusterResolver.java | 6 +-
.../resolver/MembershipNamenodeResolver.java | 10 +-
.../federation/resolver/MountTableManager.java | 28 +
.../federation/resolver/MountTableResolver.java | 83 +-
.../federation/resolver/NamenodeStatusReport.java | 75 +-
.../resolver/RouterResolveException.java | 32 +
.../resolver/order/DestinationOrder.java | 10 +-
.../federation/resolver/order/LocalResolver.java | 2 +-
.../federation/router/ConnectionManager.java | 24 +-
.../federation/router/ConnectionNullException.java | 33 +
.../server/federation/router/ConnectionPool.java | 166 +-
.../server/federation/router/ErasureCoding.java | 47 +-
.../server/federation/router/FederationUtil.java | 86 +-
.../router/MountTableRefresherService.java | 299 +
.../router/MountTableRefresherThread.java | 96 +
.../router/NamenodeHeartbeatService.java | 41 +-
.../router/NoNamenodesAvailableException.java | 33 +
.../hdfs/server/federation/router/Quota.java | 126 +-
.../server/federation/router/RBFConfigKeys.java | 81 +
.../server/federation/router/RemoteMethod.java | 23 +-
.../hdfs/server/federation/router/RemoteParam.java | 9 +
.../server/federation/router/RemoteResult.java | 84 +
.../hdfs/server/federation/router/Router.java | 186 +-
.../federation/router/RouterAdminServer.java | 210 +-
.../server/federation/router/RouterCacheAdmin.java | 173 +
.../federation/router/RouterClientProtocol.java | 663 +-
.../federation/router/RouterHeartbeatService.java | 5 +
.../server/federation/router/RouterHttpServer.java | 9 +-
.../federation/router/RouterMetricsService.java | 14 +-
.../federation/router/RouterNamenodeProtocol.java | 20 +-
.../federation/router/RouterPermissionChecker.java | 2 +-
.../federation/router/RouterQuotaManager.java | 46 +-
.../router/RouterQuotaUpdateService.java | 64 +-
.../server/federation/router/RouterQuotaUsage.java | 35 +-
.../server/federation/router/RouterRpcClient.java | 287 +-
.../server/federation/router/RouterRpcMonitor.java | 5 +
.../server/federation/router/RouterRpcServer.java | 343 +-
.../server/federation/router/RouterSnapshot.java | 208 +
.../federation/router/RouterStoragePolicy.java | 105 +
.../federation/router/RouterUserProtocol.java | 104 +
.../federation/router/RouterWebHdfsMethods.java | 167 +-
.../router/security/RouterSecurityManager.java | 259 +
.../federation/router/security/package-info.java | 28 +
.../token/ZKDelegationTokenSecretManagerImpl.java | 56 +
.../router/security/token/package-info.java | 29 +
.../server/federation/store/CachedRecordStore.java | 22 +-
.../server/federation/store/MountTableStore.java | 24 +
.../server/federation/store/StateStoreService.java | 49 +-
.../server/federation/store/StateStoreUtils.java | 26 +
.../federation/store/impl/MountTableStoreImpl.java | 31 +-
.../store/protocol/GetDestinationRequest.java | 57 +
.../store/protocol/GetDestinationResponse.java | 59 +
.../protocol/RefreshMountTableEntriesRequest.java | 34 +
.../protocol/RefreshMountTableEntriesResponse.java | 44 +
.../impl/pb/FederationProtocolPBTranslator.java | 4 +-
.../impl/pb/GetDestinationRequestPBImpl.java | 73 +
.../impl/pb/GetDestinationResponsePBImpl.java | 83 +
.../pb/RefreshMountTableEntriesRequestPBImpl.java | 67 +
.../pb/RefreshMountTableEntriesResponsePBImpl.java | 74 +
.../federation/store/records/BaseRecord.java | 46 +-
.../federation/store/records/MembershipState.java | 32 +-
.../federation/store/records/MembershipStats.java | 16 +
.../federation/store/records/MountTable.java | 56 +-
.../federation/store/records/QueryResult.java | 3 +-
.../federation/store/records/RouterState.java | 21 +
.../records/impl/pb/MembershipStatePBImpl.java | 24 +-
.../records/impl/pb/MembershipStatsPBImpl.java | 43 +
.../store/records/impl/pb/MountTablePBImpl.java | 14 +
.../store/records/impl/pb/RouterStatePBImpl.java | 14 +-
.../hadoop/hdfs/tools/federation/RouterAdmin.java | 548 +-
.../src/main/proto/FederationProtocol.proto | 25 +-
.../src/main/proto/RouterProtocol.proto | 12 +-
.../src/main/resources/hdfs-rbf-default.xml | 198 +-
.../src/main/webapps/router/federationhealth.html | 88 +-
.../src/main/webapps/router/federationhealth.js | 51 +-
.../src/main/webapps/static/rbf.css | 67 +-
.../src/site/markdown/HDFSRouterFederation.md | 66 +-
.../fs/contract/router/RouterHDFSContract.java | 26 +-
.../fs/contract/router/SecurityConfUtil.java | 170 +
.../router/TestRouterHDFSContractAppendSecure.java | 44 +
.../router/TestRouterHDFSContractConcatSecure.java | 48 +
.../router/TestRouterHDFSContractCreateSecure.java | 45 +
.../TestRouterHDFSContractDelegationToken.java | 115 +
.../router/TestRouterHDFSContractDeleteSecure.java | 44 +
.../TestRouterHDFSContractGetFileStatusSecure.java | 44 +
.../router/TestRouterHDFSContractMkdirSecure.java | 45 +
.../router/TestRouterHDFSContractOpenSecure.java | 44 +
.../router/TestRouterHDFSContractRenameSecure.java | 45 +
.../TestRouterHDFSContractRootDirectory.java | 10 +
.../TestRouterHDFSContractRootDirectorySecure.java | 70 +
.../router/TestRouterHDFSContractSeekSecure.java | 45 +
.../TestRouterHDFSContractSetTimesSecure.java | 45 +
.../contract/router/web/RouterWebHDFSContract.java | 11 +-
.../TestRouterWebHDFSContractRootDirectory.java | 10 +
.../server/federation/FederationTestUtils.java | 227 +-
.../server/federation/MiniRouterDFSCluster.java | 87 +-
.../hdfs/server/federation/MockNamenode.java | 559 +
.../hdfs/server/federation/MockResolver.java | 68 +-
.../server/federation/RouterConfigBuilder.java | 25 +
.../federation/metrics/TestFederationMetrics.java | 293 -
.../server/federation/metrics/TestMetricsBase.java | 66 +
.../server/federation/metrics/TestRBFMetrics.java | 329 +
.../resolver/TestInitializeMountTableResolver.java | 32 +-
.../resolver/TestMountTableResolver.java | 191 +-
.../resolver/order/TestLocalResolver.java | 2 +-
.../federation/router/TestConnectionManager.java | 104 +-
.../federation/router/TestDisableNameservices.java | 41 +-
.../federation/router/TestDisableRouterQuota.java | 13 +-
.../federation/router/TestFederationUtil.java | 69 +
.../hdfs/server/federation/router/TestRouter.java | 85 +
.../server/federation/router/TestRouterAdmin.java | 52 +-
.../federation/router/TestRouterAdminCLI.java | 555 +-
.../router/TestRouterAdminGenericRefresh.java | 252 +
.../router/TestRouterClientRejectOverload.java | 194 +-
.../federation/router/TestRouterFaultTolerant.java | 612 +
.../router/TestRouterMissingFolderMulti.java | 182 +
.../federation/router/TestRouterMountTable.java | 560 +-
.../router/TestRouterMountTableCacheRefresh.java | 396 +
.../router/TestRouterNamenodeMonitoring.java | 328 +-
.../router/TestRouterNamenodeWebScheme.java | 205 +
.../router/TestRouterPolicyProvider.java | 108 +
.../server/federation/router/TestRouterQuota.java | 418 +-
.../router/TestRouterRPCClientRetries.java | 6 +-
...erRPCMultipleDestinationMountTableResolver.java | 693 +
.../server/federation/router/TestRouterRpc.java | 542 +-
.../router/TestRouterRpcMultiDestination.java | 225 +-
.../federation/router/TestRouterRpcSingleNS.java | 211 +
.../TestRouterRpcStoragePolicySatisfier.java | 149 +
.../federation/router/TestRouterSafemode.java | 44 +
.../federation/router/TestRouterUserMappings.java | 398 +
.../router/TestRouterWithSecureStartup.java | 77 +
.../security/MockDelegationTokenSecretManager.java | 52 +
.../security/TestRouterHttpDelegationToken.java | 237 +
.../security/TestRouterSecurityManager.java | 204 +
.../store/FederationStateStoreTestUtils.java | 9 +-
.../store/TestStateStoreMembershipState.java | 134 +-
.../federation/store/TestStateStoreMountTable.java | 6 +-
.../store/TestStateStoreRouterState.java | 61 +-
.../store/driver/TestStateStoreDriverBase.java | 10 +-
.../store/records/TestMembershipState.java | 16 +-
.../federation/store/records/TestMountTable.java | 19 +
.../dev-support/findbugsExcludeFile.xml | 24 +-
.../jdiff/Apache_Hadoop_HDFS_2.10.0.xml | 312 +
.../dev-support/jdiff/Apache_Hadoop_HDFS_3.1.3.xml | 676 +
.../dev-support/jdiff/Apache_Hadoop_HDFS_3.2.1.xml | 674 +
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 138 +-
.../src/main/conf/user_ec_policies.xml.template | 7 +-
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 121 +-
.../main/java/org/apache/hadoop/hdfs/DFSUtil.java | 157 +-
.../apache/hadoop/hdfs/net/DFSNetworkTopology.java | 23 +-
.../hadoop/hdfs/net/DFSTopologyNodeImpl.java | 4 -
.../hadoop/hdfs/protocol/BlockListAsLongs.java | 6 +-
.../datatransfer/sasl/SaslDataTransferServer.java | 29 +-
.../hadoop/hdfs/protocolPB/AliasMapProtocolPB.java | 5 +-
...ientDatanodeProtocolServerSideTranslatorPB.java | 9 +-
...ientNamenodeProtocolServerSideTranslatorPB.java | 2 +-
.../hdfs/qjournal/client/IPCLoggerChannel.java | 20 +-
.../hadoop/hdfs/qjournal/client/QuorumCall.java | 21 +-
.../hdfs/qjournal/client/QuorumJournalManager.java | 19 +-
.../hdfs/qjournal/client/QuorumOutputStream.java | 10 +-
.../hadoop/hdfs/qjournal/server/JNStorage.java | 47 +-
.../hadoop/hdfs/qjournal/server/Journal.java | 40 +-
.../hdfs/qjournal/server/JournalNodeSyncer.java | 2 +-
.../token/block/BlockPoolTokenSecretManager.java | 17 +-
.../token/block/BlockTokenSecretManager.java | 67 +-
.../hdfs/server/aliasmap/InMemoryAliasMap.java | 5 +-
.../aliasmap/InMemoryLevelDBAliasMapServer.java | 5 -
.../hadoop/hdfs/server/balancer/Balancer.java | 88 +-
.../hdfs/server/balancer/BalancerParameters.java | 13 +
.../hadoop/hdfs/server/balancer/Dispatcher.java | 43 +-
.../hdfs/server/balancer/NameNodeConnector.java | 19 +-
.../server/blockmanagement/BlockIdManager.java | 46 +
.../hdfs/server/blockmanagement/BlockInfo.java | 6 +
.../blockmanagement/BlockInfoContiguous.java | 14 +
.../server/blockmanagement/BlockInfoStriped.java | 13 +-
.../hdfs/server/blockmanagement/BlockManager.java | 401 +-
.../blockmanagement/BlockManagerSafeMode.java | 51 +-
.../blockmanagement/BlockPlacementPolicy.java | 12 +
.../BlockPlacementPolicyDefault.java | 135 +-
.../BlockPlacementPolicyRackFaultTolerant.java | 4 +-
.../blockmanagement/BlockPlacementStatus.java | 8 +
.../BlockPlacementStatusDefault.java | 8 +
.../BlockPlacementStatusWithNodeGroup.java | 11 +
.../BlockPlacementStatusWithUpgradeDomain.java | 22 +-
.../blockmanagement/BlockReconstructionWork.java | 6 +
.../blockmanagement/BlockReportLeaseManager.java | 8 +-
.../blockmanagement/BlockStoragePolicySuite.java | 5 +
.../blockmanagement/DatanodeAdminManager.java | 167 +-
.../server/blockmanagement/DatanodeDescriptor.java | 21 +-
.../server/blockmanagement/DatanodeManager.java | 25 +-
.../server/blockmanagement/ErasureCodingWork.java | 36 +-
.../server/blockmanagement/HeartbeatManager.java | 3 +
.../server/blockmanagement/InvalidateBlocks.java | 20 +-
.../server/blockmanagement/NumberReplicas.java | 2 +
.../blockmanagement/PendingDataNodeMessages.java | 13 +-
.../PendingReconstructionBlocks.java | 12 +-
.../server/blockmanagement/ProvidedStorageMap.java | 8 +-
.../server/blockmanagement/ReplicationWork.java | 18 +-
.../hdfs/server/common/ECTopologyVerifier.java | 15 +-
.../hdfs/server/common/HdfsServerConstants.java | 2 +
.../common/HostRestrictingAuthorizationFilter.java | 443 +
.../hadoop/hdfs/server/common/JspHelper.java | 10 +-
.../apache/hadoop/hdfs/server/common/Storage.java | 33 +-
.../impl/InMemoryLevelDBAliasMapClient.java | 4 +-
.../hdfs/server/datanode/BPServiceActor.java | 43 +-
.../hadoop/hdfs/server/datanode/BlockReceiver.java | 5 +-
.../hadoop/hdfs/server/datanode/BlockSender.java | 31 +-
.../apache/hadoop/hdfs/server/datanode/DNConf.java | 30 +-
.../hadoop/hdfs/server/datanode/DataNode.java | 199 +-
.../hadoop/hdfs/server/datanode/DataStorage.java | 20 +-
.../hadoop/hdfs/server/datanode/DataXceiver.java | 30 +-
.../hdfs/server/datanode/DataXceiverServer.java | 445 +-
.../hadoop/hdfs/server/datanode/DiskBalancer.java | 30 +-
.../server/datanode/DiskFileCorruptException.java | 39 +
.../server/datanode/FSCachingGetSpaceUsed.java | 82 +
.../hdfs/server/datanode/FileIoProvider.java | 5 +-
.../hadoop/hdfs/server/datanode/LocalReplica.java | 15 +-
.../hadoop/hdfs/server/datanode/VolumeScanner.java | 11 +-
.../datanode/checker/DatasetVolumeChecker.java | 25 +-
.../datanode/checker/StorageLocationChecker.java | 8 +-
.../datanode/checker/ThrottledAsyncChecker.java | 6 +-
.../datanode/erasurecode/StripedBlockReader.java | 2 +-
.../server/datanode/fsdataset/FsDatasetSpi.java | 8 +
.../fsdataset/RoundRobinVolumeChoosingPolicy.java | 4 +
.../fsdataset/impl/AddBlockPoolException.java | 68 +
.../datanode/fsdataset/impl/BlockPoolSlice.java | 38 +-
.../server/datanode/fsdataset/impl/CacheStats.java | 212 +
.../datanode/fsdataset/impl/FsDatasetCache.java | 231 +-
.../datanode/fsdataset/impl/FsDatasetImpl.java | 74 +-
.../datanode/fsdataset/impl/FsDatasetUtil.java | 52 +-
.../datanode/fsdataset/impl/FsVolumeImpl.java | 13 +-
.../datanode/fsdataset/impl/FsVolumeList.java | 34 +-
.../datanode/fsdataset/impl/MappableBlock.java | 155 +-
.../fsdataset/impl/MappableBlockLoader.java | 193 +
.../fsdataset/impl/MappableBlockLoaderFactory.java | 51 +
.../fsdataset/impl/MemoryMappableBlockLoader.java | 125 +
.../datanode/fsdataset/impl/MemoryMappedBlock.java | 59 +
.../impl/NativePmemMappableBlockLoader.java | 192 +
.../fsdataset/impl/NativePmemMappedBlock.java | 85 +
.../fsdataset/impl/PmemMappableBlockLoader.java | 149 +
.../datanode/fsdataset/impl/PmemMappedBlock.java | 69 +
.../datanode/fsdataset/impl/PmemVolumeManager.java | 398 +
.../fsdataset/impl/ReplicaCachingGetSpaceUsed.java | 109 +
.../server/datanode/fsdataset/impl/ReplicaMap.java | 14 +
.../datanode/metrics/DataNodeDiskMetrics.java | 78 +-
.../datanode/metrics/DataNodePeerMetrics.java | 23 +-
.../server/datanode/web/DatanodeHttpServer.java | 247 +-
.../HostRestrictingAuthorizationFilterHandler.java | 240 +
.../web/RestCsrfPreventionFilterHandler.java | 64 +-
.../hdfs/server/datanode/web/package-info.java | 21 +
.../datanode/web/webhdfs/WebHdfsHandler.java | 3 +-
.../hdfs/server/diskbalancer/command/Command.java | 4 +-
.../org/apache/hadoop/hdfs/server/mover/Mover.java | 4 +-
.../hdfs/server/namenode/BackupJournalManager.java | 2 +-
.../hdfs/server/namenode/DefaultAuditLogger.java | 93 +
.../server/namenode/DirectoryWithQuotaFeature.java | 2 +-
.../server/namenode/EditLogBackupOutputStream.java | 8 +-
.../server/namenode/EditLogFileInputStream.java | 14 +-
.../server/namenode/EditLogFileOutputStream.java | 3 +-
.../hdfs/server/namenode/EditLogOutputStream.java | 16 +
.../hdfs/server/namenode/EditsDoubleBuffer.java | 10 +-
.../hadoop/hdfs/server/namenode/FSDirAclOp.java | 9 +-
.../hadoop/hdfs/server/namenode/FSDirAppendOp.java | 7 +-
.../hadoop/hdfs/server/namenode/FSDirAttrOp.java | 61 +-
.../hadoop/hdfs/server/namenode/FSDirConcatOp.java | 1 +
.../hadoop/hdfs/server/namenode/FSDirDeleteOp.java | 50 +-
.../hadoop/hdfs/server/namenode/FSDirMkdirOp.java | 18 +-
.../hadoop/hdfs/server/namenode/FSDirRenameOp.java | 69 +-
.../hdfs/server/namenode/FSDirSnapshotOp.java | 13 +-
.../server/namenode/FSDirStatAndListingOp.java | 45 +-
.../hdfs/server/namenode/FSDirWriteFileOp.java | 2 +-
.../hadoop/hdfs/server/namenode/FSDirXAttrOp.java | 21 +
.../hadoop/hdfs/server/namenode/FSDirectory.java | 9 +-
.../hadoop/hdfs/server/namenode/FSEditLog.java | 37 +-
.../hdfs/server/namenode/FSEditLogLoader.java | 17 +-
.../hadoop/hdfs/server/namenode/FSEditLogOp.java | 74 +-
.../hadoop/hdfs/server/namenode/FSImage.java | 45 +-
.../hadoop/hdfs/server/namenode/FSImageFormat.java | 9 +-
.../hdfs/server/namenode/FSImageFormatPBINode.java | 241 +-
.../server/namenode/FSImageFormatProtobuf.java | 337 +-
.../FSImagePreTransactionalStorageInspector.java | 5 +-
.../hdfs/server/namenode/FSImageSerialization.java | 4 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 1297 +-
.../hdfs/server/namenode/FSNamesystemLock.java | 154 +-
.../hdfs/server/namenode/FileJournalManager.java | 48 +-
.../hdfs/server/namenode/GlobalStateIdContext.java | 26 +-
.../hdfs/server/namenode/HdfsAuditLogger.java | 7 +-
.../apache/hadoop/hdfs/server/namenode/INode.java | 25 +
.../hdfs/server/namenode/INodeDirectory.java | 54 +-
.../hadoop/hdfs/server/namenode/INodeFile.java | 15 +
.../hadoop/hdfs/server/namenode/INodeId.java | 8 +-
.../hdfs/server/namenode/INodeReference.java | 17 +-
.../hadoop/hdfs/server/namenode/ImageServlet.java | 63 +-
.../hadoop/hdfs/server/namenode/LeaseManager.java | 8 +-
.../hadoop/hdfs/server/namenode/NNStorage.java | 24 +-
.../server/namenode/NNStorageRetentionManager.java | 46 +-
.../hadoop/hdfs/server/namenode/NameNode.java | 139 +-
.../hdfs/server/namenode/NameNodeHttpServer.java | 69 +-
.../hdfs/server/namenode/NameNodeMXBean.java | 179 +-
.../hdfs/server/namenode/NameNodeRpcServer.java | 120 +-
.../hadoop/hdfs/server/namenode/NamenodeFsck.java | 3 +-
.../hadoop/hdfs/server/namenode/QuotaCounts.java | 153 +-
.../hdfs/server/namenode/ha/BootstrapStandby.java | 3 +
.../hdfs/server/namenode/ha/EditLogTailer.java | 83 +-
.../hadoop/hdfs/server/namenode/ha/HAState.java | 6 +-
.../server/namenode/ha/StandbyCheckpointer.java | 146 +-
.../server/namenode/metrics/FSNamesystemMBean.java | 6 +
.../namenode/snapshot/AbstractINodeDiffList.java | 19 +-
.../snapshot/DirectorySnapshottableFeature.java | 19 +-
.../snapshot/DirectoryWithSnapshotFeature.java | 21 +-
.../namenode/snapshot/FSImageFormatPBSnapshot.java | 9 +-
.../namenode/snapshot/FileWithSnapshotFeature.java | 5 +
.../server/namenode/snapshot/SnapshotManager.java | 28 +-
.../namenode/startupprogress/StartupProgress.java | 35 +-
.../web/resources/NamenodeWebHdfsMethods.java | 94 +-
.../hdfs/server/protocol/NamenodeProtocol.java | 2 +
.../org/apache/hadoop/hdfs/tools/AdminHelper.java | 10 +-
.../org/apache/hadoop/hdfs/tools/CryptoAdmin.java | 4 +-
.../org/apache/hadoop/hdfs/tools/DFSAdmin.java | 57 +-
.../org/apache/hadoop/hdfs/tools/DebugAdmin.java | 4 +-
.../apache/hadoop/hdfs/tools/DiskBalancerCLI.java | 8 +
.../java/org/apache/hadoop/hdfs/tools/ECAdmin.java | 42 +-
.../OfflineEditsVisitorFactory.java | 7 +-
.../OfflineImageReconstructor.java | 20 +-
.../offlineImageViewer/OfflineImageViewer.java | 6 +-
.../offlineImageViewer/OfflineImageViewerPB.java | 6 +-
.../PBImageDelimitedTextWriter.java | 172 +-
.../offlineImageViewer/PBImageTextWriter.java | 15 +
.../tools/offlineImageViewer/PBImageXmlWriter.java | 2 +-
.../offlineImageViewer/TextWriterImageVisitor.java | 6 +-
.../apache/hadoop/hdfs/util/ConstEnumCounters.java | 98 +
.../org/apache/hadoop/hdfs/util/EnumCounters.java | 29 +-
.../org/apache/hadoop/hdfs/util/MD5FileUtils.java | 8 +-
.../org/apache/hadoop/hdfs/web/AuthFilter.java | 115 +-
.../hadoop/hdfs/web/AuthFilterInitializer.java | 69 +
.../java/org/apache/hadoop/hdfs/web/JsonUtil.java | 41 +-
.../src/main/native/tests/test-libhdfs.sh | 152 -
.../src/main/proto/AliasMapProtocol.proto | 2 +-
.../src/main/proto/DatanodeLifelineProtocol.proto | 2 +-
.../src/main/proto/DatanodeProtocol.proto | 2 +-
.../hadoop-hdfs/src/main/proto/HAZKInfo.proto | 2 +-
.../hadoop-hdfs/src/main/proto/HdfsServer.proto | 2 +-
.../src/main/proto/InterDatanodeProtocol.proto | 2 +-
.../src/main/proto/InterQJournalProtocol.proto | 2 +-
.../src/main/proto/JournalProtocol.proto | 2 +-
.../src/main/proto/NamenodeProtocol.proto | 2 +-
.../src/main/proto/QJournalProtocol.proto | 2 +-
.../hadoop-hdfs/src/main/proto/editlog.proto | 2 +-
.../hadoop-hdfs/src/main/proto/fsimage.proto | 6 +-
.../src/main/resources/hdfs-default.xml | 458 +-
.../src/main/webapps/datanode/datanode.html | 7 +-
.../src/main/webapps/hdfs/dfshealth.html | 29 +-
.../hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js | 85 +-
.../src/main/webapps/hdfs/explorer.html | 11 +-
.../hadoop-hdfs/src/main/webapps/hdfs/explorer.js | 28 +-
.../src/main/webapps/journal/index.html | 6 +-
.../src/main/webapps/secondary/status.html | 7 +-
.../static/bootstrap-3.3.7/css/bootstrap-theme.css | 587 -
.../bootstrap-3.3.7/css/bootstrap-theme.css.map | 1 -
.../bootstrap-3.3.7/css/bootstrap-theme.min.css | 6 -
.../css/bootstrap-theme.min.css.map | 1 -
.../static/bootstrap-3.3.7/css/bootstrap.css | 6757 ---
.../static/bootstrap-3.3.7/css/bootstrap.css.map | 1 -
.../static/bootstrap-3.3.7/css/bootstrap.min.css | 6 -
.../bootstrap-3.3.7/css/bootstrap.min.css.map | 1 -
.../webapps/static/bootstrap-3.3.7/js/bootstrap.js | 2377 --
.../static/bootstrap-3.3.7/js/bootstrap.min.js | 7 -
.../css/bootstrap-editable.css | 0
.../static/bootstrap-3.4.1/css/bootstrap-theme.css | 587 +
.../bootstrap-3.4.1/css/bootstrap-theme.css.map | 1 +
.../bootstrap-3.4.1/css/bootstrap-theme.min.css | 6 +
.../css/bootstrap-theme.min.css.map | 1 +
.../static/bootstrap-3.4.1/css/bootstrap.css | 6834 ++++
.../static/bootstrap-3.4.1/css/bootstrap.css.map | 1 +
.../static/bootstrap-3.4.1/css/bootstrap.min.css | 6 +
.../bootstrap-3.4.1/css/bootstrap.min.css.map | 1 +
.../fonts/glyphicons-halflings-regular.eot | Bin
.../fonts/glyphicons-halflings-regular.svg | 0
.../fonts/glyphicons-halflings-regular.ttf | Bin
.../fonts/glyphicons-halflings-regular.woff | Bin
.../fonts/glyphicons-halflings-regular.woff2 | Bin
.../js/bootstrap-editable.min.js | 0
.../webapps/static/bootstrap-3.4.1/js/bootstrap.js | 2580 ++
.../static/bootstrap-3.4.1/js/bootstrap.min.js | 6 +
.../{bootstrap-3.3.7 => bootstrap-3.4.1}/js/npm.js | 0
.../src/main/webapps/static/dfs-dust.js | 16 +
.../hadoop-hdfs/src/main/webapps/static/hadoop.css | 61 +-
.../src/main/webapps/static/jquery-3.3.1.min.js | 2 -
.../src/main/webapps/static/jquery-3.4.1.min.js | 2 +
.../site/markdown/CentralizedCacheManagement.md | 18 +-
.../hadoop-hdfs/src/site/markdown/Federation.md | 4 +-
.../hadoop-hdfs/src/site/markdown/HDFSCommands.md | 30 +-
.../src/site/markdown/HDFSErasureCoding.md | 6 +
.../site/markdown/HDFSHighAvailabilityWithNFS.md | 12 +
.../site/markdown/HDFSHighAvailabilityWithQJM.md | 12 +
.../src/site/markdown/HdfsPermissionsGuide.md | 2 +-
.../src/site/markdown/HdfsQuotaAdminGuide.md | 4 +-
.../hadoop-hdfs/src/site/markdown/HdfsUserGuide.md | 22 +
.../src/site/markdown/ObserverNameNode.md | 99 +-
.../hadoop-hdfs/src/site/markdown/WebHDFS.md | 277 +
.../fs/contract/hdfs/TestHDFSContractUnbuffer.java | 46 +
.../hadoop/fs/viewfs/TestViewFsDefaultValue.java | 1 +
.../java/org/apache/hadoop/hdfs/DFSTestUtil.java | 36 +-
.../org/apache/hadoop/hdfs/MiniDFSCluster.java | 73 +-
.../apache/hadoop/hdfs/TestBalancerBandwidth.java | 8 +
.../hadoop/hdfs/TestBlockMissingException.java | 2 -
.../hadoop/hdfs/TestBlockTokenWrappingQOP.java | 43 +-
.../apache/hadoop/hdfs/TestByteBufferPread.java | 290 +
.../java/org/apache/hadoop/hdfs/TestClose.java | 1 -
.../org/apache/hadoop/hdfs/TestDFSConfigKeys.java | 37 -
.../TestDFSInotifyEventInputStreamKerberized.java | 9 +
.../org/apache/hadoop/hdfs/TestDFSInputStream.java | 43 +
.../java/org/apache/hadoop/hdfs/TestDFSShell.java | 29 +-
.../hadoop/hdfs/TestDFSStripedInputStream.java | 84 +
.../java/org/apache/hadoop/hdfs/TestDFSUtil.java | 31 +
.../apache/hadoop/hdfs/TestDeadNodeDetection.java | 355 +
.../org/apache/hadoop/hdfs/TestDecommission.java | 158 +-
.../hadoop/hdfs/TestDecommissionWithStriped.java | 481 +-
.../hadoop/hdfs/TestDistributedFileSystem.java | 137 +-
.../apache/hadoop/hdfs/TestEncryptionZones.java | 79 +
.../org/apache/hadoop/hdfs/TestFileAppend.java | 2 -
.../apache/hadoop/hdfs/TestHAAuxiliaryPort.java | 2 +-
.../java/org/apache/hadoop/hdfs/TestHDFSTrash.java | 2 +-
.../java/org/apache/hadoop/hdfs/TestHdfsAdmin.java | 38 +-
.../apache/hadoop/hdfs/TestMaintenanceState.java | 1 +
.../apache/hadoop/hdfs/TestMissingBlocksAlert.java | 30 +
.../apache/hadoop/hdfs/TestMultipleNNPortQOP.java | 219 +
.../java/org/apache/hadoop/hdfs/TestQuota.java | 70 +-
.../org/apache/hadoop/hdfs/TestReplication.java | 111 +
.../java/org/apache/hadoop/hdfs/TestSafeMode.java | 10 +-
.../java/org/apache/hadoop/hdfs/TestSetTimes.java | 7 +
.../hdfs/TestStateAlignmentContextWithHA.java | 1 +
.../apache/hadoop/hdfs/TestStripedFileAppend.java | 149 +
.../hadoop/hdfs/net/TestDFSNetworkTopology.java | 66 +
.../hadoop/hdfs/protocolPB/TestPBHelper.java | 2 +-
.../hadoop/hdfs/qjournal/MiniJournalCluster.java | 6 +-
.../apache/hadoop/hdfs/qjournal/QJMTestUtil.java | 4 +-
.../hadoop/hdfs/qjournal/TestSecureNNWithQJM.java | 8 +
.../qjournal/client/TestQuorumJournalManager.java | 34 +-
.../client/TestQuorumJournalManagerUnit.java | 61 +-
.../hadoop/hdfs/qjournal/server/TestJournal.java | 43 +
.../hdfs/qjournal/server/TestJournalNodeSync.java | 90 +-
.../hadoop/hdfs/security/TestDelegationToken.java | 24 +-
.../hdfs/security/token/block/TestBlockToken.java | 121 +
.../hdfs/server/aliasmap/TestSecureAliasMap.java | 155 +
.../hadoop/hdfs/server/balancer/TestBalancer.java | 66 +-
.../hdfs/server/balancer/TestBalancerRPCDelay.java | 28 +-
.../hdfs/server/balancer/TestBalancerService.java | 189 +
.../balancer/TestBalancerWithHANameNodes.java | 21 +
.../server/balancer/TestBalancerWithNodeGroup.java | 1 +
.../blockmanagement/BlockManagerTestUtil.java | 54 +-
.../hdfs/server/blockmanagement/TestBlockInfo.java | 36 +
.../server/blockmanagement/TestBlockManager.java | 182 +-
.../blockmanagement/TestBlockManagerSafeMode.java | 120 +
.../TestBlockPlacementStatusDefault.java | 57 +
.../TestBlockPlacementStatusWithUpgradeDomain.java | 59 +-
.../blockmanagement/TestBlockReportLease.java | 159 +
.../TestBlocksWithNotEnoughRacks.java | 113 +-
.../blockmanagement/TestDatanodeManager.java | 149 +-
.../TestLowRedundancyBlockQueues.java | 4 +-
.../TestPendingDataNodeMessages.java | 46 +
.../blockmanagement/TestRedundancyMonitor.java | 108 +
.../blockmanagement/TestReplicationPolicy.java | 23 +
.../TestSortLocatedStripedBlock.java | 6 +-
.../TestHostRestrictingAuthorizationFilter.java | 275 +
.../hadoop/hdfs/server/common/TestJspHelper.java | 122 +-
.../hdfs/server/datanode/DataNodeTestUtils.java | 5 +
.../hdfs/server/datanode/SimulatedFSDataset.java | 27 +-
.../hdfs/server/datanode/TestBPOfferService.java | 128 +-
.../server/datanode/TestCorruptMetadataFile.java | 165 +
.../datanode/TestDataNodeHotSwapVolumes.java | 73 +-
.../server/datanode/TestDataNodePeerMetrics.java | 2 +-
.../datanode/TestDataNodeReconfiguration.java | 120 +-
.../server/datanode/TestDataNodeVolumeFailure.java | 111 +
.../TestDataNodeVolumeFailureToleration.java | 4 +-
.../datanode/TestDataXceiverBackwardsCompat.java | 2 +-
.../hdfs/server/datanode/TestDatanodeRegister.java | 56 +
.../hdfs/server/datanode/TestDirectoryScanner.java | 150 +-
.../hadoop/hdfs/server/datanode/TestDiskError.java | 4 +-
.../hdfs/server/datanode/TestFsDatasetCache.java | 636 -
.../datanode/TestFsDatasetCacheRevocation.java | 31 +-
.../hdfs/server/datanode/TestLargeBlockReport.java | 6 +-
.../hdfs/server/datanode/TestTransferRbw.java | 13 +-
.../server/datanode/TestTriggerBlockReport.java | 79 +-
.../datanode/checker/TestDatasetVolumeChecker.java | 37 +
.../checker/TestStorageLocationChecker.java | 33 +-
.../checker/TestThrottledAsyncCheckerTimeout.java | 11 +-
.../datanode/extdataset/ExternalDatasetImpl.java | 6 +
.../fsdataset/impl/FsDatasetImplTestUtils.java | 3 +-
.../fsdataset/impl/TestAddBlockPoolException.java | 80 +
.../impl/TestCacheByPmemMappableBlockLoader.java | 325 +
.../fsdataset/impl/TestFsDatasetCache.java | 639 +
.../datanode/fsdataset/impl/TestFsVolumeList.java | 30 +-
.../datanode/fsdataset/impl/TestProvidedImpl.java | 8 +
.../impl/TestReplicaCachingGetSpaceUsed.java | 203 +
.../datanode/fsdataset/impl/TestReplicaMap.java | 22 +
.../TestDataNodeOutlierDetectionViaMetrics.java | 13 +-
...tHostRestrictingAuthorizationFilterHandler.java | 178 +
.../hdfs/server/diskbalancer/TestDiskBalancer.java | 12 +-
.../command/TestDiskBalancerCommand.java | 11 +
.../hadoop/hdfs/server/namenode/FSAclBaseTest.java | 2 +-
.../hdfs/server/namenode/FSImageTestUtil.java | 42 +-
.../server/namenode/NNThroughputBenchmark.java | 7 +-
.../hdfs/server/namenode/NameNodeAdapter.java | 45 +
.../hdfs/server/namenode/TestAclConfigFlag.java | 5 +-
.../hdfs/server/namenode/TestAuditLogAtDebug.java | 4 +-
.../TestBlockPlacementPolicyRackFaultTolerant.java | 3 +-
.../hdfs/server/namenode/TestCacheDirectives.java | 64 +
.../hdfs/server/namenode/TestCheckpoint.java | 57 +-
.../hdfs/server/namenode/TestDeleteRace.java | 72 +
.../hadoop/hdfs/server/namenode/TestEditLog.java | 171 +
.../namenode/TestEditLogFileInputStream.java | 22 +
.../server/namenode/TestEditsDoubleBuffer.java | 9 +-
.../hadoop/hdfs/server/namenode/TestFSImage.java | 154 +-
.../server/namenode/TestFSImageWithSnapshot.java | 103 +-
.../hdfs/server/namenode/TestFSNamesystem.java | 14 +-
.../hdfs/server/namenode/TestFSNamesystemLock.java | 9 +
.../hadoop/hdfs/server/namenode/TestFsck.java | 8 +-
.../namenode/TestINodeAttributeProvider.java | 33 +
.../hadoop/hdfs/server/namenode/TestINodeFile.java | 35 +-
.../server/namenode/TestListCorruptFileBlocks.java | 43 +-
.../hadoop/hdfs/server/namenode/TestMetaSave.java | 60 +
.../namenode/TestNNStorageRetentionManager.java | 169 +-
.../hdfs/server/namenode/TestNameNodeMXBean.java | 14 +-
.../server/namenode/TestProtectedDirectories.java | 54 +-
.../hdfs/server/namenode/TestQuotaCounts.java | 135 +
.../hdfs/server/namenode/TestRedudantBlocks.java | 141 +
.../TestRefreshNamenodeReplicationConfig.java | 143 +
.../hadoop/hdfs/server/namenode/TestStartup.java | 27 +-
.../hadoop/hdfs/server/namenode/ha/HATestUtil.java | 20 +-
.../server/namenode/ha/TestAddBlockTailing.java | 164 +
.../TestBootstrapStandbyWithInProgressTailing.java | 46 +
.../namenode/ha/TestBootstrapStandbyWithQJM.java | 9 +-
.../namenode/ha/TestConsistentReadsObserver.java | 173 +-
.../namenode/ha/TestDelegationTokensWithHA.java | 46 +
.../hdfs/server/namenode/ha/TestEditLogTailer.java | 140 +-
.../ha/TestFailoverWithBlockTokensEnabled.java | 5 +-
.../server/namenode/ha/TestFailureToReadEdits.java | 12 +-
.../hdfs/server/namenode/ha/TestHASafeMode.java | 68 +-
.../server/namenode/ha/TestMultiObserverNode.java | 2 +
.../hdfs/server/namenode/ha/TestNNHealthCheck.java | 30 +
.../hdfs/server/namenode/ha/TestObserverNode.java | 14 +
.../namenode/ha/TestObserverReadProxyProvider.java | 29 +
.../server/namenode/ha/TestStandbyCheckpoints.java | 35 +-
.../namenode/snapshot/SnapshotTestHelper.java | 3 +-
.../TestGetContentSummaryWithSnapshot.java | 33 +-
.../namenode/snapshot/TestRenameWithSnapshots.java | 55 +-
.../server/namenode/snapshot/TestSnapshot.java | 34 +
.../namenode/snapshot/TestSnapshotDeletion.java | 2 +-
.../namenode/snapshot/TestSnapshotManager.java | 9 +-
.../startupprogress/TestStartupProgress.java | 59 +-
.../web/resources/TestWebHdfsDataLocality.java | 23 +
.../apache/hadoop/hdfs/tools/TestAdminHelper.java | 50 +
.../org/apache/hadoop/hdfs/tools/TestDFSAdmin.java | 27 +-
.../hadoop/hdfs/tools/TestDFSAdminWithHA.java | 12 +-
.../hdfs/tools/TestDFSHAAdminMiniCluster.java | 6 +-
.../hdfs/tools/TestDFSZKFailoverController.java | 41 +-
.../org/apache/hadoop/hdfs/tools/TestECAdmin.java | 235 +-
.../offlineImageViewer/TestOfflineImageViewer.java | 153 +-
.../TestOfflineImageViewerForStoragePolicy.java | 183 +
.../org/apache/hadoop/hdfs/web/TestAuthFilter.java | 125 +-
.../org/apache/hadoop/hdfs/web/TestJsonUtil.java | 66 +
.../org/apache/hadoop/hdfs/web/TestWebHDFS.java | 1767 +-
.../hdfs/web/TestWebHdfsFileSystemContract.java | 5 +-
.../hadoop/hdfs/web/TestWebHdfsTimeouts.java | 28 +
.../apache/hadoop/hdfs/web/TestWebHdfsTokens.java | 227 +-
.../org/apache/hadoop/hdfs/web/TestWebHdfsUrl.java | 99 +-
.../web/TestWebHdfsWithAuthenticationFilter.java | 18 +-
.../TestWebHdfsWithRestCsrfPreventionFilter.java | 20 +-
.../apache/hadoop/hdfs/web/WebHdfsTestUtil.java | 25 +-
.../hadoop/hdfs/web/resources/TestParam.java | 29 +
.../org/apache/hadoop/net/TestNetworkTopology.java | 96 +-
.../src/test/resources/contract/hdfs.xml | 5 +
.../hadoop-hdfs/src/test/resources/editsStored | Bin 7909 -> 7925 bytes
.../hadoop-hdfs/src/test/resources/editsStored.xml | 2 +
.../testAclCLIWithPosixAclInheritance.xml | 77 +
.../src/test/resources/testHDFSConf.xml | 8 +-
.../src/test/resources/testStoragePolicy.csv | 26 +
hadoop-hdfs-project/pom.xml | 2 +-
.../.launches/AllMapredTests.launch | 28 -
.../.eclipse.templates/.launches/JobTracker.launch | 24 -
.../.launches/SpecificTestTemplate.launch | 28 -
.../.launches/TaskTracker.launch | 24 -
.../.eclipse.templates/README.txt | 6 -
.../Apache_Hadoop_MapReduce_Common_2.10.0.xml | 113 +
.../jdiff/Apache_Hadoop_MapReduce_Core_2.10.0.xml | 27739 +++++++++++++
.../Apache_Hadoop_MapReduce_JobClient_2.10.0.xml | 16 +
.../hadoop-mapreduce-client-app/pom.xml | 7 +-
.../mapreduce/v2/app/TaskHeartbeatHandler.java | 5 +-
.../mapreduce/v2/app/job/impl/TaskAttemptImpl.java | 24 +-
.../mapreduce/v2/app/rm/RMContainerAllocator.java | 20 +-
.../mapreduce/v2/app/speculate/DataStatistics.java | 18 +-
.../v2/app/speculate/DefaultSpeculator.java | 3 +-
.../SimpleExponentialTaskRuntimeEstimator.java | 170 +
.../v2/app/speculate/StartEndTimesBase.java | 3 +-
.../v2/app/speculate/TaskRuntimeEstimator.java | 15 +
.../forecast/SimpleExponentialSmoothing.java | 196 +
.../hadoop/mapred/TestTaskAttemptListenerImpl.java | 3 +-
.../hadoop/mapreduce/jobhistory/TestEvents.java | 33 +-
.../jobhistory/TestJobHistoryEventHandler.java | 22 +-
.../v2/api/records/TestTaskAttemptReport.java | 21 +-
.../mapreduce/v2/api/records/TestTaskReport.java | 23 +-
.../hadoop/mapreduce/v2/app/TestFetchFailure.java | 5 +-
.../apache/hadoop/mapreduce/v2/app/TestMRApp.java | 4 +-
.../hadoop/mapreduce/v2/app/TestMRAppMaster.java | 5 +-
.../mapreduce/v2/app/TestMRClientService.java | 2 +-
.../hadoop/mapreduce/v2/app/TestRecovery.java | 3 +-
.../mapreduce/v2/app/TestRuntimeEstimators.java | 25 +-
.../mapreduce/v2/app/TestStagingCleanup.java | 19 +-
.../mapreduce/v2/app/TestTaskHeartbeatHandler.java | 43 +-
.../mapreduce/v2/app/job/impl/TestJobImpl.java | 3 +
.../mapreduce/v2/app/job/impl/TestTaskAttempt.java | 361 +-
.../v2/app/launcher/TestContainerLauncher.java | 5 +-
.../v2/app/local/TestLocalContainerAllocator.java | 5 +-
.../v2/app/rm/TestRMContainerAllocator.java | 25 +-
.../forecast/TestSimpleExponentialForecast.java | 120 +
.../hadoop-mapreduce-client-common/pom.xml | 41 +-
.../src/main/proto/HSAdminRefreshProtocol.proto | 3 +-
.../src/main/proto/MRClientProtocol.proto | 1 +
.../src/main/proto/mr_protos.proto | 1 +
.../src/main/proto/mr_service_protos.proto | 1 +
.../hadoop/mapred/TestLocalModeWithNewApis.java | 2 +-
.../apache/hadoop/mapreduce/TestTypeConverter.java | 6 +-
.../hadoop-mapreduce-client-core/pom.xml | 7 +-
.../org/apache/hadoop/mapred/FileInputFormat.java | 5 +-
.../hadoop/mapred/InvalidInputException.java | 4 +
.../hadoop/mapred/LocatedFileStatusFetcher.java | 75 +-
.../hadoop/mapreduce/JobResourceUploader.java | 25 +-
.../org/apache/hadoop/mapreduce/MRJobConfig.java | 33 +-
.../mapreduce/lib/input/FileInputFormat.java | 6 +-
.../mapreduce/lib/input/InvalidInputException.java | 4 +
.../mapreduce/lib/output/FileOutputCommitter.java | 3 +-
.../mapreduce/server/tasktracker/TTConfig.java | 73 +-
.../mapreduce/task/reduce/MergeManagerImpl.java | 13 +-
.../apache/hadoop/mapreduce/util/ConfigUtil.java | 43 -
.../src/main/resources/mapred-default.xml | 5 +-
.../src/site/markdown/EncryptedShuffle.md | 2 +-
.../hadoop/mapred/TestFileOutputCommitter.java | 3 +-
.../org/apache/hadoop/mapred/TestIndexCache.java | 2 +-
.../java/org/apache/hadoop/mapred/TestJobConf.java | 78 +-
.../java/org/apache/hadoop/mapred/TestJobInfo.java | 13 +-
.../apache/hadoop/mapred/TestLineRecordReader.java | 7 +-
.../java/org/apache/hadoop/mapred/TestMaster.java | 5 +-
.../apache/hadoop/mapred/TestOldMethodsJobID.java | 14 +-
.../java/org/apache/hadoop/mapred/TestQueue.java | 72 +-
.../java/org/apache/hadoop/mapred/TestTaskLog.java | 6 +-
.../apache/hadoop/mapred/TestTaskLogAppender.java | 13 +-
.../hadoop/mapred/TestTaskProgressReporter.java | 4 +-
.../hadoop/mapreduce/TestJobResourceUploader.java | 64 +-
.../lib/output/TestFileOutputCommitter.java | 3 +-
.../hadoop/mapreduce/security/TestTokenCache.java | 5 +-
.../hadoop/mapreduce/task/reduce/TestFetcher.java | 14 +-
.../mapreduce/task/reduce/TestMergeManager.java | 5 +-
.../hadoop/mapreduce/task/reduce/TestMerger.java | 13 +-
.../hadoop-mapreduce-client-hs-plugins/pom.xml | 2 +-
.../hadoop-mapreduce-client-hs/pom.xml | 9 +-
.../hadoop/mapreduce/v2/hs/CompletedJob.java | 15 +-
.../mapreduce/v2/hs/webapp/HsAttemptsPage.java | 5 +-
.../hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java | 28 +-
.../v2/hs/TestJHSDelegationTokenSecretManager.java | 9 +-
.../hadoop/mapreduce/v2/hs/TestJobHistory.java | 11 +-
.../mapreduce/v2/hs/TestJobHistoryEntities.java | 9 +-
.../mapreduce/v2/hs/TestJobHistoryEvents.java | 8 +-
.../mapreduce/v2/hs/TestJobHistoryParsing.java | 5 +-
.../hadoop/mapreduce/v2/hs/webapp/TestBlocks.java | 30 +-
.../mapreduce/v2/hs/webapp/TestHSWebApp.java | 4 +-
.../test/resources/job_1329348432655_0001_conf.xml | 14 -
.../hadoop-mapreduce-client-jobclient/pom.xml | 7 +-
.../java/org/apache/hadoop/fs/TestFileSystem.java | 17 +-
.../java/org/apache/hadoop/fs/slive/TestSlive.java | 36 +-
.../test/java/org/apache/hadoop/hdfs/NNBench.java | 14 +-
.../java/org/apache/hadoop/hdfs/TestNNBench.java | 27 +-
.../org/apache/hadoop/mapred/BigMapOutput.java | 18 +-
.../apache/hadoop/mapred/JobClientUnitTest.java | 9 +-
.../java/org/apache/hadoop/mapred/MRBench.java | 2 +-
.../hadoop/mapred/TestMRCJCFileInputFormat.java | 3 +-
.../mapred/TestMRCJCFileOutputCommitter.java | 3 +-
.../apache/hadoop/mapred/TestMiniMRChildTask.java | 22 +-
.../apache/hadoop/mapred/TestMultiFileSplit.java | 11 +-
.../mapred/TestMultipleTextOutputFormat.java | 10 +-
.../org/apache/hadoop/mapred/TestNetworkedJob.java | 9 +-
.../TestSequenceFileAsBinaryOutputFormat.java | 8 +-
.../hadoop/mapred/TestSequenceFileInputFilter.java | 3 +-
.../hadoop/mapred/TestStatisticsCollector.java | 7 +-
.../org/apache/hadoop/mapred/TestYARNRunner.java | 7 +-
.../org/apache/hadoop/mapred/lib/TestChain.java | 14 +-
.../hadoop/mapred/pipes/TestPipeApplication.java | 8 +-
.../java/org/apache/hadoop/mapreduce/SleepJob.java | 29 +-
.../lib/input/TestCombineFileInputFormat.java | 5 +-
.../lib/input/TestMRSequenceFileInputFilter.java | 3 +-
.../mapreduce/lib/join/TestJoinTupleWritable.java | 10 +-
.../lib/output/TestMRCJCFileOutputCommitter.java | 12 +-
.../TestMRSequenceFileAsBinaryOutputFormat.java | 7 +-
.../mapreduce/security/TestMRCredentials.java | 4 +-
.../mapreduce/util/TestMRAsyncDiskService.java | 5 +-
.../hadoop/mapreduce/v2/MiniMRYarnCluster.java | 4 +-
.../org/apache/hadoop/mapreduce/v2/TestMRJobs.java | 78 +-
.../mapreduce/v2/TestSpeculativeExecOnCluster.java | 935 +
.../v2/TestSpeculativeExecutionWithMRApp.java | 46 +-
.../org/apache/hadoop/mapreduce/v2/TestUberAM.java | 4 +-
.../hadoop-mapreduce-client-nativetask/pom.xml | 7 +-
.../nativetask/buffer/TestByteBufferReadWrite.java | 55 +-
.../mapred/nativetask/buffer/TestInputBuffer.java | 34 +-
.../mapred/nativetask/buffer/TestOutputBuffer.java | 21 +-
.../nativetask/combinertest/CombinerTest.java | 12 +-
.../combinertest/LargeKVCombinerTest.java | 12 +-
.../combinertest/OldAPICombinerTest.java | 15 +-
.../nativetask/compresstest/CompressTest.java | 30 +-
.../nativetask/handlers/TestCombineHandler.java | 6 +-
.../hadoop/mapred/nativetask/kvtest/KVTest.java | 13 +-
.../mapred/nativetask/nonsorttest/NonSortTest.java | 12 +-
.../nativetask/utils/TestReadWriteBuffer.java | 27 +-
.../hadoop-mapreduce-client-shuffle/pom.xml | 31 +-
.../src/main/proto/ShuffleHandlerRecovery.proto | 1 +
.../hadoop-mapreduce-client-uploader/pom.xml | 9 +-
.../mapred/uploader/TestFrameworkUploader.java | 16 +-
.../hadoop-mapreduce-client/pom.xml | 4 +-
.../hadoop-mapreduce-examples/pom.xml | 7 +-
.../org/apache/hadoop/examples/pi/TaskResult.java | 34 +-
.../hadoop/examples/terasort/TeraOutputFormat.java | 11 -
.../apache/hadoop/examples/terasort/TeraSort.java | 2 +-
.../hadoop/examples/terasort/TestTeraSort.java | 9 +-
hadoop-mapreduce-project/pom.xml | 6 +-
hadoop-maven-plugins/pom.xml | 2 +-
hadoop-minicluster/pom.xml | 2 +-
hadoop-ozone/Jenkinsfile | 116 -
hadoop-ozone/client/pom.xml | 37 -
.../org/apache/hadoop/ozone/client/BucketArgs.java | 149 -
.../apache/hadoop/ozone/client/ObjectStore.java | 425 -
.../apache/hadoop/ozone/client/OzoneBucket.java | 487 -
.../apache/hadoop/ozone/client/OzoneClient.java | 110 -
.../hadoop/ozone/client/OzoneClientException.java | 54 -
.../hadoop/ozone/client/OzoneClientFactory.java | 306 -
.../ozone/client/OzoneClientInvocationHandler.java | 62 -
.../hadoop/ozone/client/OzoneClientUtils.java | 109 -
.../org/apache/hadoop/ozone/client/OzoneKey.java | 133 -
.../hadoop/ozone/client/OzoneKeyDetails.java | 70 -
.../hadoop/ozone/client/OzoneKeyLocation.java | 82 -
.../client/OzoneMultipartUploadPartListParts.java | 107 -
.../apache/hadoop/ozone/client/OzoneVolume.java | 328 -
.../org/apache/hadoop/ozone/client/VolumeArgs.java | 150 -
.../ozone/client/io/BlockOutputStreamEntry.java | 334 -
.../hadoop/ozone/client/io/KeyInputStream.java | 338 -
.../hadoop/ozone/client/io/KeyOutputStream.java | 674 -
.../hadoop/ozone/client/io/OzoneInputStream.java | 63 -
.../hadoop/ozone/client/io/OzoneOutputStream.java | 74 -
.../hadoop/ozone/client/io/package-info.java | 23 -
.../apache/hadoop/ozone/client/package-info.java | 23 -
.../ozone/client/protocol/ClientProtocol.java | 509 -
.../hadoop/ozone/client/protocol/package-info.java | 23 -
.../client/rest/DefaultRestServerSelector.java | 36 -
.../ozone/client/rest/OzoneExceptionMapper.java | 45 -
.../hadoop/ozone/client/rest/RestClient.java | 1056 -
.../ozone/client/rest/RestServerSelector.java | 40 -
.../ozone/client/rest/exceptions/package-info.java | 22 -
.../ozone/client/rest/headers/package-info.java | 22 -
.../hadoop/ozone/client/rest/package-info.java | 23 -
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 894 -
.../hadoop/ozone/client/rpc/package-info.java | 23 -
.../hadoop/ozone/client/TestHddsClientUtils.java | 220 -
.../apache/hadoop/ozone/client/package-info.java | 23 -
.../common/dev-support/findbugsExcludeFile.xml | 22 -
hadoop-ozone/common/pom.xml | 109 -
hadoop-ozone/common/src/main/bin/ozone | 263 -
hadoop-ozone/common/src/main/bin/ozone-config.sh | 51 -
hadoop-ozone/common/src/main/bin/start-ozone.sh | 135 -
hadoop-ozone/common/src/main/bin/stop-ozone.sh | 100 -
.../apache/hadoop/hdds/protocol/StorageType.java | 64 -
.../apache/hadoop/hdds/protocol/package-info.java | 20 -
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 290 -
.../ozone/OzoneIllegalArgumentException.java | 40 -
.../org/apache/hadoop/ozone/audit/OMAction.java | 60 -
.../apache/hadoop/ozone/audit/package-info.java | 22 -
.../hadoop/ozone/client/io/LengthInputStream.java | 49 -
.../hadoop/ozone/client/io/package-info.java | 22 -
.../hadoop/ozone/client/rest/OzoneException.java | 267 -
.../hadoop/ozone/client/rest/headers/Header.java | 74 -
.../ozone/client/rest/headers/package-info.java | 22 -
.../hadoop/ozone/client/rest/package-info.java | 22 -
.../ozone/client/rest/response/BucketInfo.java | 231 -
.../hadoop/ozone/client/rest/response/KeyInfo.java | 236 -
.../ozone/client/rest/response/KeyInfoDetails.java | 107 -
.../ozone/client/rest/response/KeyLocation.java | 89 -
.../ozone/client/rest/response/VolumeInfo.java | 215 -
.../ozone/client/rest/response/VolumeOwner.java | 61 -
.../ozone/client/rest/response/package-info.java | 24 -
.../apache/hadoop/ozone/freon/OzoneGetConf.java | 271 -
.../apache/hadoop/ozone/freon/package-info.java | 21 -
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 207 -
.../apache/hadoop/ozone/om/OMMetadataManager.java | 292 -
.../apache/hadoop/ozone/om/OzoneManagerLock.java | 244 -
.../hadoop/ozone/om/codec/OmBucketInfoCodec.java | 52 -
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 52 -
.../ozone/om/codec/OmMultipartKeyInfoCodec.java | 56 -
.../hadoop/ozone/om/codec/OmVolumeArgsCodec.java | 52 -
.../hadoop/ozone/om/codec/VolumeListCodec.java | 51 -
.../apache/hadoop/ozone/om/codec/package-info.java | 24 -
.../hadoop/ozone/om/exceptions/OMException.java | 182 -
.../hadoop/ozone/om/exceptions/package-info.java | 19 -
.../hadoop/ozone/om/helpers/KeyValueUtil.java | 54 -
.../hadoop/ozone/om/helpers/OmBucketArgs.java | 263 -
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 300 -
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 230 -
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 356 -
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 166 -
.../ozone/om/helpers/OmKeyLocationInfoGroup.java | 118 -
.../helpers/OmMultipartCommitUploadPartInfo.java | 34 -
.../hadoop/ozone/om/helpers/OmMultipartInfo.java | 77 -
.../ozone/om/helpers/OmMultipartKeyInfo.java | 106 -
.../om/helpers/OmMultipartUploadCompleteInfo.java | 70 -
.../ozone/om/helpers/OmMultipartUploadList.java | 63 -
.../om/helpers/OmMultipartUploadListParts.java | 84 -
.../hadoop/ozone/om/helpers/OmOzoneAclMap.java | 111 -
.../apache/hadoop/ozone/om/helpers/OmPartInfo.java | 60 -
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 236 -
.../hadoop/ozone/om/helpers/OpenKeySession.java | 50 -
.../hadoop/ozone/om/helpers/S3SecretValue.java | 81 -
.../hadoop/ozone/om/helpers/ServiceInfo.java | 247 -
.../apache/hadoop/ozone/om/helpers/VolumeArgs.java | 140 -
.../hadoop/ozone/om/helpers/WithMetadata.java | 45 -
.../hadoop/ozone/om/helpers/package-info.java | 18 -
.../org/apache/hadoop/ozone/om/package-info.java | 21 -
.../ozone/om/protocol/OzoneManagerProtocol.java | 376 -
.../om/protocol/OzoneManagerSecurityProtocol.java | 67 -
.../hadoop/ozone/om/protocol/package-info.java | 19 -
...OzoneManagerProtocolClientSideTranslatorPB.java | 1123 -
.../om/protocolPB/OzoneManagerProtocolPB.java | 41 -
.../hadoop/ozone/om/protocolPB/package-info.java | 19 -
.../java/org/apache/hadoop/ozone/package-info.java | 22 -
.../apache/hadoop/ozone/protocolPB/OMPBHelper.java | 151 -
.../hadoop/ozone/protocolPB/OzonePBHelper.java | 30 -
.../hadoop/ozone/protocolPB/package-info.java | 24 -
.../security/OzoneBlockTokenSecretManager.java | 194 -
.../OzoneDelegationTokenSecretManager.java | 470 -
.../security/OzoneDelegationTokenSelector.java | 51 -
.../hadoop/ozone/security/OzoneSecretKey.java | 176 -
.../hadoop/ozone/security/OzoneSecretManager.java | 280 -
.../hadoop/ozone/security/OzoneSecretStore.java | 249 -
.../ozone/security/OzoneSecurityException.java | 104 -
.../ozone/security/OzoneTokenIdentifier.java | 217 -
.../ozone/security/acl/IAccessAuthorizer.java | 153 -
.../hadoop/ozone/security/acl/IOzoneObj.java | 24 -
.../ozone/security/acl/OzoneAccessAuthorizer.java | 29 -
.../ozone/security/acl/OzoneAclException.java | 71 -
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 109 -
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 118 -
.../hadoop/ozone/security/acl/RequestContext.java | 122 -
.../hadoop/ozone/security/acl/package-info.java | 22 -
.../apache/hadoop/ozone/security/package-info.java | 21 -
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 77 -
.../org/apache/hadoop/ozone/util/package-info.java | 22 -
.../hadoop/ozone/web/handlers/BucketArgs.java | 214 -
.../apache/hadoop/ozone/web/handlers/KeyArgs.java | 117 -
.../apache/hadoop/ozone/web/handlers/ListArgs.java | 142 -
.../apache/hadoop/ozone/web/handlers/UserArgs.java | 172 -
.../hadoop/ozone/web/handlers/VolumeArgs.java | 143 -
.../hadoop/ozone/web/handlers/package-info.java | 22 -
.../hadoop/ozone/web/request/OzoneQuota.java | 214 -
.../hadoop/ozone/web/request/package-info.java | 23 -
.../hadoop/ozone/web/response/BucketInfo.java | 325 -
.../apache/hadoop/ozone/web/response/KeyInfo.java | 311 -
.../hadoop/ozone/web/response/KeyInfoDetails.java | 80 -
.../hadoop/ozone/web/response/KeyLocation.java | 82 -
.../hadoop/ozone/web/response/ListBuckets.java | 154 -
.../apache/hadoop/ozone/web/response/ListKeys.java | 209 -
.../hadoop/ozone/web/response/ListVolumes.java | 152 -
.../hadoop/ozone/web/response/VolumeInfo.java | 308 -
.../hadoop/ozone/web/response/VolumeOwner.java | 60 -
.../hadoop/ozone/web/response/package-info.java | 23 -
.../apache/hadoop/ozone/web/utils/OzoneUtils.java | 227 -
.../hadoop/ozone/web/utils/package-info.java | 18 -
.../src/main/proto/OzoneManagerProtocol.proto | 737 -
.../main/resources/ozone-version-info.properties | 27 -
.../common/src/main/shellprofile.d/hadoop-ozone.sh | 21 -
.../java/org/apache/hadoop/ozone/TestOmUtils.java | 91 -
.../om/codec/TestOmMultipartKeyInfoCodec.java | 54 -
.../apache/hadoop/ozone/om/codec/package-info.java | 24 -
.../ozone/om/exceptions/TestResultCodes.java | 49 -
.../hadoop/ozone/om/helpers/TestOmBucketInfo.java | 46 -
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 52 -
.../hadoop/ozone/om/helpers/package-info.java | 21 -
.../security/TestOzoneBlockTokenSecretManager.java | 147 -
.../TestOzoneDelegationTokenSecretManager.java | 217 -
.../ozone/security/acl/TestOzoneObjInfo.java | 88 -
.../apache/hadoop/ozone/web/TestBucketInfo.java | 82 -
.../org/apache/hadoop/ozone/web/TestQuota.java | 115 -
.../org/apache/hadoop/ozone/web/TestUtils.java | 101 -
.../apache/hadoop/ozone/web/TestVolumeStructs.java | 73 -
.../org/apache/hadoop/ozone/web/package-info.java | 21 -
hadoop-ozone/datanode/pom.xml | 49 -
hadoop-ozone/dev-support/checks/acceptance.sh | 18 -
hadoop-ozone/dev-support/checks/author.sh | 22 -
hadoop-ozone/dev-support/checks/build.sh | 18 -
hadoop-ozone/dev-support/checks/checkstyle.sh | 23 -
hadoop-ozone/dev-support/checks/findbugs.sh | 34 -
hadoop-ozone/dev-support/checks/isolation.sh | 24 -
hadoop-ozone/dev-support/checks/rat.sh | 24 -
hadoop-ozone/dev-support/checks/unit.sh | 24 -
hadoop-ozone/dev-support/docker/Dockerfile | 66 -
hadoop-ozone/dist/Dockerfile | 21 -
hadoop-ozone/dist/README.md | 85 -
.../dist/dev-support/bin/dist-layout-stitching | 120 -
.../dist/dev-support/bin/dist-tar-stitching | 45 -
hadoop-ozone/dist/pom.xml | 214 -
hadoop-ozone/dist/skaffold.yaml | 25 -
hadoop-ozone/dist/src/main/blockade/README.md | 44 -
.../src/main/blockade/blockadeUtils/__init__.py | 14 -
.../src/main/blockade/blockadeUtils/blockade.py | 99 -
.../main/blockade/clusterUtils/cluster_utils.py | 300 -
hadoop-ozone/dist/src/main/blockade/conftest.py | 96 -
.../main/blockade/test_blockade_client_failure.py | 124 -
.../blockade/test_blockade_datanode_isolation.py | 111 -
.../dist/src/main/blockade/test_blockade_flaky.py | 61 -
.../main/blockade/test_blockade_mixed_failure.py | 117 -
...t_blockade_mixed_failure_three_nodes_isolate.py | 144 -
.../test_blockade_mixed_failure_two_nodes.py | 121 -
.../main/blockade/test_blockade_scm_isolation.py | 111 -
hadoop-ozone/dist/src/main/compose/README.md | 51 -
.../main/compose/common/grafana/conf/grafana.ini | 20 -
.../grafana/dashboards/Ozone - Object Metrics.json | 1344 -
.../grafana/dashboards/Ozone - RPC Metrics.json | 875 -
.../grafana/provisioning/dashboards/dashboards.yml | 22 -
.../provisioning/datasources/datasources.yml | 25 -
.../main/compose/common/prometheus/prometheus.yml | 25 -
hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env | 17 -
.../main/compose/ozone-hdfs/docker-compose.yaml | 70 -
.../dist/src/main/compose/ozone-hdfs/docker-config | 81 -
hadoop-ozone/dist/src/main/compose/ozone/.env | 17 -
.../src/main/compose/ozone/docker-compose.yaml | 50 -
.../dist/src/main/compose/ozone/docker-config | 76 -
.../main/compose/ozoneblockade/docker-compose.yaml | 58 -
.../src/main/compose/ozoneblockade/docker-config | 77 -
.../src/main/compose/ozonefs/docker-compose.yaml | 69 -
.../dist/src/main/compose/ozonefs/docker-config | 37 -
hadoop-ozone/dist/src/main/compose/ozoneperf/.env | 17 -
.../dist/src/main/compose/ozoneperf/README.md | 52 -
.../src/main/compose/ozoneperf/docker-compose.yaml | 75 -
.../dist/src/main/compose/ozoneperf/docker-config | 38 -
.../src/main/compose/ozones3/docker-compose.yaml | 59 -
.../dist/src/main/compose/ozones3/docker-config | 79 -
.../main/compose/ozonescripts/.ssh/authorized_keys | 16 -
.../dist/src/main/compose/ozonescripts/.ssh/config | 18 -
.../src/main/compose/ozonescripts/.ssh/environment | 16 -
.../dist/src/main/compose/ozonescripts/.ssh/id_rsa | 42 -
.../src/main/compose/ozonescripts/.ssh/id_rsa.pub | 16 -
.../dist/src/main/compose/ozonescripts/Dockerfile | 33 -
.../dist/src/main/compose/ozonescripts/README.md | 38 -
.../main/compose/ozonescripts/docker-compose.yaml | 42 -
.../src/main/compose/ozonescripts/docker-config | 40 -
.../dist/src/main/compose/ozonescripts/ps.sh | 17 -
.../dist/src/main/compose/ozonescripts/start.sh | 24 -
.../dist/src/main/compose/ozonescripts/stop.sh | 17 -
.../dist/src/main/compose/ozonesecure/.env | 17 -
.../dist/src/main/compose/ozonesecure/README.md | 22 -
.../main/compose/ozonesecure/docker-compose.yaml | 75 -
.../src/main/compose/ozonesecure/docker-config | 104 -
.../docker-image/docker-krb5/Dockerfile-krb5 | 34 -
.../ozonesecure/docker-image/docker-krb5/README.md | 34 -
.../ozonesecure/docker-image/docker-krb5/kadm5.acl | 20 -
.../ozonesecure/docker-image/docker-krb5/krb5.conf | 40 -
.../docker-image/docker-krb5/launcher.sh | 25 -
.../ozonesecure/docker-image/runner/Dockerfile | 39 -
.../ozonesecure/docker-image/runner/build.sh | 26 -
.../docker-image/runner/scripts/envtoconf.py | 115 -
.../docker-image/runner/scripts/krb5.conf | 38 -
.../docker-image/runner/scripts/starter.sh | 100 -
.../docker-image/runner/scripts/transformation.py | 150 -
.../main/compose/ozonetrace/docker-compose.yaml | 65 -
.../dist/src/main/compose/ozonetrace/docker-config | 84 -
.../dist/src/main/conf/dn-audit-log4j2.properties | 90 -
hadoop-ozone/dist/src/main/conf/log4j.properties | 157 -
.../dist/src/main/conf/om-audit-log4j2.properties | 90 -
hadoop-ozone/dist/src/main/conf/ozone-site.xml | 24 -
.../dist/src/main/conf/scm-audit-log4j2.properties | 90 -
.../dist/src/main/k8s/ozone/config-configmap.yaml | 34 -
.../main/k8s/ozone/datanode-public-service.yaml | 29 -
.../dist/src/main/k8s/ozone/datanode-service.yaml | 29 -
.../src/main/k8s/ozone/datanode-statefulset.yaml | 52 -
.../dist/src/main/k8s/ozone/om-public-service.yaml | 29 -
.../dist/src/main/k8s/ozone/om-service.yaml | 29 -
.../dist/src/main/k8s/ozone/om-statefulset.yaml | 72 -
.../src/main/k8s/ozone/s3g-public-service.yaml | 29 -
.../dist/src/main/k8s/ozone/s3g-service.yaml | 29 -
.../dist/src/main/k8s/ozone/s3g-statefulset.yaml | 49 -
.../src/main/k8s/ozone/scm-public-service.yaml | 29 -
.../dist/src/main/k8s/ozone/scm-service.yaml | 29 -
.../dist/src/main/k8s/ozone/scm-statefulset.yaml | 67 -
.../dist/src/main/k8s/prometheus/configmap.yaml | 51 -
.../dist/src/main/k8s/prometheus/deployment.yaml | 46 -
.../k8s/prometheus/prometheus-public-serivce.yaml | 28 -
.../dist/src/main/k8s/prometheus/role.yaml | 31 -
.../dist/src/main/k8s/prometheus/rolebindng.yaml | 27 -
.../main/k8s/prometheus/scm-public-service.yaml | 29 -
.../src/main/k8s/prometheus/service-account.yaml | 19 -
.../dist/src/main/k8s/prometheus/service.yaml | 25 -
hadoop-ozone/dist/src/main/ozone/README.txt | 51 -
hadoop-ozone/dist/src/main/smoketest/.env | 17 -
hadoop-ozone/dist/src/main/smoketest/README.md | 30 -
.../main/smoketest/auditparser/auditparser.robot | 40 -
.../dist/src/main/smoketest/basic/basic.robot | 47 -
.../src/main/smoketest/basic/ozone-shell.robot | 85 -
.../dist/src/main/smoketest/commonlib.robot | 31 -
.../dist/src/main/smoketest/ozonefs/ozonefs.robot | 112 -
.../src/main/smoketest/s3/MultipartUpload.robot | 207 -
hadoop-ozone/dist/src/main/smoketest/s3/README.md | 27 -
.../dist/src/main/smoketest/s3/__init__.robot | 21 -
.../dist/src/main/smoketest/s3/awss3.robot | 47 -
.../dist/src/main/smoketest/s3/buckethead.robot | 34 -
.../dist/src/main/smoketest/s3/bucketlist.robot | 32 -
.../dist/src/main/smoketest/s3/commonawslib.robot | 72 -
.../dist/src/main/smoketest/s3/objectcopy.robot | 66 -
.../dist/src/main/smoketest/s3/objectdelete.robot | 72 -
.../src/main/smoketest/s3/objectmultidelete.robot | 48 -
.../dist/src/main/smoketest/s3/objectputget.robot | 156 -
.../dist/src/main/smoketest/s3/webui.robot | 34 -
.../src/main/smoketest/security/ozone-secure.robot | 111 -
hadoop-ozone/dist/src/main/smoketest/test.sh | 159 -
hadoop-ozone/integration-test/pom.xml | 87 -
.../TestContainerStateManagerIntegration.java | 389 -
.../hdds/scm/pipeline/TestNode2PipelineMap.java | 122 -
.../hadoop/hdds/scm/pipeline/TestNodeFailure.java | 134 -
.../hdds/scm/pipeline/TestPipelineClose.java | 215 -
.../scm/pipeline/TestPipelineStateManager.java | 426 -
.../scm/pipeline/TestRatisPipelineProvider.java | 138 -
.../hdds/scm/pipeline/TestRatisPipelineUtils.java | 95 -
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 188 -
.../hadoop/hdds/scm/pipeline/TestSCMRestart.java | 112 -
.../scm/pipeline/TestSimplePipelineProvider.java | 103 -
.../hadoop/hdds/scm/pipeline/package-info.java | 22 -
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 451 -
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 591 -
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 261 -
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 112 -
.../org/apache/hadoop/ozone/RatisTestHelper.java | 134 -
.../hadoop/ozone/TestContainerOperations.java | 88 -
.../TestContainerStateMachineIdempotency.java | 124 -
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 267 -
.../hadoop/ozone/TestOzoneConfigurationFields.java | 51 -
.../org/apache/hadoop/ozone/TestOzoneHelper.java | 413 -
.../hadoop/ozone/TestSecureOzoneCluster.java | 595 -
.../hadoop/ozone/TestStorageContainerManager.java | 485 -
.../ozone/TestStorageContainerManagerHelper.java | 181 -
.../ozone/client/CertificateClientTestImpl.java | 151 -
.../apache/hadoop/ozone/client/package-info.java | 20 -
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 148 -
.../rpc/TestCloseContainerHandlingByClient.java | 481 -
.../client/rpc/TestContainerStateMachine.java | 152 -
.../rpc/TestContainerStateMachineFailures.java | 153 -
.../client/rpc/TestFailureHandlingByClient.java | 248 -
.../client/rpc/TestHybridPipelineOnDatanode.java | 166 -
.../ozone/client/rpc/TestOzoneRpcClient.java | 77 -
.../client/rpc/TestOzoneRpcClientAbstract.java | 2040 -
.../client/rpc/TestOzoneRpcClientWithRatis.java | 58 -
.../hadoop/ozone/client/rpc/TestReadRetries.java | 223 -
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 239 -
.../hadoop/ozone/client/rpc/package-info.java | 23 -
.../ozone/container/ContainerTestHelper.java | 700 -
.../ozone/container/TestContainerReplication.java | 179 -
.../container/common/TestBlockDeletingService.java | 442 -
.../container/common/helpers/TestBlockData.java | 132 -
.../impl/TestContainerDeletionChoosingPolicy.java | 167 -
.../common/impl/TestContainerPersistence.java | 873 -
.../commandhandler/TestBlockDeletion.java | 323 -
.../TestCloseContainerByPipeline.java | 269 -
.../commandhandler/TestCloseContainerHandler.java | 119 -
.../commandhandler/TestDeleteContainerHandler.java | 277 -
.../statemachine/commandhandler/package-info.java | 21 -
.../transport/server/ratis/TestCSMMetrics.java | 200 -
.../container/metrics/TestContainerMetrics.java | 174 -
.../container/ozoneimpl/TestOzoneContainer.java | 568 -
.../ozoneimpl/TestOzoneContainerRatis.java | 138 -
.../ozoneimpl/TestOzoneContainerWithTLS.java | 190 -
.../container/ozoneimpl/TestRatisManager.java | 124 -
.../ozoneimpl/TestSecureOzoneContainer.java | 227 -
.../container/server/TestContainerServer.java | 290 -
.../server/TestSecureContainerServer.java | 239 -
.../apache/hadoop/ozone/freon/package-info.java | 21 -
.../ozone/om/TestContainerReportWithKeys.java | 131 -
.../ozone/om/TestMultipleContainerReadWrite.java | 213 -
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 175 -
.../hadoop/ozone/om/TestOmBlockVersioning.java | 263 -
.../org/apache/hadoop/ozone/om/TestOmInit.java | 103 -
.../org/apache/hadoop/ozone/om/TestOmMetrics.java | 441 -
.../apache/hadoop/ozone/om/TestOzoneManager.java | 1388 -
.../ozone/om/TestOzoneManagerConfiguration.java | 343 -
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 156 -
.../ozone/om/TestOzoneManagerRestInterface.java | 135 -
.../apache/hadoop/ozone/om/TestScmChillMode.java | 371 -
.../org/apache/hadoop/ozone/om/package-info.java | 22 -
.../ozone/ozShell/TestOzoneDatanodeShell.java | 220 -
.../hadoop/ozone/ozShell/TestOzoneShell.java | 1400 -
.../java/org/apache/hadoop/ozone/package-info.java | 22 -
.../hadoop/ozone/scm/TestAllocateContainer.java | 84 -
.../hadoop/ozone/scm/TestContainerSmallFile.java | 209 -
.../scm/TestGetCommittedBlockLengthAndPutKey.java | 185 -
.../org/apache/hadoop/ozone/scm/TestSCMMXBean.java | 212 -
.../hadoop/ozone/scm/TestSCMNodeManagerMXBean.java | 112 -
.../hadoop/ozone/scm/TestXceiverClientManager.java | 256 -
.../hadoop/ozone/scm/TestXceiverClientMetrics.java | 179 -
.../hadoop/ozone/scm/node/TestQueryNode.java | 118 -
.../ozone/web/TestOzoneRestWithMiniCluster.java | 221 -
.../apache/hadoop/ozone/web/TestOzoneVolumes.java | 187 -
.../hadoop/ozone/web/TestOzoneWebAccess.java | 118 -
.../hadoop/ozone/web/client/TestBuckets.java | 336 -
.../hadoop/ozone/web/client/TestBucketsRatis.java | 108 -
.../apache/hadoop/ozone/web/client/TestKeys.java | 734 -
.../hadoop/ozone/web/client/TestKeysRatis.java | 126 -
.../hadoop/ozone/web/client/TestOzoneClient.java | 304 -
.../apache/hadoop/ozone/web/client/TestVolume.java | 379 -
.../hadoop/ozone/web/client/TestVolumeRatis.java | 155 -
.../hadoop/ozone/web/client/package-info.java | 22 -
.../org/apache/hadoop/ozone/web/package-info.java | 22 -
.../src/test/resources/log4j.properties | 21 -
.../integration-test/src/test/resources/ssl/ca.crt | 27 -
.../integration-test/src/test/resources/ssl/ca.key | 54 -
.../src/test/resources/ssl/client.crt | 27 -
.../src/test/resources/ssl/client.csr | 26 -
.../src/test/resources/ssl/client.key | 51 -
.../src/test/resources/ssl/client.pem | 52 -
.../src/test/resources/ssl/generate.sh | 34 -
.../src/test/resources/ssl/server.crt | 27 -
.../src/test/resources/ssl/server.csr | 26 -
.../src/test/resources/ssl/server.key | 51 -
.../src/test/resources/ssl/server.pem | 52 -
.../test/resources/webapps/ozoneManager/.gitkeep | 15 -
.../src/test/resources/webapps/scm/.gitkeep | 15 -
hadoop-ozone/objectstore-service/pom.xml | 120 -
.../hdfs/server/datanode/ObjectStoreHandler.java | 166 -
.../hadoop/hdfs/server/datanode/package-info.java | 22 -
.../org/apache/hadoop/ozone/OzoneRestUtils.java | 222 -
.../java/org/apache/hadoop/ozone/package-info.java | 22 -
.../hadoop/ozone/web/ObjectStoreApplication.java | 59 -
.../hadoop/ozone/web/OzoneHddsDatanodeService.java | 89 -
.../hadoop/ozone/web/exceptions/ErrorTable.java | 225 -
.../hadoop/ozone/web/exceptions/package-info.java | 22 -
.../hadoop/ozone/web/handlers/BucketHandler.java | 197 -
.../ozone/web/handlers/BucketProcessTemplate.java | 323 -
.../hadoop/ozone/web/handlers/KeyHandler.java | 302 -
.../ozone/web/handlers/KeyProcessTemplate.java | 235 -
.../hadoop/ozone/web/handlers/ServiceFilter.java | 61 -
.../ozone/web/handlers/StorageHandlerBuilder.java | 80 -
.../ozone/web/handlers/UserHandlerBuilder.java | 75 -
.../hadoop/ozone/web/handlers/VolumeHandler.java | 274 -
.../ozone/web/handlers/VolumeProcessTemplate.java | 276 -
.../hadoop/ozone/web/handlers/package-info.java | 22 -
.../hadoop/ozone/web/interfaces/Accounting.java | 57 -
.../apache/hadoop/ozone/web/interfaces/Bucket.java | 184 -
.../apache/hadoop/ozone/web/interfaces/Keys.java | 177 -
.../ozone/web/interfaces/StorageHandler.java | 317 -
.../hadoop/ozone/web/interfaces/UserAuth.java | 101 -
.../apache/hadoop/ozone/web/interfaces/Volume.java | 185 -
.../hadoop/ozone/web/interfaces/package-info.java | 22 -
.../ozone/web/localstorage/package-info.java | 18 -
.../LengthInputStreamMessageBodyWriter.java | 59 -
.../web/messages/StringMessageBodyWriter.java | 62 -
.../hadoop/ozone/web/messages/package-info.java | 18 -
.../ozone/web/netty/CloseableCleanupListener.java | 46 -
.../ozone/web/netty/ObjectStoreChannelHandler.java | 78 -
.../web/netty/ObjectStoreJerseyContainer.java | 348 -
.../netty/ObjectStoreJerseyContainerProvider.java | 40 -
.../ozone/web/netty/ObjectStoreRestHttpServer.java | 215 -
.../ozone/web/netty/ObjectStoreURLDispatcher.java | 61 -
.../RequestContentObjectStoreChannelHandler.java | 117 -
.../RequestDispatchObjectStoreChannelHandler.java | 103 -
.../hadoop/ozone/web/netty/package-info.java | 26 -
.../org/apache/hadoop/ozone/web/package-info.java | 22 -
.../web/storage/DistributedStorageHandler.java | 612 -
.../hadoop/ozone/web/storage/package-info.java | 27 -
.../apache/hadoop/ozone/web/userauth/Simple.java | 169 -
.../hadoop/ozone/web/userauth/package-info.java | 23 -
.../com.sun.jersey.spi.container.ContainerProvider | 16 -
.../org/apache/hadoop/ozone/web/TestErrorCode.java | 53 -
.../org/apache/hadoop/ozone/web/package-info.java | 22 -
hadoop-ozone/ozone-manager/pom.xml | 96 -
.../org/apache/hadoop/ozone/om/BucketManager.java | 79 -
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 305 -
.../apache/hadoop/ozone/om/KeyDeletingService.java | 175 -
.../org/apache/hadoop/ozone/om/KeyManager.java | 240 -
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 1135 -
.../java/org/apache/hadoop/ozone/om/OMMXBean.java | 31 -
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 613 -
.../org/apache/hadoop/ozone/om/OMNodeDetails.java | 111 -
.../java/org/apache/hadoop/ozone/om/OMStorage.java | 90 -
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 670 -
.../org/apache/hadoop/ozone/om/OmMetricsInfo.java | 43 -
.../hadoop/ozone/om/OpenKeyCleanupService.java | 117 -
.../org/apache/hadoop/ozone/om/OzoneManager.java | 2413 --
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 77 -
.../apache/hadoop/ozone/om/S3BucketManager.java | 81 -
.../hadoop/ozone/om/S3BucketManagerImpl.java | 238 -
.../apache/hadoop/ozone/om/S3SecretManager.java | 30 -
.../hadoop/ozone/om/S3SecretManagerImpl.java | 82 -
.../hadoop/ozone/om/ServiceListJSONServlet.java | 104 -
.../org/apache/hadoop/ozone/om/VolumeManager.java | 100 -
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 394 -
.../org/apache/hadoop/ozone/om/package-info.java | 21 -
.../hadoop/ozone/om/ratis/OMRatisHelper.java | 115 -
.../ozone/om/ratis/OzoneManagerRatisClient.java | 181 -
.../ozone/om/ratis/OzoneManagerRatisServer.java | 358 -
.../ozone/om/ratis/OzoneManagerStateMachine.java | 157 -
.../apache/hadoop/ozone/om/ratis/package-info.java | 22 -
...OzoneManagerProtocolServerSideTranslatorPB.java | 93 -
.../protocolPB/OzoneManagerRequestHandler.java | 879 -
.../hadoop/ozone/protocolPB/package-info.java | 22 -
.../apache/hadoop/ozone/web/ozShell/Handler.java | 57 -
.../hadoop/ozone/web/ozShell/OzoneAddress.java | 257 -
.../org/apache/hadoop/ozone/web/ozShell/Shell.java | 114 -
.../ozone/web/ozShell/bucket/BucketCommands.java | 67 -
.../web/ozShell/bucket/CreateBucketHandler.java | 70 -
.../web/ozShell/bucket/DeleteBucketHandler.java | 62 -
.../web/ozShell/bucket/InfoBucketHandler.java | 67 -
.../web/ozShell/bucket/ListBucketHandler.java | 106 -
.../ozone/web/ozShell/bucket/S3BucketMapping.java | 66 -
.../web/ozShell/bucket/UpdateBucketHandler.java | 95 -
.../ozone/web/ozShell/bucket/package-info.java | 23 -
.../ozone/web/ozShell/keys/DeleteKeyHandler.java | 67 -
.../ozone/web/ozShell/keys/GetKeyHandler.java | 115 -
.../ozone/web/ozShell/keys/InfoKeyHandler.java | 71 -
.../hadoop/ozone/web/ozShell/keys/KeyCommands.java | 67 -
.../ozone/web/ozShell/keys/ListKeyHandler.java | 108 -
.../ozone/web/ozShell/keys/PutKeyHandler.java | 119 -
.../ozone/web/ozShell/keys/RenameKeyHandler.java | 73 -
.../ozone/web/ozShell/keys/package-info.java | 23 -
.../hadoop/ozone/web/ozShell/package-info.java | 27 -
.../ozone/web/ozShell/s3/GetS3SecretHandler.java | 49 -
.../hadoop/ozone/web/ozShell/s3/S3Commands.java | 60 -
.../hadoop/ozone/web/ozShell/s3/package-info.java | 21 -
.../web/ozShell/token/CancelTokenHandler.java | 72 -
.../ozone/web/ozShell/token/GetTokenHandler.java | 77 -
.../ozone/web/ozShell/token/PrintTokenHandler.java | 71 -
.../ozone/web/ozShell/token/RenewTokenHandler.java | 75 -
.../ozone/web/ozShell/token/TokenCommands.java | 64 -
.../ozone/web/ozShell/token/package-info.java | 26 -
.../web/ozShell/volume/CreateVolumeHandler.java | 102 -
.../web/ozShell/volume/DeleteVolumeHandler.java | 59 -
.../web/ozShell/volume/InfoVolumeHandler.java | 60 -
.../web/ozShell/volume/ListVolumeHandler.java | 113 -
.../web/ozShell/volume/UpdateVolumeHandler.java | 78 -
.../ozone/web/ozShell/volume/VolumeCommands.java | 67 -
.../ozone/web/ozShell/volume/package-info.java | 23 -
.../org/apache/hadoop/ozone/web/package-info.java | 24 -
.../main/resources/webapps/ozoneManager/index.html | 70 -
.../main/resources/webapps/ozoneManager/main.css | 23 -
.../main/resources/webapps/ozoneManager/main.html | 18 -
.../resources/webapps/ozoneManager/om-metrics.html | 44 -
.../resources/webapps/ozoneManager/ozoneManager.js | 110 -
.../ozone/om/ScmBlockLocationTestIngClient.java | 181 -
.../hadoop/ozone/om/TestBucketManagerImpl.java | 382 -
.../apache/hadoop/ozone/om/TestChunkStreams.java | 167 -
.../hadoop/ozone/om/TestKeyDeletingService.java | 205 -
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 200 -
.../ozone/om/TestOzoneManagerHttpServer.java | 141 -
.../hadoop/ozone/om/TestOzoneManagerLock.java | 193 -
.../hadoop/ozone/om/TestS3BucketManager.java | 163 -
.../org/apache/hadoop/ozone/om/package-info.java | 21 -
.../om/ratis/TestOzoneManagerRatisServer.java | 155 -
.../ozone/security/TestOzoneManagerBlockToken.java | 251 -
.../ozone/security/TestOzoneTokenIdentifier.java | 300 -
.../apache/hadoop/ozone/security/package-info.java | 21 -
.../hadoop/ozone/web/ozShell/TestOzoneAddress.java | 101 -
.../hadoop/ozone/web/ozShell/package-info.java | 21 -
hadoop-ozone/ozonefs-lib-legacy/pom.xml | 111 -
.../src/main/resources/ozonefs.txt | 21 -
hadoop-ozone/ozonefs-lib/pom.xml | 96 -
hadoop-ozone/ozonefs/pom.xml | 192 -
.../org/apache/hadoop/fs/ozone/BasicKeyInfo.java | 53 -
.../java/org/apache/hadoop/fs/ozone/Constants.java | 42 -
.../hadoop/fs/ozone/FilteredClassLoader.java | 86 -
.../main/java/org/apache/hadoop/fs/ozone/OzFs.java | 44 -
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 55 -
.../hadoop/fs/ozone/OzoneClientAdapterFactory.java | 122 -
.../hadoop/fs/ozone/OzoneClientAdapterImpl.java | 246 -
.../apache/hadoop/fs/ozone/OzoneFSInputStream.java | 79 -
.../hadoop/fs/ozone/OzoneFSOutputStream.java | 58 -
.../hadoop/fs/ozone/OzoneFSStorageStatistics.java | 126 -
.../apache/hadoop/fs/ozone/OzoneFileSystem.java | 892 -
.../java/org/apache/hadoop/fs/ozone/Statistic.java | 119 -
.../org/apache/hadoop/fs/ozone/package-info.java | 30 -
.../services/org.apache.hadoop.fs.FileSystem | 16 -
.../hadoop/fs/ozone/TestOzoneFSInputStream.java | 155 -
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 275 -
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 275 -
.../hadoop/fs/ozone/TestOzoneFsRenameDir.java | 121 -
.../ozone/contract/ITestOzoneContractCreate.java | 48 -
.../ozone/contract/ITestOzoneContractDelete.java | 48 -
.../ozone/contract/ITestOzoneContractDistCp.java | 50 -
.../contract/ITestOzoneContractGetFileStatus.java | 65 -
.../fs/ozone/contract/ITestOzoneContractMkdir.java | 48 -
.../fs/ozone/contract/ITestOzoneContractOpen.java | 47 -
.../ozone/contract/ITestOzoneContractRename.java | 49 -
.../ozone/contract/ITestOzoneContractRootDir.java | 51 -
.../fs/ozone/contract/ITestOzoneContractSeek.java | 47 -
.../hadoop/fs/ozone/contract/OzoneContract.java | 123 -
.../org/apache/hadoop/fs/ozone/package-info.java | 22 -
.../ozonefs/src/test/resources/contract/ozone.xml | 113 -
.../ozonefs/src/test/resources/log4j.properties | 24 -
hadoop-ozone/pom.xml | 397 -
hadoop-ozone/s3gateway/pom.xml | 220 -
.../s3/CommonHeadersContainerResponseFilter.java | 48 -
.../java/org/apache/hadoop/ozone/s3/Gateway.java | 66 -
.../apache/hadoop/ozone/s3/GatewayApplication.java | 29 -
.../apache/hadoop/ozone/s3/HeaderPreprocessor.java | 55 -
.../hadoop/ozone/s3/OzoneClientProducer.java | 48 -
.../hadoop/ozone/s3/OzoneConfigurationHolder.java | 43 -
.../apache/hadoop/ozone/s3/RequestIdentifier.java | 48 -
.../hadoop/ozone/s3/S3GatewayConfigKeys.java | 55 -
.../hadoop/ozone/s3/S3GatewayHttpServer.java | 85 -
.../hadoop/ozone/s3/SignedChunksInputStream.java | 139 -
.../hadoop/ozone/s3/VirtualHostStyleFilter.java | 165 -
.../ozone/s3/commontypes/BucketMetadata.java | 53 -
.../hadoop/ozone/s3/commontypes/CommonPrefix.java | 47 -
.../ozone/s3/commontypes/IsoDateAdapter.java | 47 -
.../hadoop/ozone/s3/commontypes/KeyMetadata.java | 87 -
.../hadoop/ozone/s3/commontypes/package-info.java | 29 -
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 316 -
.../endpoint/CompleteMultipartUploadRequest.java | 77 -
.../endpoint/CompleteMultipartUploadResponse.java | 78 -
.../ozone/s3/endpoint/CopyObjectResponse.java | 63 -
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 216 -
.../ozone/s3/endpoint/ListBucketResponse.java | 55 -
.../ozone/s3/endpoint/ListObjectResponse.java | 180 -
.../ozone/s3/endpoint/ListPartsResponse.java | 196 -
.../ozone/s3/endpoint/MultiDeleteRequest.java | 96 -
.../endpoint/MultiDeleteRequestUnmarshaller.java | 84 -
.../ozone/s3/endpoint/MultiDeleteResponse.java | 154 -
.../endpoint/MultipartUploadInitiateResponse.java | 69 -
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 739 -
.../hadoop/ozone/s3/endpoint/RootEndpoint.java | 81 -
.../ozone/s3/endpoint/XmlNamespaceFilter.java | 54 -
.../hadoop/ozone/s3/endpoint/package-info.java | 30 -
.../hadoop/ozone/s3/exception/OS3Exception.java | 161 -
.../ozone/s3/exception/OS3ExceptionMapper.java | 50 -
.../hadoop/ozone/s3/exception/S3ErrorTable.java | 99 -
.../hadoop/ozone/s3/exception/package-info.java | 21 -
.../s3/header/AuthenticationHeaderParser.java | 65 -
.../ozone/s3/header/AuthorizationHeaderV2.java | 97 -
.../ozone/s3/header/AuthorizationHeaderV4.java | 146 -
.../apache/hadoop/ozone/s3/header/Credential.java | 94 -
.../hadoop/ozone/s3/header/package-info.java | 22 -
.../hadoop/ozone/s3/io/S3WrapperInputStream.java | 79 -
.../apache/hadoop/ozone/s3/io/package-info.java | 23 -
.../org/apache/hadoop/ozone/s3/package-info.java | 22 -
.../apache/hadoop/ozone/s3/util/ContinueToken.java | 173 -
.../apache/hadoop/ozone/s3/util/RFC1123Util.java | 98 -
.../apache/hadoop/ozone/s3/util/RangeHeader.java | 89 -
.../ozone/s3/util/RangeHeaderParserUtil.java | 95 -
.../org/apache/hadoop/ozone/s3/util/S3Consts.java | 53 -
.../apache/hadoop/ozone/s3/util/S3StorageType.java | 55 -
.../apache/hadoop/ozone/s3/util/package-info.java | 22 -
.../src/main/resources/META-INF/beans.xml | 20 -
.../s3gateway/src/main/resources/browser.html | 617 -
.../resources/webapps/s3gateway/WEB-INF/beans.xml | 20 -
.../resources/webapps/s3gateway/WEB-INF/web.xml | 36 -
.../main/resources/webapps/static/images/ozone.ico | Bin 1150 -> 0 bytes
.../src/main/resources/webapps/static/index.html | 79 -
.../hadoop/ozone/client/ObjectStoreStub.java | 244 -
.../hadoop/ozone/client/OzoneBucketStub.java | 312 -
.../hadoop/ozone/client/OzoneClientStub.java | 37 -
.../hadoop/ozone/client/OzoneOutputStreamStub.java | 73 -
.../hadoop/ozone/client/OzoneVolumeStub.java | 110 -
.../apache/hadoop/ozone/client/package-info.java | 21 -
.../ozone/s3/TestSignedChunksInputStream.java | 114 -
.../ozone/s3/TestVirtualHostStyleFilter.java | 220 -
.../s3/endpoint/TestAbortMultipartUpload.java | 83 -
.../hadoop/ozone/s3/endpoint/TestBucketDelete.java | 100 -
.../hadoop/ozone/s3/endpoint/TestBucketGet.java | 381 -
.../hadoop/ozone/s3/endpoint/TestBucketHead.java | 71 -
.../ozone/s3/endpoint/TestBucketResponse.java | 38 -
.../s3/endpoint/TestInitiateMultipartUpload.java | 79 -
.../hadoop/ozone/s3/endpoint/TestListParts.java | 129 -
.../TestMultiDeleteRequestUnmarshaller.java | 78 -
.../s3/endpoint/TestMultipartUploadComplete.java | 222 -
.../hadoop/ozone/s3/endpoint/TestObjectDelete.java | 60 -
.../hadoop/ozone/s3/endpoint/TestObjectGet.java | 90 -
.../hadoop/ozone/s3/endpoint/TestObjectHead.java | 104 -
.../ozone/s3/endpoint/TestObjectMultiDelete.java | 116 -
.../hadoop/ozone/s3/endpoint/TestObjectPut.java | 270 -
.../hadoop/ozone/s3/endpoint/TestPartUpload.java | 126 -
.../hadoop/ozone/s3/endpoint/TestRootList.java | 72 -
.../hadoop/ozone/s3/endpoint/package-info.java | 21 -
.../ozone/s3/exception/TestOS3Exception.java | 50 -
.../hadoop/ozone/s3/exception/package-info.java | 22 -
.../ozone/s3/header/TestAuthorizationHeaderV2.java | 88 -
.../ozone/s3/header/TestAuthorizationHeaderV4.java | 97 -
.../org/apache/hadoop/ozone/s3/package-info.java | 21 -
.../hadoop/ozone/s3/util/TestContinueToken.java | 50 -
.../hadoop/ozone/s3/util/TestRFC1123Util.java | 44 -
.../ozone/s3/util/TestRangeHeaderParserUtil.java | 93 -
.../tools/dev-support/findbugsExcludeFile.xml | 19 -
hadoop-ozone/tools/pom.xml | 107 -
.../hadoop/ozone/audit/parser/AuditParser.java | 55 -
.../ozone/audit/parser/common/DatabaseHelper.java | 245 -
.../ozone/audit/parser/common/ParserConsts.java | 35 -
.../ozone/audit/parser/common/package-info.java | 20 -
.../audit/parser/handler/LoadCommandHandler.java | 52 -
.../audit/parser/handler/QueryCommandHandler.java | 57 -
.../parser/handler/TemplateCommandHandler.java | 61 -
.../ozone/audit/parser/handler/package-info.java | 20 -
.../ozone/audit/parser/model/AuditEntry.java | 188 -
.../ozone/audit/parser/model/package-info.java | 20 -
.../hadoop/ozone/audit/parser/package-info.java | 20 -
.../java/org/apache/hadoop/ozone/freon/Freon.java | 46 -
.../org/apache/hadoop/ozone/freon/ProgressBar.java | 147 -
.../hadoop/ozone/freon/RandomKeyGenerator.java | 981 -
.../apache/hadoop/ozone/freon/package-info.java | 22 -
.../apache/hadoop/ozone/fsck/BlockIdDetails.java | 83 -
.../apache/hadoop/ozone/fsck/ContainerMapper.java | 134 -
.../org/apache/hadoop/ozone/fsck/package-info.java | 44 -
.../GenerateOzoneRequiredConfigurations.java | 160 -
.../apache/hadoop/ozone/genconf/package-info.java | 24 -
.../ozone/genesis/BenchMarkBlockManager.java | 150 -
.../ozone/genesis/BenchMarkContainerStateMap.java | 200 -
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 330 -
.../ozone/genesis/BenchMarkMetadataStoreReads.java | 70 -
.../genesis/BenchMarkMetadataStoreWrites.java | 62 -
.../ozone/genesis/BenchMarkOMKeyAllocation.java | 135 -
.../ozone/genesis/BenchMarkRocksDbStore.java | 119 -
.../org/apache/hadoop/ozone/genesis/Genesis.java | 62 -
.../ozone/genesis/GenesisMemoryProfiler.java | 61 -
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 100 -
.../apache/hadoop/ozone/genesis/package-info.java | 25 -
.../org/apache/hadoop/ozone/scm/cli/SQLCLI.java | 565 -
.../apache/hadoop/ozone/scm/cli/package-info.java | 22 -
.../tools/src/main/resources/commands.properties | 22 -
.../hadoop/ozone/audit/parser/TestAuditParser.java | 191 -
.../hadoop/ozone/audit/parser/package-info.java | 21 -
.../hadoop/ozone/freon/TestDataValidate.java | 121 -
.../freon/TestFreonWithDatanodeFastRestart.java | 137 -
.../ozone/freon/TestFreonWithDatanodeRestart.java | 108 -
.../ozone/freon/TestFreonWithPipelineDestroy.java | 109 -
.../apache/hadoop/ozone/freon/TestProgressBar.java | 73 -
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 112 -
.../apache/hadoop/ozone/freon/package-info.java | 22 -
.../hadoop/ozone/fsck/TestContainerMapper.java | 117 -
.../org/apache/hadoop/ozone/fsck/package-info.java | 44 -
.../TestGenerateOzoneRequiredConfigurations.java | 232 -
.../apache/hadoop/ozone/genconf/package-info.java | 22 -
.../org/apache/hadoop/ozone/om/TestOmSQLCli.java | 286 -
.../org/apache/hadoop/ozone/om/package-info.java | 22 -
.../hadoop/ozone/scm/TestContainerSQLCli.java | 260 -
.../org/apache/hadoop/ozone/scm/package-info.java | 22 -
.../org/apache/hadoop/test/OzoneTestDriver.java | 59 -
.../tools/src/test/resources/commands.properties | 22 -
.../tools/src/test/resources/testaudit.log | 15 -
hadoop-project-dist/pom.xml | 6 +-
hadoop-project/pom.xml | 295 +-
hadoop-project/src/site/site.xml | 18 +-
hadoop-submarine/dev-support/checks/rat.sh | 24 +
hadoop-submarine/hadoop-submarine-all/pom.xml | 183 +
hadoop-submarine/hadoop-submarine-core/README.md | 54 +
hadoop-submarine/hadoop-submarine-core/pom.xml | 158 +
.../ubuntu-16.04/Dockerfile.gpu.pytorch_latest | 77 +
.../src/main/docker/pytorch/build-all.sh | 30 +
.../with-cifar10-models/cifar10_tutorial.py | 354 +
.../ubuntu-16.04/Dockerfile.gpu.pytorch_latest | 21 +
.../base/ubuntu-16.04/Dockerfile.cpu.tf_1.13.1 | 71 +
.../base/ubuntu-16.04/Dockerfile.gpu.tf_1.13.1 | 85 +
.../src/main/docker/tensorflow/build-all.sh | 32 +
.../ubuntu-16.04/Dockerfile.cpu.tf_1.13.1 | 22 +
.../ubuntu-16.04/Dockerfile.gpu.tf_1.13.1 | 22 +
.../cifar10_estimator_tf_1.13.1}/README.md | 0
.../cifar10_estimator_tf_1.13.1/cifar10.py | 113 +
.../cifar10_estimator_tf_1.13.1/cifar10_main.py | 521 +
.../cifar10_estimator_tf_1.13.1}/cifar10_model.py | 0
.../cifar10_estimator_tf_1.13.1/cifar10_utils.py | 153 +
.../generate_cifar10_tfrecords.py | 118 +
.../cifar10_estimator_tf_1.13.1}/model_base.py | 0
.../zeppelin-notebook-example/Dockerfile.gpu | 0
.../zeppelin-notebook-example/run_container.sh | 0
.../zeppelin-notebook-example/shiro.ini | 0
.../zeppelin-notebook-example/zeppelin-site.xml | 0
.../yarn/submarine/client/cli/AbstractCli.java | 0
.../hadoop/yarn/submarine/client/cli/Cli.java | 106 +
.../yarn/submarine/client/cli/CliConstants.java | 65 +
.../hadoop/yarn/submarine/client/cli/CliUtils.java | 124 +
.../hadoop/yarn/submarine/client/cli/Command.java | 24 +
.../yarn/submarine/client/cli/KillJobCli.java | 113 +
.../yarn/submarine/client/cli/ShowJobCli.java | 126 +
.../submarine/client/cli/param/BaseParameters.java | 54 +
.../submarine/client/cli/param/ConfigType.java | 24 +
.../client/cli/param/KillJobParameters.java | 19 +
.../submarine/client/cli/param/Localization.java | 0
.../client/cli/param/ParametersHolder.java | 443 +
.../yarn/submarine/client/cli/param/Quicklink.java | 0
.../submarine/client/cli/param/RunParameters.java | 104 +
.../client/cli/param/ShowJobParameters.java | 0
.../submarine/client/cli/param/package-info.java | 0
.../cli/param/runjob/PyTorchRunJobParameters.java | 120 +
.../client/cli/param/runjob/RunJobParameters.java | 348 +
.../param/runjob/TensorFlowRunJobParameters.java | 213 +
.../client/cli/param/runjob/package-info.java | 20 +
.../submarine/client/cli/param/yaml/Configs.java | 107 +
.../submarine/client/cli/param/yaml/PsRole.java | 25 +
.../yarn/submarine/client/cli/param/yaml/Role.java | 91 +
.../submarine/client/cli/param/yaml/Roles.java | 41 +
.../client/cli/param/yaml/Scheduling.java | 32 +
.../submarine/client/cli/param/yaml/Security.java | 50 +
.../yarn/submarine/client/cli/param/yaml/Spec.java | 50 +
.../client/cli/param/yaml/TensorBoard.java | 41 +
.../client/cli/param/yaml/WorkerRole.java | 25 +
.../client/cli/param/yaml/YamlConfigFile.java | 77 +
.../client/cli/param/yaml/YamlParseException.java | 27 +
.../client/cli/param/yaml/package-info.java | 19 +
.../submarine/client/cli/runjob/Framework.java | 59 +
.../client/cli/runjob/RoleParameters.java | 81 +
.../submarine/client/cli/runjob/RunJobCli.java | 381 +
.../submarine/client/cli/runjob/package-info.java | 19 +
.../yarn/submarine/common/ClientContext.java | 0
.../apache/hadoop/yarn/submarine/common/Envs.java | 0
.../submarine/common/api/JobComponentStatus.java | 69 +
.../hadoop/yarn/submarine/common/api/JobState.java | 0
.../yarn/submarine/common/api/JobStatus.java | 0
.../yarn/submarine/common/api/PyTorchRole.java | 54 +
.../hadoop/yarn/submarine/common/api/Role.java | 25 +
.../hadoop/yarn/submarine/common/api/Runtime.java | 58 +
.../yarn/submarine/common/api/TensorFlowRole.java | 41 +
.../common/conf/SubmarineConfiguration.java | 0
.../yarn/submarine/common/conf/SubmarineLogs.java | 0
.../common/exception/SubmarineException.java | 0
.../exception/SubmarineRuntimeException.java | 0
.../common/fs/DefaultRemoteDirectoryManager.java | 0
.../common/fs/RemoteDirectoryManager.java | 0
.../submarine/common/resource/ResourceUtils.java | 332 +
.../common/resource/UnitsConversionUtil.java | 164 +
.../submarine/common/resource/package-info.java | 19 +
.../yarn/submarine/runtimes/RuntimeFactory.java | 103 +
.../common/FSBasedSubmarineStorageImpl.java | 0
.../yarn/submarine/runtimes/common/JobMonitor.java | 0
.../submarine/runtimes/common/JobSubmitter.java | 36 +
.../runtimes/common/StorageKeyConstants.java | 0
.../runtimes/common/SubmarineStorage.java | 0
.../src/site/markdown/Examples.md | 21 +
.../src/site/markdown/HowToInstall.md | 36 +
.../src/site/markdown/Index.md | 47 +
.../src/site/markdown/InstallationGuide.md | 594 +
.../markdown/InstallationGuideChineseVersion.md | 704 +
.../src/site/markdown/QuickStart.md | 322 +
.../markdown/RunningDistributedCifar10TFJobs.md | 164 +
.../markdown/RunningSingleNodeCifar10PTJobs.md | 62 +
.../src/site/markdown/TestAndTroubleshooting.md | 146 +
.../src/site/markdown/TonYRuntimeGuide.md | 309 +
.../src/site/markdown/WriteDockerfilePT.md | 114 +
.../src/site/markdown/WriteDockerfileTF.md | 123 +
.../src/site/resources/css/site.css | 0
.../src/site/resources/images/job-logs-ui.png | Bin
.../resources/images/multiple-tensorboard-jobs.png | Bin
.../site/resources/images/submarine-installer.gif | Bin
.../site/resources/images/tensorboard-service.png | Bin
.../hadoop-submarine-core}/src/site/site.xml | 0
.../client/cli/TestKillJobCliParsing.java | 62 +
.../client/cli/TestShowJobCliParsing.java | 0
.../submarine/client/cli/YamlConfigTestUtils.java | 66 +
.../cli/runjob/TestRunJobCliParsingCommon.java | 129 +
.../cli/runjob/TestRunJobCliParsingCommonYaml.java | 255 +
.../runjob/TestRunJobCliParsingParameterized.java | 192 +
.../pytorch/TestRunJobCliParsingPyTorch.java | 209 +
.../pytorch/TestRunJobCliParsingPyTorchYaml.java | 257 +
.../tensorflow/TestRunJobCliParsingTensorFlow.java | 170 +
.../TestRunJobCliParsingTensorFlowYaml.java | 293 +
...stRunJobCliParsingTensorFlowYamlStandalone.java | 204 +
.../yarn/submarine/common/MockClientContext.java | 38 +
.../common/fs/MockRemoteDirectoryManager.java | 170 +
.../runtimes/common/MemorySubmarineStorage.java | 0
.../common/TestFSBasedSubmarineStorage.java | 0
.../src/test/resources/core-site.xml | 0
.../src/test/resources/hdfs-site.xml | 0
.../runjob-common-yaml/empty-framework.yaml | 63 +
.../runjob-common-yaml/invalid-framework.yaml | 63 +
.../runjob-common-yaml/missing-configs.yaml | 41 +
.../runjob-common-yaml/missing-framework.yaml | 62 +
.../runjob-common-yaml/some-sections-missing.yaml | 49 +
.../runjob-common-yaml/test-false-values.yaml | 57 +
.../runjob-common-yaml/wrong-indentation.yaml | 60 +
.../runjob-common-yaml/wrong-property-name.yaml | 60 +
.../runjob-pytorch-yaml/envs-are-missing.yaml | 51 +
.../invalid-config-ps-section.yaml | 56 +
.../invalid-config-tensorboard-section.yaml | 57 +
.../security-principal-is-missing.yaml | 53 +
.../valid-config-with-overrides.yaml | 63 +
.../runjob-pytorch-yaml/valid-config.yaml | 54 +
.../runjob-pytorch-yaml/valid-gpu-config.yaml | 54 +
.../runjob-tensorflow-yaml/envs-are-missing.yaml | 60 +
.../security-principal-is-missing.yaml | 62 +
.../tensorboard-dockerimage-is-missing.yaml | 62 +
.../valid-config-with-overrides.yaml | 82 +
.../runjob-tensorflow-yaml/valid-config.yaml | 63 +
.../runjob-tensorflow-yaml/valid-gpu-config.yaml | 63 +
hadoop-submarine/hadoop-submarine-dist/pom.xml | 131 +
.../src/assembly/distribution.xml | 61 +
.../hadoop-submarine-tony-runtime/README.md | 25 +
.../hadoop-submarine-tony-runtime/pom.xml | 70 +
.../submarine/runtimes/tony/TonyJobMonitor.java | 52 +
.../submarine/runtimes/tony/TonyJobSubmitter.java | 109 +
.../runtimes/tony/TonyRuntimeFactory.java | 55 +
.../yarn/submarine/runtimes/tony/TonyUtils.java | 159 +
.../runtimes/tony/buider/JobStatusBuilder.java | 61 +
.../runtimes/tony/buider/package-info.java | 14 +
.../yarn/submarine/runtimes/tony/package-info.java | 14 +
.../src/test/java/TestTonyUtils.java | 125 +
.../README.md | 0
.../hadoop-submarine-yarnservice-runtime/pom.xml | 159 +
.../runtimes/yarnservice/AbstractComponent.java | 143 +
.../runtimes/yarnservice/AbstractServiceSpec.java | 168 +
.../runtimes/yarnservice/FileSystemOperations.java | 211 +
.../yarnservice/HadoopEnvironmentSetup.java | 176 +
.../runtimes/yarnservice/ServiceSpec.java | 27 +
.../yarnservice/ServiceSpecFileGenerator.java | 51 +
.../runtimes/yarnservice/ServiceWrapper.java | 62 +
.../yarnservice/WorkerComponentFactory.java | 71 +
.../yarnservice/YarnServiceJobMonitor.java | 58 +
.../yarnservice/YarnServiceJobSubmitter.java | 175 +
.../yarnservice/YarnServiceRuntimeFactory.java | 0
.../runtimes/yarnservice/YarnServiceUtils.java | 59 +
.../builder/JobComponentStatusBuilder.java | 44 +
.../yarnservice/builder/JobStatusBuilder.java | 63 +
.../yarnservice/command/AbstractLaunchCommand.java | 61 +
.../yarnservice/command/LaunchCommandFactory.java | 30 +
.../yarnservice/command/LaunchScriptBuilder.java | 108 +
.../command/PyTorchLaunchCommandFactory.java | 61 +
.../command/TensorFlowLaunchCommandFactory.java | 70 +
.../runtimes/yarnservice/command/package-info.java | 19 +
.../yarnservice/pytorch/PyTorchServiceSpec.java | 68 +
.../command/PyTorchWorkerLaunchCommand.java | 87 +
.../yarnservice/pytorch/command/package-info.java | 19 +
.../pytorch/component/PyTorchWorkerComponent.java | 47 +
.../pytorch/component/package-info.java | 20 +
.../runtimes/yarnservice/pytorch/package-info.java | 20 +
.../yarnservice/tensorflow/TensorFlowCommons.java | 109 +
.../tensorflow/TensorFlowServiceSpec.java | 100 +
.../command/TensorBoardLaunchCommand.java | 67 +
.../command/TensorFlowLaunchCommand.java | 91 +
.../command/TensorFlowPsLaunchCommand.java | 59 +
.../command/TensorFlowWorkerLaunchCommand.java | 59 +
.../tensorflow/command/package-info.java | 19 +
.../tensorflow/component/TensorBoardComponent.java | 100 +
.../component/TensorFlowPsComponent.java | 77 +
.../component/TensorFlowWorkerComponent.java | 46 +
.../tensorflow/component/package-info.java | 20 +
.../yarnservice/tensorflow/package-info.java | 20 +
.../yarn/submarine/utils/ClassPathUtilities.java | 57 +
.../yarn/submarine/utils/DockerUtilities.java | 33 +
.../yarn/submarine/utils/EnvironmentUtilities.java | 120 +
.../submarine/utils/KerberosPrincipalFactory.java | 95 +
.../hadoop/yarn/submarine/utils/Localizer.java | 170 +
.../submarine/utils/SubmarineResourceUtils.java | 51 +
.../hadoop/yarn/submarine/utils/ZipUtilities.java | 82 +
.../hadoop/yarn/submarine/utils/package-info.java | 19 +
.../yarn/submarine/FileUtilitiesForTests.java | 150 +
.../cli/yarnservice/ParamBuilderForTest.java | 151 +
.../cli/yarnservice/TestYarnServiceRunJobCli.java | 677 +
.../TestYarnServiceRunJobCliCommons.java | 80 +
.../TestYarnServiceRunJobCliLocalization.java | 600 +
.../cli/yarnservice/YarnServiceCliTestUtils.java | 0
.../runtimes/yarnservice/TestServiceWrapper.java | 95 +
.../yarnservice/TestTFConfigGenerator.java | 46 +
.../command/AbstractTFLaunchCommandTestHelper.java | 192 +
.../command/TestLaunchCommandFactory.java | 98 +
.../pytorch/TestPyTorchServiceSpec.java | 90 +
.../command/TestTensorBoardLaunchCommand.java | 104 +
.../command/TestTensorFlowLaunchCommand.java | 253 +
.../tensorflow/component/ComponentTestCommons.java | 109 +
.../component/TestTensorBoardComponent.java | 127 +
.../component/TestTensorFlowPsComponent.java | 168 +
.../component/TestTensorFlowWorkerComponent.java | 218 +
.../submarine/utils/TestClassPathUtilities.java | 91 +
.../submarine/utils/TestEnvironmentUtilities.java | 231 +
.../utils/TestKerberosPrincipalFactory.java | 156 +
.../utils/TestSubmarineResourceUtils.java | 112 +
.../src/test/resources/core-site.xml | 0
.../src/test/resources/hdfs-site.xml | 0
hadoop-submarine/pom.xml | 158 +
hadoop-tools/hadoop-aliyun/pom.xml | 2 +-
.../fs/aliyun/oss/AliyunOSSBlockOutputStream.java | 10 +-
.../fs/aliyun/oss/AliyunOSSCopyFileTask.java | 2 +-
.../hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java | 8 +-
.../fs/aliyun/oss/AliyunOSSFileSystemStore.java | 7 +-
.../aliyun/oss/TestAliyunOSSBlockOutputStream.java | 32 +-
.../oss/TestAliyunOSSFileSystemContract.java | 76 +
hadoop-tools/hadoop-archive-logs/pom.xml | 2 +-
hadoop-tools/hadoop-archives/pom.xml | 7 +-
.../apache/hadoop/tools/TestHadoopArchives.java | 3 +-
hadoop-tools/hadoop-aws/pom.xml | 21 +-
.../java/org/apache/hadoop/fs/s3a/Constants.java | 286 +-
.../hadoop/fs/s3a/DefaultS3ClientFactory.java | 3 +-
.../hadoop/fs/s3a/InconsistentAmazonS3Client.java | 32 +-
.../apache/hadoop/fs/s3a/InternalConstants.java | 53 -
.../java/org/apache/hadoop/fs/s3a/Invoker.java | 87 +-
.../java/org/apache/hadoop/fs/s3a/Listing.java | 117 +-
.../fs/s3a/MetadataPersistenceException.java | 40 +
.../hadoop/fs/s3a/NoVersionAttributeException.java | 44 +
.../hadoop/fs/s3a/RemoteFileChangedException.java | 83 +
.../java/org/apache/hadoop/fs/s3a/Retries.java | 21 +-
.../apache/hadoop/fs/s3a/S3ABlockOutputStream.java | 15 +-
.../org/apache/hadoop/fs/s3a/S3ADataBlocks.java | 4 +-
.../org/apache/hadoop/fs/s3a/S3AFileStatus.java | 101 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 1834 +-
.../org/apache/hadoop/fs/s3a/S3AInputStream.java | 93 +-
.../apache/hadoop/fs/s3a/S3AInstrumentation.java | 142 +-
.../apache/hadoop/fs/s3a/S3ALocatedFileStatus.java | 96 +
.../org/apache/hadoop/fs/s3a/S3AReadOpContext.java | 14 +
.../org/apache/hadoop/fs/s3a/S3ARetryPolicy.java | 63 +-
.../java/org/apache/hadoop/fs/s3a/S3AUtils.java | 169 +-
.../hadoop/fs/s3a/S3GuardExistsRetryPolicy.java | 31 +-
.../apache/hadoop/fs/s3a/S3ObjectAttributes.java | 66 +-
.../fs/s3a/SharedInstanceCredentialProvider.java | 44 +
.../java/org/apache/hadoop/fs/s3a/Statistic.java | 15 +-
.../apache/hadoop/fs/s3a/WriteOperationHelper.java | 96 +-
.../hadoop/fs/s3a/auth/AwsSignerInitializer.java | 53 +
.../s3a/auth/IAMInstanceCredentialsProvider.java | 34 +-
.../fs/s3a/auth/MarshalledCredentialBinding.java | 32 +-
.../hadoop/fs/s3a/auth/MarshalledCredentials.java | 2 +-
.../apache/hadoop/fs/s3a/auth/RolePolicies.java | 6 +-
.../hadoop/fs/s3a/auth/STSClientFactory.java | 8 +-
.../apache/hadoop/fs/s3a/auth/SignerManager.java | 147 +
.../delegation/AbstractDelegationTokenBinding.java | 14 +-
.../delegation/AbstractS3ATokenIdentifier.java | 11 +-
.../auth/delegation/DelegationTokenProvider.java | 31 +
.../delegation/FullCredentialsTokenBinding.java | 7 +-
.../delegation/FullCredentialsTokenIdentifier.java | 2 +
.../fs/s3a/auth/delegation/RoleTokenBinding.java | 7 +-
.../s3a/auth/delegation/RoleTokenIdentifier.java | 2 +
.../s3a/auth/delegation/S3ADelegationTokens.java | 24 +-
.../s3a/auth/delegation/SessionTokenBinding.java | 20 +-
.../auth/delegation/SessionTokenIdentifier.java | 6 +-
.../hadoop/fs/s3a/commit/AbstractS3ACommitter.java | 495 +-
.../fs/s3a/commit/AbstractS3ACommitterFactory.java | 2 +-
.../hadoop/fs/s3a/commit/CommitConstants.java | 30 +-
.../hadoop/fs/s3a/commit/CommitOperations.java | 163 +-
.../org/apache/hadoop/fs/s3a/commit/Duration.java | 60 -
.../apache/hadoop/fs/s3a/commit/DurationInfo.java | 90 -
.../hadoop/fs/s3a/commit/files/SuccessData.java | 6 +
.../fs/s3a/commit/magic/MagicS3GuardCommitter.java | 17 +-
.../commit/staging/DirectoryStagingCommitter.java | 45 +-
.../staging/PartitionedStagingCommitter.java | 101 +-
.../fs/s3a/commit/staging/StagingCommitter.java | 67 +-
.../commit/staging/StagingCommitterConstants.java | 2 +-
.../hadoop/fs/s3a/impl/AbstractStoreOperation.java | 49 +
.../hadoop/fs/s3a/impl/CallableSupplier.java | 126 +
.../hadoop/fs/s3a/impl/ChangeDetectionPolicy.java | 523 +
.../apache/hadoop/fs/s3a/impl/ChangeTracker.java | 317 +
.../hadoop/fs/s3a/impl/ContextAccessors.java | 76 +
.../org/apache/hadoop/fs/s3a/impl/CopyOutcome.java | 80 +
.../apache/hadoop/fs/s3a/impl/DeleteOperation.java | 577 +
.../fs/s3a/impl/ExecutingStoreOperation.java | 69 +
.../hadoop/fs/s3a/impl/InternalConstants.java | 88 +
.../apache/hadoop/fs/s3a/impl/LogExactlyOnce.java | 42 +
.../fs/s3a/impl/MultiObjectDeleteSupport.java | 250 +
.../apache/hadoop/fs/s3a/impl/NetworkBinding.java | 131 +
.../hadoop/fs/s3a/impl/OperationCallbacks.java | 198 +
.../apache/hadoop/fs/s3a/impl/RenameOperation.java | 520 +
.../apache/hadoop/fs/s3a/impl/StatusProbeEnum.java | 60 +
.../apache/hadoop/fs/s3a/impl/StoreContext.java | 354 +
.../apache/hadoop/fs/s3a/impl/package-info.java | 30 +
.../s3guard/AbstractS3GuardDynamoDBDiagnostic.java | 223 +
.../hadoop/fs/s3a/s3guard/BulkOperationState.java | 88 +
.../hadoop/fs/s3a/s3guard/DDBPathMetadata.java | 19 +-
.../fs/s3a/s3guard/DelayedUpdateRenameTracker.java | 188 +
.../hadoop/fs/s3a/s3guard/DescendantsIterator.java | 6 +-
.../hadoop/fs/s3a/s3guard/DirListingMetadata.java | 42 +-
.../fs/s3a/s3guard/DumpS3GuardDynamoTable.java | 787 +
.../fs/s3a/s3guard/DynamoDBClientFactory.java | 4 +-
.../fs/s3a/s3guard/DynamoDBMetadataStore.java | 1585 +-
.../s3guard/DynamoDBMetadataStoreTableManager.java | 712 +
.../hadoop/fs/s3a/s3guard/ITtlTimeProvider.java | 34 +
.../hadoop/fs/s3a/s3guard/InternalIterators.java | 91 +
.../hadoop/fs/s3a/s3guard/LocalMetadataStore.java | 222 +-
.../hadoop/fs/s3a/s3guard/MetadataStore.java | 203 +-
.../s3guard/MetadataStoreListFilesIterator.java | 12 +-
.../hadoop/fs/s3a/s3guard/NullMetadataStore.java | 84 +-
.../apache/hadoop/fs/s3a/s3guard/PathMetadata.java | 85 +-
.../s3guard/PathMetadataDynamoDBTranslation.java | 83 +-
.../fs/s3a/s3guard/PathOrderComparators.java | 133 +
.../fs/s3a/s3guard/ProgressiveRenameTracker.java | 245 +
.../fs/s3a/s3guard/PurgeS3GuardDynamoTable.java | 248 +
.../hadoop/fs/s3a/s3guard/RenameTracker.java | 275 +
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 481 +-
.../apache/hadoop/fs/s3a/s3guard/S3GuardFsck.java | 484 +
.../s3a/s3guard/S3GuardFsckViolationHandler.java | 346 +
.../hadoop/fs/s3a/s3guard/S3GuardTableAccess.java | 241 +
.../apache/hadoop/fs/s3a/s3guard/S3GuardTool.java | 315 +-
.../s3a/s3guard/TableDeleteTimeoutException.java | 34 +
.../fs/s3a/select/InternalSelectConstants.java | 2 +-
.../hadoop/fs/s3a/select/SelectConstants.java | 2 +-
.../apache/hadoop/fs/s3a/select/SelectTool.java | 12 +-
.../markdown/tools/hadoop-aws/assumed_roles.md | 61 +-
.../site/markdown/tools/hadoop-aws/committers.md | 22 +-
.../markdown/tools/hadoop-aws/delegation_tokens.md | 5 +-
.../src/site/markdown/tools/hadoop-aws/index.md | 195 +-
.../site/markdown/tools/hadoop-aws/performance.md | 55 +-
.../src/site/markdown/tools/hadoop-aws/s3guard.md | 671 +-
.../src/site/markdown/tools/hadoop-aws/testing.md | 145 +-
.../tools/hadoop-aws/troubleshooting_s3a.md | 237 +-
.../fs/contract/s3a/ITestS3AContractOpen.java | 9 +
.../fs/contract/s3a/ITestS3AContractRename.java | 71 +-
.../fs/contract/s3a/ITestS3AContractRootDir.java | 24 +-
.../fs/contract/s3a/ITestS3AContractSeek.java | 287 +
.../fs/contract/s3a/ITestS3AContractUnbuffer.java | 41 +
.../apache/hadoop/fs/contract/s3a/S3AContract.java | 25 +-
.../apache/hadoop/fs/s3a/AbstractS3AMockTest.java | 17 +-
.../apache/hadoop/fs/s3a/AbstractS3ATestBase.java | 22 +-
.../hadoop/fs/s3a/AbstractTestS3AEncryption.java | 80 +-
.../hadoop/fs/s3a/ITestAuthoritativePath.java | 309 +
.../fs/s3a/ITestLocatedFileStatusFetcher.java | 40 +
.../fs/s3a/ITestS3AAWSCredentialsProvider.java | 18 +-
.../org/apache/hadoop/fs/s3a/ITestS3AClosedFS.java | 18 +-
.../hadoop/fs/s3a/ITestS3AConfiguration.java | 93 +-
.../apache/hadoop/fs/s3a/ITestS3ADelayedFNF.java | 44 +-
.../hadoop/fs/s3a/ITestS3AEmptyDirectory.java | 5 +-
.../hadoop/fs/s3a/ITestS3AEncryptionSSEC.java | 46 +-
.../fs/s3a/ITestS3AEncryptionSSEKMSDefaultKey.java | 7 +-
.../ITestS3AEncryptionSSEKMSUserDefinedKey.java | 20 +-
.../hadoop/fs/s3a/ITestS3AFSMainOperations.java | 65 +
.../hadoop/fs/s3a/ITestS3AFailureHandling.java | 167 +-
.../hadoop/fs/s3a/ITestS3AFileOperationCost.java | 160 +-
.../hadoop/fs/s3a/ITestS3AInconsistency.java | 47 +-
.../s3a/ITestS3AMetadataPersistenceException.java | 152 +
.../hadoop/fs/s3a/ITestS3AMiscOperations.java | 22 +-
.../hadoop/fs/s3a/ITestS3ARemoteFileChanged.java | 1432 +
.../fs/s3a/ITestS3ATemporaryCredentials.java | 4 +-
.../org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java | 129 +
.../hadoop/fs/s3a/ITestS3GuardEmptyDirs.java | 207 +-
.../hadoop/fs/s3a/ITestS3GuardListConsistency.java | 283 +-
.../fs/s3a/ITestS3GuardOutOfBandOperations.java | 1091 +
.../org/apache/hadoop/fs/s3a/ITestS3GuardTtl.java | 317 +-
.../hadoop/fs/s3a/ITestS3GuardWriteBack.java | 1 +
.../apache/hadoop/fs/s3a/MockS3AFileSystem.java | 26 +-
.../org/apache/hadoop/fs/s3a/S3ATestConstants.java | 25 +
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 266 +-
.../java/org/apache/hadoop/fs/s3a/TestListing.java | 29 +-
.../org/apache/hadoop/fs/s3a/TestS3AUnbuffer.java | 76 +
.../hadoop/fs/s3a/TestStreamChangeTracker.java | 448 +
.../apache/hadoop/fs/s3a/auth/ITestAssumeRole.java | 275 +-
.../s3a/auth/ITestAssumedRoleCommitOperations.java | 4 +-
.../hadoop/fs/s3a/auth/ITestCustomSigner.java | 234 +
.../fs/s3a/auth/ITestRestrictedReadAccess.java | 784 +
.../apache/hadoop/fs/s3a/auth/RoleTestUtils.java | 54 +-
.../hadoop/fs/s3a/auth/TestSignerManager.java | 590 +
.../s3a/auth/delegation/AbstractDelegationIT.java | 9 +-
.../s3a/auth/delegation/ITestDelegatedMRJob.java | 43 +-
.../auth/delegation/ITestRoleDelegationTokens.java | 2 +-
.../ITestSessionDelegationInFileystem.java | 13 +-
.../delegation/ITestSessionDelegationTokens.java | 35 +-
.../delegation/TestS3ADelegationTokenSupport.java | 31 +
.../hadoop/fs/s3a/commit/AbstractCommitITest.java | 101 +-
.../fs/s3a/commit/AbstractITCommitMRJob.java | 318 -
.../fs/s3a/commit/AbstractITCommitProtocol.java | 76 +-
.../fs/s3a/commit/AbstractYarnClusterITest.java | 357 +
.../fs/s3a/commit/ITestCommitOperations.java | 136 +-
.../org/apache/hadoop/fs/s3a/commit/TestTasks.java | 2 +-
.../commit/integration/ITestS3ACommitterMRJob.java | 646 +
.../fs/s3a/commit/magic/ITMagicCommitMRJob.java | 70 -
.../s3a/commit/magic/ITestS3AHugeMagicCommits.java | 9 +-
.../fs/s3a/commit/staging/StagingTestBase.java | 100 +-
.../staging/TestDirectoryCommitterScale.java | 314 +
.../s3a/commit/staging/TestStagingCommitter.java | 35 +-
.../TestStagingDirectoryOutputCommitter.java | 94 +-
.../staging/TestStagingPartitionedJobCommit.java | 43 +-
.../staging/TestStagingPartitionedTaskCommit.java | 84 +-
.../integration/ITDirectoryCommitMRJob.java | 33 -
.../integration/ITPartitionCommitMRJob.java | 33 -
.../staging/integration/ITStagingCommitMRJob.java | 66 -
.../integration/ITStagingCommitMRJobBadDest.java | 62 -
.../integration/ITestDirectoryCommitProtocol.java | 34 +
.../fs/s3a/commit/terasort/ITestTerasortOnS3A.java | 377 +
.../fs/s3a/impl/ITestPartialRenamesDeletes.java | 833 +
.../hadoop/fs/s3a/impl/TestNeworkBinding.java | 61 +
.../fs/s3a/impl/TestPartialDeleteFailures.java | 403 +
.../s3a/s3guard/AbstractS3GuardToolTestBase.java | 212 +-
.../hadoop/fs/s3a/s3guard/DDBCapacities.java | 119 +
.../fs/s3a/s3guard/ITestDynamoDBMetadataStore.java | 1063 +-
.../s3guard/ITestDynamoDBMetadataStoreScale.java | 256 +-
.../fs/s3a/s3guard/ITestS3GuardConcurrentOps.java | 18 +-
.../s3a/s3guard/ITestS3GuardDDBRootOperations.java | 270 +
.../hadoop/fs/s3a/s3guard/ITestS3GuardFsck.java | 504 +
.../fs/s3a/s3guard/ITestS3GuardToolDynamoDB.java | 187 +-
.../fs/s3a/s3guard/ITestS3GuardToolLocal.java | 40 +-
.../fs/s3a/s3guard/MetadataStoreTestBase.java | 509 +-
.../fs/s3a/s3guard/TestDirListingMetadata.java | 60 +-
.../fs/s3a/s3guard/TestDynamoDBMiscOperations.java | 89 +-
.../fs/s3a/s3guard/TestLocalMetadataStore.java | 58 +-
.../fs/s3a/s3guard/TestNullMetadataStore.java | 5 +
.../TestObjectChangeDetectionAttributes.java | 380 +
.../TestPathMetadataDynamoDBTranslation.java | 99 +-
.../fs/s3a/s3guard/TestPathOrderComparators.java | 197 +
.../apache/hadoop/fs/s3a/s3guard/TestS3Guard.java | 239 +-
.../hadoop/fs/s3a/s3guard/ThrottleTracker.java | 134 +
.../scale/AbstractITestS3AMetadataStoreScale.java | 37 +-
.../fs/s3a/scale/ITestLocalMetadataStoreScale.java | 3 +-
.../hadoop/fs/s3a/scale/ITestS3AConcurrentOps.java | 175 +-
.../fs/s3a/scale/ITestS3ADeleteManyFiles.java | 94 +-
.../hadoop/fs/s3a/scale/S3AScaleTestBase.java | 4 +
.../hadoop/fs/s3a/select/AbstractS3SelectTest.java | 2 +-
.../apache/hadoop/fs/s3a/select/ITestS3Select.java | 6 +-
.../hadoop/fs/s3a/select/ITestS3SelectCLI.java | 12 +-
.../hadoop/fs/s3a/select/ITestS3SelectLandsat.java | 10 +-
.../hadoop/fs/s3a/select/ITestS3SelectMRJob.java | 12 +-
.../apache/hadoop/fs/s3a/test/ExtraAssertions.java | 138 +
.../mapreduce/filecache/TestS3AResourceScope.java | 64 +
.../hadoop-aws/src/test/resources/contract/s3a.xml | 5 +
.../hadoop-aws/src/test/resources/core-site.xml | 4 +-
.../hadoop-aws/src/test/resources/log4j.properties | 2 +
hadoop-tools/hadoop-azure-datalake/pom.xml | 6 +-
.../java/org/apache/hadoop/fs/adl/AdlConfKeys.java | 1 +
.../org/apache/hadoop/fs/adl/AdlFileSystem.java | 22 +
.../src/site/markdown/troubleshooting_adl.md | 10 +
.../fs/adl/live/TestAdlSdkConfiguration.java | 33 +-
hadoop-tools/hadoop-azure/pom.xml | 28 +-
.../src/config/checkstyle-suppressions.xml | 4 +-
.../hadoop-azure/src/config/checkstyle.xml | 9 +-
.../fs/azure/AzureNativeFileSystemStore.java | 9 +-
.../hadoop/fs/azure/LocalSASKeyGeneratorImpl.java | 14 +-
.../hadoop/fs/azure/NativeAzureFileSystem.java | 39 +
.../fs/azure/SecureStorageInterfaceImpl.java | 4 +
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 76 +-
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 91 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 990 +-
.../fs/azurebfs/constants/AbfsHttpConstants.java | 10 +
.../fs/azurebfs/constants/AuthConfigurations.java | 45 +
.../fs/azurebfs/constants/ConfigurationKeys.java | 20 +
.../constants/FileSystemConfigurations.java | 13 +-
.../constants/HttpHeaderConfigurations.java | 1 +
.../fs/azurebfs/constants/HttpQueryParams.java | 2 +
.../exceptions/AbfsRestOperationException.java | 6 +-
.../exceptions/TokenAccessProviderException.java | 3 +-
.../contracts/services/AbfsPerfLoggable.java | 34 +
.../fs/azurebfs/extensions/BoundDTExtension.java | 97 +
.../extensions/CustomDelegationTokenManager.java | 7 +
.../fs/azurebfs/extensions/ExtensionHelper.java | 114 +
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 147 +-
.../oauth2/CustomTokenProviderAdapter.java | 45 +-
.../fs/azurebfs/oauth2/IdentityTransformer.java | 114 +-
.../fs/azurebfs/oauth2/MsiTokenProvider.java | 14 +-
.../oauth2/RefreshTokenBasedTokenProvider.java | 12 +-
.../security/AbfsDelegationTokenIdentifier.java | 16 +
.../security/AbfsDelegationTokenManager.java | 99 +-
.../hadoop/fs/azurebfs/security/AbfsDtFetcher.java | 87 +
.../fs/azurebfs/security/AbfsTokenRenewer.java | 4 +-
.../fs/azurebfs/security/AbfssDtFetcher.java | 36 +
.../hadoop/fs/azurebfs/security/package-info.java | 2 +-
.../hadoop/fs/azurebfs/services/AbfsClient.java | 65 +-
.../fs/azurebfs/services/AbfsHttpOperation.java | 53 +-
.../fs/azurebfs/services/AbfsInputStream.java | 4 +-
.../hadoop/fs/azurebfs/services/AbfsIoUtils.java | 67 +
.../fs/azurebfs/services/AbfsOutputStream.java | 62 +-
.../hadoop/fs/azurebfs/services/AbfsPerfInfo.java | 133 +
.../fs/azurebfs/services/AbfsPerfTracker.java | 319 +
.../fs/azurebfs/services/AbfsRestOperation.java | 17 +-
.../azurebfs/services/AbfsRestOperationType.java | 3 +-
.../azurebfs/services/ExponentialRetryPolicy.java | 11 +-
.../fs/azurebfs/services/SharedKeyCredentials.java | 15 +-
.../org/apache/hadoop/fs/azurebfs/utils/CRC64.java | 60 +
.../fs/azurebfs/utils/SSLSocketFactoryEx.java | 243 -
.../org.apache.hadoop.security.token.DtFetcher | 20 +
.../hadoop-azure/src/site/markdown/abfs.md | 846 +-
.../hadoop-azure/src/site/markdown/index.md | 15 +-
.../src/site/markdown/testing_azure.md | 9 +-
.../fs/azure/AzureBlobStorageTestAccount.java | 8 +-
.../fs/azure/ITestWasbUriAndConfiguration.java | 91 +-
.../contract/NativeAzureFileSystemContract.java | 1 +
.../fs/azure/integration/AzureTestUtils.java | 9 +
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 17 +-
.../fs/azurebfs/ITestAbfsIdentityTransformer.java | 84 +-
.../azurebfs/ITestAbfsRestOperationException.java | 41 +
.../fs/azurebfs/ITestAzureBlobFileSystemCLI.java | 63 +
.../ITestAzureBlobFileSystemCheckAccess.java | 314 +
.../ITestAzureBlobFileSystemFileStatus.java | 18 +
.../fs/azurebfs/ITestAzureBlobFileSystemFlush.java | 27 +-
.../ITestAzureBlobFileSystemMainOperation.java | 78 +
...zureBlobFileSystemStoreListStatusWithRange.java | 151 +
.../fs/azurebfs/ITestFileSystemProperties.java | 4 +-
.../fs/azurebfs/ITestGetNameSpaceEnabled.java | 6 +-
.../TestAbfsConfigurationFieldsValidation.java | 24 +-
.../apache/hadoop/fs/azurebfs/TestAbfsCrc64.java | 38 +
.../azurebfs/constants/TestConfigurationKeys.java | 8 +
.../extensions/ClassicDelegationTokenManager.java | 282 +
.../extensions/ITestAbfsDelegationTokens.java | 370 +
.../azurebfs/extensions/KerberizedAbfsCluster.java | 318 +
.../extensions/StubAbfsTokenIdentifier.java | 184 +
.../extensions/StubDelegationTokenManager.java | 95 +
.../extensions/TestCustomOauthTokenProvider.java | 77 +
.../extensions/TestDTManagerLifecycle.java | 191 +
.../azurebfs/extensions/WrappingTokenProvider.java | 107 +
.../fs/azurebfs/oauth2/RetryTestTokenProvider.java | 67 +
.../fs/azurebfs/services/TestAbfsClient.java | 8 +-
.../fs/azurebfs/services/TestAbfsPerfTracker.java | 408 +
...rg.apache.hadoop.security.token.TokenIdentifier | 34 +
.../hadoop-azure/src/test/resources/azure-test.xml | 10 +
.../src/test/resources/log4j.properties | 1 +
.../hadoop-azure/src/test/resources/wasb.xml | 7 +-
hadoop-tools/hadoop-datajoin/pom.xml | 2 +-
hadoop-tools/hadoop-distcp/pom.xml | 7 +-
.../java/org/apache/hadoop/tools/CopyListing.java | 23 +
.../main/java/org/apache/hadoop/tools/DistCp.java | 9 +
.../org/apache/hadoop/tools/DistCpConstants.java | 6 +
.../org/apache/hadoop/tools/RegexCopyFilter.java | 4 +-
.../org/apache/hadoop/tools/SimpleCopyListing.java | 4 +-
.../apache/hadoop/tools/mapred/CopyCommitter.java | 31 +-
.../org/apache/hadoop/tools/mapred/CopyMapper.java | 3 +-
.../tools/mapred/RetriableFileCopyCommand.java | 86 +-
.../org/apache/hadoop/tools/util/DistCpUtils.java | 66 +-
.../hadoop-distcp/src/site/markdown/DistCp.md.vm | 178 +-
.../org/apache/hadoop/tools/TestCopyListing.java | 5 +-
.../org/apache/hadoop/tools/TestOptionsParser.java | 30 +-
.../hadoop/tools/mapred/TestCopyCommitter.java | 244 +-
.../apache/hadoop/tools/mapred/TestCopyMapper.java | 59 +-
.../tools/mapred/TestRetriableFileCopyCommand.java | 25 +-
.../apache/hadoop/tools/util/TestDistCpUtils.java | 79 +-
.../tools/util/TestDistCpUtilsWithCombineMode.java | 115 +
.../hadoop-dynamometer-blockgen/pom.xml | 131 +
.../src/main/bash/generate-block-lists.sh | 38 +
.../dynamometer/blockgenerator/BlockInfo.java | 102 +
.../blockgenerator/GenerateBlockImagesDriver.java | 140 +
.../GenerateDNBlockInfosReducer.java | 99 +
.../dynamometer/blockgenerator/XMLParser.java | 174 +
.../blockgenerator/XMLParserMapper.java | 81 +
.../dynamometer/blockgenerator/package-info.java | 26 +
.../dynamometer/blockgenerator/TestBlockGen.java | 89 +
.../dynamometer/blockgenerator/TestXMLParser.java | 93 +
.../test/resources/fsimage_0000000000000061740.xml | 553 +
.../hadoop-dynamometer-dist/pom.xml | 128 +
.../hadoop-dynamometer-infra/pom.xml | 211 +
.../src/main/bash/create-slim-hadoop-tar.sh | 53 +
.../src/main/bash/parse-metrics.sh | 84 +
.../src/main/bash/start-dynamometer-cluster.sh | 41 +
.../src/main/bash/upload-fsimage.sh | 84 +
.../apache/hadoop/tools/dynamometer/AMOptions.java | 317 +
.../dynamometer/AllowAllImpersonationProvider.java | 39 +
.../tools/dynamometer/ApplicationMaster.java | 889 +
.../BlockPlacementPolicyAlwaysSatisfied.java | 58 +
.../apache/hadoop/tools/dynamometer/Client.java | 1140 +
.../hadoop/tools/dynamometer/DynoConstants.java | 130 +
.../hadoop/tools/dynamometer/DynoInfraUtils.java | 593 +
.../hadoop/tools/dynamometer/DynoResource.java | 72 +
.../tools/dynamometer/SimulatedDataNodes.java | 185 +
.../hadoop/tools/dynamometer/package-info.java | 29 +
.../src/main/resources/start-component.sh | 348 +
.../tools/dynamometer/TestDynamometerInfra.java | 578 +
.../tools/dynamometer/TestDynoInfraUtils.java | 66 +
.../src/test/resources/blocks/dn0-a-0-r-00000 | 34 +
.../src/test/resources/blocks/dn1-a-0-r-00001 | 33 +
.../src/test/resources/blocks/dn2-a-0-r-00002 | 33 +
.../test/resources/conf/etc/hadoop/core-site.xml | 20 +
.../test/resources/conf/etc/hadoop/hdfs-site.xml | 20 +
.../resources/conf/etc/hadoop/log4j.properties | 26 +
.../src/test/resources/hadoop_3_1/VERSION | 7 +
.../hadoop_3_1/fsimage_0000000000000061740 | Bin 0 -> 13795 bytes
.../hadoop_3_1/fsimage_0000000000000061740.md5 | 1 +
.../src/test/resources/yarn-site.xml | 19 +
.../hadoop-dynamometer-workload/pom.xml | 146 +
.../src/main/bash/parse-start-timestamp.sh | 52 +
.../src/main/bash/start-workload.sh | 42 +
.../workloadgenerator/CreateFileMapper.java | 147 +
.../workloadgenerator/VirtualInputFormat.java | 57 +
.../workloadgenerator/VirtualInputSplit.java | 52 +
.../workloadgenerator/VirtualRecordReader.java | 78 +
.../workloadgenerator/WorkloadDriver.java | 203 +
.../workloadgenerator/WorkloadMapper.java | 70 +
.../audit/AuditCommandParser.java | 59 +
.../audit/AuditLogDirectParser.java | 156 +
.../audit/AuditLogHiveTableParser.java | 72 +
.../audit/AuditReplayCommand.java | 163 +
.../workloadgenerator/audit/AuditReplayMapper.java | 311 +
.../audit/AuditReplayReducer.java | 44 +
.../workloadgenerator/audit/AuditReplayThread.java | 328 +
.../workloadgenerator/audit/CountTimeWritable.java | 82 +
.../audit/NoSplitTextInputFormat.java | 47 +
.../workloadgenerator/audit/UserCommandKey.java | 111 +
.../workloadgenerator/audit/package-info.java | 27 +
.../workloadgenerator/package-info.java | 28 +
.../workloadgenerator/TestWorkloadGenerator.java | 164 +
.../audit/TestAuditLogDirectParser.java | 152 +
.../src/test/resources/audit_trace_direct/audit0 | 6 +
.../src/test/resources/audit_trace_hive/audit0 | 6 +
hadoop-tools/hadoop-dynamometer/pom.xml | 55 +
.../src/site/markdown/Dynamometer.md | 302 +
.../src/site/resources/css/site.css | 0
.../images/dynamometer-architecture-infra.png | Bin 0 -> 123874 bytes
.../images/dynamometer-architecture-replay.png | Bin 0 -> 159507 bytes
hadoop-tools/hadoop-extras/pom.xml | 2 +-
hadoop-tools/hadoop-fs2img/pom.xml | 1 -
.../hadoop/hdfs/server/namenode/FSTreeWalk.java | 54 +-
.../hadoop/hdfs/server/namenode/ImageWriter.java | 4 +-
.../hdfs/server/namenode/SingleUGIResolver.java | 5 +-
.../hadoop/hdfs/server/namenode/TreePath.java | 47 +-
.../hadoop/hdfs/server/namenode/TreeWalk.java | 2 +-
.../hadoop/hdfs/server/namenode/UGIResolver.java | 81 +-
.../namenode/ITestProvidedImplementation.java | 2 +-
.../hdfs/server/namenode/RandomTreeWalk.java | 6 +-
.../hdfs/server/namenode/TestFSTreeWalk.java | 120 +
.../server/namenode/TestSingleUGIResolver.java | 32 +
hadoop-tools/hadoop-gridmix/pom.xml | 2 +-
hadoop-tools/hadoop-kafka/pom.xml | 7 +-
.../hadoop/metrics2/impl/TestKafkaMetrics.java | 4 +-
hadoop-tools/hadoop-openstack/pom.xml | 2 +-
.../hadoop/fs/swift/http/ExceptionDiags.java | 2 -
.../fs/swift/snative/SwiftNativeFileSystem.java | 12 +-
hadoop-tools/hadoop-pipes/pom.xml | 2 +-
hadoop-tools/hadoop-resourceestimator/pom.xml | 2 +-
.../src/config/checkstyle.xml | 4 +-
hadoop-tools/hadoop-rumen/pom.xml | 2 +-
hadoop-tools/hadoop-sls/pom.xml | 4 +-
.../src/main/data/2jobs2min-rumen-jh.json | 28 -
.../src/main/html/js/thirdparty/d3-LICENSE | 26 -
.../src/main/html/showSimulationTrace.html | 507 +-
.../src/main/html/simulate.html.template | 16 +-
.../java/org/apache/hadoop/yarn/sls/SLSRunner.java | 64 +-
.../hadoop/yarn/sls/appmaster/AMSimulator.java | 51 +-
.../hadoop/yarn/sls/appmaster/DAGAMSimulator.java | 288 +
.../hadoop/yarn/sls/conf/SLSConfiguration.java | 7 +
.../hadoop/yarn/sls/nodemanager/NMSimulator.java | 2 +-
.../hadoop/yarn/sls/nodemanager/NodeInfo.java | 9 +
.../yarn/sls/scheduler/ContainerSimulator.java | 28 +-
.../hadoop/yarn/sls/scheduler/RMNodeWrapper.java | 9 +
.../org/apache/hadoop/yarn/sls/utils/SLSUtils.java | 33 +
.../apache/hadoop/yarn/sls/BaseSLSRunnerTest.java | 4 +-
.../apache/hadoop/yarn/sls/TestDagAMSimulator.java | 80 +
.../hadoop/yarn/sls/TestSLSDagAMSimulator.java | 76 +
.../org/apache/hadoop/yarn/sls/TestSLSRunner.java | 39 +
.../hadoop/yarn/sls/appmaster/TestAMSimulator.java | 100 +
.../apache/hadoop/yarn/sls/utils/TestSLSUtils.java | 32 +-
.../hadoop-sls/src/test/resources/inputsls.json | 9 +-
.../hadoop-sls/src/test/resources/sls-runner.xml | 4 +
.../hadoop-sls/src/test/resources/sls_dag.json | 62 +
hadoop-tools/hadoop-streaming/pom.xml | 2 +-
hadoop-tools/hadoop-tools-dist/pom.xml | 2 +-
hadoop-tools/pom.xml | 3 +-
.../MySQL/FederationStateStoreTables.sql | 1 +
.../SQLServer/FederationStateStoreTables.sql | 4 +
hadoop-yarn-project/hadoop-yarn/bin/yarn | 4 +
.../hadoop-yarn/conf/container-executor.cfg | 11 +-
hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh | 6 +
.../hadoop-yarn/dev-support/findbugs-exclude.xml | 41 +
.../jdiff/Apache_Hadoop_YARN_API_2.10.0.xml | 22317 ++++++++++
.../jdiff/Apache_Hadoop_YARN_Client_2.10.0.xml | 2832 ++
.../jdiff/Apache_Hadoop_YARN_Common_2.10.0.xml | 2936 ++
.../Apache_Hadoop_YARN_Server_Common_2.10.0.xml | 1422 +
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 75 +-
.../yarn/api/records/ApplicationAttemptId.java | 4 +-
.../hadoop/yarn/api/records/ApplicationId.java | 6 +-
.../hadoop/yarn/api/records/ContainerId.java | 4 +-
.../apache/hadoop/yarn/api/records/NodeLabel.java | 10 +-
.../hadoop/yarn/api/records/ReservationId.java | 6 +-
.../apache/hadoop/yarn/api/records/Resource.java | 8 +-
.../yarn/api/records/ResourceInformation.java | 6 +-
.../hadoop/yarn/api/records/ResourceOption.java | 8 +-
.../records/timeline/TimelineEntityGroupId.java | 6 +-
.../yarn/api/records/timeline/TimelineHealth.java | 82 +
.../timelineservice/FlowActivityEntity.java | 14 +-
.../api/records/timelineservice/FlowRunEntity.java | 10 +-
.../java/org/apache/hadoop/yarn/conf/HAUtil.java | 4 +-
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 421 +-
.../util/constraint/PlacementConstraintParser.java | 74 +-
.../hadoop/yarn/util/resource/ResourceUtils.java | 67 +-
.../src/main/proto/YarnCsiAdaptor.proto | 1 +
.../main/proto/application_history_client.proto | 1 +
.../main/proto/applicationclient_protocol.proto | 1 +
.../main/proto/applicationmaster_protocol.proto | 1 +
.../src/main/proto/client_SCM_protocol.proto | 1 +
.../main/proto/containermanagement_protocol.proto | 1 +
.../src/main/proto/server/SCM_Admin_protocol.proto | 1 +
.../proto/server/application_history_server.proto | 1 +
.../resourcemanager_administration_protocol.proto | 1 +
...arn_server_resourcemanager_service_protos.proto | 1 +
.../src/main/proto/yarn_csi_adaptor.proto | 4 +-
.../src/main/proto/yarn_protos.proto | 1 +
.../src/main/proto/yarn_service_protos.proto | 1 +
.../yarn/api/records/timeline/jaxb.properties | 13 +
.../hadoop/yarn/api/records/TestResource.java | 22 +-
.../apache/hadoop/yarn/api/records/TestURL.java | 8 +-
.../timelineservice/TestApplicationEntity.java | 12 +-
.../timelineservice/TestTimelineMetric.java | 10 +-
.../resource/TestPlacementConstraintParser.java | 329 +-
.../api/resource/TestPlacementConstraints.java | 55 +-
.../hadoop/yarn/conf/TestResourceInformation.java | 47 +-
.../hadoop/yarn/util/TestUnitsConversionUtil.java | 143 +-
.../hadoop-yarn-applications-catalog/README.md | 50 +
.../Dockerfile | 37 +
.../README.md | 43 +
.../pom.xml | 107 +
.../src/main/resources/jaas.config | 9 +
.../src/main/resources/samples.xml | 135 +
.../src/main/scripts/entrypoint.sh | 80 +
.../src/main/scripts/setup-image.sh | 26 +
.../.gitignore | 5 +
.../.yarnrc | 1 +
.../README.md | 51 +
.../dev-support/findbugs-exclude.xml | 0
.../package.json | 26 +
.../pom.xml | 516 +
.../yarn/appcatalog/application/AppCatalog.java | 60 +
.../application/AppCatalogInitializer.java | 54 +
.../application/AppCatalogSolrClient.java | 445 +
.../appcatalog/application/YarnServiceClient.java | 196 +
.../yarn/appcatalog/application/package-info.java | 22 +
.../controller/AppDetailsController.java | 299 +
.../appcatalog/controller/AppListController.java | 182 +
.../appcatalog/controller/AppStoreController.java | 198 +
.../yarn/appcatalog/controller/package-info.java | 22 +
.../hadoop/yarn/appcatalog/model/AppDetails.java | 76 +
.../hadoop/yarn/appcatalog/model/AppEntry.java | 72 +
.../yarn/appcatalog/model/AppStoreEntry.java | 106 +
.../hadoop/yarn/appcatalog/model/Application.java | 89 +
.../hadoop/yarn/appcatalog/model/package-info.java | 22 +
.../hadoop/yarn/appcatalog/utils/RandomWord.java | 422 +
.../yarn/appcatalog/utils/WordLengthException.java | 29 +
.../hadoop/yarn/appcatalog/utils/package-info.java | 22 +
.../src/main/javascript/app.js | 62 +
.../src/main/javascript/controllers.js | 374 +
.../src/main/javascript/filters.js | 32 +
.../src/main/javascript/routes.js | 17 +
.../src/main/javascript/services.js | 17 +
.../src/main/resources/appcatalog.properties | 1 +
.../src/main/resources/log4j.properties | 23 +
.../src/main/webapp/.gitignore | 1 +
.../src/main/webapp/WEB-INF/beans.xml | 22 +
.../src/main/webapp/WEB-INF/web.xml | 117 +
.../src/main/webapp/css/bootstrap-hadoop.css | 1526 +
.../src/main/webapp/css/bootstrap-hadoop.min.css | 18 +
.../src/main/webapp/css/img/feather.png | Bin 0 -> 6066 bytes
.../src/main/webapp/css/img/jenkins.png | Bin 0 -> 23140 bytes
.../src/main/webapp/css/img/loading.svg | 8 +
.../pattern-f61c2e99f82389a67432f54155c5f483.png | Bin 0 -> 27786 bytes
.../src/main/webapp/css/specific.css | 290 +
.../src/main/webapp/css/theme.css | 30 +
.../src/main/webapp/index.html | 89 +
.../src/main/webapp/js/bootstrap-hadoop.js | 284 +
.../src/main/webapp/js/bootstrap-hadoop.min.js | 19 +
.../src/main/webapp/partials/deploy.html | 80 +
.../src/main/webapp/partials/details.html | 65 +
.../src/main/webapp/partials/home.html | 52 +
.../src/main/webapp/partials/new.html | 111 +
.../src/main/webapp/partials/upgrade.html | 114 +
.../src/main/webapp/theme.html | 649 +
.../application/EmbeddedSolrServerFactory.java | 104 +
.../application/TestAppCatalogSolrClient.java | 149 +
.../controller/AppDetailsControllerTest.java | 157 +
.../controller/AppListControllerTest.java | 95 +
.../controller/AppStoreControllerTest.java | 97 +
.../src/test/javascript/controllersSpec.js | 249 +
.../src/test/javascript/karma.conf.js | 34 +
.../src/test/resources/configsets.tgz | Bin 0 -> 2996 bytes
.../exampleCollection/conf/lang/stopwords_en.txt | 54 +
.../configsets/exampleCollection/conf/params.json | 20 +
.../exampleCollection/conf/protwords.txt | 20 +
.../configsets/exampleCollection/conf/schema.xml | 116 +
.../exampleCollection/conf/solrconfig.xml | 36 +
.../exampleCollection/conf/stopwords.txt | 0
.../configsets/exampleCollection/conf/synonyms.txt | 28 +
.../src/test/resources/log4j.properties | 11 +
.../hadoop-yarn-applications-catalog/pom.xml | 37 +
.../pom.xml | 6 +-
.../distributedshell/ApplicationMaster.java | 52 +-
.../yarn/applications/distributedshell/Client.java | 98 +-
.../distributedshell/PlacementSpec.java | 8 +-
.../TestDSWithMultipleNodeManager.java | 6 +-
.../distributedshell/TestDistributedShell.java | 130 +-
.../dev-support/findbugs-exclude.xml | 17 +
.../dev-support/findbugs-exclude.xml | 17 +
.../hadoop-yarn-applications-mawo-core/pom.xml | 173 +
.../src/assembly/bin.xml | 41 +
.../mawo/server/common/AbstractTask.java | 246 +
.../mawo/server/common/CompositeTask.java | 42 +
.../applications/mawo/server/common/DieTask.java | 32 +
.../mawo/server/common/MawoConfiguration.java | 502 +
.../applications/mawo/server/common/NullTask.java | 33 +
.../mawo/server/common/SimpleTask.java | 57 +
.../applications/mawo/server/common/Task.java | 83 +
.../applications/mawo/server/common/TaskId.java | 149 +
.../mawo/server/common/TaskStatus.java | 347 +
.../applications/mawo/server/common/TaskType.java | 45 +
.../mawo/server/common/TeardownTask.java | 48 +
.../mawo/server/common/WorkAssignmentProtocol.java | 65 +
.../mawo/server/common/package-info.java | 20 +
.../applications/mawo/server/master/job/JobId.java | 128 +
.../mawo/server/master/job/package-info.java | 20 +
.../applications/mawo/server/worker/WorkerId.java | 162 +
.../mawo/server/worker/package-info.java | 20 +
.../src/main/resources/mawo-default.properties | 41 +
.../mawo/server/common/TestMaWoConfiguration.java | 60 +
.../src/test/resources/mawo.properties | 28 +
.../hadoop-yarn-applications-mawo/pom.xml | 37 +
.../pom.xml | 6 +-
.../hadoop-yarn-services-api/pom.xml | 2 +-
.../yarn/service/client/ApiServiceClient.java | 71 +-
.../service/client/SystemServiceManagerImpl.java | 4 +-
.../yarn/service/client/TestApiServiceClient.java | 11 +-
.../service/client/TestSecureApiServiceClient.java | 2 +-
.../examples/appcatalog/appcatalog.json | 28 +
.../hadoop-yarn-services-core/pom.xml | 41 +-
.../apache/hadoop/yarn/service/ServiceManager.java | 2 +-
.../hadoop/yarn/service/ServiceScheduler.java | 22 +-
.../hadoop/yarn/service/api/records/Artifact.java | 11 +-
.../yarn/service/api/records/BaseResource.java | 6 +-
.../hadoop/yarn/service/api/records/Component.java | 52 +-
.../yarn/service/api/records/ConfigFile.java | 42 +-
.../yarn/service/api/records/Configuration.java | 12 +-
.../hadoop/yarn/service/api/records/Container.java | 34 +-
.../hadoop/yarn/service/api/records/Error.java | 10 +-
.../service/api/records/KerberosPrincipal.java | 10 +-
.../service/api/records/PlacementConstraint.java | 32 +-
.../yarn/service/api/records/PlacementPolicy.java | 8 +-
.../yarn/service/api/records/ReadinessCheck.java | 14 +-
.../hadoop/yarn/service/api/records/Resource.java | 16 +-
.../service/api/records/ResourceInformation.java | 14 +-
.../hadoop/yarn/service/api/records/Service.java | 61 +-
.../yarn/service/api/records/ServiceStatus.java | 12 +-
.../hadoop/yarn/service/client/ServiceClient.java | 56 +-
.../hadoop/yarn/service/component/Component.java | 2 +-
.../component/instance/ComponentInstance.java | 34 +-
.../hadoop/yarn/service/conf/YarnServiceConf.java | 2 +
.../yarn/service/conf/YarnServiceConstants.java | 2 +
.../service/containerlaunch/AbstractLauncher.java | 6 +-
.../service/exceptions/RestApiErrorMessages.java | 1 +
.../yarn/service/provider/ProviderUtils.java | 63 +-
.../provider/docker/DockerProviderService.java | 6 +
.../provider/tarball/TarballProviderService.java | 4 +-
.../timelineservice/ServiceMetricsSink.java | 8 +-
.../service/utils/ApplicationReportSerDeser.java | 12 +-
.../hadoop/yarn/service/utils/CoreFileSystem.java | 21 +-
.../hadoop/yarn/service/utils/JsonSerDeser.java | 48 +-
.../yarn/service/utils/PublishedConfiguration.java | 18 +-
.../service/utils/SerializedApplicationReport.java | 8 +-
.../hadoop/yarn/service/utils/ServiceApiUtil.java | 22 +-
.../hadoop/yarn/service/utils/ServiceUtils.java | 11 +-
.../yarn/service/utils/SliderFileSystem.java | 34 +
.../hadoop/yarn/service/utils/ZookeeperUtils.java | 2 +-
.../src/main/proto/ClientAMProtocol.proto | 1 +
.../hadoop/yarn/service/ServiceTestUtils.java | 17 +-
.../yarn/service/TestYarnNativeServices.java | 54 +-
.../hadoop/yarn/service/client/TestServiceCLI.java | 13 +-
.../yarn/service/client/TestServiceClient.java | 23 +
.../provider/TestAbstractProviderService.java | 27 +-
.../yarn/service/provider/TestProviderUtils.java | 128 +-
.../yarn/service/utils/TestServiceApiUtil.java | 9 +-
.../hadoop-yarn-services/pom.xml | 2 +-
.../hadoop-yarn-submarine/pom.xml | 213 -
.../base/ubuntu-16.04/Dockerfile.cpu.tf_1.8.0 | 69 -
.../base/ubuntu-16.04/Dockerfile.gpu.tf_1.8.0 | 67 -
.../src/main/docker/build-all.sh | 32 -
.../ubuntu-16.04/Dockerfile.cpu.tf_1.8.0 | 22 -
.../ubuntu-16.04/Dockerfile.gpu.tf_1.8.0 | 22 -
.../cifar10_estimator_tf_1.8.0/cifar10.py | 113 -
.../cifar10_estimator_tf_1.8.0/cifar10_main.py | 521 -
.../cifar10_estimator_tf_1.8.0/cifar10_utils.py | 154 -
.../generate_cifar10_tfrecords.py | 114 -
.../hadoop/yarn/submarine/client/cli/Cli.java | 103 -
.../yarn/submarine/client/cli/CliConstants.java | 59 -
.../hadoop/yarn/submarine/client/cli/CliUtils.java | 124 -
.../yarn/submarine/client/cli/RunJobCli.java | 277 -
.../yarn/submarine/client/cli/ShowJobCli.java | 125 -
.../submarine/client/cli/param/BaseParameters.java | 56 -
.../client/cli/param/RunJobParameters.java | 334 -
.../submarine/client/cli/param/RunParameters.java | 103 -
.../submarine/common/api/JobComponentStatus.java | 73 -
.../hadoop/yarn/submarine/common/api/TaskType.java | 32 -
.../api/builder/JobComponentStatusBuilder.java | 44 -
.../common/api/builder/JobStatusBuilder.java | 64 -
.../yarn/submarine/runtimes/RuntimeFactory.java | 106 -
.../submarine/runtimes/common/JobSubmitter.java | 36 -
.../yarnservice/YarnServiceJobMonitor.java | 58 -
.../yarnservice/YarnServiceJobSubmitter.java | 908 -
.../runtimes/yarnservice/YarnServiceUtils.java | 133 -
.../src/site/markdown/DeveloperGuide.md | 24 -
.../src/site/markdown/Examples.md | 21 -
.../src/site/markdown/HowToInstall.md | 36 -
.../src/site/markdown/Index.md | 44 -
.../src/site/markdown/InstallationGuide.md | 615 -
.../markdown/InstallationGuideChineseVersion.md | 757 -
.../src/site/markdown/QuickStart.md | 218 -
.../markdown/RunningDistributedCifar10TFJobs.md | 162 -
.../src/site/markdown/RunningZeppelinOnYARN.md | 37 -
.../src/site/markdown/TestAndTroubleshooting.md | 165 -
.../src/site/markdown/WriteDockerfile.md | 117 -
.../submarine/client/cli/TestRunJobCliParsing.java | 201 -
.../cli/yarnservice/TestYarnServiceRunJobCli.java | 1220 -
.../yarn/submarine/common/MockClientContext.java | 61 -
.../common/fs/MockRemoteDirectoryManager.java | 168 -
.../yarnservice/TestTFConfigGenerator.java | 42 -
.../hadoop-yarn/hadoop-yarn-applications/pom.xml | 5 +-
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 50 +-
.../apache/hadoop/yarn/client/api/AMRMClient.java | 18 +-
.../yarn/client/api/async/AMRMClientAsync.java | 5 +-
.../yarn/client/api/impl/AMRMClientImpl.java | 7 +-
.../api/impl/ContainerManagementProtocolProxy.java | 26 +-
.../hadoop/yarn/client/api/impl/NMClientImpl.java | 4 +-
.../yarn/client/api/impl/RemoteRequestsTable.java | 49 +-
.../client/api/impl/SharedCacheClientImpl.java | 4 +-
.../yarn/client/api/impl/YarnClientImpl.java | 6 +-
.../hadoop/yarn/client/cli/ApplicationCLI.java | 1312 +-
.../org/apache/hadoop/yarn/client/cli/LogsCLI.java | 44 +-
.../apache/hadoop/yarn/client/cli/RMAdminCLI.java | 8 +-
.../hadoop/yarn/client/cli/SchedConfCLI.java | 157 +-
.../org/apache/hadoop/yarn/client/cli/TopCLI.java | 78 +-
.../apache/hadoop/yarn/client/TestRMFailover.java | 7 +-
.../client/api/async/impl/TestAMRMClientAsync.java | 3 +-
.../hadoop/yarn/client/api/impl/TestAHSClient.java | 3 +-
.../yarn/client/api/impl/TestAHSv2ClientImpl.java | 34 +-
.../yarn/client/api/impl/TestAMRMClient.java | 3 +-
.../api/impl/TestAMRMClientContainerRequest.java | 4 +-
.../TestOpportunisticContainerAllocationE2E.java | 3 +-
.../yarn/client/api/impl/TestYarnClient.java | 39 +-
.../api/impl/TestYarnClientWithReservation.java | 10 +-
.../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 173 +-
.../hadoop/yarn/client/cli/TestSchedConfCLI.java | 187 +-
.../apache/hadoop/yarn/client/cli/TestYarnCLI.java | 18 +-
.../src/test/proto/test_amrm_token.proto | 1 +
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 54 +-
.../yarn/FileSystemBasedConfigurationProvider.java | 22 +-
.../hadoop/yarn/YarnUncaughtExceptionHandler.java | 14 +-
.../pb/PlacementConstraintToProtoConverter.java | 22 +-
.../impl/pb/NodePublishVolumeRequestPBImpl.java | 4 +-
.../ValidateVolumeCapabilitiesRequestPBImpl.java | 4 +-
.../pb/ApplicationSubmissionContextPBImpl.java | 6 +-
.../yarn/api/records/impl/pb/ContainerPBImpl.java | 24 +-
.../api/records/impl/pb/ContainerStatusPBImpl.java | 26 +-
.../records/impl/pb/LocalizationStatusPBImpl.java | 12 +-
.../yarn/api/records/impl/pb/ProtoUtils.java | 2 +-
.../yarn/api/records/impl/pb/ResourcePBImpl.java | 13 +-
.../records/impl/pb/SchedulingRequestPBImpl.java | 23 +-
.../yarn/api/records/impl/pb/TokenPBImpl.java | 6 +-
.../org/apache/hadoop/yarn/client/AHSProxy.java | 7 +-
.../apache/hadoop/yarn/client/ClientRMProxy.java | 7 +-
.../client/ConfiguredRMFailoverProxyProvider.java | 8 +-
.../org/apache/hadoop/yarn/client/RMProxy.java | 7 +-
.../RequestHedgingRMFailoverProxyProvider.java | 8 +-
.../yarn/client/api/impl/DirectTimelineWriter.java | 8 +-
.../client/api/impl/FileSystemTimelineWriter.java | 98 +-
.../yarn/client/api/impl/TimelineClientImpl.java | 7 +-
.../yarn/client/api/impl/TimelineConnector.java | 7 +-
.../client/api/impl/TimelineReaderClientImpl.java | 8 +-
.../yarn/client/api/impl/TimelineV2ClientImpl.java | 13 +-
.../yarn/client/api/impl/TimelineWriter.java | 23 +-
.../apache/hadoop/yarn/event/AsyncDispatcher.java | 58 +-
.../apache/hadoop/yarn/event/EventDispatcher.java | 13 +-
.../factories/impl/pb/RpcClientFactoryPBImpl.java | 8 +-
.../factories/impl/pb/RpcServerFactoryPBImpl.java | 7 +-
.../apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java | 13 +-
.../java/org/apache/hadoop/yarn/ipc/YarnRPC.java | 9 +-
.../AggregatedLogDeletionService.java | 33 +-
.../yarn/logaggregation/LogAggregationUtils.java | 241 +-
.../hadoop/yarn/logaggregation/LogCLIHelpers.java | 134 +-
.../hadoop/yarn/logaggregation/LogToolUtils.java | 60 +-
.../LogAggregationFileController.java | 123 +-
.../LogAggregationFileControllerFactory.java | 43 +-
.../ifile/IndexedFileAggregatedLogsBlock.java | 64 +-
.../ifile/LogAggregationIndexedFileController.java | 33 +-
.../tfile/LogAggregationTFileController.java | 22 +-
.../tfile/TFileAggregatedLogsBlock.java | 9 +-
.../yarn/nodelabels/CommonNodeLabelsManager.java | 27 +-
.../yarn/nodelabels/FileSystemNodeLabelsStore.java | 8 +-
.../nodelabels/NonAppendableFSNodeLabelStore.java | 18 +-
.../yarn/nodelabels/store/AbstractFSNodeStore.java | 7 +-
.../hadoop/yarn/security/AMRMTokenIdentifier.java | 7 +-
.../hadoop/yarn/security/AMRMTokenSelector.java | 14 +-
.../hadoop/yarn/security/AdminACLsManager.java | 7 +-
.../yarn/security/ConfiguredYarnAuthorizer.java | 4 +-
.../yarn/security/ContainerTokenIdentifier.java | 9 +-
.../yarn/security/ContainerTokenSelector.java | 14 +-
.../hadoop/yarn/security/NMTokenIdentifier.java | 9 +-
.../hadoop/yarn/security/NMTokenSelector.java | 14 +-
.../yarn/security/YarnAuthorizationProvider.java | 9 +-
.../security/client/ClientToAMTokenSelector.java | 14 +-
.../security/client/RMDelegationTokenSelector.java | 14 +-
.../client/TimelineDelegationTokenSelector.java | 18 +-
.../server/metrics/ContainerMetricsConstants.java | 9 +
.../server/security/ApplicationACLsManager.java | 34 +-
.../java/org/apache/hadoop/yarn/state/Graph.java | 15 +-
.../yarn/util/AbstractLivelinessMonitor.java | 9 +-
.../apache/hadoop/yarn/util/AdHocLogDumper.java | 123 +-
.../yarn/util/DockerClientConfigHandler.java | 6 +-
.../org/apache/hadoop/yarn/util/FSDownload.java | 25 +-
.../hadoop/yarn/util/ProcfsBasedProcessTree.java | 43 +-
.../org/apache/hadoop/yarn/util/RackResolver.java | 74 +-
.../hadoop/yarn/util/ResourceCalculatorPlugin.java | 8 +-
.../yarn/util/ResourceCalculatorProcessTree.java | 8 +-
.../java/org/apache/hadoop/yarn/util/Times.java | 7 +-
.../hadoop/yarn/util/WindowsBasedProcessTree.java | 14 +-
.../apache/hadoop/yarn/util/YarnVersionInfo.java | 9 +-
.../util/resource/DefaultResourceCalculator.java | 24 +-
.../util/resource/DominantResourceCalculator.java | 47 +-
.../yarn/util/resource/ResourceCalculator.java | 13 +
.../hadoop/yarn/util/resource/Resources.java | 76 +-
.../hadoop/yarn/util/timeline/TimelineUtils.java | 32 +
.../org/apache/hadoop/yarn/webapp/Dispatcher.java | 9 +
.../yarn/webapp/GenericExceptionHandler.java | 10 +-
.../yarn/webapp/log/AggregatedLogsBlock.java | 14 +-
.../hadoop/yarn/webapp/util/WebAppUtils.java | 47 +-
.../yarn/webapp/util/YarnWebServiceUtils.java | 29 +-
.../apache/hadoop/yarn/webapp/view/JQueryUI.java | 2 +-
.../src/main/proto/yarn_security_token.proto | 1 +
.../webapps/static/jquery/jquery-3.3.1.min.js | 2 -
.../webapps/static/jquery/jquery-3.4.1.min.js | 2 +
.../src/main/resources/yarn-default.xml | 398 +-
.../apache/hadoop/yarn/TestContainerLaunchRPC.java | 9 +-
.../yarn/TestContainerResourceIncreaseRPC.java | 8 +-
.../hadoop/yarn/api/BasePBImplRecordsTest.java | 7 +-
.../apache/hadoop/yarn/api/TestResourcePBImpl.java | 3 +-
.../api/records/timeline/TestTimelineRecords.java | 8 +-
.../TestTimelineServiceRecords.java | 8 +-
.../api/impl/TestTimelineClientForATS1_5.java | 8 +-
.../client/api/impl/TestTimelineClientV2Impl.java | 8 +-
.../hadoop/yarn/conf/TestYarnConfiguration.java | 6 +
.../apache/hadoop/yarn/event/InlineDispatcher.java | 7 +-
.../hadoop/yarn/event/TestAsyncDispatcher.java | 57 +
.../TestAggregatedLogDeletionService.java | 182 +-
.../logaggregation/TestAggregatedLogFormat.java | 50 +-
.../logaggregation/TestAggregatedLogsBlock.java | 8 +-
.../logaggregation/TestContainerLogsUtils.java | 11 +-
.../TestLogAggregationFileController.java | 91 +
.../TestLogAggregationFileControllerFactory.java | 261 +-
.../TestLogAggregationIndexFileController.java | 444 -
.../TestLogAggregationIndexedFileController.java | 472 +
.../hadoop/yarn/util/TestAdHocLogDumper.java | 31 +-
.../java/org/apache/hadoop/yarn/util/TestApps.java | 3 +-
.../hadoop/yarn/util/TestConverterUtils.java | 9 +-
.../apache/hadoop/yarn/util/TestFSDownload.java | 7 +-
.../util/TestLog4jWarningErrorMetricsAppender.java | 32 +-
.../yarn/util/TestProcfsBasedProcessTree.java | 10 +-
.../apache/hadoop/yarn/util/TestRackResolver.java | 67 +-
.../yarn/util/TestTimelineServiceHelper.java | 4 +-
.../yarn/util/TestWindowsBasedProcessTree.java | 8 +-
.../yarn/util/resource/TestResourceCalculator.java | 189 +-
.../yarn/util/resource/TestResourceUtils.java | 236 +-
.../hadoop/yarn/util/resource/TestResources.java | 72 +-
.../yarn/util/timeline/TestShortenedFlowName.java | 52 +
.../resource-types-error-redefine-fpga-unit.xml | 45 +
.../resource-types-error-redefine-gpu-unit.xml | 45 +
.../hadoop-yarn/hadoop-yarn-csi/pom.xml | 13 +-
.../yarn/csi/adaptor/DefaultCsiAdaptorImpl.java | 20 +-
.../hadoop/yarn/csi/client/CsiGrpcClient.java | 7 +-
.../yarn/csi/adaptor/TestCsiAdaptorService.java | 13 +-
.../hadoop/yarn/csi/client/FakeCsiDriver.java | 5 +-
.../hadoop-yarn/hadoop-yarn-registry/pom.xml | 2 +-
.../pom.xml | 48 +-
.../ApplicationHistoryManagerImpl.java | 10 +
.../ApplicationHistoryManagerOnTimelineStore.java | 7 +-
.../webapp/AHSErrorsAndWarningsPage.java | 11 +-
.../webapp/AHSWebServices.java | 8 +-
.../webapp/ContextFactory.java | 133 +
.../yarn/server/timeline/LeveldbTimelineStore.java | 32 +-
.../timeline/RollingLevelDBTimelineStore.java | 101 +-
.../timeline/security/TimelineACLsManager.java | 9 +-
...elineV1DelegationTokenSecretManagerService.java | 20 +-
.../yarn_server_timelineserver_recovery.proto | 1 +
.../ApplicationHistoryStoreTestUtils.java | 6 +
.../TestApplicationHistoryManagerImpl.java | 21 +
.../webapp/TestAHSWebServices.java | 24 +-
.../timeline/webapp/TestTimelineWebServices.java | 21 +
.../hadoop-yarn-server-common/pom.xml | 52 +-
.../yarn/server/AMHeartbeatRequestHandler.java | 20 +-
.../AbstractConfigurableFederationPolicy.java | 2 +-
.../LocalityMulticastAMRMProxyPolicy.java | 13 +-
.../store/impl/SQLFederationStateStore.java | 18 +-
.../store/impl/ZookeeperFederationStateStore.java | 4 +
.../records/SubClusterPolicyConfiguration.java | 6 +-
.../utils/FederationStateStoreFacade.java | 4 +-
.../metrics/OpportunisticSchedulerMetrics.java | 149 +
...DistributedOpportunisticContainerAllocator.java | 358 +
.../scheduler/OpportunisticContainerAllocator.java | 348 +-
.../scheduler/OpportunisticContainerContext.java | 10 +
.../server/scheduler/ResourceRequestSetKey.java | 2 +-
.../security/BaseContainerTokenSecretManager.java | 8 +-
.../yarn/server/sharedcache/SharedCacheUtil.java | 4 +-
.../server/uam/UnmanagedApplicationManager.java | 8 +-
.../server/util/timeline/TimelineServerUtils.java | 17 +-
.../yarn/server/utils/YarnServerSecurityUtils.java | 2 +-
.../hadoop/yarn/server/webapp/AppAttemptBlock.java | 3 +-
.../apache/hadoop/yarn/server/webapp/AppBlock.java | 4 +-
.../hadoop/yarn/server/webapp/ContainerBlock.java | 66 +-
.../hadoop/yarn/server/webapp/LogWebService.java | 8 +-
.../hadoop/yarn/server/webapp/WebPageUtils.java | 24 +-
.../hadoop/yarn/server/webapp/dao/AppInfo.java | 16 +
.../yarn/server/webapp/dao/ContainerInfo.java | 4 +
.../src/main/proto/ResourceTracker.proto | 1 +
.../src/main/proto/SCMUploader.proto | 1 +
.../main/proto/collectornodemanager_protocol.proto | 1 +
.../proto/distributed_scheduling_am_protocol.proto | 2 +-
.../src/main/proto/yarn_server_common_protos.proto | 1 +
.../proto/yarn_server_common_service_protos.proto | 1 +
.../main/proto/yarn_server_federation_protos.proto | 1 +
.../org/apache/hadoop/yarn/lib/TestZKClient.java | 17 +-
.../api/protocolrecords/TestProtocolRecords.java | 3 +-
.../policies/manager/BasePolicyManagerTest.java | 9 +-
...DistributedOpportunisticContainerAllocator.java | 852 +
.../TestOpportunisticContainerAllocator.java | 636 -
.../yarn/server/webapp/ContainerBlockTest.java | 93 +
.../hadoop-yarn-server-nodemanager/pom.xml | 51 +-
.../src/CMakeLists.txt | 10 +-
.../yarn/server/nodemanager/ContainerExecutor.java | 44 +-
.../nodemanager/DefaultContainerExecutor.java | 82 +-
.../yarn/server/nodemanager/DeletionService.java | 7 +-
.../server/nodemanager/DirectoryCollection.java | 117 +-
.../server/nodemanager/LinuxContainerExecutor.java | 95 +-
.../nodemanager/LocalDirsHandlerService.java | 23 +-
.../yarn/server/nodemanager/NodeManager.java | 50 +-
.../server/nodemanager/NodeStatusUpdaterImpl.java | 75 +-
.../yarn/server/nodemanager/ResourceView.java | 8 +
.../WindowsSecureContainerExecutor.java | 32 +-
.../nodemanager/amrmproxy/AMRMProxyService.java | 10 +-
.../amrmproxy/AMRMProxyTokenSecretManager.java | 5 +-
.../amrmproxy/DefaultRequestInterceptor.java | 10 +-
.../amrmproxy/FederationInterceptor.java | 21 +-
.../nodemanager/api/deviceplugin/Device.java | 4 +-
.../api/deviceplugin/DevicePluginScheduler.java | 10 +-
.../nodemanager/api/impl/pb/NMProtoUtils.java | 12 +-
.../nodemanager/containermanager/AuxServices.java | 9 +-
.../containermanager/ContainerManager.java | 4 +
.../containermanager/ContainerManagerImpl.java | 69 +-
.../application/ApplicationImpl.java | 9 +-
.../containermanager/container/Container.java | 6 +
.../containermanager/container/ContainerImpl.java | 68 +-
.../container/ResourceMappings.java | 32 +-
.../deletion/task/DockerContainerDeletionTask.java | 5 +-
.../deletion/task/FileDeletionTask.java | 18 +-
.../launcher/AbstractContainersLauncher.java | 41 +
.../launcher/ContainerCleanup.java | 50 +-
.../containermanager/launcher/ContainerLaunch.java | 36 +-
.../launcher/ContainerRelaunch.java | 9 +-
.../launcher/ContainersLauncher.java | 35 +-
.../launcher/RecoverPausedContainerLaunch.java | 6 +-
.../containermanager/launcher/package-info.java | 22 +
.../linux/privileged/PrivilegedOperation.java | 5 +-
.../resources/CGroupElasticMemoryController.java | 8 +-
.../linux/resources/CGroupsHandlerImpl.java | 55 +-
.../linux/resources/CGroupsMountConfig.java | 72 +
.../linux/resources/CGroupsResourceCalculator.java | 18 +-
.../resources/CombinedResourceCalculator.java | 8 +-
.../linux/resources/DefaultOOMHandler.java | 8 +-
.../resources/NetworkPacketTaggingHandlerImpl.java | 4 +-
.../resources/NetworkTagMappingManagerFactory.java | 6 +-
.../linux/resources/ResourceHandlerModule.java | 16 +-
.../linux/resources/ResourcesExceptionUtil.java | 42 +
.../TrafficControlBandwidthHandlerImpl.java | 10 +-
.../linux/resources/TrafficController.java | 20 +-
.../resources/fpga/FpgaResourceAllocator.java | 253 +-
.../resources/fpga/FpgaResourceHandlerImpl.java | 113 +-
.../linux/resources/gpu/GpuResourceAllocator.java | 117 +-
.../resources/gpu/GpuResourceHandlerImpl.java | 31 +-
.../linux/resources/numa/NumaNodeResource.java | 7 +-
.../resources/numa/NumaResourceAllocation.java | 59 +-
.../resources/numa/NumaResourceAllocator.java | 41 +-
.../resources/numa/NumaResourceHandlerImpl.java | 8 +-
.../runtime/DelegatingLinuxContainerRuntime.java | 71 +-
.../linux/runtime/DockerLinuxContainerRuntime.java | 432 +-
.../linux/runtime/LinuxContainerRuntime.java | 14 +-
.../runtime/LinuxContainerRuntimeConstants.java | 3 +-
.../linux/runtime/OCIContainerRuntime.java | 376 +
.../linux/runtime/RuncContainerRuntime.java | 906 +
.../runtime/docker/DockerCommandExecutor.java | 19 +-
.../linux/runtime/docker/DockerImagesCommand.java | 38 +
.../linux/runtime/docker/DockerRunCommand.java | 6 +
.../runc/HdfsManifestToResourcesPlugin.java | 179 +
.../linux/runtime/runc/ImageManifest.java | 134 +
.../runtime/runc/ImageTagToManifestPlugin.java | 331 +
.../runtime/runc/RuncContainerExecutorConfig.java | 1507 +
.../runtime/runc/RuncImageTagToManifestPlugin.java | 37 +
.../runc/RuncManifestToResourcesPlugin.java | 42 +
.../linux/runtime/runc/package-info.java | 26 +
.../localizer/ContainerLocalizer.java | 11 +-
.../localizer/LocalizedResource.java | 19 +-
.../localizer/ResourceLocalizationService.java | 199 +-
.../localizer/security/LocalizerTokenSelector.java | 4 +-
.../logaggregation/AppLogAggregatorImpl.java | 68 +-
.../logaggregation/LogAggregationService.java | 79 +-
.../loghandler/NonAggregatingLogHandler.java | 6 +-
.../monitor/ContainersMonitor.java | 6 +
.../monitor/ContainersMonitorImpl.java | 180 +-
.../resourceplugin/NodeResourceUpdaterPlugin.java | 16 +-
.../resourceplugin/ResourcePluginManager.java | 98 +-
.../resourceplugin/com/nec/NECVEPlugin.java | 344 +
.../resourceplugin/com/nec/UdevUtil.java | 100 +
.../resourceplugin/com/nec/VEDeviceDiscoverer.java | 143 +
.../resourceplugin/com/nec/package-info.java | 19 +
.../com/nvidia/NvidiaGPUPluginForRuntimeV2.java | 722 +
.../resourceplugin/com/nvidia/package-info.java | 19 +
.../deviceframework/DeviceMappingManager.java | 70 +-
.../deviceframework/DevicePluginAdapter.java | 32 +-
.../DeviceResourceDockerRuntimePluginImpl.java | 218 +
.../deviceframework/DeviceResourceHandlerImpl.java | 219 +-
.../deviceframework/DeviceResourceUpdaterImpl.java | 7 +-
.../deviceframework/ShellWrapper.java | 46 +
.../fpga/AbstractFpgaVendorPlugin.java | 33 +-
.../fpga/AoclDiagnosticOutputParser.java | 165 +
.../resourceplugin/fpga/FpgaDevice.java | 142 +
.../resourceplugin/fpga/FpgaDiscoverer.java | 175 +-
.../fpga/FpgaNodeResourceUpdateHandler.java | 23 +-
.../resourceplugin/fpga/FpgaResourcePlugin.java | 19 +-
.../resourceplugin/fpga/IntelFpgaOpenclPlugin.java | 323 +-
.../AoclOutputBasedDiscoveryStrategy.java | 52 +
.../fpga/discovery/DeviceSpecParser.java | 82 +
.../fpga/discovery/FPGADiscoveryStrategy.java | 32 +
.../ScriptBasedFPGADiscoveryStrategy.java | 66 +
.../SettingsBasedFPGADiscoveryStrategy.java | 54 +
.../fpga/discovery/package-info.java | 19 +
.../resourceplugin/fpga/package-info.java | 19 +
.../gpu/GpuDeviceSpecificationException.java | 82 +
.../resourceplugin/gpu/GpuDiscoverer.java | 356 +-
.../gpu/GpuNodeResourceUpdateHandler.java | 25 +-
.../resourceplugin/gpu/GpuResourcePlugin.java | 85 +-
.../resourceplugin/gpu/NvidiaBinaryHelper.java | 63 +
.../gpu/NvidiaDockerV1CommandPlugin.java | 21 +-
.../gpu/NvidiaDockerV2CommandPlugin.java | 7 +-
.../resourceplugin/gpu/package-info.java | 20 +
.../runtime/ContainerRuntimeConstants.java | 4 +
.../AllocationBasedResourceUtilizationTracker.java | 8 +-
.../scheduler/ContainerScheduler.java | 28 +-
.../volume/csi/ContainerVolumePublisher.java | 6 +-
.../nodemanager/metrics/NodeManagerMetrics.java | 13 +
.../recovery/NMLeveldbStateStoreService.java | 82 +-
.../scheduler/DistributedScheduler.java | 4 +-
.../security/NMTokenSecretManagerInNM.java | 16 +-
.../timelineservice/NMTimelinePublisher.java | 154 +-
.../util/CgroupsLCEResourcesHandler.java | 37 +-
.../nodemanager/util/NodeManagerHardwareUtils.java | 8 +-
.../nodemanager/util/ProcessIdFileReader.java | 10 +-
.../nodemanager/webapp/ContainerLogsPage.java | 4 +-
.../server/nodemanager/webapp/NMWebServices.java | 10 +-
.../webapp/dao/gpu/GpuDeviceInformation.java | 18 +-
.../webapp/dao/gpu/GpuDeviceInformationParser.java | 54 +-
.../webapp/dao/gpu/PerGpuDeviceInformation.java | 1 -
.../webapp/dao/gpu/PerGpuMemoryUsage.java | 7 +-
.../container-executor/impl/container-executor.c | 309 +-
.../container-executor/impl/container-executor.h | 6 -
.../src/main/native/container-executor/impl/main.c | 36 +-
.../impl/modules/cgroups/cgroups-operations.c | 2 +-
.../impl/modules/devices/devices-module.c | 280 +
.../impl/modules/devices/devices-module.h | 45 +
.../src/main/native/container-executor/impl/util.c | 143 +-
.../src/main/native/container-executor/impl/util.h | 47 +-
.../container-executor/impl/utils/docker-util.c | 818 +-
.../container-executor/impl/utils/docker-util.h | 43 +-
.../container-executor/impl/utils/mount-utils.c | 356 +
.../container-executor/impl/utils/mount-utils.h | 40 +
.../test/modules/devices/test-devices-module.cc | 298 +
.../test/test-container-executor.c | 23 +
.../test/utils/test_docker_util.cc | 178 +-
.../src/main/proto/LocalizationProtocol.proto | 1 +
.../proto/yarn_server_nodemanager_recovery.proto | 1 +
.../yarn_server_nodemanager_service_protos.proto | 1 +
.../nodemanager/TestDefaultContainerExecutor.java | 39 -
.../nodemanager/TestDirectoryCollection.java | 96 +-
.../nodemanager/TestLinuxContainerExecutor.java | 8 +
.../TestLinuxContainerExecutorWithMocks.java | 46 +-
.../yarn/server/nodemanager/TestNodeManager.java | 28 +
.../server/nodemanager/TestNodeManagerMXBean.java | 8 +-
.../server/nodemanager/TestNodeStatusUpdater.java | 103 +
.../amrmproxy/TestFederationInterceptor.java | 1 +
.../containermanager/BaseContainerManagerTest.java | 1 +
.../containermanager/TestAuxServices.java | 41 +
.../TestContainerManagerRecovery.java | 14 +-
.../containermanager/container/TestContainer.java | 8 +
.../container/TestResourceMappings.java | 118 +
.../launcher/TestContainerCleanup.java | 13 +
.../launcher/TestContainerLaunch.java | 26 +-
.../TestCGroupElasticMemoryController.java | 8 +-
.../linux/resources/TestResourceHandlerModule.java | 6 +-
.../resources/fpga/TestFpgaResourceHandler.java | 460 -
.../fpga/TestFpgaResourceHandlerImpl.java | 611 +
.../resources/gpu/TestGpuResourceAllocator.java | 442 +
.../resources/gpu/TestGpuResourceHandler.java | 481 -
.../resources/gpu/TestGpuResourceHandlerImpl.java | 585 +
.../linux/runtime/TestDockerContainerRuntime.java | 132 +-
.../runtime/TestHdfsManifestToResourcesPlugin.java | 182 +
.../runtime/TestImageTagToManifestPlugin.java | 247 +
.../linux/runtime/TestRuncContainerRuntime.java | 1009 +
.../runtime/docker/TestDockerImagesCommand.java | 62 +
.../localizer/TestContainerLocalizer.java | 29 +
.../localizer/TestResourceLocalizationService.java | 241 +-
.../logaggregation/TestLogAggregationService.java | 197 +-
.../resourceplugin/TestResourcePluginManager.java | 170 +-
.../resourceplugin/com/nec/TestNECVEPlugin.java | 447 +
.../com/nec/TestVEDeviceDiscoverer.java | 283 +
.../nvidia/TestNvidiaGPUPluginForRuntimeV2.java | 849 +
.../deviceframework/FakeTestDevicePlugin1.java | 3 +-
.../deviceframework/TestDeviceMappingManager.java | 53 +-
.../deviceframework/TestDevicePluginAdapter.java | 419 +-
.../resourceplugin/fpga/TestAoclOutputParser.java | 104 +
.../resourceplugin/fpga/TestFpgaDiscoverer.java | 400 +-
.../resourceplugin/gpu/TestGpuDiscoverer.java | 485 +-
.../resourceplugin/gpu/TestGpuResourcePlugin.java | 54 +
.../scheduler/TestContainerSchedulerQueuing.java | 15 +-
.../recovery/TestNMLeveldbStateStoreService.java | 646 +-
.../scheduler/TestDistributedScheduler.java | 4 +-
.../timelineservice/TestNMTimelinePublisher.java | 157 +-
.../server/nodemanager/webapp/MockContainer.java | 11 +
.../nodemanager/webapp/TestNMWebServices.java | 268 +-
.../dao/gpu/TestGpuDeviceInformationParser.java | 157 +-
.../test/resources/nvidia-smi-output-excerpt.xml | 71 +
.../resources/nvidia-smi-output-missing-tags.xml | 28 +
.../resources/nvidia-smi-output-missing-tags2.xml | 61 +
.../test/resources/nvidia-smi-sample-output.xml | 547 +
.../test/resources/nvidia-smi-sample-xml-output | 547 -
.../resources/tensorflow-bench-result-for-GPU.csv | 2963 ++
.../conf/capacity-scheduler.xml | 24 +
.../hadoop-yarn-server-resourcemanager/pom.xml | 83 +-
.../server/resourcemanager/AMSProcessingChain.java | 7 +-
.../ActiveStandbyElectorBasedElectorService.java | 14 +-
.../yarn/server/resourcemanager/AdminService.java | 7 +-
.../resourcemanager/ApplicationMasterService.java | 27 +-
.../server/resourcemanager/ClientRMService.java | 56 +-
.../server/resourcemanager/ClusterMetrics.java | 9 +
.../CuratorBasedElectorService.java | 8 +-
.../DecommissioningNodesWatcher.java | 66 +-
.../resourcemanager/DefaultAMSProcessor.java | 29 +-
.../server/resourcemanager/NodesListManager.java | 62 +-
.../OpportunisticContainerAllocatorAMService.java | 71 +-
.../resourcemanager/RMActiveServiceContext.java | 8 +-
.../yarn/server/resourcemanager/RMAppManager.java | 262 +-
.../yarn/server/resourcemanager/RMAuditLogger.java | 65 +-
.../yarn/server/resourcemanager/RMContextImpl.java | 7 +-
.../RMCriticalThreadUncaughtExceptionHandler.java | 6 +-
.../hadoop/yarn/server/resourcemanager/RMInfo.java | 76 +
.../yarn/server/resourcemanager/RMInfoMXBean.java | 42 +
.../yarn/server/resourcemanager/RMNMInfo.java | 18 +-
.../yarn/server/resourcemanager/RMServerUtils.java | 18 +-
.../server/resourcemanager/ResourceManager.java | 143 +-
.../resourcemanager/ResourceTrackerService.java | 70 +-
.../ahs/RMApplicationHistoryWriter.java | 8 +-
.../resourcemanager/amlauncher/AMLauncher.java | 16 +-
.../amlauncher/ApplicationMasterLauncher.java | 6 +-
.../blacklist/SimpleBlacklistManager.java | 15 +-
.../metrics/TimelineServiceV1Publisher.java | 21 +-
.../metrics/TimelineServiceV2Publisher.java | 29 +-
.../resourcemanager/monitor/SchedulingMonitor.java | 7 +-
.../monitor/SchedulingMonitorManager.java | 6 +-
.../monitor/capacity/FifoCandidatesSelector.java | 16 +-
.../capacity/FifoIntraQueuePreemptionPlugin.java | 20 +-
.../capacity/IntraQueueCandidatesSelector.java | 36 +-
.../capacity/PreemptableResourceCalculator.java | 25 +-
.../ProportionalCapacityPreemptionPolicy.java | 21 +-
.../QueuePriorityContainerCandidateSelector.java | 34 +-
.../ReservedContainerCandidatesSelector.java | 23 +-
.../invariants/MetricsInvariantChecker.java | 6 +-
.../nodelabels/FileSystemNodeAttributeStore.java | 8 +-
.../nodelabels/NodeAttributesManagerImpl.java | 40 +-
.../nodelabels/NodeLabelsUtils.java | 7 +-
.../nodelabels/RMDelegatedNodeLabelsUpdater.java | 14 +-
.../nodelabels/RMNodeLabelsManager.java | 35 +-
.../placement/AppNameMappingPlacementRule.java | 14 +-
.../placement/DefaultPlacementRule.java | 113 +
.../resourcemanager/placement/FSPlacementRule.java | 177 +
.../placement/FairQueuePlacementUtils.java | 100 +
.../placement/PlacementFactory.java | 35 +-
.../placement/PlacementManager.java | 13 +-
.../resourcemanager/placement/PlacementRule.java | 57 +-
.../placement/PrimaryGroupPlacementRule.java | 103 +
.../placement/QueuePlacementRuleUtils.java | 12 +-
.../placement/RejectPlacementRule.java | 65 +
.../SecondaryGroupExistingPlacementRule.java | 102 +
.../placement/SpecifiedPlacementRule.java | 77 +
.../placement/UserGroupMappingPlacementRule.java | 170 +-
.../placement/UserPlacementRule.java | 70 +
.../preprocessor/ContextProcessor.java | 44 +
.../preprocessor/NodeLabelProcessor.java | 33 +
.../preprocessor/QueueProcessor.java | 34 +
.../SubmissionContextPreProcessor.java | 223 +
.../preprocessor/TagAddProcessor.java | 44 +
.../resourcemanager/preprocessor/package-info.java | 28 +
.../recovery/FileSystemRMStateStore.java | 32 +-
.../recovery/LeveldbRMStateStore.java | 81 +-
.../resourcemanager/recovery/RMStateStore.java | 27 +-
.../recovery/RMStateStoreFactory.java | 7 +-
.../recovery/RMStateStoreUtils.java | 7 +-
.../recovery/RMStateUpdateAppEvent.java | 7 +-
.../resourcemanager/recovery/ZKRMStateStore.java | 164 +-
.../impl/pb/ApplicationAttemptStateDataPBImpl.java | 8 +-
.../reservation/AbstractSchedulerPlanFollower.java | 12 +-
.../planning/SimpleCapacityReplanner.java | 8 +-
.../resource/DynamicResourceConfiguration.java | 23 +-
.../resource/ResourceProfilesManagerImpl.java | 8 +-
.../server/resourcemanager/rmapp/RMAppImpl.java | 338 +-
.../resourcemanager/rmapp/RMAppLogAggregation.java | 383 +
.../rmapp/attempt/RMAppAttemptImpl.java | 53 +-
.../rmapp/attempt/RMAppAttemptMetrics.java | 11 +-
.../rmapp/monitor/RMAppLifetimeMonitor.java | 7 +-
.../rmcontainer/RMContainerImpl.java | 65 +-
.../yarn/server/resourcemanager/rmnode/RMNode.java | 11 +
.../server/resourcemanager/rmnode/RMNodeImpl.java | 51 +-
.../scheduler/AbstractResourceUsage.java | 39 +-
.../scheduler/AbstractYarnScheduler.java | 198 +-
.../scheduler/ActiveUsersManager.java | 19 +-
.../resourcemanager/scheduler/Allocation.java | 19 +-
.../scheduler/AppSchedulingInfo.java | 100 +-
.../scheduler/ClusterNodeTracker.java | 25 +-
.../ConfigurationMutationACLPolicyFactory.java | 6 +-
.../FSQueueMetricsForCustomResources.java | 113 +
.../scheduler/MutableConfigurationProvider.java | 10 +
.../resourcemanager/scheduler/QueueMetrics.java | 179 +-
.../scheduler/QueueMetricsCustomResource.java | 76 +
.../scheduler/QueueMetricsForCustomResources.java | 64 +-
.../scheduler/QueueStateManager.java | 7 +-
.../resourcemanager/scheduler/ResourceUsage.java | 4 +-
.../scheduler/SchedulerAppUtils.java | 19 +-
.../scheduler/SchedulerApplicationAttempt.java | 83 +-
.../resourcemanager/scheduler/SchedulerNode.java | 73 +-
.../resourcemanager/scheduler/SchedulerUtils.java | 64 +-
.../scheduler/activities/ActivitiesLogger.java | 225 +-
.../scheduler/activities/ActivitiesManager.java | 439 +-
.../scheduler/activities/ActivitiesUtils.java | 61 +
.../activities/ActivityDiagnosticConstant.java | 119 +-
.../scheduler/activities/ActivityLevel.java | 29 +
.../scheduler/activities/ActivityNode.java | 65 +-
.../scheduler/activities/AllocationActivity.java | 49 +-
.../scheduler/activities/AppAllocation.java | 71 +-
.../scheduler/activities/DiagnosticsCollector.java | 44 +
.../activities/GenericDiagnosticsCollector.java | 85 +
.../scheduler/activities/NodeAllocation.java | 35 +-
.../capacity/AbstractAutoCreatedLeafQueue.java | 6 +-
.../scheduler/capacity/AbstractCSQueue.java | 55 +-
.../capacity/AbstractManagedParentQueue.java | 25 +-
.../AppPriorityACLConfigurationParser.java | 8 +-
.../scheduler/capacity/AutoCreatedLeafQueue.java | 5 +-
.../scheduler/capacity/CSQueue.java | 9 +
.../scheduler/capacity/CSQueueMetrics.java | 39 +
.../scheduler/capacity/CSQueueUtils.java | 74 +-
.../scheduler/capacity/CapacityScheduler.java | 426 +-
.../capacity/CapacitySchedulerConfiguration.java | 140 +-
.../capacity/CapacitySchedulerQueueManager.java | 12 +-
.../scheduler/capacity/LeafQueue.java | 214 +-
.../scheduler/capacity/ManagedParentQueue.java | 16 +-
.../scheduler/capacity/ParentQueue.java | 134 +-
.../scheduler/capacity/PlanQueue.java | 2 +-
.../scheduler/capacity/QueueCapacities.java | 30 +-
.../capacity/QueueManagementDynamicEditPolicy.java | 13 +-
.../scheduler/capacity/ReservationQueue.java | 2 +-
.../scheduler/capacity/UsersManager.java | 87 +-
.../capacity/WorkflowPriorityMappingsManager.java | 230 +
.../allocator/AbstractContainerAllocator.java | 18 +-
.../allocator/RegularContainerAllocator.java | 205 +-
.../conf/FSSchedulerConfigurationStore.java | 64 +-
.../capacity/conf/InMemoryConfigurationStore.java | 13 +
.../capacity/conf/LeveldbConfigurationStore.java | 67 +-
.../conf/MutableCSConfigurationProvider.java | 121 +-
.../capacity/conf/YarnConfigurationStore.java | 21 +-
.../conf/YarnConfigurationStoreFactory.java | 28 +-
.../capacity/conf/ZKConfigurationStore.java | 33 +-
.../capacity/preemption/PreemptionManager.java | 12 +-
.../GuaranteedOrZeroCapacityOverTimePolicy.java | 35 +-
.../scheduler/common/PendingAsk.java | 10 +-
.../scheduler/common/fica/FiCaSchedulerApp.java | 225 +-
.../scheduler/common/fica/FiCaSchedulerNode.java | 7 +-
.../MemoryPlacementConstraintManager.java | 34 +-
.../PlacementConstraintManagerService.java | 8 +-
.../constraint/PlacementConstraintsUtil.java | 93 +-
.../algorithm/LocalAllocationTagsManager.java | 4 +-
...CentralizedOpportunisticContainerAllocator.java | 340 +
.../distributed/NodeQueueLoadMonitor.java | 156 +-
.../scheduler/event/ContainerPreemptEvent.java | 4 +-
.../scheduler/fair/AllocationConfiguration.java | 46 +-
.../fair/AllocationFileLoaderService.java | 34 +-
.../scheduler/fair/ConfigurableResource.java | 5 +
.../scheduler/fair/FSAppAttempt.java | 98 +-
.../scheduler/fair/FSLeafQueue.java | 22 +-
.../scheduler/fair/FSParentQueue.java | 6 +-
.../scheduler/fair/FSPreemptionThread.java | 24 +-
.../resourcemanager/scheduler/fair/FSQueue.java | 22 +-
.../scheduler/fair/FSQueueMetrics.java | 197 +-
.../scheduler/fair/FSSchedulerNode.java | 7 +-
.../scheduler/fair/FairScheduler.java | 254 +-
.../scheduler/fair/FairSchedulerConfiguration.java | 198 +-
.../scheduler/fair/MaxRunningAppsEnforcer.java | 6 +-
.../scheduler/fair/QueueManager.java | 23 +-
.../scheduler/fair/QueuePlacementPolicy.java | 344 +-
.../scheduler/fair/QueuePlacementRule.java | 366 -
.../fair/VisitedResourceRequestTracker.java | 8 +-
.../scheduler/fair/allocation/QueueProperties.java | 3 +-
.../fair/converter/ConversionException.java | 37 +
.../FSConfigToCSConfigArgumentHandler.java | 242 +
.../converter/FSConfigToCSConfigConverter.java | 368 +
.../converter/FSConfigToCSConfigConverterMain.java | 57 +
.../FSConfigToCSConfigConverterParams.java | 131 +
.../converter/FSConfigToCSConfigRuleHandler.java | 229 +
.../scheduler/fair/converter/FSQueueConverter.java | 493 +
.../fair/converter/FSYarnSiteConverter.java | 159 +
.../fair/converter/PreconditionException.java | 36 +
.../fair/converter/QueuePlacementConverter.java | 125 +
.../converter/UnsupportedPropertyException.java | 29 +
.../scheduler/fair/converter/package-info.java | 24 +
.../scheduler/fair/policies/FairSharePolicy.java | 7 +-
.../scheduler/fair/policies/FifoPolicy.java | 7 +-
.../scheduler/fifo/FifoAppAttempt.java | 11 +-
.../scheduler/fifo/FifoScheduler.java | 18 +-
.../scheduler/placement/AppPlacementAllocator.java | 17 +
.../placement/LocalityAppPlacementAllocator.java | 47 +-
.../scheduler/placement/MultiNodeSorter.java | 7 +-
.../placement/MultiNodeSortingManager.java | 8 +-
.../SingleConstraintAppPlacementAllocator.java | 45 +-
.../policy/AbstractComparatorOrderingPolicy.java | 18 +-
.../scheduler/policy/CompoundComparator.java | 3 -
.../scheduler/policy/FairOrderingPolicy.java | 6 +
.../scheduler/policy/FifoOrderingPolicy.java | 6 +
.../policy/FifoOrderingPolicyForPendingApps.java | 6 +
.../FifoOrderingPolicyWithExclusivePartitions.java | 151 +
.../scheduler/policy/IteratorSelector.java | 48 +
.../scheduler/policy/OrderingPolicy.java | 9 +-
.../scheduler/policy/SchedulableEntity.java | 5 +
.../security/AMRMTokenSecretManager.java | 12 +-
.../security/AppPriorityACLsManager.java | 8 +-
.../security/DelegationTokenRenewer.java | 13 +-
.../security/NMTokenSecretManagerInRM.java | 20 +-
.../resourcemanager/security/QueueACLsManager.java | 7 +-
.../security/RMContainerTokenSecretManager.java | 8 +-
.../security/RMDelegationTokenSecretManager.java | 8 +-
.../security/ReservationsACLsManager.java | 2 +-
.../RMTimelineCollectorManager.java | 20 +-
.../volume/csi/VolumeManagerImpl.java | 7 +-
.../volume/csi/lifecycle/VolumeImpl.java | 13 +-
.../webapp/CapacitySchedulerPage.java | 26 +-
.../resourcemanager/webapp/DeSelectFields.java | 8 +-
.../webapp/JAXBContextResolver.java | 4 +-
.../server/resourcemanager/webapp/NavBlock.java | 8 +-
.../server/resourcemanager/webapp/NodesPage.java | 12 +-
.../webapp/RMAppLogAggregationStatusBlock.java | 8 +-
.../server/resourcemanager/webapp/RMAppsBlock.java | 5 +
.../webapp/RMErrorsAndWarningsPage.java | 11 +-
.../server/resourcemanager/webapp/RMWSConsts.java | 38 +-
.../server/resourcemanager/webapp/RMWebApp.java | 8 +-
.../resourcemanager/webapp/RMWebAppFilter.java | 5 +-
.../resourcemanager/webapp/RMWebAppUtil.java | 11 +-
.../webapp/RMWebServiceProtocol.java | 32 +-
.../resourcemanager/webapp/RMWebServices.java | 307 +-
.../resourcemanager/webapp/SchedulerPageUtil.java | 4 +-
.../resourcemanager/webapp/dao/ActivitiesInfo.java | 61 +-
.../webapp/dao/ActivityNodeInfo.java | 112 +-
.../webapp/dao/AppActivitiesInfo.java | 63 +-
.../webapp/dao/AppAllocationInfo.java | 84 +-
.../server/resourcemanager/webapp/dao/AppInfo.java | 4 +
.../webapp/dao/AppRequestAllocationInfo.java | 79 +
.../webapp/dao/CapacitySchedulerInfo.java | 62 +-
.../webapp/dao/CapacitySchedulerLeafQueueInfo.java | 37 +-
.../webapp/dao/CapacitySchedulerQueueInfo.java | 55 +-
.../webapp/dao/ClusterMetricsInfo.java | 12 +
.../webapp/dao/ConfigVersionInfo.java | 44 +
.../webapp/dao/FairSchedulerQueueInfo.java | 10 +-
.../webapp/dao/NodeAllocationInfo.java | 37 +-
.../webapp/dao/PartitionQueueCapacitiesInfo.java | 3 +-
.../resourcemanager/webapp/dao/QueueAclInfo.java | 48 +
.../resourcemanager/webapp/dao/QueueAclsInfo.java | 47 +
.../resourcemanager/webapp/dao/ResourceInfo.java | 1 -
.../yarn_server_resourcemanager_recovery.proto | 1 +
.../yarn/server/resourcemanager/ACLsTestBase.java | 7 +-
.../server/resourcemanager/AppManagerTestBase.java | 7 +
.../yarn/server/resourcemanager/Application.java | 42 +-
.../ApplicationMasterServiceTestBase.java | 66 +-
.../hadoop/yarn/server/resourcemanager/MockAM.java | 6 +-
.../hadoop/yarn/server/resourcemanager/MockNM.java | 60 +-
.../yarn/server/resourcemanager/MockNodes.java | 9 +
.../hadoop/yarn/server/resourcemanager/MockRM.java | 28 +-
.../yarn/server/resourcemanager/NodeManager.java | 23 +-
.../hadoop/yarn/server/resourcemanager/Task.java | 7 +-
.../resourcemanager/TestAMAuthorization.java | 7 +-
.../server/resourcemanager/TestAppManager.java | 211 +-
.../TestAppManagerWithFairScheduler.java | 289 +-
.../resourcemanager/TestApplicationACLs.java | 9 +-
.../resourcemanager/TestApplicationCleanup.java | 22 +-
.../TestApplicationMasterLauncher.java | 18 +-
.../TestApplicationMasterServiceInterceptor.java | 8 +-
.../TestCapacitySchedulerMetrics.java | 12 +-
.../resourcemanager/TestClientRMService.java | 208 +-
.../server/resourcemanager/TestClientRMTokens.java | 7 +-
.../server/resourcemanager/TestClusterMetrics.java | 8 +-
.../TestContainerResourceUsage.java | 8 +-
.../TestDecommissioningNodesWatcher.java | 101 +-
.../TestKillApplicationWithRMHA.java | 11 +-
.../resourcemanager/TestLeaderElectorService.java | 10 +-
...stOpportunisticContainerAllocatorAMService.java | 350 +-
.../hadoop/yarn/server/resourcemanager/TestRM.java | 20 +-
.../server/resourcemanager/TestRMAdminService.java | 40 +-
.../server/resourcemanager/TestRMAuditLogger.java | 21 +-
.../resourcemanager/TestRMEmbeddedElector.java | 8 +-
.../yarn/server/resourcemanager/TestRMHA.java | 11 +-
.../resourcemanager/TestRMHAForNodeLabels.java | 8 +-
.../server/resourcemanager/TestRMHAMetrics.java | 24 +
.../resourcemanager/TestRMNodeTransitions.java | 18 +
.../yarn/server/resourcemanager/TestRMRestart.java | 49 +-
.../server/resourcemanager/TestRMServerUtils.java | 9 +-
.../resourcemanager/TestRMStoreCommands.java | 35 +
.../resourcemanager/TestRMTimelineService.java | 59 +-
.../resourcemanager/TestResourceManager.java | 46 +-
.../resourcemanager/TestResourceManagerMXBean.java | 8 +-
.../TestResourceTrackerService.java | 7 +-
.../resourcemanager/TestSignalContainer.java | 18 +-
.../TestSubmitApplicationWithRMHA.java | 8 +-
.../TestWorkPreservingRMRestart.java | 23 +-
.../TestWorkPreservingUnmanagedAM.java | 8 +-
.../applicationsmanager/TestAMLaunchFailure.java | 9 +-
.../applicationsmanager/TestAMRestart.java | 148 +-
.../applicationsmanager/TestASMStateMachine.java | 9 +-
.../TestApplicationMasterExpiry.java | 9 +-
.../metrics/TestSystemMetricsPublisher.java | 13 +-
.../metrics/TestSystemMetricsPublisherForV2.java | 50 +-
...ionalCapacityPreemptionPolicyMockFramework.java | 8 +-
...alCapacityPreemptionPolicyPreemptToBalance.java | 4 +-
.../invariants/TestMetricsInvariantChecker.java | 10 +-
.../TestFileSystemNodeAttributeStore.java | 13 +-
.../nodelabels/TestRMNodeLabelsManager.java | 76 +-
.../placement/TestFairQueuePlacementUtils.java | 137 +
.../placement/TestPlacementFactory.java | 75 +
.../placement/TestPlacementRuleFS.java | 212 +
.../TestUserGroupMappingPlacementRule.java | 74 +-
.../preprocessor/TestContextProcessor.java | 63 +
.../preprocessor/TestNodeLabelProcessor.java | 45 +
.../preprocessor/TestQueueProcessor.java | 43 +
.../preprocessor/TestTagAddProcessor.java | 47 +
.../recovery/RMStateStoreTestBase.java | 7 +-
.../recovery/TestFSRMStateStore.java | 12 +-
.../recovery/TestZKRMStateStore.java | 16 +-
.../recovery/TestZKRMStateStorePerf.java | 7 +-
.../TestZKRMStateStoreZKClientConnections.java | 8 +-
.../reservation/TestReservationInputValidator.java | 8 +-
.../reservation/TestReservationSystemUtil.java | 9 +-
.../reservation/TestSchedulerPlanFollowerBase.java | 12 +-
.../reservation/planning/TestAlignedPlanner.java | 33 +-
.../resourcetracker/TestNMExpiry.java | 7 +-
.../rmapp/TestApplicationLifetimeMonitor.java | 15 +-
.../rmapp/TestNodesListManager.java | 17 +-
.../rmapp/TestRMAppTransitions.java | 44 +-
.../rmapp/attempt/TestRMAppAttemptTransitions.java | 8 +-
.../rmcontainer/TestRMContainerImpl.java | 68 +-
.../scheduler/ResourceMetricsChecker.java | 67 +-
.../scheduler/TestAbstractYarnScheduler.java | 93 +
.../scheduler/TestAppSchedulingInfo.java | 9 +-
.../scheduler/TestQueueMetrics.java | 60 +-
.../TestQueueMetricsForCustomResources.java | 59 +-
.../scheduler/TestResourceUsage.java | 7 +-
.../scheduler/TestSchedulerApplicationAttempt.java | 8 +-
.../scheduler/TestSchedulerOvercommit.java | 735 +
.../scheduler/TestSchedulerUtils.java | 162 +-
.../activities/TestActivitiesManager.java | 500 +
.../TestAbsoluteResourceConfiguration.java | 49 +
.../scheduler/capacity/TestApplicationLimits.java | 33 +-
.../capacity/TestApplicationLimitsByPartition.java | 85 +
.../capacity/TestApplicationPriority.java | 8 +-
.../scheduler/capacity/TestCapacityScheduler.java | 426 +-
.../TestCapacitySchedulerAutoCreatedQueueBase.java | 35 +-
.../TestCapacitySchedulerAutoQueueCreation.java | 17 +-
.../TestCapacitySchedulerDynamicBehavior.java | 8 +-
.../TestCapacitySchedulerLazyPreemption.java | 11 +-
.../capacity/TestCapacitySchedulerMultiNodes.java | 147 +-
.../capacity/TestCapacitySchedulerOvercommit.java | 52 +
.../capacity/TestCapacitySchedulerPerf.java | 12 +-
.../TestCapacitySchedulerQueueMappingFactory.java | 190 +-
.../TestCapacitySchedulerSurgicalPreemption.java | 7 +
...estCapacitySchedulerWithMultiResourceTypes.java | 154 +-
...stCapacitySchedulerWorkflowPriorityMapping.java | 151 +
.../scheduler/capacity/TestChildQueueOrder.java | 7 +-
.../capacity/TestContainerAllocation.java | 8 +-
.../scheduler/capacity/TestContainerResizing.java | 7 +-
.../scheduler/capacity/TestLeafQueue.java | 166 +-
.../capacity/TestNodeLabelContainerAllocation.java | 165 +-
.../scheduler/capacity/TestParentQueue.java | 7 +-
.../scheduler/capacity/TestQueueCapacities.java | 7 +-
.../TestQueueManagementDynamicEditPolicy.java | 2 +-
.../scheduler/capacity/TestQueueMappings.java | 7 +-
.../scheduler/capacity/TestQueueParsing.java | 7 +-
.../scheduler/capacity/TestQueueState.java | 4 +-
.../scheduler/capacity/TestReservations.java | 15 +-
.../TestSchedulingRequestContainerAllocation.java | 63 +-
...tSchedulingRequestContainerAllocationAsync.java | 11 +-
.../scheduler/capacity/TestUtils.java | 8 +-
.../conf/TestFSSchedulerConfigurationStore.java | 73 +
.../conf/TestLeveldbConfigurationStore.java | 8 +-
.../conf/TestMutableCSConfigurationProvider.java | 45 +
.../capacity/conf/TestZKConfigurationStore.java | 34 +-
.../constraint/TestPlacementConstraintsUtil.java | 19 +-
.../constraint/TestPlacementProcessor.java | 8 +-
...CentralizedOpportunisticContainerAllocator.java | 669 +
.../distributed/TestNodeQueueLoadMonitor.java | 181 +
.../scheduler/fair/FairSchedulerTestBase.java | 28 +-
.../scheduler/fair/PrimaryGroupMapping.java | 47 +
.../scheduler/fair/SimpleGroupsMapping.java | 3 -
.../fair/TestAllocationFileLoaderService.java | 152 +-
.../scheduler/fair/TestAppRunnability.java | 35 +-
.../fair/TestApplicationMasterServiceWithFS.java | 8 +-
.../scheduler/fair/TestConfigurableResource.java | 13 +-
.../scheduler/fair/TestContinuousScheduling.java | 36 +-
.../scheduler/fair/TestFSAppAttempt.java | 48 +-
.../scheduler/fair/TestFSLeafQueue.java | 144 +-
.../scheduler/fair/TestFSParentQueue.java | 20 +-
.../scheduler/fair/TestFSQueueMetrics.java | 247 +-
.../scheduler/fair/TestFairScheduler.java | 327 +-
.../fair/TestFairSchedulerConfiguration.java | 120 +-
.../fair/TestFairSchedulerOvercommit.java | 46 +
.../fair/TestFairSchedulerPreemption.java | 12 +
.../TestFairSchedulerWithMultiResourceTypes.java | 9 +
.../scheduler/fair/TestMaxRunningAppsEnforcer.java | 26 +-
.../scheduler/fair/TestQueueManager.java | 46 +-
.../scheduler/fair/TestQueuePlacementPolicy.java | 515 +-
.../scheduler/fair/TestSchedulingPolicy.java | 18 +-
.../converter/FSConfigConverterTestCommons.java | 185 +
.../TestFSConfigToCSConfigArgumentHandler.java | 382 +
.../converter/TestFSConfigToCSConfigConverter.java | 460 +
.../TestFSConfigToCSConfigConverterMain.java | 135 +
.../TestFSConfigToCSConfigRuleHandler.java | 144 +
.../fair/converter/TestFSQueueConverter.java | 427 +
.../fair/converter/TestFSYarnSiteConverter.java | 141 +
.../converter/TestQueuePlacementConverter.java | 265 +
.../scheduler/fifo/TestFifoScheduler.java | 34 +-
.../TestSingleConstraintAppPlacementAllocator.java | 23 +-
.../scheduler/policy/MockSchedulableEntity.java | 15 +-
.../scheduler/policy/TestFairOrderingPolicy.java | 12 +-
.../scheduler/policy/TestFifoOrderingPolicy.java | 12 +-
.../TestFifoOrderingPolicyForPendingApps.java | 13 +-
...tFifoOrderingPolicyWithExclusivePartitions.java | 244 +
.../resourcemanager/security/TestAMRMTokens.java | 7 +-
.../security/TestDelegationTokenRenewer.java | 12 +-
.../security/TestRMDelegationTokens.java | 40 +-
.../volume/csi/TestVolumeProcessor.java | 9 +-
.../webapp/ActivitiesTestUtils.java | 318 +
.../resourcemanager/webapp/TestRMWebServices.java | 109 +-
.../webapp/TestRMWebServicesCapacitySched.java | 48 +-
.../TestRMWebServicesConfigurationMutation.java | 161 +
.../webapp/TestRMWebServicesContainers.java | 170 +
...RMWebServicesDelegationTokenAuthentication.java | 23 +-
.../TestRMWebServicesForCSWithPartitions.java | 99 +-
.../webapp/TestRMWebServicesNodeLabels.java | 8 +-
.../webapp/TestRMWebServicesNodes.java | 5 +-
.../webapp/TestRMWebServicesReservation.java | 3 +-
.../TestRMWebServicesSchedulerActivities.java | 1149 +-
...esSchedulerActivitiesWithMultiNodesEnabled.java | 642 +
.../webapp/dao/TestFairSchedulerQueueInfo.java | 24 +-
.../src/test/proto/test_client_tokens.proto | 1 +
.../src/test/resources/conversion-rules.properties | 24 +
.../test/resources/fair-scheduler-conversion.xml | 94 +
.../src/test/resources/fair-scheduler-invalid.xml | 21 +
.../fair-scheduler-max-resources-percentage.xml | 90 +
.../fair-scheduler-orderingpolicy-mixed.xml | 89 +
.../resources/fair-scheduler-orderingpolicy.xml | 89 +
.../resources/fair-scheduler-sameleafqueue.xml | 90 +
.../hadoop-metrics2-resourcemanager.properties | 23 +
.../src/test/resources/hadoop-metrics2.properties | 23 +
.../src/test/resources/invariants_jdk9.txt | 54 +
.../yarn-site-with-allocation-file-ref.xml | 23 +
.../yarn-site-with-invalid-allocation-file-ref.xml | 37 +
.../hadoop-yarn-server-router/pom.xml | 2 +-
.../clientrm/FederationClientInterceptor.java | 15 +-
.../webapp/DefaultRequestInterceptorREST.java | 16 +-
.../router/webapp/FederationInterceptorREST.java | 13 +-
.../yarn/server/router/webapp/FederationPage.java | 10 +-
.../yarn/server/router/webapp/NodesPage.java | 10 +-
.../server/router/webapp/RouterWebServices.java | 36 +-
.../router/webapp/BaseRouterWebServicesTest.java | 6 +-
.../router/webapp/MockRESTRequestInterceptor.java | 13 +-
.../webapp/PassThroughRESTRequestInterceptor.java | 19 +-
.../metrics/ClientSCMMetrics.java | 2 +-
.../metrics/SharedCacheUploaderMetrics.java | 2 +-
.../hadoop-yarn-server-tests/pom.xml | 37 +-
.../server/ContainerTokenIdentifierForTest.java | 2 +-
.../apache/hadoop/yarn/server/MiniYARNCluster.java | 53 +-
.../yarn/server/NMTokenIdentifierNewForTest.java | 2 +-
.../yarn/server/TestContainerManagerSecurity.java | 14 +-
.../security/TestTimelineAuthFilterForV2.java | 181 +-
.../src/test/proto/test_token.proto | 1 +
.../pom.xml | 7 +-
.../timeline/EntityGroupFSTimelineStore.java | 87 +-
.../server/timeline/LevelDBCacheTimelineStore.java | 8 +
.../timeline/TestEntityGroupFSTimelineStore.java | 84 +-
.../timeline/TestLevelDBCacheTimelineStore.java | 13 +
.../pom.xml | 178 +
.../DocumentStoreCollectionCreator.java | 66 +
.../DocumentStoreTimelineReaderImpl.java | 134 +
.../DocumentStoreTimelineWriterImpl.java | 285 +
.../documentstore/DocumentStoreUtils.java | 498 +
.../documentstore/collection/CollectionType.java | 44 +
.../document/NoDocumentFoundException.java | 39 +
.../collection/document/TimelineDocument.java | 37 +
.../document/entity/TimelineEntityDocument.java | 250 +
.../document/entity/TimelineEventSubDoc.java | 91 +
.../document/entity/TimelineMetricSubDoc.java | 161 +
.../collection/document/entity/package-info.java | 30 +
.../flowactivity/FlowActivityDocument.java | 131 +
.../document/flowactivity/FlowActivitySubDoc.java | 73 +
.../document/flowactivity/package-info.java | 29 +
.../document/flowrun/FlowRunDocument.java | 244 +
.../collection/document/flowrun/package-info.java | 29 +
.../collection/document/package-info.java | 30 +
.../documentstore/collection/package-info.java | 30 +
.../documentstore/lib/DocumentStoreFactory.java | 96 +
.../lib/DocumentStoreNotSupportedException.java | 35 +
.../documentstore/lib/DocumentStoreVendor.java | 39 +
.../documentstore/lib/package-info.java | 30 +
.../documentstore/package-info.java | 29 +
.../documentstore/reader/DocumentStoreReader.java | 45 +
.../reader/TimelineCollectionReader.java | 220 +
.../cosmosdb/CosmosDBDocumentStoreReader.java | 253 +
.../reader/cosmosdb/package-info.java | 28 +
.../documentstore/reader/package-info.java | 29 +
.../documentstore/writer/DocumentStoreWriter.java | 35 +
.../writer/TimelineCollectionWriter.java | 146 +
.../cosmosdb/CosmosDBDocumentStoreWriter.java | 288 +
.../writer/cosmosdb/package-info.java | 28 +
.../documentstore/writer/package-info.java | 29 +
.../documentstore/DocumentStoreTestUtils.java | 81 +
.../timelineservice/documentstore/JsonUtils.java | 59 +
.../TestDocumentStoreCollectionCreator.java | 64 +
.../TestDocumentStoreTimelineReaderImpl.java | 407 +
.../TestDocumentStoreTimelineWriterImpl.java | 90 +
.../collection/TestDocumentOperations.java | 177 +
.../reader/DummyDocumentStoreReader.java | 118 +
.../cosmosdb/TestCosmosDBDocumentStoreReader.java | 67 +
.../writer/DummyDocumentStoreWriter.java | 46 +
.../MockedCosmosDBDocumentStoreWriter.java | 47 +
.../cosmosdb/TestCosmosDBDocumentStoreWriter.java | 96 +
.../test/resources/documents/flowactivity-doc.json | 20 +
.../src/test/resources/documents/flowrun-doc.json | 126 +
.../documents/test-timeline-entities-doc.json | 185 +
.../test/resources/documents/timeline-app-doc.json | 203 +
.../resources/documents/timeline-entities.json | 119 +
.../pom.xml | 8 +-
.../storage/TestTimelineReaderHBaseDown.java | 22 +-
.../storage/TestTimelineWriterHBaseDown.java | 117 +
.../flow/TestHBaseStorageFlowRunCompaction.java | 4 +-
.../pom.xml | 15 +-
.../storage/HBaseStorageMonitor.java | 90 +
.../storage/HBaseTimelineReaderImpl.java | 108 +-
.../storage/HBaseTimelineSchemaCreator.java | 45 +-
.../storage/HBaseTimelineWriterImpl.java | 93 +-
.../storage/apptoflow/AppToFlowTableRW.java | 8 +-
.../storage/common/ColumnRWHelper.java | 4 +-
.../storage/common/HBaseTimelineStorageUtils.java | 32 +-
.../storage/flow/FlowActivityTableRW.java | 2 +-
.../storage/flow/FlowRunTableRW.java | 2 +-
.../storage/reader/TimelineEntityReader.java | 8 +-
.../common/TestHBaseTimelineStorageUtils.java | 74 +
.../pom.xml | 4 +-
.../storage/common/HBaseTimelineSchemaUtils.java | 10 +-
.../timelineservice/storage/common/Separator.java | 4 +-
.../storage/flow/FlowRunRowKey.java | 12 +-
.../pom.xml | 4 +-
.../timelineservice/storage/flow/FlowScanner.java | 4 +-
.../pom.xml | 4 +-
.../timelineservice/storage/flow/FlowScanner.java | 12 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../hadoop-yarn-server-timelineservice/pom.xml | 13 +-
.../server/timelineservice/TimelineContext.java | 9 +-
.../collector/NodeTimelineCollectorManager.java | 16 +-
.../collector/TimelineCollector.java | 42 +-
.../collector/TimelineCollectorWebService.java | 4 +
.../PerNodeAggTimelineCollectorMetrics.java | 2 +-
.../reader/TimelineReaderManager.java | 12 +-
.../reader/TimelineReaderServer.java | 11 +-
.../reader/TimelineReaderWebServices.java | 48 +-
.../storage/FileSystemTimelineReaderImpl.java | 19 +
.../storage/FileSystemTimelineWriterImpl.java | 219 +-
.../storage/NoOpTimelineReaderImpl.java | 19 +-
.../storage/NoOpTimelineWriterImpl.java | 16 +-
.../timelineservice/storage/TimelineReader.java | 8 +
.../storage/TimelineStorageMonitor.java | 102 +
.../collector/TestTimelineCollector.java | 48 +-
.../reader/TestTimelineReaderWebServices.java | 19 +
.../reader/TestTimelineReaderWebServicesACL.java | 179 +
...TimelineReaderWhitelistAuthorizationFilter.java | 58 +-
.../storage/TestFileSystemTimelineWriterImpl.java | 170 +-
.../hadoop-yarn-server-web-proxy/pom.xml | 11 +-
.../hadoop/yarn/server/webproxy/ProxyCA.java | 4 +-
.../hadoop/yarn/server/webproxy/ProxyUtils.java | 4 +-
.../yarn/server/webproxy/WebAppProxyServlet.java | 8 +-
.../yarn/server/webproxy/amfilter/AmIpFilter.java | 27 +-
.../server/webproxy/TestWebAppProxyServlet.java | 5 +-
.../server/webproxy/amfilter/TestAmFilter.java | 5 +-
.../webproxy/amfilter/TestSecureAmFilter.java | 14 +-
.../hadoop-yarn/hadoop-yarn-server/pom.xml | 3 +-
.../hadoop-yarn/hadoop-yarn-site/pom.xml | 2 +-
.../src/site/markdown/CapacityScheduler.md | 12 +-
.../site/markdown/DevelopYourOwnDevicePlugin.md | 177 +
.../src/site/markdown/DockerContainers.md | 207 +-
.../src/site/markdown/FairScheduler.md | 14 +-
.../src/site/markdown/NodeManager.md | 8 +-
.../src/site/markdown/NodeManagerRest.md | 2 +-
.../src/site/markdown/PlacementConstraints.md.vm | 39 +-
.../src/site/markdown/PluggableDeviceFramework.md | 151 +
.../src/site/markdown/ResourceManagerRest.md | 135 +
.../src/site/markdown/ResourceModel.md | 2 +-
.../src/site/markdown/RuncContainers.md | 853 +
.../src/site/markdown/TimelineServiceV2.md | 88 +-
.../src/site/markdown/UsingFPGA.md | 2 +-
.../src/site/markdown/YarnApplicationSecurity.md | 8 +-
.../src/site/markdown/YarnCommands.md | 2 +
.../hadoop-yarn-site/src/site/markdown/YarnUI2.md | 2 +-
.../site/markdown/yarn-service/Configurations.md | 1 +
.../src/site/markdown/yarn-service/Examples.md | 77 +
.../src/site/markdown/yarn-service/MasterWorker.md | 36 +
.../src/site/markdown/yarn-service/Overview.md | 2 +-
.../hadoop-yarn/hadoop-yarn-ui/pom.xml | 2 +-
.../hadoop-yarn-ui/src/main/webapp/.gitignore | 4 +
.../main/webapp/app/adapters/timeline-health.js | 30 +
.../main/webapp/app/components/nodes-heatmap.js | 6 +-
.../main/webapp/app/components/tree-selector.js | 43 +-
.../webapp/app/controllers/app-table-columns.js | 12 +-
.../src/main/webapp/app/controllers/application.js | 7 +
.../main/webapp/app/controllers/yarn-app/logs.js | 9 +-
.../webapp/app/controllers/yarn-container-log.js | 5 +-
.../main/webapp/app/controllers/yarn-flow/runs.js | 2 +-
.../main/webapp/app/controllers/yarn-node-app.js | 5 +
.../main/webapp/app/controllers/yarn-node-apps.js | 5 +
.../webapp/app/controllers/yarn-node-container.js | 5 +
.../webapp/app/controllers/yarn-node-containers.js | 5 +
.../src/main/webapp/app/models/timeline-health.js | 27 +
.../main/webapp/app/models/yarn-flowrun-brief.js | 9 +-
.../src/main/webapp/app/models/yarn-user.js | 11 +-
.../src/main/webapp/app/routes/application.js | 4 +
.../src/main/webapp/app/routes/timeline-error.js | 3 +
.../src/main/webapp/app/routes/yarn-app/logs.js | 5 +-
.../main/webapp/app/routes/yarn-container-log.js | 8 +-
.../src/main/webapp/app/routes/yarn-node-app.js | 8 +-
.../main/webapp/app/serializers/timeline-health.js | 31 +
.../src/main/webapp/app/serializers/yarn-app.js | 3 +
.../app/serializers/yarn-queue/capacity-queue.js | 11 +
.../src/main/webapp/app/styles/app.scss | 18 +
.../src/main/webapp/app/templates/application.hbs | 4 +-
.../app/templates/components/nodes-heatmap.hbs | 1 -
.../yarn-queue/fair-queue-conf-table.hbs | 2 +-
.../templates/components/yarn-queue/fair-queue.hbs | 12 +
.../main/webapp/app/templates/yarn-app/logs.hbs | 71 +-
.../main/webapp/app/templates/yarn-node-app.hbs | 2 +-
.../main/webapp/app/templates/yarn-node-apps.hbs | 2 +-
.../webapp/app/templates/yarn-node-container.hbs | 2 +-
.../webapp/app/templates/yarn-node-containers.hbs | 4 +-
.../src/main/webapp/app/templates/yarn-queue.hbs | 44 +-
.../main/webapp/app/templates/yarn-queue/apps.hbs | 47 +-
.../tests/unit/models/cluster-metric-test.js | 2 +-
hadoop-yarn-project/hadoop-yarn/pom.xml | 4 +-
hadoop-yarn-project/pom.xml | 4 +-
licenses-binary/LICENSE-angular-nvd3.txt | 16 +
licenses-binary/LICENSE-angular.txt | 21 +
licenses-binary/LICENSE-asm.txt | 27 +
licenses-binary/LICENSE-azure-cosmosdb.txt | 21 +
licenses-binary/LICENSE-azure-data-lake-store.txt | 24 +
licenses-binary/LICENSE-azure-keyvault.txt | 21 +
licenses-binary/LICENSE-bloomfilter.txt | 28 +
licenses-binary/LICENSE-bouncycastle.txt | 7 +
licenses-binary/LICENSE-cddl-gplv2-ce.txt | 759 +
licenses-binary/LICENSE-checker-framework.txt | 408 +
licenses-binary/LICENSE-d3.txt | 26 +
licenses-binary/LICENSE-datatables.txt | 7 +
licenses-binary/LICENSE-dnsjava.txt | 24 +
licenses-binary/LICENSE-dust.txt | 19 +
licenses-binary/LICENSE-freebsd.txt | 27 +
licenses-binary/LICENSE-go.txt | 32 +
licenses-binary/LICENSE-gtest.txt | 28 +
licenses-binary/LICENSE-hamcrest.txt | 27 +
licenses-binary/LICENSE-hsql.txt | 66 +
licenses-binary/LICENSE-jaf.txt | 28 +
licenses-binary/LICENSE-jdom.txt | 55 +
licenses-binary/LICENSE-jline.txt | 34 +
licenses-binary/LICENSE-jquery.txt | 20 +
licenses-binary/LICENSE-jsch.txt | 30 +
licenses-binary/LICENSE-jsr305.txt | 8 +
licenses-binary/LICENSE-jstree.txt | 22 +
licenses-binary/LICENSE-junit.txt | 213 +
licenses-binary/LICENSE-leveldb.txt | 27 +
licenses-binary/LICENSE-leveldbjni.txt | 27 +
licenses-binary/LICENSE-lz4.txt | 31 +
licenses-binary/LICENSE-moment.txt | 22 +
licenses-binary/LICENSE-mssql-jdbc.txt | 14 +
licenses-binary/LICENSE-nvd3.txt | 10 +
licenses-binary/LICENSE-ojalgo.txt | 21 +
licenses-binary/LICENSE-paranamer.txt | 28 +
licenses-binary/LICENSE-protobuf.txt | 32 +
licenses-binary/LICENSE-re2j.txt | 32 +
licenses-binary/LICENSE-slf4j.txt | 21 +
licenses-binary/LICENSE-slicing-by-8.txt | 45 +
licenses-binary/LICENSE-tree.txt | 22 +
licenses-binary/LICENSE-xtermjs.txt | 24 +
licenses/LICENSE-angular-nvd3.txt | 16 +
licenses/LICENSE-angular.txt | 21 +
licenses/LICENSE-bloomfilter.txt | 28 +
licenses/LICENSE-d3.txt | 26 +
licenses/LICENSE-datatables.txt | 7 +
licenses/LICENSE-dust.txt | 19 +
licenses/LICENSE-freebsd.txt | 27 +
licenses/LICENSE-gtest.txt | 28 +
licenses/LICENSE-jquery.txt | 20 +
licenses/LICENSE-jstree.txt | 22 +
licenses/LICENSE-lz4.txt | 31 +
licenses/LICENSE-moment.txt | 22 +
licenses/LICENSE-nvd3.txt | 10 +
licenses/LICENSE-slicing-by-8.txt | 45 +
licenses/LICENSE-tree.txt | 22 +
pom.xml | 46 +-
start-build-env.sh | 1 +
4817 files changed, 327162 insertions(+), 257968 deletions(-)
create mode 100644 .github/pull_request_template.md
create mode 100644 Jenkinsfile
create mode 100644 LICENSE-binary
create mode 100644 NOTICE-binary
create mode 100644 hadoop-assemblies/src/main/resources/assemblies/hadoop-dynamometer-blockgen.xml
create mode 100644 hadoop-assemblies/src/main/resources/assemblies/hadoop-dynamometer-infra.xml
create mode 100644 hadoop-assemblies/src/main/resources/assemblies/hadoop-dynamometer-workload.xml
create mode 100644 hadoop-assemblies/src/main/resources/assemblies/hadoop-dynamometer.xml
rename hadoop-assemblies/src/main/resources/assemblies/{hadoop-src-with-hdds.xml => hadoop-src-submarine.xml} (100%)
copy hadoop-hdfs-project/hadoop-hdfs-nfs/dev-support/findbugsExcludeFile.xml => hadoop-cloud-storage-project/hadoop-cos/dev-support/findbugs-exclude.xml (100%)
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/pom.xml
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/BufferPool.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/ByteBufferInputStream.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/ByteBufferOutputStream.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/ByteBufferWrapper.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/Constants.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosN.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNConfigKeys.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNCopyFileContext.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNCopyFileTask.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNFileReadTask.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNFileSystem.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNInputStream.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNOutputStream.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNUtils.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNativeFileSystemStore.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/FileMetadata.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/NativeFileSystemStore.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/PartialListing.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/Unit.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/COSCredentialProviderList.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/EnvironmentVariableCredentialProvider.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/NoAuthWithCOSException.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/SimpleCredentialProvider.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/auth/package-info.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/package-info.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/site/markdown/cloud-storage/index.md
copy {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-cloud-storage-project/hadoop-cos}/src/site/resources/css/site.css (100%)
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/CosNTestConfigKey.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/CosNTestUtils.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/TestCosNInputStream.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/TestCosNOutputStream.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/CosNContract.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractCreate.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractDelete.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractDistCp.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractGetFileStatus.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractMkdir.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractOpen.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractRename.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractRootDir.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/TestCosNContractSeek.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/java/org/apache/hadoop/fs/cosn/contract/package-info.java
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/resources/contract/cosn.xml
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/resources/core-site.xml
create mode 100644 hadoop-cloud-storage-project/hadoop-cos/src/test/resources/log4j.properties
create mode 100644 hadoop-common-project/hadoop-common/dev-support/jdiff/Apache_Hadoop_Common_2.10.0.xml
mode change 100644 => 100755 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ByteBufferPositionedReadable.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonPathCapabilities.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathCapabilities.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FsLinkResolution.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FunctionsRaisingIOE.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/PathCapabilitiesSupport.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/PrometheusServlet.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/WebServlet.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CostProvider.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/DefaultCostProvider.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProcessingDetails.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WeightedTimeCostProvider.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/PrometheusMetricsSink.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DomainNameResolver.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DomainNameResolverFactory.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/package-info.java
delete mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/package.html
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/FastSaslClientFactory.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/FastSaslServerFactory.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilter.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilterInitializer.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/package-info.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DurationInfo.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/OperationDuration.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/pmdk_load.c
create mode 100644 hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/pmdk_load.h
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/filesystem/pathcapabilities.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.10.0/CHANGES.2.10.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3/CHANGES.3.1.3.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3/RELEASENOTES.3.1.3.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.2.1/CHANGELOG.3.2.1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.2.1/RELEASENOTES.3.2.1.md
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestChecksumFs.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsUrlConnectionPath.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/impl/TestFutureIO.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpnego.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestProcessingDetails.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestWeightedTimeCostProvider.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/metrics/TestRpcMetrics.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/sink/TestPrometheusMetricsSink.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/MockDomainNameResolver.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestMockDomainNameResolver.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithBindUserSwitch.java
delete mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithSecurityOn.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/authentication/server/TestProxyUserAuthenticationFilter.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestDelegatingSSLSocketFactory.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/AbstractHadoopTestBase.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestDurationInfo.java
delete mode 100644 hadoop-common-project/hadoop-common/src/test/resources/kdc/killKdc.sh
delete mode 100644 hadoop-common-project/hadoop-common/src/test/resources/kdc/ldif/users.ldif
delete mode 100644 hadoop-common-project/hadoop-common/src/test/resources/kdc/server.xml
create mode 100644 hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/KMSBenchmark.java
create mode 100644 hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSMDCFilter.java
delete mode 100644 hadoop-hdds/client/pom.xml
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/ClientCredentialInterceptor.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientGrpc.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientMetrics.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientRatis.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/ContainerOperationClient.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/package-info.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/package-info.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/package-info.java
delete mode 100644 hadoop-hdds/common/dev-support/findbugsExcludeFile.xml
delete mode 100644 hadoop-hdds/common/pom.xml
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsConfigKeys.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsIdFactory.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsUtils.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/GenericCli.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/GenericParentCommand.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/HddsVersionProvider.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/MissingSubcommandException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/BlockID.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/ContainerBlockID.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/OzoneQuota.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/ReplicationFactor.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/ReplicationType.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/HddsConfServlet.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/OzoneConfiguration.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/DatanodeDetails.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/SCMSecurityProtocol.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/SCMSecurityProtocolClientSideTranslatorPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/SCMSecurityProtocolPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/SCMSecurityProtocolServerSideTranslatorPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfigKeys.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmInfo.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientReply.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientSpi.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/client/ScmClient.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/client/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerID.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerInfo.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerNotFoundException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaNotFoundException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/AllocatedBlock.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/BlockNotCommittedException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ContainerNotOpenException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ContainerWithPipeline.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/DeleteBlockResult.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/InvalidContainerStateException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/StorageContainerException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineID.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineNotFoundException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/UnknownPipelineStateException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/LocatedContainer.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/ScmBlockLocationProtocol.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/ScmLocatedBlock.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocol.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/ScmBlockLocationProtocolClientSideTranslatorPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/ScmBlockLocationProtocolPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolClientSideTranslatorPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/exception/SCMSecurityException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/exception/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/BlockTokenException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/BlockTokenVerifier.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/OzoneBlockTokenIdentifier.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/OzoneBlockTokenSelector.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/TokenVerifier.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/SecurityConfig.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/BaseApprover.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/CertificateApprover.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/CertificateServer.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/DefaultApprover.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/DefaultCAServer.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/DefaultCAProfile.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/DefaultProfile.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/PKIProfile.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/CertificateClient.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/DNCertificateClient.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/DefaultCertificateClient.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/OMCertificateClient.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/CertificateCodec.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/CertificateSignRequest.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/SelfSignedCertificate.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/exceptions/CertificateException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/exceptions/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/HDDSKeyGenerator.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/KeyCodec.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/SecurityUtil.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/GrpcClientInterceptor.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/GrpcServerInterceptor.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/StringCodec.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/TraceAllMethod.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/TracingUtil.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneAcl.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneSecurityUtil.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditAction.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditEventStatus.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditLogger.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditLoggerType.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditMarker.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditMessage.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/Auditable.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/Auditor.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/DNAction.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/SCMAction.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/BlockGroup.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChecksumData.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/DeleteBlockGroupResult.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/InconsistentStorageStateException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/OzoneChecksumException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Storage.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/StorageInfo.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/statemachine/InvalidStateTransitionException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/statemachine/StateMachine.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/statemachine/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/BlockData.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ChunkInfo.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerCommandRequestPBHelper.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/Lease.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseAlreadyExistException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseCallbackExecutor.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseExpiredException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseManager.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseManagerNotRunningException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/LeaseNotFoundException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lease/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/ActiveLock.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/LockManager.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/PooledLockFactory.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/protocolPB/ScmBlockLocationProtocolServerSideTranslatorPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/protocolPB/StorageContainerLocationProtocolServerSideTranslatorPB.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/protocolPB/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/web/utils/JsonUtils.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/web/utils/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/BackgroundService.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/BackgroundTask.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/BackgroundTaskQueue.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/BackgroundTaskResult.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/BatchOperation.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/EntryConsumer.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/HddsVersionInfo.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/LevelDBStore.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/LevelDBStoreIterator.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/MetaStoreIterator.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/MetadataKeyFilters.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/MetadataStore.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/MetadataStoreBuilder.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/RetriableTask.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/RocksDBStore.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/RocksDBStoreIterator.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/RocksDBStoreMBean.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/VersionInfo.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/BatchOperation.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/ByteArrayKeyValue.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/Codec.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/CodecRegistry.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/DBConfigFromFile.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/DBProfile.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/DBStore.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/DBStoreBuilder.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RDBBatchOperation.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RDBStore.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RDBStoreIterator.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RDBTable.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/StringCodec.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/Table.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/TableConfig.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/TableIterator.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/TypedTable.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/ratis/RatisHelper.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/ratis/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/proto/DatanodeContainerProtocol.proto
delete mode 100644 hadoop-hdds/common/src/main/proto/SCMSecurityProtocol.proto
delete mode 100644 hadoop-hdds/common/src/main/proto/ScmBlockLocationProtocol.proto
delete mode 100644 hadoop-hdds/common/src/main/proto/StorageContainerLocationProtocol.proto
delete mode 100644 hadoop-hdds/common/src/main/proto/hdds.proto
delete mode 100644 hadoop-hdds/common/src/main/resources/hdds-version-info.properties
delete mode 100644 hadoop-hdds/common/src/main/resources/ozone-default.xml
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/token/TestOzoneBlockTokenIdentifier.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/token/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/MockApprover.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/TestDefaultCAServer.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/TestDefaultProfile.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/client/TestCertificateClientInit.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/client/TestDefaultCertificateClient.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/utils/TestCertificateCodec.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/utils/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/TestCertificateSignRequest.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/TestRootCertificate.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/TestHDDSKeyGenerator.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/TestKeyCodec.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/TestOzoneAcls.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/audit/DummyAction.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/audit/DummyEntity.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/audit/TestOzoneAuditLogger.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/audit/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/common/TestChecksum.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/common/TestStateMachine.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lease/TestLeaseManager.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lease/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lock/TestLockManager.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lock/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/TestHddsIdFactory.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/TestMetadataStore.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/TestRetriableTask.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/TestRocksDBStoreMBean.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestDBConfigFromFile.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestDBStoreBuilder.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestRDBStore.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestRDBTableStore.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestTypedRDBTableStore.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/package-info.java
delete mode 100644 hadoop-hdds/common/src/test/resources/log4j2.properties
delete mode 100644 hadoop-hdds/common/src/test/resources/test.db.ini
delete mode 100644 hadoop-hdds/container-service/dev-support/findbugsExcludeFile.xml
delete mode 100644 hadoop-hdds/container-service/pom.xml
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/scm/HddsServerUtil.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/scm/VersionInfo.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/scm/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeService.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DataNodeLayoutVersion.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerMetrics.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/DatanodeVersionFile.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/DeletedContainerBlocksSummary.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ChunkLayOutVersion.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerData.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerDataYaml.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerSet.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/OpenContainerBlockMap.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/RandomContainerDeletionChoosingPolicy.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/StorageLocationReport.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/TopNOrderedContainerDeletionChoosingPolicy.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/BlockIterator.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Container.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerDeletionChoosingPolicy.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerDispatcher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerLocationManager.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerLocationManagerMXBean.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerPacker.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/StorageLocationReportMXBean.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/VolumeChoosingPolicy.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/CommandStatusReportPublisher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/ContainerReportPublisher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/NodeReportPublisher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/PipelineReportPublisher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/ReportManager.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/ReportPublisher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/ReportPublisherFactory.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeStateMachine.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/EndpointStateMachine.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/EndpointStateMachineMBean.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/SCMConnectionManager.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/SCMConnectionManagerMXBean.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/CloseContainerCommandHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/CommandDispatcher.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/CommandHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/DeleteBlocksCommandHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/DeleteContainerCommandHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/ReplicateContainerCommandHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/DatanodeState.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/datanode/InitDatanodeState.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/datanode/RunningDatanodeState.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/datanode/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/RegisterEndpointTask.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/VersionEndpointTask.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/GrpcXceiverService.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ServerCredentialInterceptor.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServer.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServerGrpc.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServerSpi.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/CSMMetrics.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/DispatcherContext.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/ContainerCache.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/HddsVolumeUtil.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/AbstractFuture.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolume.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeChecker.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/RoundRobinVolumeChoosingPolicy.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/ThrottledAsyncChecker.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/TimeoutFuture.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeIOStats.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfo.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeSet.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeUsage.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueBlockIterator.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerData.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/TarContainerPacker.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/BlockUtils.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/ChunkUtils.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerLocationUtil.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/SmallFileUtils.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/BlockManagerImpl.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/interfaces/BlockManager.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/interfaces/ChunkManager.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/interfaces/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/statemachine/background/BlockDeletingService.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/statemachine/background/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerController.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerReader.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerDownloader.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerReplicationSource.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerReplicator.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerStreamingOutput.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/DownloadAndImportReplicator.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationClient.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationService.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/OnDemandContainerReplicationSource.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ReplicationSupervisor.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ReplicationTask.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SimpleContainerDownloader.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/StorageContainerDatanodeProtocol.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/StorageContainerNodeProtocol.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/VersionResponse.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CloseContainerCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CommandForDatanode.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CommandStatus.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/DeleteBlockCommandStatus.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/DeleteBlocksCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/DeleteContainerCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/RegisteredCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/ReplicateContainerCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/ReregisterCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/SCMCommand.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocolPB/StorageContainerDatanodeProtocolClientSideTranslatorPB.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocolPB/StorageContainerDatanodeProtocolPB.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocolPB/StorageContainerDatanodeProtocolServerSideTranslatorPB.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocolPB/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/proto/StorageContainerDatanodeProtocol.proto
delete mode 100644 hadoop-hdds/container-service/src/main/resources/META-INF/services/com.sun.jersey.spi.container.ContainerProvider
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/TestHddsDatanodeService.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/ContainerTestUtils.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/SCMTestUtils.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/ScmTestMock.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestChunkLayOutVersion.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestDatanodeLayOutVersion.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestDatanodeStateMachine.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestKeyValueContainerData.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/helpers/TestDatanodeVersionFile.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestContainerDataYaml.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestContainerSet.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestHddsDispatcher.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/interfaces/TestHandler.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/report/TestReportManager.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/report/TestReportPublisher.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/report/TestReportPublisherFactory.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/report/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerCommandHandler.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTask.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestHddsVolume.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestHddsVolumeChecker.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestRoundRobinVolumeChoosingPolicy.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestVolumeSet.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestVolumeSetDiskChecks.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestBlockManagerImpl.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestChunkManagerImpl.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueBlockIterator.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainer.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerMarkUnhealthy.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandlerWithUnhealthyContainer.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestTarContainerPacker.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainer.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/testutils/BlockDeletingServiceTestImpl.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/testutils/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/resources/additionalfields.container
delete mode 100644 hadoop-hdds/container-service/src/test/resources/incorrect.checksum.container
delete mode 100644 hadoop-hdds/container-service/src/test/resources/incorrect.container
delete mode 100644 hadoop-hdds/container-service/src/test/resources/log4j.properties
delete mode 100644 hadoop-hdds/docs/README.md
delete mode 100644 hadoop-hdds/docs/archetypes/default.md
delete mode 100644 hadoop-hdds/docs/config.yaml
delete mode 100644 hadoop-hdds/docs/content/AuditParser.md
delete mode 100644 hadoop-hdds/docs/content/BucketCommands.md
delete mode 100644 hadoop-hdds/docs/content/BuildingSources.md
delete mode 100644 hadoop-hdds/docs/content/CommandShell.md
delete mode 100644 hadoop-hdds/docs/content/Concepts.md
delete mode 100644 hadoop-hdds/docs/content/Dozone.md
delete mode 100644 hadoop-hdds/docs/content/Freon.md
delete mode 100644 hadoop-hdds/docs/content/Hdds.md
delete mode 100644 hadoop-hdds/docs/content/JavaApi.md
delete mode 100644 hadoop-hdds/docs/content/KeyCommands.md
delete mode 100644 hadoop-hdds/docs/content/OzoneFS.md
delete mode 100644 hadoop-hdds/docs/content/OzoneManager.md
delete mode 100644 hadoop-hdds/docs/content/Prometheus.md
delete mode 100644 hadoop-hdds/docs/content/RealCluster.md
delete mode 100644 hadoop-hdds/docs/content/Rest.md
delete mode 100644 hadoop-hdds/docs/content/RunningViaDocker.md
delete mode 100644 hadoop-hdds/docs/content/RunningWithHDFS.md
delete mode 100644 hadoop-hdds/docs/content/S3.md
delete mode 100644 hadoop-hdds/docs/content/S3Commands.md
delete mode 100644 hadoop-hdds/docs/content/SCMCLI.md
delete mode 100644 hadoop-hdds/docs/content/Settings.md
delete mode 100644 hadoop-hdds/docs/content/VolumeCommands.md
delete mode 100644 hadoop-hdds/docs/content/_index.md
delete mode 100644 hadoop-hdds/docs/content/genconf.md
delete mode 100755 hadoop-hdds/docs/dev-support/bin/generate-site.sh
delete mode 100644 hadoop-hdds/docs/pom.xml
delete mode 100644 hadoop-hdds/docs/static/NOTES.md
delete mode 100644 hadoop-hdds/docs/static/OzoneOverview.png
delete mode 100644 hadoop-hdds/docs/static/OzoneOverview.svg
delete mode 100644 hadoop-hdds/docs/static/SCMBlockDiagram.png
delete mode 100644 hadoop-hdds/docs/static/ozone-logo.png
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/index.html
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/footer.html
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/navbar.html
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/css/bootstrap-theme.min.css
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/css/bootstrap-theme.min.css.map
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/css/bootstrap.min.css
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/css/bootstrap.min.css.map
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/css/ozonedoc.css
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.eot
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.svg
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.ttf
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.woff
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.woff2
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/js/bootstrap.min.js
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/js/jquery.min.js
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/js/ozonedoc.js
delete mode 100644 hadoop-hdds/docs/themes/ozonedoc/theme.toml
delete mode 100644 hadoop-hdds/framework/README.md
delete mode 100644 hadoop-hdds/framework/pom.xml
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/BaseHttpServer.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/PrometheusMetricsSink.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/PrometheusServlet.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServerUtils.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServiceRuntimeInfo.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/ServiceRuntimeInfoImpl.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/Event.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventExecutor.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventHandler.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventPublisher.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventWatcher.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventWatcherMetrics.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/IdentifiableEventPayload.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/SingleThreadExecutor.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/TypedEvent.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/package-info.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/package-info.java
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/datanode/dn.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/angular-1.6.4.min.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/angular-nvd3-1.0.9.min.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/angular-route-1.6.4.min.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/d3-3.5.17.min.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/dfs-dust.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/nvd3-1.8.5.min.css
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/nvd3-1.8.5.min.css.map
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/nvd3-1.8.5.min.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/nvd3-1.8.5.min.js.map
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/ozone.css
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/ozone.js
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/templates/config.html
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/templates/jvm.html
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/templates/menu.html
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/templates/overview.html
delete mode 100644 hadoop-hdds/framework/src/main/resources/webapps/static/templates/rpc-metrics.html
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/TestBaseHttpServer.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/TestPrometheusMetricsSink.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/events/EventHandlerStub.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/events/TestEventQueue.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/events/TestEventQueueChain.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/events/TestEventWatcher.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/events/package-info.java
delete mode 100644 hadoop-hdds/framework/src/test/resources/ozone-site.xml
delete mode 100644 hadoop-hdds/pom.xml
delete mode 100644 hadoop-hdds/server-scm/pom.xml
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ScmUtils.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/BlockManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/BlockManagerImpl.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/BlockmanagerMXBean.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DatanodeDeletedBlockTransactions.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLog.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogImpl.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteStatusList.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/SCMBlockDeletingService.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModeExitRule.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModePrecheck.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModeRestrictedOps.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ContainerChillModeRule.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/DataNodeChillModeRule.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/PipelineChillModeRule.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/Precheck.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/SCMChillModeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/command/CommandStatusReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/command/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/CloseContainerEventHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerActionsHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplica.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerStateManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/DeleteContainerCommandWatcher.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/IncrementalContainerReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReportHandlerHelper.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/closer/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/ContainerPlacementPolicy.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMCommonPolicy.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementCapacity.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRandom.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/ContainerStat.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/DatanodeMetric.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/LongMetric.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/NodeStat.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/SCMMetrics.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/SCMNodeMetric.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/SCMNodeStat.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationActivityStatus.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationActivityStatusMXBean.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationCommandWatcher.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationQueue.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationRequest.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/ContainerAttribute.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/ContainerQueryKey.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/ContainerState.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/ContainerStateMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/events/SCMEvents.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/events/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/exceptions/SCMException.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/exceptions/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/DeletedBlocksTransactionCodec.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/LongCodec.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/SCMMetadataStore.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/SCMMetadataStoreRDBImpl.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/CommandQueue.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DeadNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NewNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeManagerMXBean.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeStateManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NonHealthyToHealthyNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeStorageStatMXBean.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeStorageStatMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/StaleNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/StorageReportResult.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2ContainerMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2ObjectsMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2PipelineMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/NodeAlreadyExistsException.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/NodeException.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/NodeNotFoundException.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/NodeStateMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/ReportResult.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineActionHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineFactory.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineProvider.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineUtils.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SimplePipelineProvider.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ratis/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMBlockProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMConfigurator.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMDatanodeHeartbeatDispatcher.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMDatanodeProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMMXBean.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMSecurityProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorageConfig.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManagerHttpServer.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/protocol/commands/RetriableDatanodeEventWatcher.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/protocol/commands/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/resources/webapps/scm/index.html
delete mode 100644 hadoop-hdds/server-scm/src/main/resources/webapps/scm/main.html
delete mode 100644 hadoop-hdds/server-scm/src/main/resources/webapps/scm/scm-overview.html
delete mode 100644 hadoop-hdds/server-scm/src/main/resources/webapps/scm/scm.js
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/HddsServerUtilTest.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/HddsTestUtils.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/HddsWhiteboxTestUtils.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestHddsServerUtils.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestStorageContainerManagerHttpServer.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestUtils.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/block/TestBlockManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/block/TestDeletedBlockLog.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/block/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/chillmode/TestSCMChillModeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/command/TestCommandStatusReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/command/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/MockNodeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestCloseContainerEventHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerActionsHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHelper.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerStateManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestIncrementalContainerReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestSCMContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/closer/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/TestSCMContainerPlacementCapacity.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/TestSCMContainerPlacementRandom.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationActivityStatus.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationQueue.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/states/TestContainerAttribute.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/states/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestContainerPlacement.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestNodeReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeStorageStatMap.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/states/TestNode2ContainerMap.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/states/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMClientProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMDatanodeHeartbeatDispatcher.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMSecurityProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/common/TestEndPoint.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/common/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/placement/TestContainerPlacement.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/placement/TestDatanodeMetrics.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/placement/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/replication/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/testutils/ReplicationNodeManagerMock.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/testutils/package-info.java
delete mode 100644 hadoop-hdds/tools/pom.xml
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ChillModeCheckSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ChillModeCommands.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ChillModeExitSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/SCMCLI.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/CloseSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/CreateSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/DeleteSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/InfoSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ListSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/package-info.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/package-info.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/ClosePipelineSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/ListPipelinesSubcommand.java
delete mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/package-info.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DeadNodeDetector.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/DfsPathCapabilities.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/CorruptMetaHeaderException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/SSLConnectionConfigurator.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/NameSpaceQuotaParam.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/StorageSpaceQuotaParam.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/StorageTypeParam.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/conf/httpfs-signature.secret
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerWebServerWithRandomSecret.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-examples/CMakeLists.txt
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-examples/README.md
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-examples/libhdfs_read.c
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-examples/libhdfs_write.c
create mode 100755 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-examples/test-libhdfs.sh
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_htable.c
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_read.c
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_write.c
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/common/htable.c
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/common/htable.h
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jclasses.c
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jclasses.h
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/conf/hdfs-rbf-site.xml
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/protocolPB/RouterAdminProtocol.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/protocolPB/RouterPolicyProvider.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/FederationMetrics.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/NullStateStoreMetrics.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RouterMBean.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/RouterResolveException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionNullException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/MountTableRefresherService.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/MountTableRefresherThread.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/NoNamenodesAvailableException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RemoteResult.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterCacheAdmin.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterSnapshot.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterStoragePolicy.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterUserProtocol.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/RouterSecurityManager.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/package-info.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/ZKDelegationTokenSecretManagerImpl.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/GetDestinationRequest.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/GetDestinationResponse.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/RefreshMountTableEntriesRequest.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/RefreshMountTableEntriesResponse.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/GetDestinationRequestPBImpl.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/GetDestinationResponsePBImpl.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/RefreshMountTableEntriesRequestPBImpl.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/protocol/impl/pb/RefreshMountTableEntriesResponsePBImpl.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/SecurityConfUtil.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractAppendSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractConcatSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractCreateSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractDelegationToken.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractDeleteSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractGetFileStatusSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractMkdirSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractOpenSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractRenameSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractRootDirectorySecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractSeekSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/fs/contract/router/TestRouterHDFSContractSetTimesSecure.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/MockNamenode.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/metrics/TestFederationMetrics.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/metrics/TestRBFMetrics.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestFederationUtil.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAdminGenericRefresh.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterFaultTolerant.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterMissingFolderMulti.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterMountTableCacheRefresh.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterNamenodeWebScheme.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterPolicyProvider.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRPCMultipleDestinationMountTableResolver.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpcSingleNS.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpcStoragePolicySatisfier.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterUserMappings.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterWithSecureStartup.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/security/MockDelegationTokenSecretManager.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/security/TestRouterHttpDelegationToken.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/security/TestRouterSecurityManager.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_2.10.0.xml
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_3.1.3.xml
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_3.2.1.xml
mode change 100644 => 100755 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
mode change 100644 => 100755 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/InvalidateBlocks.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/HostRestrictingAuthorizationFilter.java
mode change 100644 => 100755 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskFileCorruptException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSCachingGetSpaceUsed.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/AddBlockPoolException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/CacheStats.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MappableBlockLoaderFactory.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MemoryMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MemoryMappedBlock.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/NativePmemMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/NativePmemMappedBlock.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemMappedBlock.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemVolumeManager.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaCachingGetSpaceUsed.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/HostRestrictingAuthorizationFilterHandler.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/package-info.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/DefaultAuditLogger.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/ConstEnumCounters.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilterInitializer.java
delete mode 100755 hadoop-hdfs-project/hadoop-hdfs/src/main/native/tests/test-libhdfs.sh
mode change 100644 => 100755 hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap-theme.css
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap-theme.css.map
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap-theme.min.css
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap-theme.min.css.map
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap.css
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap.css.map
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap.min.css
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/css/bootstrap.min.css.map
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/js/bootstrap.js
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7/js/bootstrap.min.js
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/css/bootstrap-editable.css (100%)
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap-theme.css
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap-theme.css.map
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap-theme.min.css
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap-theme.min.css.map
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap.css
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap.css.map
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap.min.css
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/css/bootstrap.min.css.map
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/fonts/glyphicons-halflings-regular.eot (100%)
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/fonts/glyphicons-halflings-regular.svg (100%)
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/fonts/glyphicons-halflings-regular.ttf (100%)
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/fonts/glyphicons-halflings-regular.woff (100%)
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/fonts/glyphicons-halflings-regular.woff2 (100%)
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/js/bootstrap-editable.min.js (100%)
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap.js
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap.min.js
rename hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/{bootstrap-3.3.7 => bootstrap-3.4.1}/js/npm.js (100%)
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery-3.3.1.min.js
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery-3.4.1.min.js
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/contract/hdfs/TestHDFSContractUnbuffer.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestByteBufferPread.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSConfigKeys.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDeadNodeDetection.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMultipleNNPortQOP.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestStripedFileAppend.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/TestSecureAliasMap.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerService.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockPlacementStatusDefault.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRedundancyMonitor.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestHostRestrictingAuthorizationFilter.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCorruptMetadataFile.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestAddBlockPoolException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestCacheByPmemMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetCache.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestReplicaCachingGetSpaceUsed.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/web/TestHostRestrictingAuthorizationFilterHandler.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestQuotaCounts.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestRedudantBlocks.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestRefreshNamenodeReplicationConfig.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestAddBlockTailing.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandbyWithInProgressTailing.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestAdminHelper.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForStoragePolicy.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testStoragePolicy.csv
delete mode 100644 hadoop-mapreduce-project/.eclipse.templates/.launches/AllMapredTests.launch
delete mode 100644 hadoop-mapreduce-project/.eclipse.templates/.launches/JobTracker.launch
delete mode 100644 hadoop-mapreduce-project/.eclipse.templates/.launches/SpecificTestTemplate.launch
delete mode 100644 hadoop-mapreduce-project/.eclipse.templates/.launches/TaskTracker.launch
delete mode 100644 hadoop-mapreduce-project/.eclipse.templates/README.txt
create mode 100644 hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Common_2.10.0.xml
create mode 100644 hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Core_2.10.0.xml
create mode 100644 hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_JobClient_2.10.0.xml
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/SimpleExponentialTaskRuntimeEstimator.java
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/forecast/SimpleExponentialSmoothing.java
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/speculate/forecast/TestSimpleExponentialForecast.java
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestSpeculativeExecOnCluster.java
delete mode 100644 hadoop-ozone/Jenkinsfile
delete mode 100644 hadoop-ozone/client/pom.xml
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/BucketArgs.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/ObjectStore.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneClient.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneClientException.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneClientFactory.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneClientInvocationHandler.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneClientUtils.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneKey.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneKeyDetails.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneKeyLocation.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneMultipartUploadPartListParts.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneVolume.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/VolumeArgs.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/BlockOutputStreamEntry.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyInputStream.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyOutputStream.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/OzoneInputStream.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/OzoneOutputStream.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/package-info.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/package-info.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/protocol/ClientProtocol.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/protocol/package-info.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/DefaultRestServerSelector.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/OzoneExceptionMapper.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/RestClient.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/RestServerSelector.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/exceptions/package-info.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/headers/package-info.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rest/package-info.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/package-info.java
delete mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/TestHddsClientUtils.java
delete mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/package-info.java
delete mode 100644 hadoop-ozone/common/dev-support/findbugsExcludeFile.xml
delete mode 100644 hadoop-ozone/common/pom.xml
delete mode 100755 hadoop-ozone/common/src/main/bin/ozone
delete mode 100755 hadoop-ozone/common/src/main/bin/ozone-config.sh
delete mode 100755 hadoop-ozone/common/src/main/bin/start-ozone.sh
delete mode 100755 hadoop-ozone/common/src/main/bin/stop-ozone.sh
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/hdds/protocol/StorageType.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/hdds/protocol/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OzoneIllegalArgumentException.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/audit/OMAction.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/audit/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/io/LengthInputStream.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/io/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/OzoneException.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/headers/Header.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/headers/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/BucketInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/KeyInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/KeyInfoDetails.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/KeyLocation.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/VolumeInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/VolumeOwner.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/client/rest/response/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/freon/OzoneGetConf.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/freon/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerLock.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmBucketInfoCodec.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmKeyInfoCodec.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmMultipartKeyInfoCodec.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmVolumeArgsCodec.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/VolumeListCodec.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/exceptions/OMException.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/exceptions/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/KeyValueUtil.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmBucketArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmBucketInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyLocationInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyLocationInfoGroup.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartCommitUploadPartInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartKeyInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadCompleteInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadList.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadListParts.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmOzoneAclMap.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmPartInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmVolumeArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OpenKeySession.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/S3SecretValue.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/ServiceInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/VolumeArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/WithMetadata.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/OzoneManagerProtocol.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/OzoneManagerSecurityProtocol.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OzoneManagerProtocolPB.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/protocolPB/OMPBHelper.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/protocolPB/OzonePBHelper.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/protocolPB/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneBlockTokenSecretManager.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneDelegationTokenSecretManager.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneDelegationTokenSelector.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretKey.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretManager.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretStore.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecurityException.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneTokenIdentifier.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/IAccessAuthorizer.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/IOzoneObj.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneAccessAuthorizer.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneAclException.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneObj.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneObjInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/RequestContext.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/OzoneVersionInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/handlers/BucketArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/handlers/KeyArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/handlers/ListArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/handlers/UserArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/handlers/VolumeArgs.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/handlers/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/request/OzoneQuota.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/request/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/BucketInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/KeyInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/KeyInfoDetails.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/KeyLocation.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/ListBuckets.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/ListKeys.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/ListVolumes.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/VolumeInfo.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/VolumeOwner.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/response/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/utils/OzoneUtils.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/web/utils/package-info.java
delete mode 100644 hadoop-ozone/common/src/main/proto/OzoneManagerProtocol.proto
delete mode 100644 hadoop-ozone/common/src/main/resources/ozone-version-info.properties
delete mode 100644 hadoop-ozone/common/src/main/shellprofile.d/hadoop-ozone.sh
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/TestOmUtils.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmMultipartKeyInfoCodec.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/codec/package-info.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/exceptions/TestResultCodes.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/helpers/TestOmBucketInfo.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/helpers/TestOmKeyInfo.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/helpers/package-info.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestOzoneBlockTokenSecretManager.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestOzoneDelegationTokenSecretManager.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/acl/TestOzoneObjInfo.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/web/TestBucketInfo.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/web/TestQuota.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/web/TestUtils.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/web/TestVolumeStructs.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/web/package-info.java
delete mode 100644 hadoop-ozone/datanode/pom.xml
delete mode 100755 hadoop-ozone/dev-support/checks/acceptance.sh
delete mode 100755 hadoop-ozone/dev-support/checks/author.sh
delete mode 100755 hadoop-ozone/dev-support/checks/build.sh
delete mode 100755 hadoop-ozone/dev-support/checks/checkstyle.sh
delete mode 100755 hadoop-ozone/dev-support/checks/findbugs.sh
delete mode 100755 hadoop-ozone/dev-support/checks/isolation.sh
delete mode 100755 hadoop-ozone/dev-support/checks/rat.sh
delete mode 100755 hadoop-ozone/dev-support/checks/unit.sh
delete mode 100644 hadoop-ozone/dev-support/docker/Dockerfile
delete mode 100644 hadoop-ozone/dist/Dockerfile
delete mode 100644 hadoop-ozone/dist/README.md
delete mode 100755 hadoop-ozone/dist/dev-support/bin/dist-layout-stitching
delete mode 100755 hadoop-ozone/dist/dev-support/bin/dist-tar-stitching
delete mode 100644 hadoop-ozone/dist/pom.xml
delete mode 100644 hadoop-ozone/dist/skaffold.yaml
delete mode 100644 hadoop-ozone/dist/src/main/blockade/README.md
delete mode 100644 hadoop-ozone/dist/src/main/blockade/blockadeUtils/__init__.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/blockadeUtils/blockade.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/clusterUtils/cluster_utils.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/conftest.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_client_failure.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_datanode_isolation.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_flaky.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure_three_nodes_isolate.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure_two_nodes.py
delete mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_scm_isolation.py
delete mode 100644 hadoop-ozone/dist/src/main/compose/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/grafana/conf/grafana.ini
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone - Object Metrics.json
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone - RPC Metrics.json
delete mode 100755 hadoop-ozone/dist/src/main/compose/common/grafana/provisioning/dashboards/dashboards.yml
delete mode 100755 hadoop-ozone/dist/src/main/compose/common/grafana/provisioning/datasources/datasources.yml
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/prometheus/prometheus.yml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozone/.env
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozone/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozoneblockade/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozoneblockade/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonefs/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonefs/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/.env
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozones3/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozones3/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/authorized_keys
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/environment
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/id_rsa
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/id_rsa.pub
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/Dockerfile
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/docker-config
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonescripts/ps.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonescripts/start.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonescripts/stop.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/.env
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/Dockerfile-krb5
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/kadm5.acl
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/krb5.conf
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/launcher.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/Dockerfile
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/build.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/envtoconf.py
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/krb5.conf
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/starter.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/transformation.py
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonetrace/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/ozonetrace/docker-config
delete mode 100644 hadoop-ozone/dist/src/main/conf/dn-audit-log4j2.properties
delete mode 100644 hadoop-ozone/dist/src/main/conf/log4j.properties
delete mode 100644 hadoop-ozone/dist/src/main/conf/om-audit-log4j2.properties
delete mode 100644 hadoop-ozone/dist/src/main/conf/ozone-site.xml
delete mode 100644 hadoop-ozone/dist/src/main/conf/scm-audit-log4j2.properties
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/config-configmap.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/datanode-public-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/datanode-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/datanode-statefulset.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/om-public-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/om-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/om-statefulset.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/s3g-public-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/s3g-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/s3g-statefulset.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/scm-public-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/scm-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/scm-statefulset.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/configmap.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/deployment.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/prometheus-public-serivce.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/role.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/rolebindng.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/scm-public-service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/service-account.yaml
delete mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/service.yaml
delete mode 100644 hadoop-ozone/dist/src/main/ozone/README.txt
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/.env
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/README.md
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/auditparser/auditparser.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/basic/basic.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/basic/ozone-shell.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/commonlib.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/ozonefs/ozonefs.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/README.md
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/__init__.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/awss3.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/buckethead.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/bucketlist.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/commonawslib.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectcopy.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectdelete.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectmultidelete.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectputget.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/webui.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/security/ozone-secure.robot
delete mode 100755 hadoop-ozone/dist/src/main/smoketest/test.sh
delete mode 100644 hadoop-ozone/integration-test/pom.xml
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerStateManagerIntegration.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestNode2PipelineMap.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestNodeFailure.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineClose.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineStateManager.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineProvider.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineUtils.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSCMPipelineManager.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSCMRestart.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSimplePipelineProvider.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneCluster.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneHAClusterImpl.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/OzoneTestUtils.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/RatisTestHelper.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestContainerOperations.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestContainerStateMachineIdempotency.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestMiniOzoneCluster.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestOzoneConfigurationFields.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestOzoneHelper.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestSecureOzoneCluster.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManagerHelper.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/CertificateClientTestImpl.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestBCSID.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestCloseContainerHandlingByClient.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachine.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineFailures.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestFailureHandlingByClient.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestHybridPipelineOnDatanode.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClient.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientAbstract.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithRatis.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestReadRetries.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestSecureOzoneRpcClient.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/TestContainerReplication.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/TestBlockDeletingService.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/helpers/TestBlockData.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestContainerDeletionChoosingPolicy.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestContainerPersistence.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestBlockDeletion.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerByPipeline.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerHandler.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestDeleteContainerHandler.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/TestCSMMetrics.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/metrics/TestContainerMetrics.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestRatisManager.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestSecureOzoneContainer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestContainerReportWithKeys.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestMultipleContainerReadWrite.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmAcls.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmBlockVersioning.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmInit.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmMetrics.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManager.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHA.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerRestInterface.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestScmChillMode.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/ozShell/TestOzoneDatanodeShell.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/ozShell/TestOzoneShell.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestAllocateContainer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestContainerSmallFile.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestGetCommittedBlockLengthAndPutKey.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestSCMMXBean.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestSCMNodeManagerMXBean.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestXceiverClientManager.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestXceiverClientMetrics.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/node/TestQueryNode.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/TestOzoneRestWithMiniCluster.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/TestOzoneVolumes.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/TestOzoneWebAccess.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestBuckets.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestBucketsRatis.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestKeys.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestKeysRatis.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestOzoneClient.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestVolume.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestVolumeRatis.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/log4j.properties
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/ca.crt
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/ca.key
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.crt
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.csr
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.key
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.pem
delete mode 100755 hadoop-ozone/integration-test/src/test/resources/ssl/generate.sh
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.crt
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.csr
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.key
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.pem
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/webapps/ozoneManager/.gitkeep
delete mode 100644 hadoop-ozone/integration-test/src/test/resources/webapps/scm/.gitkeep
delete mode 100644 hadoop-ozone/objectstore-service/pom.xml
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/hdfs/server/datanode/ObjectStoreHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/hdfs/server/datanode/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/OzoneRestUtils.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/ObjectStoreApplication.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/OzoneHddsDatanodeService.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/exceptions/ErrorTable.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/exceptions/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/BucketHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/BucketProcessTemplate.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/KeyHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/KeyProcessTemplate.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/ServiceFilter.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/StorageHandlerBuilder.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/UserHandlerBuilder.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/VolumeHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/VolumeProcessTemplate.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/handlers/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/Accounting.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/Bucket.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/Keys.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/StorageHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/UserAuth.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/Volume.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/localstorage/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/messages/LengthInputStreamMessageBodyWriter.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/messages/StringMessageBodyWriter.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/messages/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/CloseableCleanupListener.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/ObjectStoreChannelHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/ObjectStoreJerseyContainer.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/ObjectStoreJerseyContainerProvider.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/ObjectStoreRestHttpServer.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/ObjectStoreURLDispatcher.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/RequestContentObjectStoreChannelHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/RequestDispatchObjectStoreChannelHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/netty/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/storage/DistributedStorageHandler.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/storage/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/userauth/Simple.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/userauth/package-info.java
delete mode 100644 hadoop-ozone/objectstore-service/src/main/resources/META-INF/services/com.sun.jersey.spi.container.ContainerProvider
delete mode 100644 hadoop-ozone/objectstore-service/src/test/java/org/apache/hadoop/ozone/web/TestErrorCode.java
delete mode 100644 hadoop-ozone/objectstore-service/src/test/java/org/apache/hadoop/ozone/web/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/pom.xml
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/BucketManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/BucketManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMMXBean.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMMetrics.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMNodeDetails.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMStorage.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetricsInfo.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OpenKeyCleanupService.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerHttpServer.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3SecretManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3SecretManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ServiceListJSONServlet.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OMRatisHelper.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerRatisClient.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerRatisServer.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerProtocolServerSideTranslatorPB.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/Handler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/OzoneAddress.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/Shell.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/BucketCommands.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/CreateBucketHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/DeleteBucketHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/InfoBucketHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/ListBucketHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/S3BucketMapping.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/UpdateBucketHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/DeleteKeyHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/GetKeyHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/InfoKeyHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/KeyCommands.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/ListKeyHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/PutKeyHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/RenameKeyHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/GetS3SecretHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/S3Commands.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/CancelTokenHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/GetTokenHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/PrintTokenHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/RenewTokenHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/TokenCommands.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/CreateVolumeHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/DeleteVolumeHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/InfoVolumeHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/ListVolumeHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/UpdateVolumeHandler.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/VolumeCommands.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/resources/webapps/ozoneManager/index.html
delete mode 100644 hadoop-ozone/ozone-manager/src/main/resources/webapps/ozoneManager/main.css
delete mode 100644 hadoop-ozone/ozone-manager/src/main/resources/webapps/ozoneManager/main.html
delete mode 100644 hadoop-ozone/ozone-manager/src/main/resources/webapps/ozoneManager/om-metrics.html
delete mode 100644 hadoop-ozone/ozone-manager/src/main/resources/webapps/ozoneManager/ozoneManager.js
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ScmBlockLocationTestIngClient.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestBucketManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestChunkStreams.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestKeyDeletingService.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestKeyManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHttpServer.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerLock.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestS3BucketManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerRatisServer.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/TestOzoneManagerBlockToken.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/TestOzoneTokenIdentifier.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell/TestOzoneAddress.java
delete mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell/package-info.java
delete mode 100644 hadoop-ozone/ozonefs-lib-legacy/pom.xml
delete mode 100644 hadoop-ozone/ozonefs-lib-legacy/src/main/resources/ozonefs.txt
delete mode 100644 hadoop-ozone/ozonefs-lib/pom.xml
delete mode 100644 hadoop-ozone/ozonefs/pom.xml
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicKeyInfo.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/Constants.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/FilteredClassLoader.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzFs.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientAdapter.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientAdapterFactory.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientAdapterImpl.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSInputStream.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSOutputStream.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSStorageStatistics.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFileSystem.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/Statistic.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/package-info.java
delete mode 100644 hadoop-ozone/ozonefs/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSInputStream.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfaces.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFsRenameDir.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractCreate.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractDelete.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractDistCp.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractGetFileStatus.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractMkdir.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractOpen.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractRename.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractRootDir.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractSeek.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/contract/OzoneContract.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/package-info.java
delete mode 100644 hadoop-ozone/ozonefs/src/test/resources/contract/ozone.xml
delete mode 100644 hadoop-ozone/ozonefs/src/test/resources/log4j.properties
delete mode 100644 hadoop-ozone/pom.xml
delete mode 100644 hadoop-ozone/s3gateway/pom.xml
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/CommonHeadersContainerResponseFilter.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/Gateway.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/GatewayApplication.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/HeaderPreprocessor.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneClientProducer.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneConfigurationHolder.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/RequestIdentifier.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/S3GatewayConfigKeys.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/S3GatewayHttpServer.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/SignedChunksInputStream.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/VirtualHostStyleFilter.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/BucketMetadata.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/CommonPrefix.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/IsoDateAdapter.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/KeyMetadata.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadRequest.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CopyObjectResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/EndpointBase.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListBucketResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListObjectResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListPartsResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultiDeleteRequest.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultiDeleteRequestUnmarshaller.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultiDeleteResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultipartUploadInitiateResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/RootEndpoint.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/XmlNamespaceFilter.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/OS3Exception.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/OS3ExceptionMapper.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/S3ErrorTable.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthenticationHeaderParser.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthorizationHeaderV2.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthorizationHeaderV4.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/Credential.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io/S3WrapperInputStream.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/ContinueToken.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/RFC1123Util.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/RangeHeader.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/RangeHeaderParserUtil.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Consts.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3StorageType.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/resources/META-INF/beans.xml
delete mode 100644 hadoop-ozone/s3gateway/src/main/resources/browser.html
delete mode 100644 hadoop-ozone/s3gateway/src/main/resources/webapps/s3gateway/WEB-INF/beans.xml
delete mode 100644 hadoop-ozone/s3gateway/src/main/resources/webapps/s3gateway/WEB-INF/web.xml
delete mode 100755 hadoop-ozone/s3gateway/src/main/resources/webapps/static/images/ozone.ico
delete mode 100644 hadoop-ozone/s3gateway/src/main/resources/webapps/static/index.html
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/ObjectStoreStub.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneBucketStub.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneClientStub.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneOutputStreamStub.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneVolumeStub.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestSignedChunksInputStream.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestVirtualHostStyleFilter.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketDelete.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketGet.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketHead.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketResponse.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestInitiateMultipartUpload.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestListParts.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestMultiDeleteRequestUnmarshaller.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestMultipartUploadComplete.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectDelete.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectGet.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectHead.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectMultiDelete.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectPut.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestPartUpload.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestRootList.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/exception/TestOS3Exception.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/exception/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/header/TestAuthorizationHeaderV2.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/header/TestAuthorizationHeaderV4.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/package-info.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestContinueToken.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestRFC1123Util.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestRangeHeaderParserUtil.java
delete mode 100644 hadoop-ozone/tools/dev-support/findbugsExcludeFile.xml
delete mode 100644 hadoop-ozone/tools/pom.xml
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/AuditParser.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/DatabaseHelper.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/ParserConsts.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/LoadCommandHandler.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/QueryCommandHandler.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/TemplateCommandHandler.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/model/AuditEntry.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/model/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/Freon.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/ProgressBar.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/RandomKeyGenerator.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/fsck/BlockIdDetails.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/fsck/ContainerMapper.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/fsck/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genconf/GenerateOzoneRequiredConfigurations.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genconf/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkBlockManager.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkContainerStateMap.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkDatanodeDispatcher.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreReads.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreWrites.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOMKeyAllocation.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkRocksDbStore.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/Genesis.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisMemoryProfiler.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/GenesisUtil.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/scm/cli/SQLCLI.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/scm/cli/package-info.java
delete mode 100644 hadoop-ozone/tools/src/main/resources/commands.properties
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/audit/parser/TestAuditParser.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/audit/parser/package-info.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestDataValidate.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithDatanodeFastRestart.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithDatanodeRestart.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithPipelineDestroy.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestProgressBar.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestRandomKeyGenerator.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/package-info.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/fsck/TestContainerMapper.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/fsck/package-info.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/genconf/TestGenerateOzoneRequiredConfigurations.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/genconf/package-info.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/om/TestOmSQLCli.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/om/package-info.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/scm/TestContainerSQLCli.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/scm/package-info.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/test/OzoneTestDriver.java
delete mode 100644 hadoop-ozone/tools/src/test/resources/commands.properties
delete mode 100644 hadoop-ozone/tools/src/test/resources/testaudit.log
create mode 100755 hadoop-submarine/dev-support/checks/rat.sh
create mode 100644 hadoop-submarine/hadoop-submarine-all/pom.xml
create mode 100644 hadoop-submarine/hadoop-submarine-core/README.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/pom.xml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/pytorch/base/ubuntu-16.04/Dockerfile.gpu.pytorch_latest
create mode 100755 hadoop-submarine/hadoop-submarine-core/src/main/docker/pytorch/build-all.sh
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/pytorch/with-cifar10-models/cifar10_tutorial.py
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/pytorch/with-cifar10-models/ubuntu-16.04/Dockerfile.gpu.pytorch_latest
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/base/ubuntu-16.04/Dockerfile.cpu.tf_1.13.1
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/base/ubuntu-16.04/Dockerfile.gpu.tf_1.13.1
create mode 100755 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/build-all.sh
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/Dockerfile.cpu.tf_1.13.1
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/Dockerfile.gpu.tf_1.13.1
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0 => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1}/README.md (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1/cifar10.py
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1/cifar10_main.py
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0 => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1}/cifar10_model.py (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1/cifar10_utils.py
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1/generate_cifar10_tfrecords.py
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0 => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.13.1}/model_base.py (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow}/zeppelin-notebook-example/Dockerfile.gpu (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow}/zeppelin-notebook-example/run_container.sh (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow}/zeppelin-notebook-example/shiro.ini (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker => hadoop-submarine/hadoop-submarine-core/src/main/docker/tensorflow}/zeppelin-notebook-example/zeppelin-site.xml (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/AbstractCli.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/Cli.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/CliConstants.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/CliUtils.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/Command.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/KillJobCli.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/ShowJobCli.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/BaseParameters.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/ConfigType.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/KillJobParameters.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/Localization.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/ParametersHolder.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/Quicklink.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/RunParameters.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/ShowJobParameters.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/package-info.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/runjob/PyTorchRunJobParameters.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/runjob/RunJobParameters.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/runjob/TensorFlowRunJobParameters.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/runjob/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/Configs.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/PsRole.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/Role.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/Roles.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/Scheduling.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/Security.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/Spec.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/TensorBoard.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/WorkerRole.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/YamlConfigFile.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/YamlParseException.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/yaml/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/Framework.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/RoleParameters.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/RunJobCli.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/package-info.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/ClientContext.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/Envs.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/api/JobComponentStatus.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/api/JobState.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/api/JobStatus.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/api/PyTorchRole.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/api/Role.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/api/Runtime.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/api/TensorFlowRole.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/conf/SubmarineConfiguration.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/conf/SubmarineLogs.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/exception/SubmarineException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/exception/SubmarineRuntimeException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/fs/DefaultRemoteDirectoryManager.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/common/fs/RemoteDirectoryManager.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/resource/ResourceUtils.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/resource/UnitsConversionUtil.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/common/resource/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/RuntimeFactory.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/common/FSBasedSubmarineStorageImpl.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/common/JobMonitor.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/common/JobSubmitter.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/common/StorageKeyConstants.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/common/SubmarineStorage.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/Examples.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/HowToInstall.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/Index.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/InstallationGuide.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/InstallationGuideChineseVersion.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/QuickStart.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/RunningDistributedCifar10TFJobs.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/RunningSingleNodeCifar10PTJobs.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/TestAndTroubleshooting.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/TonYRuntimeGuide.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/WriteDockerfilePT.md
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/site/markdown/WriteDockerfileTF.md
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/site/resources/css/site.css (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/site/resources/images/job-logs-ui.png (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/site/resources/images/multiple-tensorboard-jobs.png (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/site/resources/images/submarine-installer.gif (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/site/resources/images/tensorboard-service.png (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/site/site.xml (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/TestKillJobCliParsing.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/TestShowJobCliParsing.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/YamlConfigTestUtils.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/TestRunJobCliParsingCommon.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/TestRunJobCliParsingCommonYaml.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/TestRunJobCliParsingParameterized.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/pytorch/TestRunJobCliParsingPyTorch.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/pytorch/TestRunJobCliParsingPyTorchYaml.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/tensorflow/TestRunJobCliParsingTensorFlow.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/tensorflow/TestRunJobCliParsingTensorFlowYaml.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/runjob/tensorflow/TestRunJobCliParsingTensorFlowYamlStandalone.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/common/MockClientContext.java
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/java/org/apache/hadoop/yarn/submarine/common/fs/MockRemoteDirectoryManager.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/common/MemorySubmarineStorage.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/common/TestFSBasedSubmarineStorage.java (100%)
copy {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/test/resources/core-site.xml (100%)
copy {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-core}/src/test/resources/hdfs-site.xml (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/empty-framework.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/invalid-framework.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/missing-configs.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/missing-framework.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/some-sections-missing.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/test-false-values.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/wrong-indentation.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-common-yaml/wrong-property-name.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/envs-are-missing.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/invalid-config-ps-section.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/invalid-config-tensorboard-section.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/security-principal-is-missing.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/valid-config-with-overrides.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/valid-config.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-pytorch-yaml/valid-gpu-config.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-tensorflow-yaml/envs-are-missing.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-tensorflow-yaml/security-principal-is-missing.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-tensorflow-yaml/tensorboard-dockerimage-is-missing.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-tensorflow-yaml/valid-config-with-overrides.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-tensorflow-yaml/valid-config.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-core/src/test/resources/runjob-tensorflow-yaml/valid-gpu-config.yaml
create mode 100644 hadoop-submarine/hadoop-submarine-dist/pom.xml
create mode 100644 hadoop-submarine/hadoop-submarine-dist/src/assembly/distribution.xml
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/README.md
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/pom.xml
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyJobMonitor.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyJobSubmitter.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyRuntimeFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/TonyUtils.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/buider/JobStatusBuilder.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/buider/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/tony/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-tony-runtime/src/test/java/TestTonyUtils.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-yarnservice-runtime}/README.md (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/pom.xml
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/AbstractComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/AbstractServiceSpec.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/FileSystemOperations.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/HadoopEnvironmentSetup.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/ServiceSpec.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/ServiceSpecFileGenerator.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/ServiceWrapper.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/WorkerComponentFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceJobMonitor.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceJobSubmitter.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-yarnservice-runtime}/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceRuntimeFactory.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceUtils.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/builder/JobComponentStatusBuilder.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/builder/JobStatusBuilder.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/AbstractLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/LaunchCommandFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/LaunchScriptBuilder.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/PyTorchLaunchCommandFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/TensorFlowLaunchCommandFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/PyTorchServiceSpec.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/command/PyTorchWorkerLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/command/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/component/PyTorchWorkerComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/component/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/TensorFlowCommons.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/TensorFlowServiceSpec.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/TensorBoardLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/TensorFlowLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/TensorFlowPsLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/TensorFlowWorkerLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/TensorBoardComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/TensorFlowPsComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/TensorFlowWorkerComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/ClassPathUtilities.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/DockerUtilities.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/EnvironmentUtilities.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/KerberosPrincipalFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/Localizer.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/SubmarineResourceUtils.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/ZipUtilities.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/main/java/org/apache/hadoop/yarn/submarine/utils/package-info.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/FileUtilitiesForTests.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/yarnservice/ParamBuilderForTest.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/yarnservice/TestYarnServiceRunJobCli.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/yarnservice/TestYarnServiceRunJobCliCommons.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/yarnservice/TestYarnServiceRunJobCliLocalization.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-yarnservice-runtime}/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/yarnservice/YarnServiceCliTestUtils.java (100%)
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/TestServiceWrapper.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/TestTFConfigGenerator.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/AbstractTFLaunchCommandTestHelper.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/command/TestLaunchCommandFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/pytorch/TestPyTorchServiceSpec.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/TestTensorBoardLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/command/TestTensorFlowLaunchCommand.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/ComponentTestCommons.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/TestTensorBoardComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/TestTensorFlowPsComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/tensorflow/component/TestTensorFlowWorkerComponent.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/utils/TestClassPathUtilities.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/utils/TestEnvironmentUtilities.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/utils/TestKerberosPrincipalFactory.java
create mode 100644 hadoop-submarine/hadoop-submarine-yarnservice-runtime/src/test/java/org/apache/hadoop/yarn/submarine/utils/TestSubmarineResourceUtils.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-yarnservice-runtime}/src/test/resources/core-site.xml (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine => hadoop-submarine/hadoop-submarine-yarnservice-runtime}/src/test/resources/hdfs-site.xml (100%)
create mode 100644 hadoop-submarine/pom.xml
delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/InternalConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/MetadataPersistenceException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/NoVersionAttributeException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/RemoteFileChangedException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ALocatedFileStatus.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/SharedInstanceCredentialProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/AwsSignerInitializer.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/SignerManager.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/DelegationTokenProvider.java
delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/Duration.java
delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/DurationInfo.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AbstractStoreOperation.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CallableSupplier.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ChangeDetectionPolicy.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ChangeTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ContextAccessors.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CopyOutcome.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/DeleteOperation.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ExecutingStoreOperation.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/InternalConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/LogExactlyOnce.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/MultiObjectDeleteSupport.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/NetworkBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OperationCallbacks.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/RenameOperation.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/StatusProbeEnum.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/StoreContext.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/package-info.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/AbstractS3GuardDynamoDBDiagnostic.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/BulkOperationState.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DelayedUpdateRenameTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DumpS3GuardDynamoTable.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DynamoDBMetadataStoreTableManager.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ITtlTimeProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/InternalIterators.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/PathOrderComparators.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ProgressiveRenameTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/PurgeS3GuardDynamoTable.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/RenameTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardFsck.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardFsckViolationHandler.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardTableAccess.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/TableDeleteTimeoutException.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractUnbuffer.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestAuthoritativePath.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestLocatedFileStatusFetcher.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFSMainOperations.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AMetadataPersistenceException.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARemoteFileChanged.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardOutOfBandOperations.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AUnbuffer.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestStreamChangeTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestCustomSigner.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestRestrictedReadAccess.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/TestSignerManager.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/AbstractITCommitMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/AbstractYarnClusterITest.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/integration/ITestS3ACommitterMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/magic/ITMagicCommitMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/TestDirectoryCommitterScale.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITDirectoryCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITPartitionCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITStagingCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITStagingCommitMRJobBadDest.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/terasort/ITestTerasortOnS3A.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestNeworkBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestPartialDeleteFailures.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/DDBCapacities.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardDDBRootOperations.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardFsck.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/TestObjectChangeDetectionAttributes.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/TestPathOrderComparators.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ThrottleTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/test/ExtraAssertions.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/mapreduce/filecache/TestS3AResourceScope.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AuthConfigurations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/AbfsPerfLoggable.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/BoundDTExtension.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/ExtensionHelper.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDtFetcher.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfssDtFetcher.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsIoUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsPerfInfo.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsPerfTracker.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/CRC64.java
delete mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/SSLSocketFactoryEx.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/resources/META-INF/services/org.apache.hadoop.security.token.DtFetcher
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCLI.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCheckAccess.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemMainOperation.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemStoreListStatusWithRange.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsCrc64.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/ClassicDelegationTokenManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/ITestAbfsDelegationTokens.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/KerberizedAbfsCluster.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/StubAbfsTokenIdentifier.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/StubDelegationTokenManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/TestCustomOauthTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/TestDTManagerLifecycle.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/WrappingTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/oauth2/RetryTestTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsPerfTracker.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier
create mode 100644 hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtilsWithCombineMode.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/pom.xml
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/bash/generate-block-lists.sh
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/java/org/apache/hadoop/tools/dynamometer/blockgenerator/BlockInfo.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/java/org/apache/hadoop/tools/dynamometer/blockgenerator/GenerateBlockImagesDriver.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/java/org/apache/hadoop/tools/dynamometer/blockgenerator/GenerateDNBlockInfosReducer.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/java/org/apache/hadoop/tools/dynamometer/blockgenerator/XMLParser.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/java/org/apache/hadoop/tools/dynamometer/blockgenerator/XMLParserMapper.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/main/java/org/apache/hadoop/tools/dynamometer/blockgenerator/package-info.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/test/java/org/apache/hadoop/tools/dynamometer/blockgenerator/TestBlockGen.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/test/java/org/apache/hadoop/tools/dynamometer/blockgenerator/TestXMLParser.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-blockgen/src/test/resources/fsimage_0000000000000061740.xml
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-dist/pom.xml
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/pom.xml
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/bash/create-slim-hadoop-tar.sh
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/bash/parse-metrics.sh
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/bash/start-dynamometer-cluster.sh
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/bash/upload-fsimage.sh
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/AMOptions.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/AllowAllImpersonationProvider.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/ApplicationMaster.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/BlockPlacementPolicyAlwaysSatisfied.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/Client.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/DynoConstants.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/DynoInfraUtils.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/DynoResource.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/SimulatedDataNodes.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/package-info.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/resources/start-component.sh
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/java/org/apache/hadoop/tools/dynamometer/TestDynamometerInfra.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/java/org/apache/hadoop/tools/dynamometer/TestDynoInfraUtils.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/blocks/dn0-a-0-r-00000
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/blocks/dn1-a-0-r-00001
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/blocks/dn2-a-0-r-00002
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/conf/etc/hadoop/core-site.xml
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/conf/etc/hadoop/hdfs-site.xml
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/conf/etc/hadoop/log4j.properties
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/hadoop_3_1/VERSION
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/hadoop_3_1/fsimage_0000000000000061740
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/hadoop_3_1/fsimage_0000000000000061740.md5
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/test/resources/yarn-site.xml
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/pom.xml
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/bash/parse-start-timestamp.sh
create mode 100755 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/bash/start-workload.sh
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/CreateFileMapper.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/VirtualInputFormat.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/VirtualInputSplit.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/VirtualRecordReader.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/WorkloadDriver.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/WorkloadMapper.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditCommandParser.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditLogDirectParser.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditLogHiveTableParser.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditReplayCommand.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditReplayMapper.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditReplayReducer.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/AuditReplayThread.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/CountTimeWritable.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/NoSplitTextInputFormat.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/UserCommandKey.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/package-info.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/main/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/package-info.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/test/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/TestWorkloadGenerator.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/test/java/org/apache/hadoop/tools/dynamometer/workloadgenerator/audit/TestAuditLogDirectParser.java
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/test/resources/audit_trace_direct/audit0
create mode 100644 hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-workload/src/test/resources/audit_trace_hive/audit0
create mode 100644 hadoop-tools/hadoop-dynamometer/pom.xml
create mode 100644 hadoop-tools/hadoop-dynamometer/src/site/markdown/Dynamometer.md
copy {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site => hadoop-tools/hadoop-dynamometer}/src/site/resources/css/site.css (100%)
create mode 100644 hadoop-tools/hadoop-dynamometer/src/site/resources/images/dynamometer-architecture-infra.png
create mode 100644 hadoop-tools/hadoop-dynamometer/src/site/resources/images/dynamometer-architecture-replay.png
create mode 100644 hadoop-tools/hadoop-fs2img/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSTreeWalk.java
delete mode 100644 hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE
create mode 100644 hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/DAGAMSimulator.java
create mode 100644 hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestDagAMSimulator.java
create mode 100644 hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/TestSLSDagAMSimulator.java
create mode 100644 hadoop-tools/hadoop-sls/src/test/resources/sls_dag.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_API_2.10.0.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Client_2.10.0.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Common_2.10.0.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Server_Common_2.10.0.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelineHealth.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/resources/org/apache/hadoop/yarn/api/records/timeline/jaxb.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/README.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/Dockerfile
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/README.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/resources/jaas.config
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/resources/samples.xml
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/scripts/entrypoint.sh
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/scripts/setup-image.sh
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/.gitignore
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/.yarnrc
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/README.md
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/{hadoop-yarn-services/hadoop-yarn-services-api => hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp}/dev-support/findbugs-exclude.xml (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/package.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/AppCatalog.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/AppCatalogInitializer.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/AppCatalogSolrClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/YarnServiceClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/AppDetailsController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/AppListController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/AppStoreController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/AppDetails.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/AppEntry.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/AppStoreEntry.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/Application.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/utils/RandomWord.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/utils/WordLengthException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/utils/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/app.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/controllers.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/filters.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/routes.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/services.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/resources/appcatalog.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/resources/log4j.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/.gitignore
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/WEB-INF/beans.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/WEB-INF/web.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/bootstrap-hadoop.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/bootstrap-hadoop.min.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/feather.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/jenkins.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/loading.svg
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/pattern-f61c2e99f82389a67432f54155c5f483.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/specific.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/theme.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/index.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/js/bootstrap-hadoop.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/js/bootstrap-hadoop.min.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/deploy.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/details.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/home.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/new.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/upgrade.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/theme.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/application/EmbeddedSolrServerFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/application/TestAppCatalogSolrClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/controller/AppDetailsControllerTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/controller/AppListControllerTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/controller/AppStoreControllerTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/javascript/controllersSpec.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/javascript/karma.conf.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets.tgz
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/lang/stopwords_en.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/params.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/protwords.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/schema.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/solrconfig.xml
rename hadoop-ozone/dist/src/main/blockade/clusterUtils/__init__.py => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/stopwords.txt (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/synonyms.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/log4j.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/dev-support/findbugs-exclude.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/dev-support/findbugs-exclude.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/assembly/bin.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/AbstractTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/CompositeTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/DieTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/MawoConfiguration.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/NullTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/SimpleTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/Task.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/TaskId.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/TaskStatus.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/TaskType.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/TeardownTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/WorkAssignmentProtocol.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/common/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/master/job/JobId.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/master/job/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/worker/WorkerId.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/java/org/apache/hadoop/applications/mawo/server/worker/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/main/resources/mawo-default.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/test/java/org/apache/hadoop/applications/mawo/server/common/TestMaWoConfiguration.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core/src/test/resources/mawo.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/pom.xml
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/examples/appcatalog/appcatalog.json
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/pom.xml
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/base/ubuntu-16.04/Dockerfile.cpu.tf_1.8.0
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/base/ubuntu-16.04/Dockerfile.gpu.tf_1.8.0
delete mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/build-all.sh
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/Dockerfile.cpu.tf_1.8.0
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/Dockerfile.gpu.tf_1.8.0
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10.py
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10_main.py
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10_utils.py
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/generate_cifar10_tfrecords.py
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/Cli.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/CliConstants.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/CliUtils.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/RunJobCli.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/ShowJobCli.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/BaseParameters.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/RunJobParameters.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/RunParameters.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/common/api/JobComponentStatus.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/common/api/TaskType.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/common/api/builder/JobComponentStatusBuilder.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/common/api/builder/JobStatusBuilder.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/RuntimeFactory.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/common/JobSubmitter.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceJobMonitor.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceJobSubmitter.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/YarnServiceUtils.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/DeveloperGuide.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/Examples.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/HowToInstall.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/Index.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/InstallationGuide.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/InstallationGuideChineseVersion.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/QuickStart.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/RunningDistributedCifar10TFJobs.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/RunningZeppelinOnYARN.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/TestAndTroubleshooting.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/WriteDockerfile.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/TestRunJobCliParsing.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/test/java/org/apache/hadoop/yarn/submarine/client/cli/yarnservice/TestYarnServiceRunJobCli.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/test/java/org/apache/hadoop/yarn/submarine/common/MockClientContext.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/test/java/org/apache/hadoop/yarn/submarine/common/fs/MockRemoteDirectoryManager.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/test/java/org/apache/hadoop/yarn/submarine/runtimes/yarnservice/TestTFConfigGenerator.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-3.3.1.min.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery/jquery-3.4.1.min.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/TestLogAggregationFileController.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/TestLogAggregationIndexFileController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/TestLogAggregationIndexedFileController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/timeline/TestShortenedFlowName.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-error-redefine-fpga-unit.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-error-redefine-gpu-unit.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/ContextFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/OpportunisticSchedulerMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/scheduler/DistributedOpportunisticContainerAllocator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/scheduler/TestDistributedOpportunisticContainerAllocator.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/scheduler/TestOpportunisticContainerAllocator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/webapp/ContainerBlockTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/AbstractContainersLauncher.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsMountConfig.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourcesExceptionUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/OCIContainerRuntime.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/RuncContainerRuntime.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerImagesCommand.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/HdfsManifestToResourcesPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/ImageManifest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/ImageTagToManifestPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/RuncContainerExecutorConfig.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/RuncImageTagToManifestPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/RuncManifestToResourcesPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/runc/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nec/NECVEPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nec/UdevUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nec/VEDeviceDiscoverer.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nec/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nvidia/NvidiaGPUPluginForRuntimeV2.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nvidia/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/DeviceResourceDockerRuntimePluginImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/ShellWrapper.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/AoclDiagnosticOutputParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/FpgaDevice.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/AoclOutputBasedDiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/DeviceSpecParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/FPGADiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/ScriptBasedFPGADiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/SettingsBasedFPGADiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuDeviceSpecificationException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/NvidiaBinaryHelper.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/devices/devices-module.c
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/devices/devices-module.h
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/mount-utils.c
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/mount-utils.h
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/modules/devices/test-devices-module.cc
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestResourceMappings.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/fpga/TestFpgaResourceHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/fpga/TestFpgaResourceHandlerImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceAllocator.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceHandlerImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestHdfsManifestToResourcesPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestImageTagToManifestPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestRuncContainerRuntime.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/TestDockerImagesCommand.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nec/TestNECVEPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nec/TestVEDeviceDiscoverer.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/com/nvidia/TestNvidiaGPUPluginForRuntimeV2.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/TestAoclOutputParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/TestGpuResourcePlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-xml-output
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/tensorflow-bench-result-for-GPU.csv
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMInfoMXBean.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/DefaultPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/FSPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/FairQueuePlacementUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/PrimaryGroupPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/RejectPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/SecondaryGroupExistingPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/SpecifiedPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/UserPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/ContextProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/NodeLabelProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/QueueProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/SubmissionContextPreProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/TagAddProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppLogAggregation.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/FSQueueMetricsForCustomResources.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsCustomResource.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/activities/ActivitiesUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/activities/ActivityLevel.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/activities/DiagnosticsCollector.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/activities/GenericDiagnosticsCollector.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/WorkflowPriorityMappingsManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/distributed/CentralizedOpportunisticContainerAllocator.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/ConversionException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigArgumentHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverterMain.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverterParams.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigRuleHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSYarnSiteConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/PreconditionException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/QueuePlacementConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/UnsupportedPropertyException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FifoOrderingPolicyWithExclusivePartitions.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/IteratorSelector.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppRequestAllocationInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ConfigVersionInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/QueueAclInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/QueueAclsInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestFairQueuePlacementUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestPlacementFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestPlacementRuleFS.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/TestContextProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/TestNodeLabelProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/TestQueueProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/preprocessor/TestTagAddProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerOvercommit.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/activities/TestActivitiesManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerOvercommit.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerWorkflowPriorityMapping.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/distributed/TestCentralizedOpportunisticContainerAllocator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/PrimaryGroupMapping.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerOvercommit.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigConverterTestCommons.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigArgumentHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigConverterMain.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigRuleHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSQueueConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSYarnSiteConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestQueuePlacementConverter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFifoOrderingPolicyWithExclusivePartitions.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/ActivitiesTestUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesContainers.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesSchedulerActivitiesWithMultiNodesEnabled.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/conversion-rules.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-conversion.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-max-resources-percentage.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-orderingpolicy-mixed.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-orderingpolicy.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-sameleafqueue.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/hadoop-metrics2-resourcemanager.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/hadoop-metrics2.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/invariants_jdk9.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-allocation-file-ref.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/pom.xml
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreCollectionCreator.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreTimelineReaderImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreTimelineWriterImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreUtils.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/CollectionType.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/NoDocumentFoundException.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/TimelineDocument.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/TimelineEntityDocument.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/TimelineEventSubDoc.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/TimelineMetricSubDoc.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowactivity/FlowActivityDocument.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowactivity/FlowActivitySubDoc.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowactivity/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowrun/FlowRunDocument.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowrun/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/DocumentStoreFactory.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/DocumentStoreNotSupportedException.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/DocumentStoreVendor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/DocumentStoreReader.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/TimelineCollectionReader.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/cosmosdb/CosmosDBDocumentStoreReader.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/cosmosdb/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/DocumentStoreWriter.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/TimelineCollectionWriter.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/cosmosdb/CosmosDBDocumentStoreWriter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/cosmosdb/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreTestUtils.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/JsonUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/TestDocumentStoreCollectionCreator.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/TestDocumentStoreTimelineReaderImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/TestDocumentStoreTimelineWriterImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/TestDocumentOperations.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/DummyDocumentStoreReader.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/cosmosdb/TestCosmosDBDocumentStoreReader.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/DummyDocumentStoreWriter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/cosmosdb/MockedCosmosDBDocumentStoreWriter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/cosmosdb/TestCosmosDBDocumentStoreWriter.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/flowactivity-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/flowrun-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/test-timeline-entities-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/timeline-app-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/timeline-entities.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestTimelineWriterHBaseDown.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseStorageMonitor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineStorageMonitor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesACL.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DevelopYourOwnDevicePlugin.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/PluggableDeviceFramework.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/RuncContainers.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/MasterWorker.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/.gitignore
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/timeline-health.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/timeline-health.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/timeline-health.js
create mode 100644 licenses-binary/LICENSE-angular-nvd3.txt
create mode 100644 licenses-binary/LICENSE-angular.txt
create mode 100644 licenses-binary/LICENSE-asm.txt
create mode 100644 licenses-binary/LICENSE-azure-cosmosdb.txt
create mode 100644 licenses-binary/LICENSE-azure-data-lake-store.txt
create mode 100644 licenses-binary/LICENSE-azure-keyvault.txt
create mode 100644 licenses-binary/LICENSE-bloomfilter.txt
create mode 100644 licenses-binary/LICENSE-bouncycastle.txt
create mode 100644 licenses-binary/LICENSE-cddl-gplv2-ce.txt
create mode 100644 licenses-binary/LICENSE-checker-framework.txt
create mode 100644 licenses-binary/LICENSE-d3.txt
create mode 100644 licenses-binary/LICENSE-datatables.txt
create mode 100644 licenses-binary/LICENSE-dnsjava.txt
create mode 100644 licenses-binary/LICENSE-dust.txt
create mode 100644 licenses-binary/LICENSE-freebsd.txt
create mode 100644 licenses-binary/LICENSE-go.txt
create mode 100644 licenses-binary/LICENSE-gtest.txt
create mode 100644 licenses-binary/LICENSE-hamcrest.txt
create mode 100644 licenses-binary/LICENSE-hsql.txt
create mode 100644 licenses-binary/LICENSE-jaf.txt
create mode 100644 licenses-binary/LICENSE-jdom.txt
create mode 100644 licenses-binary/LICENSE-jline.txt
create mode 100644 licenses-binary/LICENSE-jquery.txt
create mode 100644 licenses-binary/LICENSE-jsch.txt
create mode 100644 licenses-binary/LICENSE-jsr305.txt
create mode 100644 licenses-binary/LICENSE-jstree.txt
create mode 100644 licenses-binary/LICENSE-junit.txt
create mode 100644 licenses-binary/LICENSE-leveldb.txt
create mode 100644 licenses-binary/LICENSE-leveldbjni.txt
create mode 100644 licenses-binary/LICENSE-lz4.txt
create mode 100644 licenses-binary/LICENSE-moment.txt
create mode 100644 licenses-binary/LICENSE-mssql-jdbc.txt
create mode 100644 licenses-binary/LICENSE-nvd3.txt
create mode 100644 licenses-binary/LICENSE-ojalgo.txt
create mode 100644 licenses-binary/LICENSE-paranamer.txt
create mode 100644 licenses-binary/LICENSE-protobuf.txt
create mode 100644 licenses-binary/LICENSE-re2j.txt
create mode 100644 licenses-binary/LICENSE-slf4j.txt
create mode 100644 licenses-binary/LICENSE-slicing-by-8.txt
create mode 100644 licenses-binary/LICENSE-tree.txt
create mode 100644 licenses-binary/LICENSE-xtermjs.txt
create mode 100644 licenses/LICENSE-angular-nvd3.txt
create mode 100644 licenses/LICENSE-angular.txt
create mode 100644 licenses/LICENSE-bloomfilter.txt
create mode 100644 licenses/LICENSE-d3.txt
create mode 100644 licenses/LICENSE-datatables.txt
create mode 100644 licenses/LICENSE-dust.txt
create mode 100644 licenses/LICENSE-freebsd.txt
create mode 100644 licenses/LICENSE-gtest.txt
create mode 100644 licenses/LICENSE-jquery.txt
create mode 100644 licenses/LICENSE-jstree.txt
create mode 100644 licenses/LICENSE-lz4.txt
create mode 100644 licenses/LICENSE-moment.txt
create mode 100644 licenses/LICENSE-nvd3.txt
create mode 100644 licenses/LICENSE-slicing-by-8.txt
create mode 100644 licenses/LICENSE-tree.txt
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/04: HDFS-13310. The DatanodeProtocol should have a
DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs.
Followup work and fixed contributed by Virajith Jalaparthi.
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 4f7bd416853143d9becaa51ed0d2158b4613a988
Author: Ewan Higgs <ew...@wdc.com>
AuthorDate: Mon Jul 23 13:14:04 2018 +0200
HDFS-13310. The DatanodeProtocol should have a DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
---
.../protocol/BlockSyncTaskExecutionFeedback.java | 67 +++++++
.../server/protocol/SyncTaskExecutionOutcome.java | 25 +++
.../server/protocol/SyncTaskExecutionResult.java | 46 +++++
.../DatanodeProtocolClientSideTranslatorPB.java | 8 +-
.../DatanodeProtocolServerSideTranslatorPB.java | 6 +-
.../apache/hadoop/hdfs/protocolPB/PBHelper.java | 208 ++++++++++++++++++++-
.../server/blockmanagement/DatanodeManager.java | 4 +-
.../hdfs/server/datanode/BPServiceActor.java | 9 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 8 +-
.../hdfs/server/namenode/NameNodeRpcServer.java | 8 +-
.../hadoop/hdfs/server/protocol/BlockSyncTask.java | 83 ++++++++
.../protocol/BulkSyncTaskExecutionFeedback.java | 36 ++++
.../hdfs/server/protocol/DatanodeProtocol.java | 20 +-
.../hadoop/hdfs/server/protocol/SyncCommand.java | 39 ++++
.../src/main/proto/DatanodeProtocol.proto | 88 ++++++++-
.../blockmanagement/TestDatanodeManager.java | 2 +-
.../TestNameNodePrunesMissingStorages.java | 2 +-
.../server/datanode/InternalDataNodeTestUtils.java | 1 +
.../hdfs/server/datanode/TestBPOfferService.java | 5 +-
.../hdfs/server/datanode/TestBlockRecovery.java | 1 +
.../hdfs/server/datanode/TestDataNodeLifeline.java | 9 +-
.../datanode/TestDatanodeProtocolRetryPolicy.java | 1 +
.../hdfs/server/datanode/TestStorageReport.java | 4 +-
.../fsdataset/impl/TestFsDatasetCache.java | 4 +-
.../server/namenode/NNThroughputBenchmark.java | 8 +-
.../hdfs/server/namenode/NameNodeAdapter.java | 5 +-
.../hdfs/server/namenode/TestDeadDatanode.java | 4 +-
27 files changed, 653 insertions(+), 48 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTaskExecutionFeedback.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTaskExecutionFeedback.java
new file mode 100644
index 0000000..2e5393e
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTaskExecutionFeedback.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.util.UUID;
+
+/**
+ * Feedback for a BlockSyncTask.
+ */
+public class BlockSyncTaskExecutionFeedback {
+
+ private UUID syncTaskId;
+ private SyncTaskExecutionOutcome outcome;
+ private SyncTaskExecutionResult result;
+ private String syncMountId;
+
+ public BlockSyncTaskExecutionFeedback(UUID syncTaskId,
+ SyncTaskExecutionOutcome outcome, SyncTaskExecutionResult result,
+ String syncMountId) {
+ this.syncTaskId = syncTaskId;
+ this.outcome = outcome;
+ this.result = result;
+ this.syncMountId = syncMountId;
+ }
+
+ public static BlockSyncTaskExecutionFeedback finishedSuccessfully(
+ UUID syncTaskId, String syncMountId, SyncTaskExecutionResult result) {
+ return new BlockSyncTaskExecutionFeedback(syncTaskId,
+ SyncTaskExecutionOutcome.FINISHED_SUCCESSFULLY, result, syncMountId);
+ }
+
+ public static BlockSyncTaskExecutionFeedback failedWithException(
+ UUID syncTaskId, String syncMountId, Exception e) {
+ return new BlockSyncTaskExecutionFeedback(syncTaskId,
+ SyncTaskExecutionOutcome.EXCEPTION, null, syncMountId);
+ }
+
+ public UUID getSyncTaskId() {
+ return syncTaskId;
+ }
+
+ public SyncTaskExecutionOutcome getOutcome() {
+ return outcome;
+ }
+
+ public SyncTaskExecutionResult getResult() {
+ return result;
+ }
+
+ public String getSyncMountId() {
+ return syncMountId;
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionOutcome.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionOutcome.java
new file mode 100644
index 0000000..492575b
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionOutcome.java
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+/**
+ * SyncTaskExecutionOutcome is whether the SyncTask was successful or not.
+ */
+public enum SyncTaskExecutionOutcome {
+ FINISHED_SUCCESSFULLY,
+ EXCEPTION
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionResult.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionResult.java
new file mode 100644
index 0000000..b623dc5
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionResult.java
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.nio.ByteBuffer;
+
+/**
+ * Result of a SyncTask.
+ */
+public class SyncTaskExecutionResult {
+
+ /** result is the opaque byte stream result of a task. e.g. PartHandle */
+ private ByteBuffer result;
+ private Long numberOfBytes;
+
+ public SyncTaskExecutionResult(ByteBuffer result, Long numberOfBytes) {
+ this.result = result;
+ this.numberOfBytes = numberOfBytes;
+ }
+
+ public static SyncTaskExecutionResult emptyResult() {
+ return new SyncTaskExecutionResult(ByteBuffer.wrap(new byte[0]), 0L);
+ }
+
+ public ByteBuffer getResult() {
+ return result;
+ }
+
+ public Long getNumberOfBytes() {
+ return numberOfBytes;
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
index e4125dc..20b314c 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
@@ -48,6 +48,7 @@ import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.StorageBlock
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.StorageReceivedDeletedBlocksProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.VersionRequestProto;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
@@ -138,8 +139,8 @@ public class DatanodeProtocolClientSideTranslatorPB implements
VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
- throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback feedback) throws IOException {
HeartbeatRequestProto.Builder builder = HeartbeatRequestProto.newBuilder()
.setRegistration(PBHelper.convert(registration))
.setXmitsInProgress(xmitsInProgress).setXceiverCount(xceiverCount)
@@ -162,6 +163,9 @@ public class DatanodeProtocolClientSideTranslatorPB implements
if (slowDisks.haveSlowDisks()) {
builder.addAllSlowDisks(PBHelper.convertSlowDiskInfo(slowDisks));
}
+ if(feedback != null && !feedback.getFeedbacks().isEmpty()) {
+ builder.setBulkSyncTaskExecutionFeedback(PBHelper.convert(feedback));
+ }
HeartbeatResponseProto resp;
try {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
index 5cba284..a51ce85 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
@@ -109,6 +109,7 @@ public class DatanodeProtocolServerSideTranslatorPB implements
@Override
public HeartbeatResponseProto sendHeartbeat(RpcController controller,
HeartbeatRequestProto request) throws ServiceException {
+
HeartbeatResponse response;
try {
final StorageReport[] report = PBHelperClient.convertStorageReports(
@@ -122,7 +123,10 @@ public class DatanodeProtocolServerSideTranslatorPB implements
request.getXceiverCount(), request.getFailedVolumes(),
volumeFailureSummary, request.getRequestFullBlockReportLease(),
PBHelper.convertSlowPeerInfo(request.getSlowPeersList()),
- PBHelper.convertSlowDiskInfo(request.getSlowDisksList()));
+ PBHelper.convertSlowDiskInfo(request.getSlowDisksList()),
+ PBHelper.convertBulkSyncTaskExecutionFeedback(
+ request.getBulkSyncTaskExecutionFeedback())
+ );
} catch (IOException e) {
throw new ServiceException(e);
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
index baec6fa..a62a00d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
@@ -17,6 +17,9 @@
*/
package org.apache.hadoop.hdfs.protocolPB;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
@@ -24,9 +27,10 @@ import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.UUID;
+import java.util.stream.Collectors;
import com.google.protobuf.ByteString;
-
import org.apache.hadoop.fs.StorageType;
import org.apache.hadoop.ha.HAServiceProtocol.HAServiceState;
import org.apache.hadoop.hdfs.DFSUtilClient;
@@ -43,38 +47,44 @@ import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockCommand
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockECReconstructionCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockIdCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockRecoveryCommandProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockReportContextProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockSyncTaskProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BulkSyncTaskExecutionFeedbackProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.DatanodeCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.DatanodeRegistrationProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.FinalizeCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.KeyUpdateCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.ReceivedDeletedBlockInfoProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.RegisterCommandProto;
-import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos
- .SlowDiskReportProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SlowDiskReportProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SlowPeerReportProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncCommandProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskExecutionFeedbackProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskExecutionOutcomeProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskExecutionResultProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskIdProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.VolumeFailureSummaryProto;
-import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockReportContextProto;
import org.apache.hadoop.hdfs.protocol.proto.ErasureCodingProtos.BlockECReconstructionInfoProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.BlockProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ExtendedBlockProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ProvidedStorageLocationProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageUuidsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.DatanodeInfosProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ExtendedBlockProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.LocatedBlockProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ProvidedStorageLocationProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageTypeProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageTypesProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageUuidsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlockKeyProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlockWithLocationsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlocksWithLocationsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.CheckpointCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.CheckpointSignatureProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.ExportedBlockKeysProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NNHAStatusHeartbeatProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamenodeCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamenodeRegistrationProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamenodeRegistrationProto.NamenodeRoleProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamespaceInfoProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NNHAStatusHeartbeatProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.RecoveringBlockProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.RemoteEditLogManifestProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.RemoteEditLogProto;
@@ -89,18 +99,23 @@ import org.apache.hadoop.hdfs.server.common.HdfsServerConstants.NodeType;
import org.apache.hadoop.hdfs.server.common.HdfsServerConstants.ReplicaState;
import org.apache.hadoop.hdfs.server.common.StorageInfo;
import org.apache.hadoop.hdfs.server.namenode.CheckpointSignature;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTaskExecutionFeedback;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionOutcome;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
import org.apache.hadoop.hdfs.server.protocol.BalancerBandwidthCommand;
import org.apache.hadoop.hdfs.server.protocol.BlockCommand;
import org.apache.hadoop.hdfs.server.protocol.BlockECReconstructionCommand;
+import org.apache.hadoop.hdfs.server.protocol.BlockECReconstructionCommand.BlockECReconstructionInfo;
import org.apache.hadoop.hdfs.server.protocol.BlockIdCommand;
import org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand;
-import org.apache.hadoop.hdfs.server.protocol.BlockECReconstructionCommand.BlockECReconstructionInfo;
import org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand.RecoveringBlock;
import org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand.RecoveringStripedBlock;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.CheckpointCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol;
@@ -119,7 +134,9 @@ import org.apache.hadoop.hdfs.server.protocol.RemoteEditLog;
import org.apache.hadoop.hdfs.server.protocol.RemoteEditLogManifest;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import org.apache.hadoop.hdfs.server.protocol.SlowPeerReports;
+import org.apache.hadoop.hdfs.server.protocol.SyncCommand;
import org.apache.hadoop.hdfs.server.protocol.VolumeFailureSummary;
+import org.apache.hadoop.ipc.ClientId;
/**
* Utilities for converting protobuf classes to and from implementation classes
@@ -469,11 +486,52 @@ public class PBHelper {
return PBHelper.convert(proto.getBlkIdCmd());
case BlockECReconstructionCommand:
return PBHelper.convert(proto.getBlkECReconstructionCmd());
+ case SyncCommand:
+ return PBHelper.convert(proto.getSyncCommand());
default:
return null;
}
}
-
+
+ private static SyncCommand convert(SyncCommandProto backupCommand) {
+ List<BlockSyncTaskProto> syncTasksProtoList =
+ backupCommand.getSyncTasksList();
+ List<BlockSyncTask> syncTasksList =
+ new ArrayList(syncTasksProtoList.size());
+ for (BlockSyncTaskProto syncTaskProto : syncTasksProtoList) {
+ syncTasksList.add(convertSyncTask(syncTaskProto));
+ }
+
+ return new SyncCommand(DatanodeProtocol.DNA_BACKUP, syncTasksList);
+ }
+
+ private static BlockSyncTask convertSyncTask(
+ BlockSyncTaskProto syncTaskProto) {
+ SyncTaskIdProto syncTaskIdProto = syncTaskProto.getSyncTaskId();
+ UUID syncTaskId = convert(syncTaskIdProto);
+ try {
+ return new BlockSyncTask(syncTaskId,
+ new URI(syncTaskProto.getUri()),
+ PBHelperClient.convertLocatedBlocks(
+ syncTaskProto.getLocatedBlocksList()),
+ syncTaskProto.getPartNumber(),
+ syncTaskProto.getUploadHandle().toByteArray(),
+ syncTaskProto.getOffset(),
+ syncTaskProto.getLength(),
+ syncTaskIdProto.getSyncMountId());
+ } catch (URISyntaxException e) {
+ throw new IllegalArgumentException();
+ }
+ }
+
+ public static UUID convert(SyncTaskIdProto syncTaskIdProto) {
+ byte[] clientId = syncTaskIdProto.getSyncTaskId().toByteArray();
+ long syncTaskIdMsb = ClientId.getMsb(clientId);
+ long syncTaskIdLsb = ClientId.getLsb(clientId);
+ return new UUID(syncTaskIdMsb, syncTaskIdLsb);
+ }
+
+
public static BalancerBandwidthCommandProto convert(
BalancerBandwidthCommand bbCmd) {
return BalancerBandwidthCommandProto.newBuilder()
@@ -603,6 +661,10 @@ public class PBHelper {
.setBlkECReconstructionCmd(
convert((BlockECReconstructionCommand) datanodeCommand));
break;
+ case DatanodeProtocol.DNA_BACKUP:
+ builder.setCmdType(DatanodeCommandProto.Type.SyncCommand)
+ .setSyncCommand(convert((SyncCommand) datanodeCommand));
+ break;
case DatanodeProtocol.DNA_UNKNOWN: //Not expected
default:
builder.setCmdType(DatanodeCommandProto.Type.NullDatanodeCommand);
@@ -1128,4 +1190,130 @@ public class PBHelper {
return new FileRegion(block, providedStorageLocation);
}
+
+ private static SyncCommandProto convert(SyncCommand syncCommand) {
+ SyncCommandProto.Builder builder = SyncCommandProto.newBuilder();
+
+ List<BlockSyncTaskProto> syncTaskProtos = syncCommand.getSyncTasks()
+ .stream()
+ .map(syncTask -> convert(syncTask))
+ .collect(Collectors.toList());
+
+ builder.addAllSyncTasks(syncTaskProtos);
+
+ return builder.build();
+ }
+
+ private static BlockSyncTaskProto convert(BlockSyncTask blockSyncTask) {
+ BlockSyncTaskProto.Builder builder = BlockSyncTaskProto.newBuilder();
+ builder.addAllLocatedBlocks(
+ PBHelperClient.convertLocatedBlocks2(blockSyncTask.getLocatedBlocks()));
+ builder.setUploadHandle(
+ ByteString.copyFrom(blockSyncTask.getUploadHandle()));
+ builder.setPartNumber(blockSyncTask.getPartNumber());
+ builder.setUri(blockSyncTask.getRemoteURI().toString());
+ builder.setOffset(blockSyncTask.getOffset());
+ builder.setLength(blockSyncTask.getLength());
+
+ return builder.build();
+ }
+
+ public static SyncTaskIdProto convert(UUID syncTaskId, String syncMountId) {
+ SyncTaskIdProto.Builder builder = SyncTaskIdProto.newBuilder();
+ ByteBuffer syncTaskIdBytes = ByteBuffer.wrap(new byte[16]);
+ syncTaskIdBytes.putLong(syncTaskId.getMostSignificantBits());
+ syncTaskIdBytes.putLong(syncTaskId.getLeastSignificantBits());
+ builder.setSyncTaskId(ByteString.copyFrom(syncTaskIdBytes.array()));
+ builder.setSyncMountId(syncMountId);
+ return builder.build();
+ }
+
+
+ public static BulkSyncTaskExecutionFeedbackProto convert(
+ BulkSyncTaskExecutionFeedback bulkFeedback) {
+ return BulkSyncTaskExecutionFeedbackProto.newBuilder()
+ .addAllFeedbacks(bulkFeedback.getFeedbacks().stream()
+ .map(f -> convert(f)).collect(Collectors.toList()))
+ .build();
+ }
+
+ public static SyncTaskExecutionFeedbackProto convert(
+ BlockSyncTaskExecutionFeedback feedback) {
+ SyncTaskExecutionFeedbackProto.Builder builder =
+ SyncTaskExecutionFeedbackProto.newBuilder()
+ .setSyncTaskId(
+ convert(feedback.getSyncTaskId(), feedback.getSyncMountId()))
+ .setOutcome(convert(feedback.getOutcome()));
+ if (feedback.getResult() != null) {
+ builder.setResult(convert(feedback.getResult()));
+ }
+ return builder.build();
+ }
+
+ public static SyncTaskExecutionOutcomeProto convert(
+ SyncTaskExecutionOutcome outcome) {
+ switch (outcome) {
+ case FINISHED_SUCCESSFULLY:
+ return SyncTaskExecutionOutcomeProto.FINISHED_SUCCESSFULLY;
+ case EXCEPTION:
+ return SyncTaskExecutionOutcomeProto.EXCEPTION;
+ default:
+ throw new IllegalArgumentException(
+ "Unknown SyncTaskExecutionOutcome: " + outcome);
+ }
+ }
+
+ public static SyncTaskExecutionResultProto convert(
+ SyncTaskExecutionResult result) {
+ SyncTaskExecutionResultProto.Builder builder =
+ SyncTaskExecutionResultProto.newBuilder();
+ if (result.getResult() != null) {
+ builder.setResult(ByteString.copyFrom(result.getResult()));
+ }
+ if (result.getNumberOfBytes() != null) {
+ builder.setNumberOfBytes(result.getNumberOfBytes());
+ }
+ return builder.build();
+ }
+
+ public static BulkSyncTaskExecutionFeedback convertBulkSyncTaskExecutionFeedback(
+ BulkSyncTaskExecutionFeedbackProto bulkSyncTaskExecutionFeedback) {
+ return new BulkSyncTaskExecutionFeedback(
+ bulkSyncTaskExecutionFeedback.getFeedbacksList().stream()
+ .map(feedback -> convert(feedback)).collect(Collectors.toList()));
+ }
+
+ public static BlockSyncTaskExecutionFeedback convert(
+ SyncTaskExecutionFeedbackProto feedback) {
+ return new BlockSyncTaskExecutionFeedback(convert(feedback.getSyncTaskId()),
+ convert(feedback.getOutcome()),
+ feedback.hasResult() ? convert(feedback.getResult()) : null,
+ feedback.getSyncTaskId().getSyncMountId());
+ }
+
+ public static SyncTaskExecutionOutcome convert(
+ SyncTaskExecutionOutcomeProto outcome) {
+ switch (outcome) {
+ case FINISHED_SUCCESSFULLY:
+ return SyncTaskExecutionOutcome.FINISHED_SUCCESSFULLY;
+ case EXCEPTION:
+ return SyncTaskExecutionOutcome.EXCEPTION;
+ default:
+ throw new IllegalArgumentException(
+ "Unknown SyncTaskExecutionOutcomeProto: " + outcome);
+ }
+ }
+
+ public static SyncTaskExecutionResult convert(
+ SyncTaskExecutionResultProto result) {
+ byte[] bytes = null;
+ if (result.getResult() != null) {
+ bytes = result.getResult().toByteArray();
+ }
+
+ ByteBuffer byteBuffer =
+ (bytes == null) ? null : ByteBuffer.wrap(bytes).asReadOnlyBuffer();
+ return new SyncTaskExecutionResult(byteBuffer, result.getNumberOfBytes());
+ }
+
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
index 8adb03d..1c6a1f0 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
@@ -1660,7 +1660,9 @@ public class DatanodeManager {
int maxTransfers, int failedVolumes,
VolumeFailureSummary volumeFailureSummary,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks) throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
+ throws IOException {
final DatanodeDescriptor nodeinfo;
try {
nodeinfo = getDatanode(nodeReg);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
index 495035e..0d2f514 100755
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
@@ -50,6 +50,7 @@ import org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB;
import org.apache.hadoop.hdfs.server.common.IncorrectVersionException;
import org.apache.hadoop.hdfs.server.namenode.FSNamesystem;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
import org.apache.hadoop.hdfs.server.protocol.DatanodeStorage;
@@ -534,6 +535,10 @@ class BPServiceActor implements Runnable {
SlowDiskReports.create(dn.getDiskMetrics().getDiskOutliersStats()) :
SlowDiskReports.EMPTY_REPORT;
+ // TODO - collect feedback from SyncTasks here.
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback =
+ new BulkSyncTaskExecutionFeedback(Collections.emptyList());
+
HeartbeatResponse response = bpNamenode.sendHeartbeat(bpRegistration,
reports,
dn.getFSDataset().getCacheCapacity(),
@@ -544,13 +549,13 @@ class BPServiceActor implements Runnable {
volumeFailureSummary,
requestBlockReportLease,
slowPeers,
- slowDisks);
+ slowDisks,
+ bulkSyncTaskExecutionFeedback);
if (outliersReportDue) {
// If the report was due and successfully sent, schedule the next one.
scheduler.scheduleNextOutlierReport();
}
-
return response;
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
index 8d1884e..d78d940 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
@@ -107,6 +107,7 @@ import org.apache.hadoop.hdfs.protocol.SnapshotDiffReportListing;
import org.apache.hadoop.hdfs.protocol.SnapshotDiffReport;
import org.apache.hadoop.hdfs.server.common.ECTopologyVerifier;
import org.apache.hadoop.hdfs.server.namenode.metrics.ReplicatedBlocksMBean;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import org.apache.hadoop.util.Time;
import static org.apache.hadoop.util.Time.now;
@@ -3988,8 +3989,9 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
- throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
+ throws IOException {
readLock();
try {
//get datanode commands
@@ -3998,7 +4000,7 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
DatanodeCommand[] cmds = blockManager.getDatanodeManager().handleHeartbeat(
nodeReg, reports, getBlockPoolId(), cacheCapacity, cacheUsed,
xceiverCount, maxTransfer, failedVolumes, volumeFailureSummary,
- slowPeers, slowDisks);
+ slowPeers, slowDisks, bulkSyncTaskExecutionFeedback);
long blockReportLeaseId = 0;
if (requestFullBlockReportLease) {
blockReportLeaseId = blockManager.requestBlockReportLeaseId(nodeReg);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
index e483961..c6e00de 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
@@ -160,6 +160,7 @@ import org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics;
import org.apache.hadoop.hdfs.server.namenode.sps.StoragePolicySatisfyManager;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
@@ -1564,14 +1565,15 @@ public class NameNodeRpcServer implements NamenodeProtocols {
int failedVolumes, VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
- throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
+ throws IOException {
checkNNStartup();
verifyRequest(nodeReg);
return namesystem.handleHeartbeat(nodeReg, report,
dnCacheCapacity, dnCacheUsed, xceiverCount, xmitsInProgress,
failedVolumes, volumeFailureSummary, requestFullBlockReportLease,
- slowPeers, slowDisks);
+ slowPeers, slowDisks, bulkSyncTaskExecutionFeedback);
}
@Override // DatanodeProtocol
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTask.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTask.java
new file mode 100644
index 0000000..875a409
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTask.java
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import java.net.URI;
+import java.util.List;
+import java.util.UUID;
+
+/**
+ * A BlockSyncTask is an operation that is sent to the datanodes to copy
+ * blocks to an external storage endpoint as a part of an orchestrated
+ * synchronization across multiple datanodes.
+ * BlockSyncTask is intended to be an immutable POJO.
+ */
+public class BlockSyncTask {
+ private final UUID syncTaskId;
+ private final URI remoteURI;
+ private final List<LocatedBlock> locatedBlocks;
+ private String syncMountId;
+ private final int partNumber;
+ private byte[] uploadHandle;
+ private final int offset;
+ private final long length;
+
+ public BlockSyncTask(UUID syncTaskId, URI remoteURI,
+ List<LocatedBlock> locatedBlocks, Integer partNumber, byte[] uploadHandle,
+ int offset, long length, String syncMountId) {
+ this.syncTaskId = syncTaskId;
+ this.remoteURI = remoteURI;
+ this.locatedBlocks = locatedBlocks;
+ this.syncMountId = syncMountId;
+ this.partNumber = partNumber;
+ this.uploadHandle = uploadHandle;
+ this.offset = offset;
+ this.length = length;
+ }
+
+ public int getPartNumber() {
+ return partNumber;
+ }
+
+ public byte[] getUploadHandle() {
+ return uploadHandle;
+ }
+
+ public int getOffset() {
+ return offset;
+ }
+
+ public long getLength() {
+ return length;
+ }
+
+ public UUID getSyncTaskId() {
+ return syncTaskId;
+ }
+
+ public URI getRemoteURI() {
+ return remoteURI;
+ }
+
+ public List<LocatedBlock> getLocatedBlocks() {
+ return locatedBlocks;
+ }
+
+ public String getSyncMountId() {
+ return syncMountId;
+ }
+}
\ No newline at end of file
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BulkSyncTaskExecutionFeedback.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BulkSyncTaskExecutionFeedback.java
new file mode 100644
index 0000000..0d459e8
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BulkSyncTaskExecutionFeedback.java
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.util.Collection;
+
+/**
+ * Feedback for a collection of {@link BlockSyncTask}s.
+ */
+public class BulkSyncTaskExecutionFeedback {
+
+ private Collection<BlockSyncTaskExecutionFeedback> feedbacks;
+
+ public BulkSyncTaskExecutionFeedback(
+ Collection<BlockSyncTaskExecutionFeedback> feedbacks) {
+ this.feedbacks = feedbacks;
+ }
+
+ public Collection<BlockSyncTaskExecutionFeedback> getFeedbacks() {
+ return feedbacks;
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
index 5680ef3..692a2e8 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
@@ -81,6 +81,7 @@ public interface DatanodeProtocol {
final static int DNA_ERASURE_CODING_RECONSTRUCTION = 11; // erasure coding reconstruction command
int DNA_BLOCK_STORAGE_MOVEMENT = 12; // block storage movement command
int DNA_DROP_SPS_WORK_COMMAND = 13; // drop sps work command
+ final static int DNA_BACKUP = 14; // back up data to PROVIDED stores.
/**
* Register Datanode.
@@ -112,20 +113,19 @@ public interface DatanodeProtocol {
* @param slowPeers Details of peer DataNodes that were detected as being
* slow to respond to packet writes. Empty report if no
* slow peers were detected by the DataNode.
+ * @param bulkSyncTaskExecutionFeedback Result of the execution of the
+ * sync tasks.
* @throws IOException on error
*/
@Idempotent
public HeartbeatResponse sendHeartbeat(DatanodeRegistration registration,
- StorageReport[] reports,
- long dnCacheCapacity,
- long dnCacheUsed,
- int xmitsInProgress,
- int xceiverCount,
- int failedVolumes,
- VolumeFailureSummary volumeFailureSummary,
- boolean requestFullBlockReportLease,
- @Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
+ StorageReport[] reports, long cacheCapacity, long cacheUsed,
+ int xmitsInProgress, int xceiverCount, int failedVolumes,
+ VolumeFailureSummary volumeFailureSummary,
+ boolean requestFullBlockReportLease,
+ @Nonnull SlowPeerReports slowPeers,
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
throws IOException;
/**
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncCommand.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncCommand.java
new file mode 100644
index 0000000..7e2c242
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncCommand.java
@@ -0,0 +1,39 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.util.List;
+
+/**
+ * A SyncCommand is an instruction to a DataNode to move the
+ * give file to specified target DataNodes provided storage.
+ */
+public class SyncCommand extends DatanodeCommand {
+
+ private final List<BlockSyncTask> syncTasks;
+
+ public SyncCommand(int action, List<BlockSyncTask> syncTasks) {
+ super(action);
+ this.syncTasks = syncTasks;
+ }
+
+ public List<BlockSyncTask> getSyncTasks() {
+ return syncTasks;
+ }
+
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
index 0e24130..66df8ed 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
@@ -60,6 +60,7 @@ message DatanodeCommandProto {
NullDatanodeCommand = 7;
BlockIdCommand = 8;
BlockECReconstructionCommand = 9;
+ SyncCommand = 10;
}
required Type cmdType = 1; // Type of the command
@@ -74,6 +75,7 @@ message DatanodeCommandProto {
optional RegisterCommandProto registerCmd = 7;
optional BlockIdCommandProto blkIdCmd = 8;
optional BlockECReconstructionCommandProto blkECReconstructionCmd = 9;
+ optional SyncCommandProto syncCommand = 10;
}
/**
@@ -154,6 +156,89 @@ message BlockECReconstructionCommandProto {
repeated BlockECReconstructionInfoProto blockECReconstructioninfo = 1;
}
+message SyncTaskIdProto {
+ required bytes syncTaskId = 1;
+ required string syncMountId = 2;
+}
+
+/**
+ * Instruct the datanode to perform a backup command
+ */
+message SyncCommandProto {
+ repeated BlockSyncTaskProto syncTasks = 1;
+}
+
+/**
+ * A block synchronization task as part of an orchestrated synchronization
+ * across potentially multiple datanodes (i.e. multipart put part).
+ */
+message BlockSyncTaskProto {
+ required SyncTaskIdProto syncTaskId = 1;
+
+ required bytes uploadHandle = 2;
+ required int32 partNumber = 3;
+ repeated LocatedBlockProto locatedBlocks = 4;
+ required string uri = 5;
+ required int32 offset = 6;
+ required int64 length = 7;
+}
+
+/**
+ * Block storage movement information
+ */
+message BlockMovingInfoProto {
+ required BlockProto block = 1;
+ required DatanodeInfoProto sourceDnInfo = 2;
+ required DatanodeInfoProto targetDnInfo = 3;
+ required StorageTypeProto sourceStorageType = 4;
+ required StorageTypeProto targetStorageType = 5;
+}
+
+/**
+ * Blocks for which storage movements has been attempted and finished
+ * with either success or failure.
+ */
+message BlocksStorageMoveAttemptFinishedProto {
+ repeated BlockProto blocks = 1;
+}
+
+/**
+ * A collection of feedbacks for a collection of sync tasks.
+ */
+message BulkSyncTaskExecutionFeedbackProto {
+ repeated SyncTaskExecutionFeedbackProto feedbacks = 1;
+}
+
+/**
+ * Feedback for a sync task that has been executed.
+ * syncTaskId - identifier for the task.
+ * outcome - success/error.
+ * operation - the type of operation.
+ * result - if the outcome is successful, the results of the sync task.
+ */
+message SyncTaskExecutionFeedbackProto {
+ required SyncTaskIdProto syncTaskId = 1;
+ required SyncTaskExecutionOutcomeProto outcome = 2;
+ optional SyncTaskExecutionResultProto result = 3;
+}
+
+/**
+ * Success of failure indication of a sync task.
+ */
+enum SyncTaskExecutionOutcomeProto {
+ FINISHED_SUCCESSFULLY = 0;
+ EXCEPTION = 1;
+}
+
+/**
+ * result - the opaque result data from the sync task.
+ * numberOfBytes - the number of bytes copied.
+ */
+message SyncTaskExecutionResultProto {
+ optional bytes result = 1;
+ optional int64 numberOfBytes = 2;
+}
+
/**
* registration - Information of the datanode registering with the namenode
*/
@@ -211,6 +296,7 @@ message HeartbeatRequestProto {
optional bool requestFullBlockReportLease = 9 [ default = false ];
repeated SlowPeerReportProto slowPeers = 10;
repeated SlowDiskReportProto slowDisks = 11;
+ optional BulkSyncTaskExecutionFeedbackProto bulkSyncTaskExecutionFeedback = 12;
}
/**
@@ -276,7 +362,7 @@ message StorageBlockReportProto {
*/
message BlockReportResponseProto {
optional DatanodeCommandProto cmd = 1;
-}
+}
/**
* registration - datanode registration information
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
index e8e6b94..6d24321 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
@@ -737,7 +737,7 @@ public class TestDatanodeManager {
Mockito.when(dm.getDatanode(dnReg)).thenReturn(nodeInfo);
DatanodeCommand[] cmds = dm.handleHeartbeat(
dnReg, new StorageReport[1], "bp-123", 0, 0, 10, maxTransfers, 0, null,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT);
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT, null);
long expectedNumCmds = Arrays.stream(
new int[]{numReplicationTasks, numECTasks})
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
index f64767a..d8e53ae 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
@@ -117,7 +117,7 @@ public class TestNameNodePrunesMissingStorages {
cluster.stopDataNode(0);
cluster.getNameNodeRpc().sendHeartbeat(dnReg, prunedReports, 0L, 0L, 0, 0,
0, null, true, SlowPeerReports.EMPTY_REPORT,
- SlowDiskReports.EMPTY_REPORT);
+ SlowDiskReports.EMPTY_REPORT, null);
// Check that the missing storage was pruned.
assertThat(dnDescriptor.getStorageInfos().length, is(expectedStoragesAfterTest));
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
index 97302b5..5586f89 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
@@ -163,6 +163,7 @@ public class InternalDataNodeTestUtils {
Mockito.anyInt(), Mockito.any(),
Mockito.anyBoolean(),
Mockito.any(),
+ Mockito.any(),
Mockito.any())).thenReturn(
new HeartbeatResponse(new DatanodeCommand[0], new NNHAStatusHeartbeat(
HAServiceState.ACTIVE, 1), null, ThreadLocalRandom.current()
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
index 2353992..26c07d5 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
@@ -18,6 +18,8 @@
package org.apache.hadoop.hdfs.server.datanode;
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY;
+
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
@@ -160,7 +162,8 @@ public class TestBPOfferService {
Mockito.any(VolumeFailureSummary.class),
Mockito.anyBoolean(),
Mockito.any(SlowPeerReports.class),
- Mockito.any(SlowDiskReports.class));
+ Mockito.any(SlowDiskReports.class),
+ Mockito.any(BulkSyncTaskExecutionFeedback.class));
mockHaStatuses[nnIdx] = new NNHAStatusHeartbeat(HAServiceState.STANDBY, 0);
datanodeCommands[nnIdx] = new DatanodeCommand[0];
return mock;
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
index f12285c..fb8768d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
@@ -228,6 +228,7 @@ public class TestBlockRecovery {
Mockito.any(),
Mockito.anyBoolean(),
Mockito.any(),
+ Mockito.any(),
Mockito.any()))
.thenReturn(new HeartbeatResponse(
new DatanodeCommand[0],
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
index dcfbf02..7c62edc 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
@@ -25,13 +25,14 @@ import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_HEARTBEAT_RECHEC
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_LIFELINE_RPC_ADDRESS_KEY;
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_STALE_DATANODE_INTERVAL_KEY;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.apache.hadoop.test.MetricsAsserts.getLongCounter;
import static org.apache.hadoop.test.MetricsAsserts.getMetrics;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
-import static org.mockito.Mockito.any;
+import static org.mockito.Matchers.any;
import static org.mockito.Mockito.anyBoolean;
import static org.mockito.Mockito.anyInt;
import static org.mockito.Mockito.anyLong;
@@ -172,7 +173,8 @@ public class TestDataNodeLifeline {
any(),
anyBoolean(),
any(SlowPeerReports.class),
- any(SlowDiskReports.class));
+ any(SlowDiskReports.class),
+ any(BulkSyncTaskExecutionFeedback.class));
// Intercept lifeline to trigger latch count-down on each call.
doAnswer(new LatchCountingAnswer<Void>(lifelinesSent))
@@ -241,7 +243,8 @@ public class TestDataNodeLifeline {
any(),
anyBoolean(),
any(SlowPeerReports.class),
- any(SlowDiskReports.class));
+ any(SlowDiskReports.class),
+ any(BulkSyncTaskExecutionFeedback.class));
// While waiting on the latch for the expected number of heartbeat messages,
// poll DataNode tracking information. We expect that the DataNode always
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
index 93f9f44..bb4c125 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
@@ -216,6 +216,7 @@ public class TestDatanodeProtocolRetryPolicy {
Mockito.any(),
Mockito.anyBoolean(),
Mockito.any(),
+ Mockito.any(),
Mockito.any());
dn = new DataNode(conf, locations, null, null) {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
index 9df6209..875cef7 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
@@ -29,6 +29,7 @@ import org.apache.hadoop.hdfs.HdfsConfiguration;
import org.apache.hadoop.hdfs.MiniDFSCluster;
import org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB;
import org.apache.hadoop.hdfs.server.namenode.NameNode;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
import org.apache.hadoop.hdfs.server.protocol.DatanodeStorage;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
@@ -110,7 +111,8 @@ public class TestStorageReport {
anyLong(), anyLong(), anyInt(), anyInt(), anyInt(),
any(), Mockito.anyBoolean(),
Mockito.any(SlowPeerReports.class),
- Mockito.any(SlowDiskReports.class));
+ Mockito.any(SlowDiskReports.class),
+ Mockito.any(BulkSyncTaskExecutionFeedback.class));
StorageReport[] reports = captor.getValue();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetCache.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetCache.java
index ae42bd8..0689095 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetCache.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetCache.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.hdfs.server.datanode.fsdataset.impl;
import net.jcip.annotations.NotThreadSafe;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.apache.hadoop.test.MetricsAsserts.getMetrics;
import static org.junit.Assert.assertEquals;
@@ -210,7 +211,8 @@ public class TestFsDatasetCache {
(StorageReport[]) any(), anyLong(), anyLong(),
anyInt(), anyInt(), anyInt(), (VolumeFailureSummary) any(),
anyBoolean(), any(SlowPeerReports.class),
- any(SlowDiskReports.class));
+ any(SlowDiskReports.class),
+ any(BulkSyncTaskExecutionFeedback.class));
} finally {
lock.writeLock().unlock();
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
index f501673..6cf3aa6 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
@@ -964,8 +964,8 @@ public class NNThroughputBenchmark implements Tool {
DF_CAPACITY, DF_USED, DF_CAPACITY - DF_USED, DF_USED, 0L) };
DatanodeCommand[] cmds = dataNodeProto.sendHeartbeat(dnRegistration, rep,
0L, 0L, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT)
- .getCommands();
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ null).getCommands();
if(cmds != null) {
for (DatanodeCommand cmd : cmds ) {
if(LOG.isDebugEnabled()) {
@@ -1015,8 +1015,8 @@ public class NNThroughputBenchmark implements Tool {
false, DF_CAPACITY, DF_USED, DF_CAPACITY - DF_USED, DF_USED, 0) };
DatanodeCommand[] cmds = dataNodeProto.sendHeartbeat(dnRegistration,
rep, 0L, 0L, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT)
- .getCommands();
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ null).getCommands();
if (cmds != null) {
for (DatanodeCommand cmd : cmds) {
if (cmd.getAction() == DatanodeProtocol.DNA_TRANSFER) {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
index f1b26ab..1c672cd 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
@@ -20,11 +20,13 @@ package org.apache.hadoop.hdfs.server.namenode;
import org.apache.hadoop.ha.HAServiceProtocol.HAServiceState;
import org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo;
import org.apache.hadoop.hdfs.server.blockmanagement.BlockManager;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.mockito.Mockito.spy;
import java.io.File;
import java.io.IOException;
+import java.util.ArrayList;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.commons.lang3.reflect.FieldUtils;
@@ -136,7 +138,8 @@ public class NameNodeAdapter {
return namesystem.handleHeartbeat(nodeReg,
BlockManagerTestUtil.getStorageReportsForDatanode(dd),
dd.getCacheCapacity(), dd.getCacheRemaining(), 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT);
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ new BulkSyncTaskExecutionFeedback(new ArrayList<>()));
}
public static boolean setReplication(final FSNamesystem ns,
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
index 46b8db1..6212292 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
@@ -140,8 +140,8 @@ public class TestDeadDatanode {
false, 0, 0, 0, 0, 0) };
DatanodeCommand[] cmd =
dnp.sendHeartbeat(reg, rep, 0L, 0L, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT)
- .getCommands();
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ null).getCommands();
assertEquals(1, cmd.length);
assertEquals(cmd[0].getAction(), RegisterCommand.REGISTER
.getAction());
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 03/04: HDFS-13794. [PROVIDED Phase 2] Teach
BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 30607ec809d5b422c9e26c7b207c63a6a9c8d579
Author: Virajith Jalaparti <vi...@apache.org>
AuthorDate: Tue Feb 12 13:43:06 2019 -0800
HDFS-13794. [PROVIDED Phase 2] Teach BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
---
.../main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 1 -
.../AliasMapProtocolServerSideTranslatorPB.java | 16 ++++++++++++++++
.../InMemoryAliasMapProtocolClientSideTranslatorPB.java | 15 +++++++++++++++
.../hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java | 8 ++++++--
.../hdfs/server/aliasmap/InMemoryAliasMapProtocol.java | 9 +++++++++
.../server/aliasmap/InMemoryLevelDBAliasMapServer.java | 5 +++++
.../hdfs/server/common/blockaliasmap/BlockAliasMap.java | 14 +++++++++++++-
.../impl/InMemoryLevelDBAliasMapClient.java | 10 ++++++++++
.../blockaliasmap/impl/LevelDBFileRegionAliasMap.java | 6 ++++++
.../blockaliasmap/impl/TextFileRegionAliasMap.java | 6 ++++++
.../hadoop-hdfs/src/main/proto/AliasMapProtocol.proto | 8 ++++++++
.../hdfs/server/aliasmap/ITestInMemoryAliasMap.java | 10 +++++++---
.../impl/TestInMemoryLevelDBAliasMapClient.java | 13 ++++++++-----
.../impl/TestLevelDbMockAliasMapClient.java | 3 ++-
.../hadoop/hdfs/server/namenode/NullBlockAliasMap.java | 7 ++++++-
15 files changed, 117 insertions(+), 14 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
index 1c3a71f..e2f2f38 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
@@ -98,7 +98,6 @@ public class DFSConfigKeys extends CommonConfigurationKeys {
HdfsClientConfigKeys.DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS;
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS_DEFAULT = "0.0.0.0:50200";
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_BIND_HOST = "dfs.provided.aliasmap.inmemory.rpc.bind-host";
-
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_LEVELDB_DIR = "dfs.provided.aliasmap.inmemory.leveldb.dir";
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_BATCH_SIZE = "dfs.provided.aliasmap.inmemory.batch-size";
public static final int DFS_PROVIDED_ALIASMAP_INMEMORY_BATCH_SIZE_DEFAULT = 500;
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java
index 8d89c40..48da058 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java
@@ -57,6 +57,9 @@ public class AliasMapProtocolServerSideTranslatorPB
private static final WriteResponseProto VOID_WRITE_RESPONSE =
WriteResponseProto.newBuilder().build();
+ private static final RemoveResponseProto VOID_REMOVE_RESPONSE =
+ RemoveResponseProto.newBuilder().build();
+
@Override
public WriteResponseProto write(RpcController controller,
WriteRequestProto request) throws ServiceException {
@@ -72,6 +75,19 @@ public class AliasMapProtocolServerSideTranslatorPB
}
@Override
+ public RemoveResponseProto remove(RpcController controller,
+ RemoveRequestProto request) throws ServiceException {
+ try {
+ Block toRemove =
+ PBHelperClient.convert(request.getKey());
+ aliasMap.remove(toRemove);
+ return VOID_REMOVE_RESPONSE;
+ } catch (IOException e) {
+ throw new ServiceException(e);
+ }
+ }
+
+ @Override
public ReadResponseProto read(RpcController controller,
ReadRequestProto request) throws ServiceException {
try {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java
index d9e984b..dc5bd3b 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java
@@ -214,6 +214,21 @@ public class InMemoryAliasMapProtocolClientSideTranslatorPB
}
@Override
+ public void remove(@Nonnull Block block) throws IOException {
+ RemoveRequestProto request =
+ RemoveRequestProto
+ .newBuilder()
+ .setKey(PBHelperClient.convert(block))
+ .build();
+
+ try {
+ rpcProxy.remove(null, request);
+ } catch (ServiceException e) {
+ throw ProtobufHelper.getRemoteException(e);
+ }
+ }
+
+ @Override
public String getBlockPoolId() throws IOException {
try {
BlockPoolResponseProto response = rpcProxy.getBlockPoolId(null,
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java
index 8df27cd..777e673 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java
@@ -180,6 +180,11 @@ public class InMemoryAliasMap implements InMemoryAliasMapProtocol,
levelDb.put(extendedBlockDbFormat, providedStorageLocationDbFormat);
}
+ public void remove(@Nonnull Block block) throws IOException {
+ byte[] extendedBlockDbFormat = toProtoBufBytes(block);
+ levelDb.delete(extendedBlockDbFormat);
+ }
+
@Override
public String getBlockPoolId() {
return blockPoolID;
@@ -218,8 +223,7 @@ public class InMemoryAliasMap implements InMemoryAliasMapProtocol,
public static byte[] toProtoBufBytes(@Nonnull Block block)
throws IOException {
- BlockProto blockProto =
- PBHelperClient.convert(block);
+ BlockProto blockProto = PBHelperClient.convert(block);
ByteArrayOutputStream blockOutputStream = new ByteArrayOutputStream();
blockProto.writeTo(blockOutputStream);
return blockOutputStream.toByteArray();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java
index c3824e5..170d1f6 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java
@@ -99,6 +99,15 @@ public interface InMemoryAliasMapProtocol {
throws IOException;
/**
+ * Removed the block and it's associated {@link ProvidedStorageLocation}
+ * from the alias map.
+ * As this is for in memory alias map, we do not require the bpid.
+ * @param block
+ * @throws IOException
+ */
+ void remove(@Nonnull Block block) throws IOException;
+
+ /**
* Get the associated block pool id.
* @return the block pool id associated with the Namenode running
* the in-memory alias map.
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java
index 6a2fa56..ea6a2d8 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java
@@ -122,6 +122,11 @@ public class InMemoryLevelDBAliasMapServer implements InMemoryAliasMapProtocol,
}
@Override
+ public void remove(@Nonnull Block block) throws IOException {
+ aliasMap.remove(block);
+ }
+
+ @Override
public String getBlockPoolId() {
return blockPoolId;
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
index 897aefd..9905039 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
@@ -88,8 +88,20 @@ public abstract class BlockAliasMap<T extends BlockAlias> {
*/
public interface Options { }
- public abstract void store(U token) throws IOException;
+ /**
+ * Store the BlockAlias; this may be a destructive store, replacing old
+ * data.
+ * @param blockAlias
+ * @throws IOException
+ */
+ public abstract void store(U blockAlias) throws IOException;
+ /**
+ * Remove the block from the AliasMap.
+ * @param block
+ * @throws IOException
+ */
+ public abstract void remove(Block block) throws IOException;
}
/**
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java
index cacf8f1..7677258 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java
@@ -28,6 +28,7 @@ import org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol;
import org.apache.hadoop.hdfs.server.common.blockaliasmap.BlockAliasMap;
import org.apache.hadoop.hdfs.server.common.FileRegion;
import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.security.UserGroupInformation;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -144,11 +145,20 @@ public class InMemoryLevelDBAliasMapClient extends BlockAliasMap<FileRegion>
}
@Override
+ public void remove(Block block) throws IOException {
+ aliasMap.remove(block);
+ }
+
+ @Override
public void close() throws IOException {
}
}
InMemoryLevelDBAliasMapClient() {
+ if (UserGroupInformation.isSecurityEnabled()) {
+ throw new UnsupportedOperationException("Unable to start "
+ + "InMemoryLevelDBAliasMapClient as security is enabled");
+ }
aliasMaps = new ArrayList<>();
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java
index 6afe6bb..81756e1 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java
@@ -265,6 +265,12 @@ public class LevelDBFileRegionAliasMap
}
@Override
+ public void remove(Block block) throws IOException {
+ byte[] key = toProtoBufBytes(block);
+ db.delete(key);
+ }
+
+ @Override
public void close() throws IOException {
if (db != null) {
db.close();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java
index 4d65142..fb73b41 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java
@@ -458,6 +458,12 @@ public class TextFileRegionAliasMap
}
@Override
+ public void remove(Block block) throws IOException {
+ throw new RuntimeException("TextFileWriter does not support " +
+ "block removal");
+ }
+
+ @Override
public void close() throws IOException {
out.close();
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto
index 8050f35..16936c8 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto
@@ -36,6 +36,13 @@ message WriteRequestProto {
message WriteResponseProto {
}
+message RemoveRequestProto {
+ required BlockProto key = 1;
+}
+
+message RemoveResponseProto {
+}
+
message ReadRequestProto {
required BlockProto key = 1;
}
@@ -62,6 +69,7 @@ message BlockPoolResponseProto {
service AliasMapProtocolService {
rpc write(WriteRequestProto) returns(WriteResponseProto);
+ rpc remove(RemoveRequestProto) returns(RemoveResponseProto);
rpc read(ReadRequestProto) returns(ReadResponseProto);
rpc list(ListRequestProto) returns(ListResponseProto);
rpc getBlockPoolId(BlockPoolRequestProto) returns(BlockPoolResponseProto);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java
index 2785f68..a9f5c73 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java
@@ -64,7 +64,7 @@ public class ITestInMemoryAliasMap {
}
@Test
- public void readNotFoundReturnsNothing() throws IOException {
+ public void testReadNotFoundReturnsNothing() throws IOException {
Block block = new Block(42, 43, 44);
Optional<ProvidedStorageLocation> actualProvidedStorageLocationOpt
@@ -74,7 +74,7 @@ public class ITestInMemoryAliasMap {
}
@Test
- public void readWrite() throws Exception {
+ public void testReadWriteRemove() throws Exception {
Block block = new Block(42, 43, 44);
Path path = new Path("eagle", "mouse");
@@ -92,14 +92,18 @@ public class ITestInMemoryAliasMap {
Optional<ProvidedStorageLocation> actualProvidedStorageLocationOpt
= aliasMap.read(block);
+
assertTrue(actualProvidedStorageLocationOpt.isPresent());
assertEquals(expectedProvidedStorageLocation,
actualProvidedStorageLocationOpt.get());
+ aliasMap.remove(block);
+ actualProvidedStorageLocationOpt = aliasMap.read(block);
+ assertFalse(actualProvidedStorageLocationOpt.isPresent());
}
@Test
- public void list() throws IOException {
+ public void testIteration() throws IOException {
Block block1 = new Block(42, 43, 44);
Block block2 = new Block(43, 44, 45);
Block block3 = new Block(44, 45, 46);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java
index 1e42a88..c4fea99 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java
@@ -28,7 +28,7 @@ import org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap;
import org.apache.hadoop.hdfs.server.aliasmap.InMemoryLevelDBAliasMapServer;
import org.apache.hadoop.hdfs.server.common.blockaliasmap.BlockAliasMap;
import org.apache.hadoop.hdfs.server.common.FileRegion;
-import org.apache.hadoop.test.LambdaTestUtils;
+import org.apache.hadoop.net.NetUtils;
import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
@@ -36,6 +36,7 @@ import org.junit.Test;
import org.junit.rules.ExpectedException;
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_SERVICE_RPC_BIND_HOST_KEY;
+import static org.apache.hadoop.test.LambdaTestUtils.assertOptionalUnset;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
@@ -70,7 +71,7 @@ public class TestInMemoryLevelDBAliasMapClient {
@Before
public void setUp() throws IOException {
conf = new Configuration();
- int port = 9876;
+ int port = NetUtils.getFreeSocketPort();
conf.set(DFSConfigKeys.DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS,
"localhost:" + port);
@@ -92,7 +93,7 @@ public class TestInMemoryLevelDBAliasMapClient {
}
@Test
- public void writeRead() throws Exception {
+ public void writeReadRemove() throws Exception {
levelDBAliasMapServer.setConf(conf);
levelDBAliasMapServer.start();
inMemoryLevelDBAliasMapClient.setConf(conf);
@@ -110,6 +111,8 @@ public class TestInMemoryLevelDBAliasMapClient {
Optional<FileRegion> fileRegion = reader.resolve(block);
assertEquals(new FileRegion(block, providedStorageLocation),
fileRegion.get());
+ writer.remove(block);
+ assertOptionalUnset("Block should not exist", reader.resolve(block));
}
@Test
@@ -354,7 +357,7 @@ public class TestInMemoryLevelDBAliasMapClient {
@Test
public void testServerBindHost() throws Exception {
conf.set(DFS_NAMENODE_SERVICE_RPC_BIND_HOST_KEY, "0.0.0.0");
- writeRead();
+ writeReadRemove();
}
@Test
@@ -375,7 +378,7 @@ public class TestInMemoryLevelDBAliasMapClient {
BlockAliasMap.Reader<FileRegion> reader =
inMemoryLevelDBAliasMapClient.getReader(null, BPID);
- LambdaTestUtils.assertOptionalUnset("Expected empty BlockAlias",
+ assertOptionalUnset("Expected empty BlockAlias",
reader.resolve(block1));
}
}
\ No newline at end of file
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java
index 534bc36..ab1fa63 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java
@@ -26,6 +26,7 @@ import org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation;
import org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap;
import org.apache.hadoop.hdfs.server.aliasmap.InMemoryLevelDBAliasMapServer;
import org.apache.hadoop.hdfs.server.common.FileRegion;
+import org.apache.hadoop.net.NetUtils;
import org.iq80.leveldb.DBException;
import org.junit.After;
import org.junit.Before;
@@ -56,7 +57,7 @@ public class TestLevelDbMockAliasMapClient {
levelDBAliasMapServer = new InMemoryLevelDBAliasMapServer(
(config, blockPoolID) -> aliasMapMock, bpid);
conf = new Configuration();
- int port = 9877;
+ int port = NetUtils.getFreeSocketPort();
conf.set(DFSConfigKeys.DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS,
"localhost:" + port);
diff --git a/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java b/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java
index 41e202d..e439bab 100644
--- a/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java
+++ b/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java
@@ -74,7 +74,12 @@ public class NullBlockAliasMap extends BlockAliasMap<FileRegion> {
throws IOException {
return new Writer<FileRegion>() {
@Override
- public void store(FileRegion token) throws IOException {
+ public void store(FileRegion blockAlias) throws IOException {
+ // do nothing
+ }
+
+ @Override
+ public void remove(Block block) throws IOException {
// do nothing
}
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/04: HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP
command in Datanode. Contributed by Ewan Higgs.
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 1abb8f3dab1d5258c233c0bdbfef7980f9ceaa4d
Author: Virajith Jalaparti <vi...@apache.org>
AuthorDate: Wed Aug 1 12:13:31 2018 -0700
HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode. Contributed by Ewan Higgs.
---
.../org/apache/hadoop/hdfs/BlockInputStream.java | 52 +++++++++
.../hdfs/server/datanode/BPOfferService.java | 6 +
.../hadoop/hdfs/server/datanode/DataNode.java | 18 +++
.../SyncServiceSatisfierDatanodeWorker.java | 97 ++++++++++++++++
.../SyncTaskExecutionFeedbackCollector.java | 54 +++++++++
.../executor/BlockSyncOperationExecutor.java | 122 +++++++++++++++++++++
.../executor/BlockSyncReaderFactory.java | 92 ++++++++++++++++
.../syncservice/executor/BlockSyncTaskRunner.java | 69 ++++++++++++
.../apache/hadoop/hdfs/TestBlockInputStream.java | 84 ++++++++++++++
.../executor/TestBlockSyncOperationExecutor.java | 94 ++++++++++++++++
10 files changed, 688 insertions(+)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockInputStream.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockInputStream.java
new file mode 100644
index 0000000..152f83e
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockInputStream.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs;
+
+import java.io.IOException;
+import java.io.InputStream;
+
+/**
+ * Facade around BlockReader that indeed implements the InputStream interface.
+ */
+public class BlockInputStream extends InputStream {
+ private final BlockReader blockReader;
+
+ public BlockInputStream(BlockReader blockReader) {
+ this.blockReader = blockReader;
+ }
+
+ @Override
+ public int read() throws IOException {
+ byte[] b = new byte[1];
+ int c = blockReader.read(b, 0, b.length);
+ if (c > 0) {
+ return b[0];
+ } else {
+ return -1;
+ }
+ }
+
+ @Override
+ public int read(byte b[], int off, int len) throws IOException {
+ return blockReader.read(b, off, len);
+ }
+
+ @Override
+ public long skip(long n) throws IOException {
+ return blockReader.skip(n);
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
index 3233e2c..1d63fa9 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
@@ -797,6 +797,12 @@ class BPOfferService {
((BlockECReconstructionCommand) cmd).getECTasks();
dn.getErasureCodingWorker().processErasureCodingTasks(ecTasks);
break;
+ case DatanodeProtocol.DNA_BACKUP:
+ LOG.info("DatanodeCommand action: DNA_BACKUP");
+ Collection<BlockSyncTask> backupTasks =
+ ((SyncCommand) cmd).getSyncTasks();
+ dn.getSyncServiceSatisfierDatanodeWorker().processSyncTasks(backupTasks);
+ break;
default:
LOG.warn("Unknown DatanodeCommand action: " + cmd.getAction());
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
index f322119..c2085a1 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
@@ -394,6 +394,7 @@ public class DataNode extends ReconfigurableBase
private String dnUserName = null;
private BlockRecoveryWorker blockRecoveryWorker;
private ErasureCodingWorker ecWorker;
+ private SyncServiceSatisfierDatanodeWorker syncServiceSatisfierDatanodeWorker;
private final Tracer tracer;
private final TracerConfigurationManager tracerConfigurationManager;
private static final int NUM_CORES = Runtime.getRuntime()
@@ -1456,6 +1457,9 @@ public class DataNode extends ReconfigurableBase
ecWorker = new ErasureCodingWorker(getConf(), this);
blockRecoveryWorker = new BlockRecoveryWorker(this);
+ syncServiceSatisfierDatanodeWorker =
+ new SyncServiceSatisfierDatanodeWorker(getConf(), this);
+ syncServiceSatisfierDatanodeWorker.start();
blockPoolManager = new BlockPoolManager(this);
blockPoolManager.refreshNamenodes(getConf());
@@ -2037,6 +2041,11 @@ public class DataNode extends ReconfigurableBase
}
}
+ // stop syncServiceSatisfierDatanodeWorker
+ if (syncServiceSatisfierDatanodeWorker != null) {
+ syncServiceSatisfierDatanodeWorker.stop();
+ }
+
List<BPOfferService> bposArray = (this.blockPoolManager == null)
? new ArrayList<BPOfferService>()
: this.blockPoolManager.getAllNamenodeThreads();
@@ -2192,6 +2201,11 @@ public class DataNode extends ReconfigurableBase
notifyAll();
}
tracer.close();
+
+ // Waiting to finish backup SPS worker thread.
+ if (syncServiceSatisfierDatanodeWorker != null) {
+ syncServiceSatisfierDatanodeWorker.waitToFinishWorkerThread();
+ }
}
/**
@@ -3739,6 +3753,10 @@ public class DataNode extends ReconfigurableBase
return this.diskBalancer;
}
+ public SyncServiceSatisfierDatanodeWorker getSyncServiceSatisfierDatanodeWorker() {
+ return syncServiceSatisfierDatanodeWorker;
+ }
+
/**
* Construct DataTransfer in {@link DataNode#transferBlock}, the
* BlockConstructionStage is PIPELINE_SETUP_CREATE and clientName is "".
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SyncServiceSatisfierDatanodeWorker.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SyncServiceSatisfierDatanodeWorker.java
new file mode 100644
index 0000000..7216e8f
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SyncServiceSatisfierDatanodeWorker.java
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.SyncTaskExecutionFeedbackCollector;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.executor.BlockSyncOperationExecutor;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.executor.BlockSyncReaderFactory;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.executor.BlockSyncTaskRunner;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.util.concurrent.HadoopExecutors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.RejectedExecutionException;
+import java.util.concurrent.TimeUnit;
+
+public class SyncServiceSatisfierDatanodeWorker {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(SyncServiceSatisfierDatanodeWorker.class);
+
+ private ExecutorService executorService;
+ private BlockSyncOperationExecutor syncOperationExecutor;
+ private SyncTaskExecutionFeedbackCollector syncTaskExecutionFeedbackCollector;
+
+ public SyncServiceSatisfierDatanodeWorker(Configuration conf, DataNode dataNode) throws IOException {
+ this.executorService = HadoopExecutors.newFixedThreadPool(4);
+ this.syncOperationExecutor =
+ BlockSyncOperationExecutor.createOnDataNode(conf,
+ (locatedBlock, config) -> {
+ try {
+ return BlockSyncReaderFactory.createBlockReader(dataNode, locatedBlock, config);
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ );
+ this.syncTaskExecutionFeedbackCollector = new SyncTaskExecutionFeedbackCollector();
+ }
+
+
+ public void start() {
+ this.executorService = HadoopExecutors.newFixedThreadPool(4);
+ }
+
+ public void stop() {
+ this.executorService.shutdown();
+ }
+
+ public void waitToFinishWorkerThread() {
+ try {
+ this.executorService.awaitTermination(3, TimeUnit.MINUTES);
+ } catch (InterruptedException e) {
+ LOG.warn("SyncServiceSatisfierDatanodeWorker interrupted during waiting for finalization.");
+ Thread.currentThread().interrupt();
+ }
+ }
+
+ public void processSyncTasks(Collection<BlockSyncTask> blockSyncTasks) {
+
+ LOG.debug("Received SyncTasks: {}", blockSyncTasks);
+ for (BlockSyncTask blockSyncTask : blockSyncTasks) {
+ try {
+ executorService.submit(new BlockSyncTaskRunner(blockSyncTask,
+ syncOperationExecutor,
+ syncTaskExecutionFeedback -> syncTaskExecutionFeedbackCollector
+ .addFeedback(syncTaskExecutionFeedback)));
+ } catch (RejectedExecutionException e) {
+ LOG.warn("BlockSyncTask {} for {} was rejected: {}",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI(),
+ e.getCause());
+ }
+ }
+ }
+
+ public SyncTaskExecutionFeedbackCollector getSyncTaskExecutionFeedbackCollector() {
+ return syncTaskExecutionFeedbackCollector;
+ }
+
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/SyncTaskExecutionFeedbackCollector.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/SyncTaskExecutionFeedbackCollector.java
new file mode 100644
index 0000000..41cd441
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/SyncTaskExecutionFeedbackCollector.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice;
+
+import com.google.common.collect.Lists;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTaskExecutionFeedback;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
+
+import java.util.List;
+
+/**
+ * DatanodeSyncTaskExecutionFeedbackCollector collects feedback for the
+ * sync service tracker to determine what has happened and report statistics.
+ */
+public class SyncTaskExecutionFeedbackCollector {
+
+ private List<BlockSyncTaskExecutionFeedback> collectedFeedback;
+
+ public SyncTaskExecutionFeedbackCollector() {
+ this.collectedFeedback = Lists.newArrayList();
+ }
+
+ public void addFeedback(BlockSyncTaskExecutionFeedback feedback) {
+ synchronized (this) {
+ collectedFeedback.add(feedback);
+ }
+ }
+
+ public BulkSyncTaskExecutionFeedback packageFeedbackForHeartbeat() {
+
+ List<BlockSyncTaskExecutionFeedback> feedbackForHeartbeat;
+
+ synchronized (this) {
+ feedbackForHeartbeat = collectedFeedback;
+ collectedFeedback = Lists.newArrayList();
+ }
+ return new BulkSyncTaskExecutionFeedback(feedbackForHeartbeat);
+
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncOperationExecutor.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncOperationExecutor.java
new file mode 100644
index 0000000..7fde230
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncOperationExecutor.java
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.BBUploadHandle;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.MultipartUploader;
+import org.apache.hadoop.fs.MultipartUploaderFactory;
+import org.apache.hadoop.fs.PartHandle;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hdfs.BlockInputStream;
+import org.apache.hadoop.hdfs.BlockReader;
+import org.apache.hadoop.hdfs.DFSInputStream;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.SequenceInputStream;
+import java.net.URI;
+import java.nio.ByteBuffer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.Enumeration;
+import java.util.List;
+import java.util.Vector;
+
+/**
+ * BlockSyncOperationExecutor writes the blocks to the sync service remote
+ * endpoint.
+ */
+public class BlockSyncOperationExecutor {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(BlockSyncOperationExecutor.class);
+
+ private Configuration conf;
+ private BiFunction<LocatedBlock, Configuration, BlockReader>
+ createBlockReader;
+ private Function<FileSystem, MultipartUploader> multipartUploaderSupplier;
+
+ @VisibleForTesting
+ BlockSyncOperationExecutor(Configuration conf,
+ BiFunction<LocatedBlock, Configuration, BlockReader> createBlockReader,
+ Function<FileSystem, MultipartUploader> multipartUploaderSupplier) {
+ this.conf = conf;
+ this.createBlockReader = createBlockReader;
+ this.multipartUploaderSupplier = multipartUploaderSupplier;
+ }
+
+ public static BlockSyncOperationExecutor createOnDataNode(Configuration conf,
+ BiFunction<LocatedBlock, Configuration, BlockReader> createBlockReader) {
+ return new BlockSyncOperationExecutor(conf,
+ createBlockReader,
+ fs -> {
+ try {
+ return MultipartUploaderFactory.get(fs, conf);
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ });
+ }
+
+ public SyncTaskExecutionResult execute(BlockSyncTask blockSyncTask)
+ throws Exception {
+ LOG.info("Executing MetadataSyncTask {} (on {})",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI());
+
+ return doMultiPartPart(
+ blockSyncTask.getRemoteURI(),
+ blockSyncTask.getLocatedBlocks(),
+ blockSyncTask.getPartNumber(),
+ blockSyncTask.getUploadHandle(),
+ blockSyncTask.getOffset(),
+ blockSyncTask.getLength());
+ }
+
+ private SyncTaskExecutionResult doMultiPartPart(URI uri,
+ List<LocatedBlock> locatedBlocks, int partNumber, byte[] uploadHandle,
+ int offset, long length) throws IOException {
+ FileSystem fs = FileSystem.get(uri, conf);
+ Path filePath = new Path(uri);
+ Vector<InputStream> inputStreams = new Vector<>(locatedBlocks.size());
+ for (int i = 0; i < locatedBlocks.size(); ++i) {
+ LocatedBlock locatedBlock = locatedBlocks.get(i);
+ BlockReader reader = createBlockReader.apply(locatedBlock, conf);
+ if (i == 0) {
+ reader.skip(offset);
+ }
+ BlockInputStream inputStream = new BlockInputStream(reader);
+ inputStreams.add(inputStream);
+ }
+ Enumeration<InputStream> streamEnumeration = inputStreams.elements();
+ SequenceInputStream inputStream =
+ new SequenceInputStream(streamEnumeration);
+ MultipartUploader mpu = multipartUploaderSupplier.apply(fs);
+ PartHandle partHandle = mpu.putPart(filePath, inputStream,
+ partNumber, BBUploadHandle.from(ByteBuffer.wrap(uploadHandle)), length);
+ return new SyncTaskExecutionResult(partHandle.bytes(), length);
+ }
+}
+
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncReaderFactory.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncReaderFactory.java
new file mode 100644
index 0000000..cc5eb5c
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncReaderFactory.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FsTracer;
+import org.apache.hadoop.hdfs.BlockReader;
+import org.apache.hadoop.hdfs.ClientContext;
+import org.apache.hadoop.hdfs.DFSUtilClient;
+import org.apache.hadoop.hdfs.client.impl.BlockReaderFactory;
+import org.apache.hadoop.hdfs.client.impl.DfsClientConf;
+import org.apache.hadoop.hdfs.net.Peer;
+import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
+import org.apache.hadoop.hdfs.protocol.HdfsConstants;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier;
+import org.apache.hadoop.hdfs.server.datanode.CachingStrategy;
+import org.apache.hadoop.hdfs.server.datanode.DataNode;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.security.token.Token;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.util.EnumSet;
+
+/**
+ * BlockSyncReaderFactory constructs a block reader in the Datanode for the
+ * Sync Command to read blocks that will be written to the synchronization
+ * remote endpoint.
+ */
+public class BlockSyncReaderFactory {
+
+ public static BlockReader createBlockReader(DataNode dataNode,
+ LocatedBlock locatedBlock, Configuration conf) throws IOException {
+ ClientContext clientContext = ClientContext.getFromConf(conf);
+ Token<BlockTokenIdentifier> accessToken = dataNode.getBlockAccessToken(
+ locatedBlock.getBlock(),
+ EnumSet.of(BlockTokenIdentifier.AccessMode.READ),
+ locatedBlock.getStorageTypes(), locatedBlock.getStorageIDs());
+
+ DatanodeInfo datanodeInfo = locatedBlock.getLocations()[0];
+
+ Socket socked = NetUtils.getDefaultSocketFactory(conf).createSocket();
+ InetSocketAddress resolvedAddress =
+ datanodeInfo.getResolvedAddress();
+ socked.connect(resolvedAddress);
+
+ return new BlockReaderFactory(new DfsClientConf(conf))
+ .setConfiguration(conf)
+ .setBlock(locatedBlock.getBlock())
+ .setBlockToken(accessToken)
+ .setStartOffset(0)
+ .setLength(locatedBlock.getBlock().getNumBytes())
+ .setInetSocketAddress(datanodeInfo.getResolvedAddress())
+ .setVerifyChecksum(true)
+ .setDatanodeInfo(datanodeInfo)
+ .setClientName("BlockSyncOperationExecutor")
+ .setCachingStrategy(CachingStrategy.newDefaultStrategy())
+ .setRemotePeerFactory((addr, blockToken, datanodeId) -> {
+ Peer peer = null;
+ Socket sock = NetUtils.getDefaultSocketFactory(conf).createSocket();
+ try {
+ sock.connect(addr, HdfsConstants.READ_TIMEOUT);
+ sock.setSoTimeout(HdfsConstants.READ_TIMEOUT);
+ peer = DFSUtilClient.peerFromSocket(sock);
+ } finally {
+ if (peer == null) {
+ IOUtils.closeQuietly(sock);
+ }
+ }
+ return peer;
+ })
+ .setClientCacheContext(clientContext)
+ .build();
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncTaskRunner.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncTaskRunner.java
new file mode 100644
index 0000000..660e39e
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncTaskRunner.java
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTaskExecutionFeedback;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * BlockSyncTaskRunner glues together the sync task and the feedback reporting.
+ */
+import java.util.function.Consumer;
+
+public class BlockSyncTaskRunner implements Runnable {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(BlockSyncTaskRunner.class);
+
+ private BlockSyncTask blockSyncTask;
+ private BlockSyncOperationExecutor syncOperationExecutor;
+
+ private Consumer<BlockSyncTaskExecutionFeedback> publishOutcomeCallback;
+
+ public BlockSyncTaskRunner(BlockSyncTask blockSyncTask,
+ BlockSyncOperationExecutor syncOperationExecutor,
+ Consumer<BlockSyncTaskExecutionFeedback> publishOutcomeCallback) {
+ this.blockSyncTask = blockSyncTask;
+ this.syncOperationExecutor = syncOperationExecutor;
+ this.publishOutcomeCallback = publishOutcomeCallback;
+ }
+
+ @Override
+ public void run() {
+ LOG.info("Executing BlockyncTask {} (on {})",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI());
+ try {
+ SyncTaskExecutionResult result =
+ syncOperationExecutor.execute(blockSyncTask);
+ publishOutcomeCallback.accept(BlockSyncTaskExecutionFeedback
+ .finishedSuccessfully(blockSyncTask.getSyncTaskId(),
+ blockSyncTask.getSyncMountId(),
+ result));
+ } catch (Exception e) {
+ LOG.error(
+ String.format("Exception executing MetadataSyncTask %s (on %s)",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI()), e);
+ publishOutcomeCallback.accept(BlockSyncTaskExecutionFeedback
+ .failedWithException(blockSyncTask.getSyncTaskId(),
+ blockSyncTask.getSyncMountId(), e));
+ }
+ }
+
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockInputStream.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockInputStream.java
new file mode 100644
index 0000000..43d4881
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockInputStream.java
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.Mock;
+import org.mockito.runners.MockitoJUnitRunner;
+
+import java.io.IOException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+/**
+ * Test the BlockInputStream facade.
+ */
+@RunWith(MockitoJUnitRunner.class)
+public class TestBlockInputStream {
+ @Mock
+ private BlockReader blockReaderMock;
+
+ @Test
+ public void testBlockInputStreamReadChar() {
+ BlockInputStream is = new BlockInputStream(blockReaderMock);
+
+ try {
+ when(blockReaderMock.read(any(), eq(0), eq(1)))
+ .thenReturn(32);
+ // Making the mock perform the side effect of writing to buf is nasty.
+ is.read();
+ verify(blockReaderMock, times(1)).read(any(), eq(0), eq(1));
+ } catch (IOException e) {
+ fail("Could not even mock out read function.");
+ }
+ }
+
+ @Test
+ public void testBlockInputStreamReadBuf() {
+ BlockInputStream is = new BlockInputStream(blockReaderMock);
+
+ try {
+ byte[] buf = new byte[1024];
+ when(blockReaderMock.read(buf, 0, buf.length)).thenReturn(1024);
+ is.read(buf, 0, buf.length);
+ verify(blockReaderMock, times(1)).read(buf, 0, buf.length);
+ } catch (IOException e) {
+ fail("Could not even mock out read function.");
+ }
+ }
+
+ @Test
+ public void testBlockInputStreamSkip() {
+ BlockInputStream is = new BlockInputStream(blockReaderMock);
+
+ try {
+ when(blockReaderMock.skip(10)).thenReturn(10L);
+ long ret = is.skip(10);
+ assertEquals(10, ret);
+ verify(blockReaderMock, times(1)).skip(10L);
+ } catch (IOException e) {
+ fail("Could not even mock out skip function.");
+ }
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/TestBlockSyncOperationExecutor.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/TestBlockSyncOperationExecutor.java
new file mode 100644
index 0000000..e16d086
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/TestBlockSyncOperationExecutor.java
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.BBPartHandle;
+import org.apache.hadoop.fs.MultipartUploader;
+import org.apache.hadoop.fs.PartHandle;
+import org.apache.hadoop.hdfs.BlockReader;
+import org.apache.hadoop.hdfs.protocol.Block;
+import org.apache.hadoop.hdfs.protocol.ExtendedBlock;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.Mock;
+import org.mockito.runners.MockitoJUnitRunner;
+
+import java.net.URI;
+import java.nio.ByteBuffer;
+import java.util.List;
+import java.util.UUID;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.anyInt;
+import static org.mockito.Matchers.anyLong;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+@RunWith(MockitoJUnitRunner.class)
+public class TestBlockSyncOperationExecutor {
+
+ @Mock
+ private BlockReader blockReaderMock;
+
+ @Mock
+ private MultipartUploader multipartUploaderMock;
+
+ @Test
+ public void executeMultipartPutFileSyncTask() throws Exception {
+ long blockLength = 42L;
+ Configuration conf = new Configuration();
+ BlockSyncOperationExecutor blockSyncOperationExecutor =
+ new BlockSyncOperationExecutor(conf,
+ (locatedBlock, config) -> blockReaderMock,
+ fs -> multipartUploaderMock);
+ String uploadHandleStr = "uploadHandle";
+ byte[] uploadHandle = uploadHandleStr.getBytes();
+ ByteBuffer byteBuffer = ByteBuffer.wrap(uploadHandle);
+ PartHandle partHandle = BBPartHandle.from(byteBuffer);
+ when(multipartUploaderMock.putPart(any(), any(), anyInt(), any(),
+ anyLong())).thenReturn(partHandle);
+ UUID syncTaskId = UUID.randomUUID();
+ URI remoteUri = new URI("remoteUri");
+ String syncMountId = "syncMountId";
+ Block block = new Block(42L, blockLength, 44L);
+ ExtendedBlock extendedBlock1 = new ExtendedBlock("poolId", block);
+ LocatedBlock locatedBlock = new LocatedBlock(extendedBlock1, null);
+ List<LocatedBlock> locatedBlocks = Lists.newArrayList(locatedBlock);
+ Integer partNumber = 85;
+ final int offset = 0;
+ final long length = locatedBlock.getBlockSize();
+
+
+ BlockSyncTask blockSyncTask = new BlockSyncTask(syncTaskId, remoteUri,
+ locatedBlocks, partNumber, uploadHandle, offset, length, syncMountId);
+
+ SyncTaskExecutionResult result =
+ blockSyncOperationExecutor.execute(blockSyncTask);
+
+ assertThat(result).isNotNull();
+ Long actualLength = result.getNumberOfBytes();
+ assertThat(actualLength).isEqualTo(blockLength);
+ assertThat(result.getResult()).isEqualTo(partHandle.bytes());
+ }
+}
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 04/04: HDFS-12090. Fixup TestBlockReportLease and
TestBPOfferService in branch rebase.
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit afdb20b1e16fb12bd22042ec1c69a34646f7ab26
Author: Ewan Higgs <ew...@wdc.com>
AuthorDate: Mon Dec 2 17:53:37 2019 +0100
HDFS-12090. Fixup TestBlockReportLease and TestBPOfferService in branch rebase.
---
.../hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java | 2 +-
.../org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java
index 40408b1..6fbae09 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockReportLease.java
@@ -92,7 +92,7 @@ public class TestBlockReportLease {
// Send heartbeat and request full block report lease
HeartbeatResponse hbResponse = rpcServer.sendHeartbeat(
dnRegistration, storages, 0, 0, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT);
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT, null);
DelayAnswer delayer = new DelayAnswer(BlockManager.LOG);
doAnswer(delayer).when(spyBlockManager).processReport(
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
index 26c07d5..822677a 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
@@ -998,7 +998,8 @@ public class TestBPOfferService {
Mockito.any(VolumeFailureSummary.class),
Mockito.anyBoolean(),
Mockito.any(SlowPeerReports.class),
- Mockito.any(SlowDiskReports.class)))
+ Mockito.any(SlowDiskReports.class),
+ Mockito.any(BulkSyncTaskExecutionFeedback.class)))
//heartbeat to old NN instance
.thenAnswer(new HeartbeatAnswer(0))
//heartbeat to new NN instance with Register Command
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org