You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by aa...@apache.org on 2019/04/04 02:00:59 UTC
[hadoop] branch HDFS-13891 updated (dea3798 -> 2577e3e)
This is an automated email from the ASF dual-hosted git repository.
aajisaka pushed a change to branch HDFS-13891
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
discard dea3798 HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
discard 42dc40e HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
discard 0c686b7 HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
discard c359a52 HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
discard bddb4cb HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
discard e1d3494 HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
discard 1d1dc8e HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
discard fc8dd2e HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
discard 82c6610 HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
discard cb54d8b HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
discard 5d54818 HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
discard fd40b6c HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
discard 0bbe324 HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
discard a1e44c7 HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
discard 9b1c43e HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
discard 2d6473a HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
discard 63e160f HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
discard 1edeaa5 HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
discard 531ed36 HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
discard 5928923 HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.
discard dc0e90d HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
discard f16f666 HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
discard 4ffdb3e HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
discard b8fb3df HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
discard 89836cd HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
discard e087ea1 HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
discard 89f1884 HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
discard d5f5256 HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
discard a4cb78c HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
discard 7d61701 HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
discard 8724b68 HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
discard 8493efa HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
discard 8693e6a HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
discard 0022423 HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
discard dff8dea HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
discard bd1e6b4 HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
discard e0fe409 HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
discard da42dca HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
discard 0ca0ea2 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
discard 3886266 Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
discard a78ba70 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
discard 1048530 HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
discard 58466f8 HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
discard b77cc2a HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
discard afe9ac9 HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
discard 20e0514 HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
discard a046221 HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
discard 4db58c6 HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
discard 4519935 HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
discard ee6ea6a HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
discard 0420816 HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
discard 57163c4 HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
discard 4cba216 HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
discard a54839e Revert "HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng."
discard fe6b2b2 YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin.
discard bc6fe7a Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client."
discard 5a15f7b HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen.
add eea40cb HDDS-1204. Fix ClassNotFound issue with javax.xml.bind.DatatypeConverter used by DefaultProfile. Contributed by Xiaoyu Yao.
add 8b72aea Merge pull request #542 from xiaoyuyao/HDDS-1204
add 4a3cdde HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar.
add 6fef6fc HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao.
add 78c8a33 Fix checkstyle issue
add 9e53088 Revert "HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contr…" (#544)
add fc17ba1 HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen.
add b18c1c2 Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client."
add e20b5ef YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin.
add 6c4d566 Revert "HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng."
add bd8d299 YARN-7477. Moving logging APIs over to slf4j in hadoop-yarn-common. Contributed by Prabhu Joseph.
add 18ea0c1 HDFS-14182. Datanode usage histogram is clicked to show ip list. Contributed by fengchuang.
add 15098df Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
add eed8b79 HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao. (#545)
add 387dbe5 HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star.
add 10b802b HDDS-1136 : Add metric counters to capture the RocksDB checkpointing statistics. Contributed by Aravindan Vijayan.
add cb0fa0c HDFS-14321. Fix -Xcheck:jni issues in libhdfs, run ctest with -Xcheck:jni enabled. Contributed by Sahil Takiar.
add 90c37ac HDDS-623. On SCM UI, Node Manager info is empty (#523)
add 9fcd89a HADOOP-16148. Cleanup LineReader Unit Test.
add fe7551f HADOOP-16162. Remove unused Job Summary Appender configurations from log4j.properties
add e40e2d6 YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
add 4b7313e HDFS-14336. Fix checkstyle for NameNodeMXBean. Contributed by Danny Becker.
add 0aefe28 YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
add f940ab2 HDFS-7663. Erasure Coding: Append on striped file. Contributed by Ayush Saxena.
add 05df151 HADOOP-16163. NPE in setup/teardown of ITestAbfsDelegationTokens.
add 686c014 HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately.
add 2576401 YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
add 353b90d HDDS-1156. testDelegationToken is failing in TestSecureOzoneCluster. Contributed by Ajay Kumar.
add 5fcea54 HDDS-935. Avoid creating an already created container on a datanode in case of disk removal followed by datanode restart. Contributed by Shashikant Banerjee.
add 7b42e0e YARN-7266. Fixed deadlock in Timeline Server thread initialization. Contributed by Prabhu Joseph
add 12402b7 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client.
add 02da06d HDDS-1222. Remove TestContainerSQLCli unit test stub. Contributed by Elek, Marton.
add 549d54f Merge pull request #556 from elek/HDDS-1222
add 296259e HDDS-1219. TestContainerActionsHandler.testCloseContainerAction has an intermittent failure. Contributed by Elek, Marton.
add df99137 Merge pull request #555 from elek/HDDS-1219
add 3ef1235 HDDS-1218. Do the dist-layout-stitching for Ozone after the test-compile phase. Contributed by Marton Elek.
add 3c5b713 HADOOP-16150. Added concat method to ChecksumFS as unsupported operation. Contributed by Steve Loughran
add 7fd8901 HDDS-1171. Add benchmark for OM and OM client in Genesis. Contributed by Lokesh Jain.
add 313e8b9 HDDS-1193. Refactor ContainerChillModeRule and DatanodeChillMode rule. (#534)
add 7f636b4 HDDS-919. Enable prometheus endpoints for Ozone datanodes (#502)
add 1ad5bfc HDDS-1188. Implement a skeleton patch for Recon server with initial set of interfaces. Contributed by Siddharth Wagle.
add 945b504 HDFS-14326. Add CorruptFilesCount to JMX. Contributed by Danny Becker.
add 62e89dc HDDS-1184. Parallelization of write chunks in datanodes is broken. Contributed by Shashikant Banerjee.
add 46045c5 YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth.
add c79f139 YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB.
add 129fd5d HDDS-1208. ContainerStateMachine should set chunk data as state machine data for ratis. Contributed by Lokesh Jain.
add 9d87247 HDDS-1216. Change name of ozoneManager service in docker compose files to om.
add 2c3ec37 HDDS-1210. Ratis pipeline creation doesn't check raft client reply status during initialization. Contributed by Mukul Kumar Singh.
add 45f976f HDDS-1113. Remove default dependencies from hadoop-ozone project. Contributed by Elek, Marton.
add 01ada40 YARN-9348. Application catalog build system bug fixes. Contributed by Eric Yang
add f5a4b43 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
add 6192c1f Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
add 618e009 HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
add a55fc36 HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values.
add bb12e81 HDDS-1175. Serve read requests directly from RocksDB. (#557)
add 09a9938 SUBMARINE-41. Fix ASF warnings related to submarine target files. Contributed by Dinesh Chitlangia. This closes #559.
add 491313a YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
add 0eba407 HDDS-1225. Provide docker-compose for OM HA.
add 2b94e51 HADOOP-16114. NetUtils#canonicalizeHost gives different value for same host.
add 475011b HDDS-1226. ozone-filesystem jar missing in hadoop classpath
add 1bc282e HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam.
add 39b4a37 YARN-9341. Fixed enentrant lock usage in YARN project. Contributed by Prabhu Joseph
add 064f38b HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547)
add 373705f YARN-9239. Document docker registry deployment with Ozone CSI driver. Contributed by Eric Yang.
add fb851c9 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan.
add de15a66 YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko.
add e026041 HADOOP-16169. ABFS: Bug fix for getPathProperties.
add 341c076 HDDS-1196. Add a ReplicationStartTimer class. (#567)
add c072458 HDDS-1213. Support plain text S3 MPU initialization request.
add 0cbe9ad HADOOP-16109. Parquet reading S3AFileSystem causes EOF
add 8458ced HDDS-1238. Fix Ratis Snapshot creation error if the snapshot file already exists in ContainerStateMachine. Contributed by Shashikant Banerjee.
add 11db469 HDDS-1235. BaseHttpServer NPE is HTTP policy is HTTPS_ONLY. Contributed by Xiaoyu Yao.
add bb8ad09 HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore (Contributed by Shweta Yakkali via Daniel Templeton)
add ddede7a HADOOP-16174. Disable wildfly logs to the console.
add 73e1801 HDDS-1242. In S3 when bucket already exists, it should just return location.
add 68aa2cc HDDS-1240. Fix check style issues caused by HDDS-1196.
add f060514 MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang.
add ebb5fa1 HDDS-1220. KeyManager#openKey should release the bucket lock before doing an allocateBlock. Contributed by Lokesh Jain.
add 4fc493e HDDS-1236. Fix incorrect Ozone ClientProtocol KerberosInfo annotation. Contributed by Xiaoyu Yao. (#573)
add b4aa24d HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee.
add fa7a0b2 YARN-8805. Automatically convert the launch command to the exec form when using entrypoint support
add 4ad295a HDFS-14270.[SBN Read] Add trace level logging for stateId in RPC Server. Contributed by Shweta Yakkali.
add 599e926 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar.
add 2a54fea HDFS-14347. [SBN Read] Restore a comment line mistakenly removed in ProtobufRpcEngine. Contributed by Fengnan Li.
add c730786 HDDS-1245. OM delegation expiration time should use Time.now instead … (#587)
add d17e31e HDDS-1214. Enable tracing for the datanode read/write path. Contributed by Elek, Marton.
add daf4660 HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh.
add c24af4b YARN-9336. JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController. Contributed by Tarun Parimi.
add 2778259 HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593)
add aab7b77 YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
add 34b1406 HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell.
add dcb0de8 HDDS-1043. Enable token based authentication for S3 api
add c449cde HDDS-807. Period should be an invalid character in bucket names. Contributed by Siddharth Wagle.
add 024b3ba Revert "YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang"
add 55069bf Revert "YARN-9348. Application catalog build system bug fixes."
add ab574ff Revert "YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang"
add 24793d2 HDDS-1163. Basic framework for Ozone Data Scrubber. Contributed by Supratim Deka.
add 8e1539e YARN-9266. General improvements in IntelFpgaOpenclPlugin. Contributed by Peter Bacsko.
add 67cc24a YARN-9381. The yarn-default.xml has two identical property named yarn.timeline-service.http-cross-origin.enabled. Contributed by Abhishek Modi.
add 17a3e14 YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi.
add 159a715 MAPREDUCE-7192. JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable. Contributed by Jiandan Yang.
add d602d46 HDDS-1256. Fix failure in TestKeyValueContainerCheck. Contributed by Supratim Deka.
add 0d62753 HDDS-1128. Create stateful manager class for the pipeline creation scheduling.
add c1649bf HDDS-1226. Addendum. ozone-filesystem jar missing in hadoop classpath
add 1f47fb7 HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline.
add f74159c HADOOP-16166. TestRawLocalFileSystemContract fails with build Docker container running on Mac. Also provided similar fix for Windows.
add 86d508c HDDS-1087. Fix TestDefaultCertificateClient#testSignDataStream. Contributed by Xiaoyu Yao. (#596)
add 4fa0099 HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598)
add 6635757 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.
add 6fa2298 HADOOP-15625. S3A input stream to use etags/version number to detect changed source files.
add f235a94 YARN-9016 DocumentStore as a backend for ATSv2. Contributed by Sushil Ks.
add ea9fb07 HDDS-1241. Update ozone to latest ratis snapshot build (0.4.0-5680cf5-SNAPSHOT). Contributed by Mukul Kumar Singh.
add 3d140a5 HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton.
add 983b78a HDDS-1237. Fix test TestSecureContainerServer.testClientServerRatisGrpc. Contributed by Mukul Kumar Singh.
add d60673c HDDS-1257. Incorrect object because of mismatch in block lengths. Contributed by Shashikant Banerjee.
add 688b177 YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
add 091a664 HDDS-917. Expose NodeManagerMXBean as a MetricsSource. Contributed by Siddharth Wagle.
add 2627dad HDDS-1265. ozone sh s3 getsecret throws Null Pointer Exception for unsecured clusters.
add 9001508 HDDS-761. Create S3 subcommand to run S3 related operations.
add ba50a36 HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He.
add 155ab6d HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee.
add a7f5e74 HDDS-1088. Add blockade Tests to test Replica Manager. Contributed by Nilotpal Nandi.
add dc21655 HDDS-1259. OzoneFS classpath separation is broken by the token validation. Contributed by Elek Marton.
add d1afa03 HDDS-1138. Ozone Client should avoid talking to SCM directly. Contributed by Xiaoyu Yao and Mukul Kumar Singh.
add ff06ef0 HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun.
add 16b7862 HDDS-1283. Fix the dynamic documentation of basic s3 client usage. Contributed by Elek, Marton. (#605)
add 44b8451 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)
add 03f3c8a YARN-4404. Corrected typo in javadoc. Contributed by Yesha Vora
add af2dfc9 HDDS-1263. SCM CLI does not list container with id 1.
add 5cfb88a HDDS-1246. Add ozone delegation token utility subcmd for Ozone CLI. Contributed by Xiaoyu Yao. (#594)
add 2064ca0 YARN-9349. Changed logging to use slf4j api. Contributed by Prabhu Joseph
add 926d548 HDDS-1281. Fix the findbug issue caused by HDDS-1163. Contributed by Aravindan Vijayan.
add 4d2a116 HDDS-699. Detect Ozone Network topology. Contributed by Sammi Chen.
add cb4d911 MAPREDUCE-7188. [Clean-up] Remove NULL check before instanceof and fix checkstyle issue in TaskResult (Contributed by Shweta Yakkali via Daniel Templeton)
add 1edf191 HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file.
add 0e7e901 YARN-9340. [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey (Contributed by Shweta Yakkali via Daniel Templeton)
add 2db38ab HDFS-14328. [Clean-up] Remove NULL check before instanceof in TestGSet (Contributed by Shweta Yakkali via Daniel Templeton)
add 8c95cb9 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed.
add 66a104b HDDS-1296. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#616)
add 5446e3c HADOOP-16167. Fixed Hadoop shell script for Ubuntu 18. Contributed by Daniel Templeton
add 19b22c4 YARN-9385. Fixed ApiServiceClient to use current UGI. Contributed by Eric Yang
add 5f6e225 YARN-9363. Replaced debug logging with SLF4J parameterized log message. Contributed by Prabhu Joseph
add c0427c8 HADOOP-16124. Extend documentation in testing.md about S3 endpoint constants.
add 93db5da HDDS-1250. In OM HA AllocateBlock call where connecting to SCM from OM should not happen on Ratis.
add ae3a2c3 Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)"
add 09eabda YARN-9364. Remove commons-logging dependency from YARN. Contributed by Prabhu Joseph
add 568d3ab HADOOP-16191. AliyunOSS: improvements for copyFile/copyDirectory and logging. Contributed by wujinhu.
add f10d493 HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601)
add c9e50c4 YARN-9365. Fix table creation command in TimelineServiceV2.md
add 55b3a71 HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen.
add 992489c HDDS-1215. Change hadoop-runner and apache/hadoop base image to use Java8. Contributed by Xiaoyu Yao.
add 5d8bd0e YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
add 1639071 HDDS-1307. Test ScmChillMode testChillModeOperations failed. (#622)
add 310ebf5 HDDS-1308. Fix asf license errors. (#623)
add 072750c HDDS-1306. TestContainerStateManagerIntegration fails in Ratis shutdown. Contributed by Lokesh Jain.
add c1a4eeb YARN-9389. FlowActivity and FlowRun table prefix is wrong. Contributed by Prabhu Joseph.
add f6e60e9 YARN-9387. Update document for ATS HBase Custom tablenames (-entityTableName). Contributed by Prabhu Joseph.
add 0d24684 YARN-9357. Modify HBase Liveness monitor log to debug. Contributed by Prabhu Joseph.
add b3b0e33 YARN-9299. TestTimelineReaderWhitelistAuthorizationFilter ignores Http Errors. Contributed by Prabhu Joseph.
add 399563f HDFS-14176. Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.
add f2b862c YARN-9398. Fixed javadoc errors for FPGA related java files. Contributed by Peter Bacsko
add 506502b YARN-9370. Added logging for recovering assigned GPU devices. Contributed by Yesha Vora
add 60cdd4c HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan.
add 9f1c017 HADOOP-16058. S3A tests to include Terasort.
add a99eb80 YARN-9267. General improvements in FpgaResourceHandlerImpl. Contributed by Peter Bacsko.
add 548997d YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
add 2828f8c HDDS-1321. TestOzoneManagerHttpServer depends on hard-coded port numbers. Contributed by Arpit Agarwal. (#633)
add 90afc9a HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh.
add 246ab77 HADOOP-16196. Path Parameterize Comparable.
add d18d085 HADOOP-16181. HadoopExecutors shutdown Cleanup.
add faba359 HADOOP-16147. Allow CopyListing sequence file keys and values to be more easily customized.
add ce5eb9c YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322 (Contributed by Zoltan Siegl via Daniel Templeton)
add ae2eb2d HADOOP-16201: S3AFileSystem#innerMkdirs builds needless lists (#636)
add 1d389ec HDDS-1323. Ignore unit test TestFailureHandlingByClient. Contributed by Supratim Deka.
add 509b20b YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
add f854a89 HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED containers. Contributed by Nanda kumar. (#620)
add dea6f2a HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading from stale pipeline. Contributed by Xiaoyu Yao. (#639)
add 73f7b04 HDDS-1302. Fix SCM CLI does not list container with id 1.
add efad571 HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri.
add 771ea6b Revert "HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri."
add 43e421a HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth Wagle.
add 6a34c9b HDDS-1299. Support TokenIssuer interface for running jobs with OzoneFileSystem. Contributed by Xiaoyu Yao.
add 128dd91 HDDS-1331. In DatanodeStateMachine join check for not null.
add d4e4a7d HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee.
add 67dd45f HDDS-1185. Optimize GetFileStatus in OzoneFileSystem by reducing the number of rpc call to OM. Contributed by Mukul Kumar Singh.
add e5d72f5 HDDS-1234. Iterate the OM DB snapshot and populate the recon container DB. Contributed by Aravindan Vijayan.
add 3c45762 YARN-9391. Fixed node manager environment leaks into Docker containers. Contributed by Jim Brennan
add 8739693 HDDS-1217. Refactor ChillMode rules and chillmode manager. (#558)
add eeda689 YARN-9268. General improvements in FpgaDevice. Contributed by Peter Bacsko.
add 710cbc9 YARN-9405. Fixed flaky tests in TestYarnNativeServices. Contributed by Prabhu Joseph
add 3f6d6d2 HDFS-14359. Inherited ACL permissions masked when parent directory does not exist (mkdir -p) (Contributed by Stephen O'Donnell via Daniel Templeton)
add c99b107 HDDS-1334. Fix asf license errors in newly added files by HDDS-1234. Contributed by Aravindan Vijayan.
add 5257f50 YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
add 5c0a81a HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails.
add 82d4772 HDDS-939. Add S3 access check to Ozone manager. Contributed by Ajay Kumar. (#634)
add 55fb3c3 HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory.
add ce4bafd HADOOP-16037. DistCp: Document usage of Sync (-diff option) in detail.
add fe29b39 HDFS-14348: Fix JNI exception handling issues in libhdfs
add f426b7c HDDS-139. Output of createVolume can be improved. Contributed by Shweta.
add 18c57cf HDFS-14304: High lock contention on hdfsHashMutex in libhdfs
add eef8cae HDDS-1262. In OM HA OpenKey call Should happen only leader OM. (#626)
add b226958 HDDS-1264. Remove Parametrized in TestOzoneShell. (#614)
add b4ed81c HDDS-1332. Add some logging for flaky test testStartStopDatanodeStateMachine. Contributed by Arpit Agarwal. (#649)
add a4cd75e YARN-9269. Minor cleanup in FpgaResourceAllocator. Contributed by Peter Bacsko.
add 9cd6619 YARN-9411. TestYarnNativeServices fails sporadically with bind address in use. Contributed by Prabhu Joseph.
add 8a59efe HDDS-1346. Remove hard-coded version ozone-0.5.0 from ReadMe of ozonesecure-mr docker-compose. Contributed by Xiaoyu Yao.
add 15d38b1 HDFS-14295. Add Threadpool for DataTransfers. Contributed by David Mollitor.
add cfb0186 HADOOP-16186. S3Guard: NPE in DynamoDBMetadataStore.lambda$listChildren.
add b5db238 HADOOP-15999. S3Guard: Better support for out-of-band operations.
add df578c0 HADOOP-16195 MarshalledCredentials toString
add 49b02d4 HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor.
add f3f5128 HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R
add ca5e4ce HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao.
add ac4010b HDDS-1293. ExcludeList#getProtoBuf throws ArrayIndexOutOfBoundsException. Contributed by Shashikant Banerjee.
add 4cceeb2 HDDS-1309 . change logging from warn to debug in XceiverClient. Contributed by Nilotpal Nandi. (#638)
add d7a2f94 HDDS-1350. Fix checkstyle issue in TestDatanodeStateMachine. Contributed by Xiaoyu Yao. (#656)
add f41f938 HADOOP-16199. KMSLoadBlanceClientProvider does not select token correctly. Contributed by Xiaoyu Yao.
add 82769ba HDDS-1352. Remove unused call in TestStorageContainerManagerHttpServer.
add 6186ed9 HDDS-1300. Optimize non-recursive ozone filesystem apis. Contributed by Lokesh Jain.
add 7dc0ecc [HDDS-1351] NoClassDefFoundError when running ozone genconf (). Contributed by Doroszlai Attila.
add 56f1e13 YARN-9270. Minor cleanup in TestFpgaDiscoverer. Contributed by Peter Bacsko.
add 7ee32a2 YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
add d9e9e56 YARN-9348. Application catalog build system bug fixes. Contributed by Eric Yang
add ec82e4c HDDS-1357. ozone s3 shell command has confusing subcommands (#663)
add bf3b7fd HDDS-1288. SCM - Failing test on trunk that waits for HB report processing. Contributed by Siddharth Wagle.
add 35ff31d HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He.
add 53a86e2 HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project
add 509f31b HDDS-1295. Add MiniOzoneChaosCluster to mimic long running workload in a unit test environment. Contributed by Mukul Kumar Singh.
add ef5de29 HDDS-1312. Add more unit tests to verify BlockOutputStream functionalities. Contributed by Shashikant Banerjee.
add 8bfef21 HDDS-1360. Invalid metric type due to fully qualified class name. Contributed by Doroszlai, Attila.
add 0df9492 HDDS-1067 . freon run on client gets hung when two of the datanodes are down in 3 datanode cluster. Contributed by Nilotpal Nandi.
add 5f951ea HDDS-1255. Refactor ozone acceptance test to allow run in secure mode
add 3675670 HDDS-1322. Hugo errors when building Ozone (#671)
add 35b0a38 YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
add 04f1db8 HDDS-1260. Create Recon Server lifecyle integration with Ozone. (#643)
add 856cbf6 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal.
add b0d24ef YARN-9227. DistributedShell RelativePath is not removed at end. Contributed by Prabhu Joseph.
add 332cab5 YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
add da7f8c2 YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
add ab2bda5 YARN-9428. Add metrics for paused containers in NodeManager. Contributed by Abhishek Modi.
add ebd0d21 HADOOP-16225. Fix links to the developer mailing lists in DownstreamDev.md. Contributed by Wanqiang Ji.
add 2f75283 YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
add 61d1911 HADOOP-16218. Findbugs warning of null param to non-nullable method in Configuration with Guava update. (#655)
add d31c868 HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee.
add aeb43df HDDS-1153. Make tracing instrumentation configurable
add bfc90bd HDDS-1355. Only FQDN is accepted for OM rpc address in secure environment. Contributed by Ajay Kumar.
add cf26811 HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain.
add aaaf856 HADOOP-16226. new Path(String str) does not remove all the trailing slashes of str
add f96fb05 HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka.
add 8b6deeb HADOOP-16232. Fix errors in the checkstyle configration xmls. Contributed by Wanqiang Ji.
add 002dcc4 YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
add 7cd7045 HDDS-1164. Add New blockade Tests to test Replica Manager. Contributed by Nilotpal Nandi.
add 3b0c501 HDFS-14394: Add -std=c99 / -std=gnu99 to libhdfs compile flags
add be488b6 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang.
add d6c233f HDDS-1377. OM failed to start with incorrect hostname set as ip address in CSR. (#683)
add d797907 HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
add 59022b2 HDDS-1330 : Add a docker compose for Ozone deployment with Recon. (#669)
add 366186d HADOOP-16233. S3AFileStatus to declare that isEncrypted() is always true (#685)
add e62cbcb HADOOP-16011. OsSecureRandom very slow compared to other SecureRandom implementations. Contributed by Siyao Meng.
add 8ff41d6 HDDS-1358 : Recon Server REST API not working as expected. (#668)
add 358f7f9 HDDS-1211. Test SCMChillMode failing randomly in Jenkins run (#543)
add 4e64f8d HDDS-1324. TestOzoneManagerHA tests are flaky (#676)
add 7b5b783 HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li.
new ee6c231 HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
new 33762ec HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
new e81ba43 HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
new 9fb5bc4 HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
new fbe3201 HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
new fb65537 HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
new 79f6fda HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
new 5870f70 HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
new 0273420 HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
new 30381b7 HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
new 03baa6d HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
new 6d28616 HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
new cf474b5 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
new 83a1beb Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
new 9c16a45 HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
new 1555e5c HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
new c6eb9ee HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
new aa6cff0 HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
new e3ab7a4 HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
new 2b48aa0 HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
new 1372cc8 HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
new 2a7998b HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
new 64689a2 HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
new 03d98f5 HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
new 692c581 HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
new e86634a HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
new 7c01d25 HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
new 9c22816 HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
new 4257376 HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
new 3864de4 HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
new 29fa6ae HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
new 8dc059d HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
new 4feb3ae HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
new e43400c HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.
new 08a4e69 HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
new 4585b97 HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
new 7bbe35e HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
new 49d489c HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
new 8478112 HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
new 1761b90 HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
new f330e6f HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
new d897cee HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
new 9d809a2 HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
new 58c5457 HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
new 3e01881 HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
new 5634f14 HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
new e6eacbd HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
new 4a21db8 HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
new 64ad3d6 HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
new ba429bc2 HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
new 0b0c334 HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
new d10765d HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
new 2577e3e HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (dea3798)
\
N -- N -- N refs/heads/HDFS-13891 (2577e3e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 53 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../main/resources/assemblies/hadoop-yarn-dist.xml | 5 +
.../src/main/resources/checkstyle/checkstyle.xml | 4 +-
.../src/main/resources/checkstyle/suppressions.xml | 4 +-
.../hadoop-common/HadoopCommon.cmake | 4 +-
.../dev-support/findbugsExcludeFile.xml | 7 +
.../hadoop-common/src/main/bin/hadoop-functions.sh | 12 +
.../hadoop-common/src/main/conf/log4j.properties | 19 -
.../java/org/apache/hadoop/conf/Configuration.java | 65 +-
.../apache/hadoop/crypto/CryptoInputStream.java | 1 +
.../hadoop/crypto/OpensslAesCtrCryptoCodec.java | 4 +-
.../key/kms/LoadBalancingKMSClientProvider.java | 3 +
.../org/apache/hadoop/fs/ChecksumFileSystem.java | 6 +
.../apache/hadoop/fs/DelegationTokenRenewer.java | 4 +-
.../src/main/java/org/apache/hadoop/fs/Path.java | 17 +-
.../org/apache/hadoop/fs/StreamCapabilities.java | 6 +
.../src/main/java/org/apache/hadoop/fs/Trash.java | 6 +
.../java/org/apache/hadoop/fs/TrashPolicy.java | 5 +
.../org/apache/hadoop/fs/TrashPolicyDefault.java | 18 +-
.../java/org/apache/hadoop/fs/shell/Delete.java | 25 +-
.../java/org/apache/hadoop/fs/shell/Display.java | 27 +-
.../org/apache/hadoop/ipc/CallQueueManager.java | 11 +-
.../org/apache/hadoop/ipc/ProtobufRpcEngine.java | 1 +
.../main/java/org/apache/hadoop/ipc/Server.java | 3 +
.../apache/hadoop/net/DNSDomainNameResolver.java | 33 +-
.../org/apache/hadoop/net/DomainNameResolver.java | 23 +
.../main/java/org/apache/hadoop/net/NetUtils.java | 4 +-
.../java/org/apache/hadoop/security/Groups.java | 2 +-
.../org/apache/hadoop/security/ProviderUtils.java | 5 +-
.../apache/hadoop/util/NodeHealthScriptRunner.java | 2 +-
.../hadoop/util/concurrent/HadoopExecutors.java | 58 +-
.../src/main/resources/core-default.xml | 44 +-
.../src/site/markdown/DownstreamDev.md | 9 +-
.../src/site/markdown/FileSystemShell.md | 11 +-
.../src/site/markdown/filesystem/filesystem.md | 11 +
.../org/apache/hadoop/conf/TestConfiguration.java | 5 +
.../kms/TestLoadBalancingKMSClientProvider.java | 35 +-
.../test/java/org/apache/hadoop/fs/TestPath.java | 2 +
.../hadoop/fs/TestRawLocalFileSystemContract.java | 42 +-
.../test/java/org/apache/hadoop/fs/TestTrash.java | 65 +-
.../AbstractContractGetFileStatusTest.java | 4 +-
.../fs/contract/AbstractContractOpenTest.java | 12 +-
.../fs/contract/AbstractContractSeekTest.java | 4 +-
.../TestLocalFSContractMultipartUploader.java | 8 +
.../apache/hadoop/ipc/TestCallQueueManager.java | 13 +
.../test/java/org/apache/hadoop/ipc/TestRPC.java | 5 +-
.../apache/hadoop/net/MockDomainNameResolver.java | 36 +-
.../org/apache/hadoop/test/GenericTestUtils.java | 9 +
.../test/java/org/apache/hadoop/util/TestGSet.java | 3 +-
.../org/apache/hadoop/util/TestLineReader.java | 222 +--
.../hadoop-common/src/test/resources/testConf.xml | 2 +-
hadoop-hdds/client/pom.xml | 4 +-
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 25 +-
.../hadoop/hdds/scm/XceiverClientMetrics.java | 20 +
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 109 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 392 +++---
.../apache/hadoop/hdds/scm/storage/BufferPool.java | 114 ++
hadoop-hdds/common/pom.xml | 6 +-
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 33 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 34 +
.../org/apache/hadoop/hdds/cli/GenericCli.java | 3 +-
.../org/apache/hadoop/hdds/client/BlockID.java | 4 +-
.../hadoop/hdds/conf/OzoneConfiguration.java | 29 +
.../hadoop/hdds/protocol/DatanodeDetails.java | 44 +-
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 26 +-
.../scm/container/common/helpers/ExcludeList.java | 12 +-
.../org/apache/hadoop/hdds/scm/net/InnerNode.java | 84 ++
.../apache/hadoop/hdds/scm/net/InnerNodeImpl.java | 495 +++++++
.../apache/hadoop/hdds/scm/net/NetConstants.java | 67 +
.../org/apache/hadoop/hdds/scm/net/NetUtils.java | 156 ++
.../hadoop/hdds/scm/net/NetworkTopology.java | 250 ++++
.../hadoop/hdds/scm/net/NetworkTopologyImpl.java | 778 ++++++++++
.../java/org/apache/hadoop/hdds/scm/net/Node.java | 89 ++
.../org/apache/hadoop/hdds/scm/net/NodeImpl.java | 200 +++
.../org/apache/hadoop/hdds/scm/net/NodeSchema.java | 148 ++
.../hadoop/hdds/scm/net/NodeSchemaLoader.java | 388 +++++
.../hadoop/hdds/scm/net/NodeSchemaManager.java | 136 ++
.../apache/hadoop/hdds/scm/net/package-info.java | 21 +
.../scm/protocol/ScmBlockLocationProtocol.java | 9 +-
...lockLocationProtocolClientSideTranslatorPB.java | 24 +-
.../hdds/security/token/BlockTokenVerifier.java | 45 +-
.../security/token/OzoneBlockTokenIdentifier.java | 13 +
.../hadoop/hdds/security/x509/SecurityConfig.java | 12 +
.../certificate/authority/CertificateApprover.java | 7 +-
.../certificate/authority/DefaultApprover.java | 36 +-
.../certificate/authority/DefaultCAServer.java | 28 +-
.../authority/PKIProfiles/DefaultProfile.java | 7 +-
.../x509/certificate/client/CertificateClient.java | 21 +-
.../certificate/client/DNCertificateClient.java | 25 +-
.../client/DefaultCertificateClient.java | 257 +++-
.../certificate/client/OMCertificateClient.java | 25 +-
.../x509/certificate/utils/CertificateCodec.java | 26 +
.../certificates/utils/CertificateSignRequest.java | 25 +-
.../x509/exceptions/CertificateException.java | 4 +-
.../hadoop/hdds/security/x509/keys/KeyCodec.java | 11 +
.../apache/hadoop/hdds/tracing/StringCodec.java | 9 +-
.../apache/hadoop/hdds/tracing/TracingUtil.java | 26 +-
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 34 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 5 +
.../org/apache/hadoop/ozone/OzoneSecurityUtil.java | 58 +
.../org/apache/hadoop/ozone/common/Storage.java | 12 +
...lockLocationProtocolServerSideTranslatorPB.java | 32 +-
.../apache/hadoop/utils/LevelDBStoreIterator.java | 1 +
.../java/org/apache/hadoop/utils/Scheduler.java | 17 +-
.../org/apache/hadoop/utils/db/CodecRegistry.java | 1 +
.../org/apache/hadoop/utils/db/DBCheckpoint.java | 5 +
.../java/org/apache/hadoop/utils/db/DBStore.java | 6 +
.../org/apache/hadoop/utils/db/DBStoreBuilder.java | 8 +-
.../org/apache/hadoop/utils/db/IntegerCodec.java | 38 +
.../java/org/apache/hadoop/utils/db/LongCodec.java | 46 +
.../hadoop/utils/db/RDBCheckpointManager.java | 49 +-
.../java/org/apache/hadoop/utils/db/RDBStore.java | 23 +-
.../apache/hadoop/utils/db/RocksDBCheckpoint.java | 81 ++
.../main/java/org/apache/ratis/RatisHelper.java | 33 +-
.../src/main/proto/DatanodeContainerProtocol.proto | 6 +
.../src/main/proto/ScmBlockLocationProtocol.proto | 22 +-
hadoop-hdds/common/src/main/proto/hdds.proto | 1 +
.../main/resources/network-topology-default.xml | 68 +
.../main/resources/network-topology-nodegroup.xml | 74 +
.../common/src/main/resources/ozone-default.xml | 376 ++++-
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 143 ++
.../org/apache/hadoop/hdds/conf/package-info.java | 22 +
.../hdds/scm/net/TestNetworkTopologyImpl.java | 922 ++++++++++++
.../hadoop/hdds/scm/net/TestNodeSchemaLoader.java | 103 ++
.../hadoop/hdds/scm/net/TestNodeSchemaManager.java | 101 ++
.../x509/certificate/authority/MockApprover.java | 7 +-
.../certificate/authority/TestDefaultCAServer.java | 84 +-
.../client/TestCertificateClientInit.java | 55 +-
.../client/TestDefaultCertificateClient.java | 217 ++-
.../certificates/TestCertificateSignRequest.java | 18 -
.../hadoop/hdds/tracing/TestStringCodec.java | 52 +
.../apache/hadoop/hdds/tracing/package-info.java | 21 +
.../org/apache/hadoop/utils/TestMetadataStore.java | 1 +
.../org/apache/hadoop/utils/db/TestRDBStore.java | 43 +
.../networkTopologyTestFiles/enforce-error.xml | 47 +
.../resources/networkTopologyTestFiles/good.xml | 49 +
.../networkTopologyTestFiles/invalid-cost.xml | 43 +
.../networkTopologyTestFiles/invalid-version.xml | 43 +
.../networkTopologyTestFiles/multiple-leaf.xml | 43 +
.../networkTopologyTestFiles/multiple-root.xml | 43 +
.../networkTopologyTestFiles/multiple-topology.xml | 47 +
.../resources/networkTopologyTestFiles/no-leaf.xml | 43 +
.../resources/networkTopologyTestFiles/no-root.xml | 43 +
.../networkTopologyTestFiles/no-topology.xml | 39 +
.../path-layers-size-mismatch.xml | 43 +
.../path-with-id-reference-failure.xml | 43 +
.../unknown-layer-type.xml | 43 +
.../wrong-path-order-1.xml | 43 +
.../wrong-path-order-2.xml | 43 +
hadoop-hdds/container-service/pom.xml | 4 +-
.../hadoop/ozone/HddsDatanodeHttpServer.java | 86 ++
.../apache/hadoop/ozone/HddsDatanodeService.java | 170 ++-
.../ozone/container/common/impl/ContainerSet.java | 22 +-
.../container/common/impl/HddsDispatcher.java | 85 +-
.../container/common/interfaces/Container.java | 5 +
.../common/interfaces/ContainerDispatcher.java | 8 +
.../common/statemachine/DatanodeStateMachine.java | 22 +-
.../CloseContainerCommandHandler.java | 2 +
.../common/transport/server/XceiverServer.java | 14 +-
.../common/transport/server/XceiverServerGrpc.java | 26 +-
.../server/ratis/ContainerStateMachine.java | 314 +++--
.../transport/server/ratis/DispatcherContext.java | 33 +-
.../transport/server/ratis/XceiverServerRatis.java | 45 +-
.../container/keyvalue/KeyValueContainer.java | 72 +-
.../container/keyvalue/KeyValueContainerCheck.java | 295 ++++
.../container/ozoneimpl/ContainerScrubber.java | 158 +++
.../ozone/container/ozoneimpl/OzoneContainer.java | 44 +-
.../main/resources/webapps/hddsDatanode/.gitkeep | 17 +
.../hadoop/ozone/TestHddsSecureDatanodeInit.java | 271 ++++
.../container/common/TestDatanodeStateMachine.java | 16 +-
.../TestCloseContainerCommandHandler.java | 10 +-
.../container/keyvalue/TestKeyValueContainer.java | 13 +
.../keyvalue/TestKeyValueContainerCheck.java | 197 +++
.../container/keyvalue/TestKeyValueHandler.java | 3 +
.../container/ozoneimpl/TestOzoneContainer.java | 2 +-
hadoop-hdds/docs/content/BucketCommands.md | 4 +-
hadoop-hdds/docs/content/OzoneFS.md | 19 +-
hadoop-hdds/docs/content/S3.md | 2 +-
hadoop-hdds/docs/content/S3Commands.md | 2 +-
hadoop-hdds/docs/content/SparkOzoneFSK8S.md | 6 +-
hadoop-hdds/docs/pom.xml | 6 +-
hadoop-hdds/framework/pom.xml | 4 +-
.../apache/hadoop/hdds/server/BaseHttpServer.java | 10 +-
.../org/apache/hadoop/hdds/server/ServerUtils.java | 39 +-
.../src/main/resources/webapps/static/ozone.js | 2 -
hadoop-hdds/pom.xml | 6 +-
hadoop-hdds/server-scm/pom.xml | 4 +-
.../apache/hadoop/hdds/scm/block/BlockManager.java | 7 +
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 29 +-
.../hdds/scm/chillmode/ChillModeExitRule.java | 88 +-
.../hdds/scm/chillmode/ChillModeHandler.java | 106 ++
.../hdds/scm/chillmode/ContainerChillModeRule.java | 49 +-
.../hdds/scm/chillmode/DataNodeChillModeRule.java | 41 +-
.../chillmode/HealthyPipelineChillModeRule.java | 114 +-
.../chillmode/OneReplicaPipelineChillModeRule.java | 83 +-
.../hdds/scm/chillmode/SCMChillModeManager.java | 134 +-
.../hdds/scm/container/ContainerManager.java | 27 +
.../hdds/scm/container/ContainerStateManager.java | 60 +-
.../hdds/scm/container/ReplicationManager.java | 748 ++++++++++
.../hdds/scm/container/SCMContainerManager.java | 27 +
.../replication/ReplicationActivityStatus.java | 54 +-
.../scm/container/states/ContainerStateMap.java | 9 +-
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 5 +-
.../hadoop/hdds/scm/node/NewNodeHandler.java | 4 +-
.../scm/node/NonHealthyToHealthyNodeHandler.java | 3 +-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 4 +-
.../hadoop/hdds/scm/node/SCMNodeMetrics.java | 76 +-
.../hadoop/hdds/scm/node/StaleNodeHandler.java | 4 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 110 ++
.../hdds/scm/pipeline/PipelineActionHandler.java | 4 +-
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 13 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 9 +-
.../hadoop/hdds/scm/pipeline/PipelineProvider.java | 1 -
.../hdds/scm/pipeline/PipelineReportHandler.java | 9 -
.../hdds/scm/pipeline/RatisPipelineProvider.java | 19 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 144 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 145 +-
.../hdds/scm/pipeline/SimplePipelineProvider.java | 5 -
.../hdds/scm/server/SCMBlockProtocolServer.java | 13 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 32 +-
.../hdds/scm/server/SCMContainerMetrics.java | 88 ++
.../hdds/scm/server/StorageContainerManager.java | 51 +-
.../main/resources/webapps/scm/scm-overview.html | 18 +-
.../src/main/resources/webapps/scm/scm.js | 4 -
.../scm/TestStorageContainerManagerHttpServer.java | 11 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 53 +-
.../hadoop/hdds/scm/block/TestBlockManager.java | 23 +-
.../hdds/scm/chillmode/TestChillModeHandler.java | 104 ++
.../TestHealthyPipelineChillModeRule.java | 25 +-
.../TestOneReplicaPipelineChillModeRule.java | 11 +-
.../scm/chillmode/TestSCMChillModeManager.java | 219 ++-
.../container/TestCloseContainerEventHandler.java | 10 +-
.../scm/container/TestContainerActionsHandler.java | 2 +-
.../scm/container/TestContainerReportHandler.java | 36 +-
.../scm/container/TestContainerReportHelper.java | 40 -
.../TestIncrementalContainerReportHandler.java | 6 +-
.../hdds/scm/container/TestReplicationManager.java | 625 ++++++++
.../replication/TestReplicationActivityStatus.java | 36 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 11 +
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 2 +-
.../scm/pipeline/MockRatisPipelineProvider.java | 40 +
.../scm/server/TestSCMClientProtocolServer.java | 12 +-
.../hdds/scm/server/TestSCMContainerMetrics.java | 81 ++
.../ozone/container/common/TestEndPoint.java | 10 +-
hadoop-hdds/tools/pom.xml | 4 +-
.../org/apache/hadoop/hdds/scm/cli/SCMCLI.java | 2 +-
.../org/apache/hadoop/hdfs/DFSInputStream.java | 1 +
.../org/apache/hadoop/hdfs/DFSOutputStream.java | 16 +-
.../apache/hadoop/hdfs/DFSStripedOutputStream.java | 20 +-
.../java/org/apache/hadoop/hdfs/DataStreamer.java | 25 +-
.../hadoop/hdfs/client/HdfsClientConfigKeys.java | 2 +
.../hadoop/hdfs/client/impl/DfsClientConf.java | 2 +-
.../ha/AbstractNNFailoverProxyProvider.java | 18 +-
.../namenode/ha/ObserverReadProxyProvider.java | 64 +
.../hadoop/hdfs/web/SSLConnectionConfigurator.java | 72 +
.../hadoop/hdfs/web/URLConnectionFactory.java | 43 +-
.../apache/hadoop/hdfs/web/WebHdfsFileSystem.java | 3 +
.../ha/TestConfiguredFailoverProxyProvider.java | 47 +-
.../hadoop/hdfs/web/TestURLConnectionFactory.java | 53 +
.../hadoop-hdfs-native-client/pom.xml | 1 +
.../src/main/native/libhdfs-tests/CMakeLists.txt | 5 +-
.../main/native/libhdfs-tests/native_mini_dfs.c | 42 +-
.../src/main/native/libhdfs-tests/test_htable.c | 100 --
.../src/main/native/libhdfs/CMakeLists.txt | 2 +-
.../src/main/native/libhdfs/common/htable.c | 287 ----
.../src/main/native/libhdfs/common/htable.h | 161 ---
.../src/main/native/libhdfs/exception.c | 6 +-
.../src/main/native/libhdfs/hdfs.c | 542 +++----
.../src/main/native/libhdfs/jclasses.c | 136 ++
.../src/main/native/libhdfs/jclasses.h | 112 ++
.../src/main/native/libhdfs/jni_helper.c | 231 +--
.../src/main/native/libhdfs/jni_helper.h | 37 +-
.../src/main/native/libhdfs/os/mutexes.h | 6 +-
.../src/main/native/libhdfs/os/posix/mutexes.c | 2 +-
.../native/libhdfs/os/posix/thread_local_storage.c | 94 +-
.../src/main/native/libhdfs/os/windows/mutexes.c | 4 +-
.../main/native/libhdfspp/tests/hdfs_ext_test.cc | 5 +-
.../federation/metrics/NamenodeBeanMetrics.java | 5 +
.../federation/resolver/order/LocalResolver.java | 2 +-
.../federation/router/RouterClientProtocol.java | 10 +-
.../federation/router/RouterWebHdfsMethods.java | 2 +
.../dev-support/findbugsExcludeFile.xml | 24 +-
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 22 +-
.../hdfs/qjournal/client/IPCLoggerChannel.java | 2 +-
.../hadoop/hdfs/qjournal/client/QuorumCall.java | 3 +-
.../hdfs/qjournal/client/QuorumJournalManager.java | 1 +
.../hadoop/hdfs/qjournal/server/Journal.java | 11 +-
.../hadoop/hdfs/server/balancer/Balancer.java | 4 +-
.../hdfs/server/blockmanagement/BlockManager.java | 56 +-
.../BlockPlacementPolicyDefault.java | 3 +-
.../hdfs/server/datanode/BPServiceActor.java | 7 +-
.../apache/hadoop/hdfs/server/datanode/DNConf.java | 40 +-
.../hadoop/hdfs/server/datanode/DataNode.java | 80 +-
.../datanode/checker/DatasetVolumeChecker.java | 13 +-
.../datanode/checker/ThrottledAsyncChecker.java | 2 +-
.../fsdataset/impl/AddBlockPoolException.java | 45 +
.../datanode/fsdataset/impl/FsDatasetCache.java | 209 ++-
.../datanode/fsdataset/impl/FsDatasetImpl.java | 20 +-
.../datanode/fsdataset/impl/FsDatasetUtil.java | 27 +
.../datanode/fsdataset/impl/FsVolumeList.java | 29 +-
.../datanode/fsdataset/impl/MappableBlock.java | 155 +-
.../fsdataset/impl/MappableBlockLoader.java | 129 ++
.../datanode/fsdataset/impl/MemoryCacheStats.java | 212 +++
.../fsdataset/impl/MemoryMappableBlockLoader.java | 185 +++
.../datanode/fsdataset/impl/MemoryMappedBlock.java | 54 +
.../fsdataset/impl/PmemMappableBlockLoader.java | 230 +++
.../datanode/fsdataset/impl/PmemMappedBlock.java | 70 +
.../datanode/fsdataset/impl/PmemVolumeManager.java | 306 ++++
.../org/apache/hadoop/hdfs/server/mover/Mover.java | 4 +-
.../hadoop/hdfs/server/namenode/FSDirAppendOp.java | 10 +-
.../hadoop/hdfs/server/namenode/FSDirMkdirOp.java | 18 +-
.../hadoop/hdfs/server/namenode/FSEditLog.java | 3 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 11 +-
.../hdfs/server/namenode/NameNodeMXBean.java | 179 +--
.../hdfs/server/namenode/ha/EditLogTailer.java | 23 +-
.../src/main/resources/hdfs-default.xml | 50 +-
.../hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js | 47 +-
.../src/site/markdown/ObserverNameNode.md | 48 +-
.../org/apache/hadoop/hdfs/MiniDFSCluster.java | 9 +-
.../java/org/apache/hadoop/hdfs/TestDFSShell.java | 26 +
.../apache/hadoop/hdfs/TestStripedFileAppend.java | 114 ++
.../hdfs/server/datanode/SimulatedFSDataset.java | 12 +-
.../hdfs/server/datanode/TestBPOfferService.java | 125 +-
.../server/datanode/TestDataNodeVolumeFailure.java | 110 ++
.../hdfs/server/datanode/TestFsDatasetCache.java | 634 ---------
.../datanode/TestFsDatasetCacheRevocation.java | 31 +-
.../checker/TestThrottledAsyncCheckerTimeout.java | 11 +-
.../fsdataset/impl/FsDatasetImplTestUtils.java | 3 +-
.../impl/TestCacheByPmemMappableBlockLoader.java | 329 +++++
.../fsdataset/impl/TestFsDatasetCache.java | 636 +++++++++
.../hadoop/hdfs/server/namenode/FSAclBaseTest.java | 2 +-
.../server/namenode/TestListCorruptFileBlocks.java | 43 +-
.../hdfs/server/namenode/TestNameNodeMXBean.java | 5 +
.../namenode/ha/TestConsistentReadsObserver.java | 56 +-
.../hdfs/server/namenode/ha/TestEditLogTailer.java | 90 +-
.../server/namenode/ha/TestFailureToReadEdits.java | 12 +-
.../testAclCLIWithPosixAclInheritance.xml | 77 +
.../hadoop/mapred/LocatedFileStatusFetcher.java | 9 +-
.../mapreduce/task/reduce/MergeManagerImpl.java | 13 +-
.../hadoop/mapreduce/v2/hs/CompletedJob.java | 15 +-
.../mapreduce/v2/hs/webapp/HsAttemptsPage.java | 5 +-
.../hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java | 28 +-
.../hadoop/mapreduce/v2/hs/webapp/TestBlocks.java | 30 +-
.../org/apache/hadoop/mapred/BigMapOutput.java | 18 +-
.../java/org/apache/hadoop/mapred/MRBench.java | 2 +-
.../java/org/apache/hadoop/mapreduce/SleepJob.java | 29 +-
.../org/apache/hadoop/mapreduce/v2/TestMRJobs.java | 15 +
.../org/apache/hadoop/examples/pi/TaskResult.java | 34 +-
.../hadoop/examples/terasort/TeraOutputFormat.java | 11 -
.../apache/hadoop/examples/terasort/TeraSort.java | 2 +-
.../hadoop/examples/terasort/TestTeraSort.java | 6 +-
hadoop-ozone/client/pom.xml | 9 +-
.../apache/hadoop/ozone/client/ObjectStore.java | 19 +-
.../apache/hadoop/ozone/client/OzoneBucket.java | 61 +
.../hadoop/ozone/client/OzoneClientUtils.java | 13 +
.../ozone/client/io/BlockOutputStreamEntry.java | 29 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 5 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 304 ++--
.../ozone/client/protocol/ClientProtocol.java | 101 +-
.../hadoop/ozone/client/rest/RestClient.java | 57 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 240 +++-
.../hadoop/ozone/client/rpc/ha/OMProxyInfo.java | 49 -
.../ozone/client/rpc/ha/OMProxyProvider.java | 177 ---
.../hadoop/ozone/client/rpc/ha/package-info.java | 23 -
hadoop-ozone/common/pom.xml | 63 +-
hadoop-ozone/common/src/main/bin/ozone | 18 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 6 +
.../org/apache/hadoop/ozone/audit/OMAction.java | 10 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 7 +
.../apache/hadoop/ozone/om/OMMetadataManager.java | 8 +
.../apache/hadoop/ozone/om/S3SecretManager.java | 36 +
.../hadoop/ozone/om/S3SecretManagerImpl.java | 113 ++
.../ozone/om/codec/TokenIdentifierCodec.java | 52 +
.../ozone/om/exceptions/NotLeaderException.java | 51 +
.../hadoop/ozone/om/exceptions/OMException.java | 11 +
.../ozone/om/ha/OMFailoverProxyProvider.java | 285 ++++
.../apache/hadoop/ozone/om/ha/package-info.java | 23 +
.../hadoop/ozone/om/helpers/OMRatisHelper.java | 110 ++
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 16 +-
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 49 +-
.../hadoop/ozone/om/helpers/OzoneFileStatus.java | 115 ++
.../hadoop/ozone/om/helpers/S3SecretValue.java | 19 +-
.../ozone/om/protocol/OzoneManagerHAProtocol.java | 85 ++
.../ozone/om/protocol/OzoneManagerProtocol.java | 66 +
.../om/protocol/OzoneManagerServerProtocol.java | 28 +
...OzoneManagerProtocolClientSideTranslatorPB.java | 228 ++-
.../hadoop/ozone/security/AWSV4AuthValidator.java | 116 ++
.../security/OzoneBlockTokenSecretManager.java | 17 +-
.../OzoneDelegationTokenSecretManager.java | 146 +-
.../security/OzoneDelegationTokenSelector.java | 4 +-
.../hadoop/ozone/security/OzoneSecretManager.java | 5 -
.../hadoop/ozone/security/OzoneSecretStore.java | 198 +--
.../ozone/security/OzoneSecurityException.java | 1 +
.../ozone/security/OzoneTokenIdentifier.java | 119 +-
.../src/main/proto/OzoneManagerProtocol.proto | 115 +-
.../ozone/security/TestAWSV4AuthValidator.java | 78 +
.../security/TestOzoneBlockTokenSecretManager.java | 186 ---
.../TestOzoneDelegationTokenSecretManager.java | 274 ----
hadoop-ozone/datanode/pom.xml | 4 +-
hadoop-ozone/dev-support/docker/Dockerfile | 3 +
.../dist/dev-support/bin/dist-layout-stitching | 3 +-
hadoop-ozone/dist/pom.xml | 19 +-
hadoop-ozone/dist/src/main/blockade/README.md | 10 +
.../src/main/blockade/blockadeUtils/blockade.py | 5 +-
.../main/blockade/clusterUtils/cluster_utils.py | 542 +++----
hadoop-ozone/dist/src/main/blockade/conftest.py | 139 +-
.../main/blockade/test_blockade_client_failure.py | 11 +-
.../blockade/test_blockade_datanode_isolation.py | 152 +-
.../main/blockade/test_blockade_mixed_failure.py | 152 +-
...t_blockade_mixed_failure_three_nodes_isolate.py | 255 ++--
.../test_blockade_mixed_failure_two_nodes.py | 188 ++-
.../main/blockade/test_blockade_scm_isolation.py | 162 ++-
hadoop-ozone/dist/src/main/compose/README.md | 6 +
.../main/compose/ozone-hdfs/docker-compose.yaml | 4 +-
.../dist/src/main/compose/ozone-hdfs/docker-config | 4 +-
.../src/main/compose/{ozone => ozone-om-ha}/.env | 0
.../main/compose/ozone-om-ha/docker-compose.yaml | 82 ++
.../src/main/compose/ozone-om-ha/docker-config | 81 ++
.../src/main/compose/{ozone => ozone-recon}/.env | 0
.../main/compose/ozone-recon/docker-compose.yaml | 66 +
.../src/main/compose/ozone-recon/docker-config | 80 ++
.../src/main/compose/ozone/docker-compose.yaml | 11 +-
.../dist/src/main/compose/ozone/docker-config | 4 +-
.../main/compose/ozoneblockade/docker-compose.yaml | 4 +-
.../src/main/compose/ozoneblockade/docker-config | 4 +-
.../src/main/compose/ozonefs/docker-compose.yaml | 8 +-
.../dist/src/main/compose/ozonefs/docker-config | 4 +-
.../src/main/compose/ozoneperf/docker-compose.yaml | 4 +-
.../dist/src/main/compose/ozoneperf/docker-config | 4 +-
.../src/main/compose/ozones3/docker-compose.yaml | 4 +-
.../dist/src/main/compose/ozones3/docker-config | 2 +-
.../compose/{ozonesecure => ozonesecure-mr}/.env | 0
.../dist/src/main/compose/ozonesecure-mr/README.md | 73 +
.../compose/ozonesecure-mr/docker-compose.yaml | 114 ++
.../src/main/compose/ozonesecure-mr/docker-config | 177 +++
.../docker-image/docker-krb5/Dockerfile-krb5 | 35 +
.../docker-image/docker-krb5/README.md | 34 +
.../docker-image/docker-krb5/kadm5.acl | 0
.../docker-image/docker-krb5/krb5.conf | 0
.../docker-image/docker-krb5/launcher.sh | 0
.../main/compose/ozonesecure/docker-compose.yaml | 15 +-
.../src/main/compose/ozonesecure/docker-config | 8 +-
.../main/compose/ozonetrace/docker-compose.yaml | 4 +-
.../dist/src/main/compose/ozonetrace/docker-config | 2 +-
.../dist/src/main/smoketest/__init__.robot | 18 +
.../main/smoketest/auditparser/auditparser.robot | 23 +-
.../dist/src/main/smoketest/basic/basic.robot | 2 +-
.../src/main/smoketest/basic/ozone-shell.robot | 2 +-
.../dist/src/main/smoketest/commonlib.robot | 27 +-
.../dist/src/main/smoketest/ozonefs/ozonefs.robot | 30 +-
.../src/main/smoketest/s3/MultipartUpload.robot | 135 +-
.../dist/src/main/smoketest/s3/bucketcreate.robot | 34 +
.../dist/src/main/smoketest/s3/commonawslib.robot | 50 +-
.../dist/src/main/smoketest/s3/objectputget.robot | 4 +-
.../main/smoketest/security/ozone-secure-fs.robot | 49 +
.../main/smoketest/security/ozone-secure-s3.robot | 44 +
.../src/main/smoketest/security/ozone-secure.robot | 111 --
hadoop-ozone/dist/src/main/smoketest/test.sh | 14 +-
hadoop-ozone/integration-test/pom.xml | 32 +-
.../TestContainerStateManagerIntegration.java | 55 +
.../hdds/scm/pipeline/TestNode2PipelineMap.java | 4 +-
.../hdds/scm/pipeline/TestPipelineClose.java | 74 +-
.../scm/pipeline/TestRatisPipelineProvider.java | 2 +-
.../hdds/scm/pipeline/TestRatisPipelineUtils.java | 39 +-
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 41 +-
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 224 +++
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 9 +
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 42 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 38 +-
.../hadoop/ozone/MiniOzoneLoadGenerator.java | 160 +++
.../org/apache/hadoop/ozone/RatisTestHelper.java | 5 +-
.../hadoop/ozone/TestMiniChaosOzoneCluster.java | 116 ++
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 18 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 4 +
.../hadoop/ozone/TestSecureOzoneCluster.java | 275 +++-
.../ozone/TestStorageContainerManagerHelper.java | 1 +
.../ozone/client/CertificateClientTestImpl.java | 39 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 1 +
.../ozone/client/rpc/TestBlockOutputStream.java | 690 +++++++++
.../rpc/TestBlockOutputStreamWithFailures.java | 1241 ++++++++++++++++
.../rpc/TestCloseContainerHandlingByClient.java | 80 +-
.../client/rpc/TestContainerStateMachine.java | 8 +-
.../rpc/TestContainerStateMachineFailures.java | 43 +-
.../client/rpc/TestFailureHandlingByClient.java | 10 +
.../client/rpc/TestOzoneAtRestEncryption.java | 1 +
.../rpc/TestOzoneClientRetriesOnException.java | 213 +++
.../ozone/client/rpc/TestOzoneRpcClient.java | 22 -
.../client/rpc/TestOzoneRpcClientAbstract.java | 33 +-
.../hadoop/ozone/client/rpc/TestReadRetries.java | 2 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 21 +-
.../ozone/container/ContainerTestHelper.java | 119 +-
.../ozone/container/TestContainerReplication.java | 2 -
.../commandhandler/TestBlockDeletion.java | 8 +-
.../TestCloseContainerByPipeline.java | 8 +-
.../commandhandler/TestCloseContainerHandler.java | 4 +-
.../commandhandler/TestDeleteContainerHandler.java | 4 +-
.../transport/server/ratis/TestCSMMetrics.java | 7 +-
.../container/metrics/TestContainerMetrics.java | 3 +-
.../container/ozoneimpl/TestOzoneContainer.java | 48 +-
.../ozoneimpl/TestOzoneContainerWithTLS.java | 3 +-
.../ozoneimpl/TestSecureOzoneContainer.java | 86 +-
.../container/server/TestContainerServer.java | 18 +-
.../server/TestSecureContainerServer.java | 71 +-
.../ozone/om/TestContainerReportWithKeys.java | 1 +
.../hadoop/ozone/om/TestOMDbCheckpointServlet.java | 176 +++
.../hadoop/ozone/om/TestOmBlockVersioning.java | 5 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 406 +++++-
.../apache/hadoop/ozone/om/TestScmChillMode.java | 117 +-
.../hadoop/ozone/om/TestSecureOzoneManager.java | 241 ++++
.../hadoop/ozone/ozShell/TestOzoneShell.java | 124 +-
.../apache/hadoop/ozone/ozShell/TestS3Shell.java | 292 ++++
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 39 +
hadoop-ozone/objectstore-service/pom.xml | 10 +-
.../hdfs/server/datanode/ObjectStoreHandler.java | 8 +-
.../web/storage/DistributedStorageHandler.java | 15 +-
hadoop-ozone/ozone-manager/pom.xml | 35 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 47 +-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 778 +++++++---
.../hadoop/ozone/om/OMDBCheckpointServlet.java | 26 +-
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 103 ++
.../java/org/apache/hadoop/ozone/om/OMStorage.java | 17 +
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 149 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 452 +++++-
.../apache/hadoop/ozone/om/S3SecretManager.java | 30 -
.../hadoop/ozone/om/S3SecretManagerImpl.java | 82 --
.../java/org/apache/hadoop/ozone/om/ScmClient.java | 44 +
.../apache/hadoop/ozone/om/fs/OzoneManagerFS.java | 40 +
.../apache/hadoop/ozone/om/fs/package-info.java | 21 +
.../hadoop/ozone/om/ratis/OMRatisHelper.java | 117 --
.../ozone/om/ratis/OzoneManagerRatisClient.java | 53 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 167 ++-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 228 ++-
...OzoneManagerProtocolServerSideTranslatorPB.java | 53 +-
.../protocolPB/OzoneManagerRequestHandler.java | 163 ++-
.../hadoop/ozone/protocolPB/RequestHandler.java | 48 +
.../hadoop/ozone/web/ozShell/OzoneShell.java | 64 +
.../org/apache/hadoop/ozone/web/ozShell/Shell.java | 39 +-
.../ozone/web/ozShell/bucket/BucketCommands.java | 3 +-
.../ozone/web/ozShell/s3/GetS3SecretHandler.java | 26 +-
.../hadoop/ozone/web/ozShell/s3/S3Commands.java | 60 -
.../hadoop/ozone/web/ozShell/s3/S3Shell.java | 56 +
.../ozone/om/ScmBlockLocationTestIngClient.java | 5 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 408 ++++--
.../ozone/om/TestOzoneManagerHttpServer.java | 14 +-
.../om/ratis/TestOzoneManagerRatisServer.java | 23 -
.../om/ratis/TestOzoneManagerStateMachine.java | 281 ++++
.../security/TestOzoneBlockTokenSecretManager.java | 186 +++
.../TestOzoneDelegationTokenSecretManager.java | 380 +++++
hadoop-ozone/ozone-recon/pom.xml | 179 +++
.../ozone/recon/OzoneConfigurationProvider.java | 43 +
.../apache/hadoop/ozone/recon/ReconConstants.java | 41 +
.../hadoop/ozone/recon/ReconControllerModule.java | 51 +
.../recon/ReconGuiceServletContextListener.java | 40 +
.../apache/hadoop/ozone/recon/ReconHttpServer.java | 88 ++
.../hadoop/ozone/recon/ReconRestServletModule.java | 134 ++
.../org/apache/hadoop/ozone/recon/ReconServer.java | 124 ++
.../hadoop/ozone/recon/ReconServerConfigKeys.java | 95 ++
.../org/apache/hadoop/ozone/recon/ReconUtils.java | 178 +++
.../ozone/recon/api/ContainerKeyService.java | 152 ++
.../hadoop/ozone/recon/api/package-info.java | 23 +
.../ozone/recon/api/types/ContainerKeyPrefix.java | 89 ++
.../ozone/recon/api/types/ContainerMetadata.java | 73 +
.../ozone/recon/api/types/IsoDateAdapter.java | 48 +
.../hadoop/ozone/recon/api/types/KeyMetadata.java | 147 ++
.../hadoop/ozone/recon/api/types/package-info.java | 22 +
.../apache/hadoop/ozone/recon/package-info.java | 22 +
.../recon/recovery/ReconOMMetadataManager.java | 38 +
.../recon/recovery/ReconOmMetadataManagerImpl.java | 97 ++
.../hadoop/ozone/recon/recovery/package-info.java | 22 +
.../recon/spi/ContainerDBServiceProvider.java | 69 +
.../recon/spi/HddsDatanodeServiceProvider.java | 25 +
.../recon/spi/OzoneManagerServiceProvider.java | 45 +
.../recon/spi/StorageContainerServiceProvider.java | 25 +
.../spi/impl/ContainerDBServiceProviderImpl.java | 166 +++
.../recon/spi/impl/ContainerKeyPrefixCodec.java | 87 ++
.../spi/impl/OzoneManagerServiceProviderImpl.java | 194 +++
.../recon/spi/impl/ReconContainerDBProvider.java | 80 ++
.../hadoop/ozone/recon/spi/impl/package-info.java | 22 +
.../hadoop/ozone/recon/spi/package-info.java | 24 +
.../ozone/recon/tasks/ContainerKeyMapperTask.java | 112 ++
.../hadoop/ozone/recon/tasks/package-info.java | 22 +
.../main/resources/webapps.recon.WEB-INF/web.xml | 28 +
.../src/main/resources/webapps/recon/index.html | 59 +
.../ozone/recon/AbstractOMMetadataManagerTest.java | 190 +++
.../apache/hadoop/ozone/recon/TestReconCodecs.java | 58 +
.../apache/hadoop/ozone/recon/TestReconUtils.java | 135 ++
.../ozone/recon/api/TestContainerKeyService.java | 226 +++
.../hadoop/ozone/recon/api/package-info.java | 21 +
.../apache/hadoop/ozone/recon/package-info.java | 21 +
.../recovery/TestReconOmMetadataManagerImpl.java | 148 ++
.../hadoop/ozone/recon/recovery/package-info.java | 21 +
.../impl/TestContainerDBServiceProviderImpl.java | 206 +++
.../impl/TestOzoneManagerServiceProviderImpl.java | 184 +++
.../spi/impl/TestReconContainerDBProvider.java | 87 ++
.../hadoop/ozone/recon/spi/impl/package-info.java | 21 +
.../recon/tasks/TestContainerKeyMapperTask.java | 194 +++
.../hadoop/ozone/recon/tasks/package-info.java | 22 +
hadoop-ozone/ozonefs-lib-current/pom.xml | 96 ++
hadoop-ozone/ozonefs-lib-legacy/pom.xml | 4 +-
hadoop-ozone/ozonefs-lib/pom.xml | 96 --
hadoop-ozone/ozonefs/pom.xml | 29 +-
.../org/apache/hadoop/fs/ozone/O3fsDtFetcher.java | 84 ++
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 28 +-
.../hadoop/fs/ozone/OzoneClientAdapterImpl.java | 242 +++-
.../apache/hadoop/fs/ozone/OzoneFileSystem.java | 225 ++-
.../org.apache.hadoop.security.token.DtFetcher | 19 +
...rg.apache.hadoop.security.token.TokenIdentifier | 17 +
.../org.apache.hadoop.security.token.TokenRenewer | 19 +
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 51 +-
.../fs/ozone/TestOzoneFileSystemWithMocks.java | 111 ++
hadoop-ozone/pom.xml | 82 +-
hadoop-ozone/s3gateway/pom.xml | 25 +-
.../org/apache/hadoop/ozone/s3/AWSAuthParser.java | 78 +
.../apache/hadoop/ozone/s3/AWSV4AuthParser.java | 300 ++++
.../hadoop/ozone/s3/OzoneClientProducer.java | 90 +-
.../hadoop/ozone/s3/OzoneServiceProvider.java | 52 +
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 7 +-
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 2 +-
.../endpoint/PlainTextMultipartUploadReader.java | 66 +
.../hadoop/ozone/s3/endpoint/RootEndpoint.java | 7 +-
.../hadoop/ozone/s3/exception/S3ErrorTable.java | 8 +
.../hadoop/ozone/s3/header/AWSConstants.java | 67 -
.../ozone/s3/header/AuthorizationHeaderV4.java | 35 +-
.../apache/hadoop/ozone/s3/header/Credential.java | 16 +-
.../apache/hadoop/ozone/s3/util/OzoneS3Util.java | 36 +
.../src/main/resources/webapps/static/index.html | 2 +-
.../hadoop/ozone/s3/TestOzoneClientProducer.java | 144 ++
.../hadoop/ozone/s3/endpoint/TestRootList.java | 4 +-
.../ozone/s3/header/TestAuthorizationHeaderV4.java | 2 +-
hadoop-ozone/tools/pom.xml | 28 +-
.../ozone/genesis/BenchMarkBlockManager.java | 168 ---
.../hadoop/ozone/genesis/BenchMarkOMClient.java | 153 ++
.../ozone/genesis/BenchMarkOzoneManager.java | 185 +++
.../apache/hadoop/ozone/genesis/BenchMarkSCM.java | 120 ++
.../org/apache/hadoop/ozone/genesis/Genesis.java | 25 +-
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 96 ++
.../ozone/freon/TestFreonWithPipelineDestroy.java | 4 +-
.../hadoop/ozone/scm/TestContainerSQLCli.java | 262 ----
hadoop-project/pom.xml | 50 +-
.../fs/aliyun/oss/AliyunOSSCopyFileTask.java | 2 +-
.../hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java | 8 +-
.../fs/aliyun/oss/AliyunOSSFileSystemStore.java | 7 +-
.../oss/TestAliyunOSSFileSystemContract.java | 76 +
hadoop-tools/hadoop-aws/pom.xml | 4 +
.../java/org/apache/hadoop/fs/s3a/Constants.java | 82 +-
.../hadoop/fs/s3a/NoVersionAttributeException.java | 44 +
.../hadoop/fs/s3a/RemoteFileChangedException.java | 49 +
.../org/apache/hadoop/fs/s3a/S3AFileStatus.java | 8 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 84 +-
.../org/apache/hadoop/fs/s3a/S3AInputStream.java | 29 +-
.../apache/hadoop/fs/s3a/S3AInstrumentation.java | 14 +
.../org/apache/hadoop/fs/s3a/S3AReadOpContext.java | 14 +
.../org/apache/hadoop/fs/s3a/S3ARetryPolicy.java | 7 +
.../java/org/apache/hadoop/fs/s3a/Statistic.java | 4 +-
.../hadoop/fs/s3a/auth/MarshalledCredentials.java | 2 +-
.../hadoop/fs/s3a/impl/ChangeDetectionPolicy.java | 376 +++++
.../apache/hadoop/fs/s3a/impl/ChangeTracker.java | 196 +++
.../apache/hadoop/fs/s3a/impl/LogExactlyOnce.java | 42 +
.../apache/hadoop/fs/s3a/impl/package-info.java | 30 +
.../fs/s3a/s3guard/DynamoDBMetadataStore.java | 42 +-
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 20 +-
.../src/site/markdown/tools/hadoop-aws/index.md | 125 ++
.../src/site/markdown/tools/hadoop-aws/s3guard.md | 64 +-
.../src/site/markdown/tools/hadoop-aws/testing.md | 12 +-
.../tools/hadoop-aws/troubleshooting_s3a.md | 113 +-
.../fs/contract/s3a/ITestS3AContractOpen.java | 9 +
.../fs/contract/s3a/ITestS3AContractSeek.java | 275 ++++
.../hadoop/fs/s3a/ITestS3AFailureHandling.java | 67 +-
.../hadoop/fs/s3a/ITestS3ARemoteFileChanged.java | 197 +++
.../fs/s3a/ITestS3GuardOutOfBandOperations.java | 633 +++++++++
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 13 +
.../hadoop/fs/s3a/TestStreamChangeTracker.java | 255 ++++
.../s3a/auth/delegation/AbstractDelegationIT.java | 5 +-
.../s3a/auth/delegation/ITestDelegatedMRJob.java | 43 +-
.../hadoop/fs/s3a/commit/AbstractCommitITest.java | 64 +
.../fs/s3a/commit/AbstractITCommitMRJob.java | 156 +-
.../fs/s3a/commit/AbstractYarnClusterITest.java | 256 ++++
.../fs/s3a/commit/magic/ITMagicCommitMRJob.java | 70 -
.../fs/s3a/commit/magic/ITestMagicCommitMRJob.java | 98 ++
.../integration/ITDirectoryCommitMRJob.java | 33 -
.../integration/ITPartitionCommitMRJob.java | 33 -
.../staging/integration/ITStagingCommitMRJob.java | 66 -
.../integration/ITStagingCommitMRJobBadDest.java | 62 -
.../integration/ITestDirectoryCommitMRJob.java | 61 +
.../integration/ITestPartitionCommitMRJob.java | 62 +
.../integration/ITestStagingCommitMRJob.java | 94 ++
.../ITestStagingCommitMRJobBadDest.java | 89 ++
.../commit/terasort/AbstractCommitTerasortIT.java | 241 ++++
.../terasort/ITestTerasortDirectoryCommitter.java | 62 +
.../terasort/ITestTerasortMagicCommitter.java | 73 +
.../fs/s3a/s3guard/TestDynamoDBMiscOperations.java | 18 +
.../mapreduce/filecache/TestS3AResourceScope.java | 64 +
hadoop-tools/hadoop-azure/pom.xml | 2 +
.../src/config/checkstyle-suppressions.xml | 4 +-
.../hadoop-azure/src/config/checkstyle.xml | 4 +-
.../fs/azure/AzureNativeFileSystemStore.java | 4 +-
.../fs/azurebfs/AzureBlobFileSystemStore.java | 12 +-
.../fs/azurebfs/constants/HttpQueryParams.java | 1 +
.../hadoop/fs/azurebfs/services/AbfsClient.java | 7 +-
.../fs/azurebfs/services/AbfsOutputStream.java | 21 +-
.../fs/azurebfs/utils/SSLSocketFactoryEx.java | 5 +-
.../fs/azurebfs/ITestFileSystemProperties.java | 4 +-
.../java/org/apache/hadoop/tools/CopyListing.java | 23 +
.../org/apache/hadoop/tools/SimpleCopyListing.java | 4 +-
.../hadoop-distcp/src/site/markdown/DistCp.md.vm | 120 ++
.../fs/swift/snative/SwiftNativeFileSystem.java | 12 +-
.../src/config/checkstyle.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 10 +-
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 9 +
.../hadoop/yarn/util/resource/ResourceUtils.java | 29 +-
.../yarn/api/records/timeline/jaxb.properties | 13 +
.../hadoop-yarn-applications-catalog/README.md | 50 +
.../Dockerfile | 37 +
.../README.md | 43 +
.../pom.xml | 108 ++
.../src/main/resources/jaas.config | 9 +
.../src/main/resources/samples.xml | 135 ++
.../src/main/scripts/entrypoint.sh | 45 +
.../src/main/scripts/setup-image.sh | 21 +
.../.gitignore | 5 +
.../.yarnrc | 1 +
.../README.md | 51 +
.../dev-support/findbugs-exclude.xml | 0
.../package.json | 26 +
.../pom.xml | 491 +++++++
.../yarn/appcatalog/application/AppCatalog.java | 60 +
.../application/AppCatalogSolrClient.java | 404 ++++++
.../appcatalog/application/YarnServiceClient.java | 174 +++
.../yarn/appcatalog/application/package-info.java | 22 +
.../controller/AppDetailsController.java | 265 ++++
.../appcatalog/controller/AppListController.java | 182 +++
.../appcatalog/controller/AppStoreController.java | 198 +++
.../yarn/appcatalog/controller/package-info.java | 22 +
.../hadoop/yarn/appcatalog/model/AppDetails.java | 76 +
.../hadoop/yarn/appcatalog/model/AppEntry.java | 72 +
.../yarn/appcatalog/model/AppStoreEntry.java | 106 ++
.../hadoop/yarn/appcatalog/model/Application.java | 89 ++
.../hadoop/yarn/appcatalog/model/package-info.java | 22 +
.../hadoop/yarn/appcatalog/utils/RandomWord.java | 422 ++++++
.../yarn/appcatalog/utils/WordLengthException.java | 29 +
.../hadoop/yarn/appcatalog/utils/package-info.java | 22 +
.../src/main/javascript/app.js | 59 +
.../src/main/javascript/controllers.js | 320 +++++
.../src/main/javascript/filters.js | 32 +
.../src/main/javascript/routes.js | 17 +
.../src/main/javascript/services.js | 17 +
.../src/main/resources/appcatalog.properties | 1 +
.../src/main/resources/log4j.properties | 23 +
.../src/main/webapp/.gitignore | 1 +
.../src/main/webapp/WEB-INF/beans.xml | 22 +
.../src/main/webapp/WEB-INF/web.xml | 75 +
.../src/main/webapp/css/bootstrap-hadoop.css | 1488 ++++++++++++++++++++
.../src/main/webapp/css/bootstrap-hadoop.min.css | 18 +
.../src/main/webapp/css/img/feather.png | Bin 0 -> 6066 bytes
.../src/main/webapp/css/img/jenkins.png | Bin 0 -> 23140 bytes
.../src/main/webapp/css/img/loading.svg | 8 +
.../pattern-f61c2e99f82389a67432f54155c5f483.png | Bin 0 -> 27786 bytes
.../src/main/webapp/css/specific.css | 252 ++++
.../src/main/webapp/css/theme.css | 30 +
.../src/main/webapp/index.html | 82 ++
.../src/main/webapp/js/bootstrap-hadoop.js | 284 ++++
.../src/main/webapp/js/bootstrap-hadoop.min.js | 19 +
.../src/main/webapp/partials/deploy.html | 80 ++
.../src/main/webapp/partials/details.html | 64 +
.../src/main/webapp/partials/home.html | 52 +
.../src/main/webapp/partials/new.html | 111 ++
.../src/main/webapp/theme.html | 649 +++++++++
.../application/EmbeddedSolrServerFactory.java | 104 ++
.../application/TestAppCatalogSolrClient.java | 130 ++
.../controller/AppDetailsControllerTest.java | 138 ++
.../controller/AppListControllerTest.java | 95 ++
.../controller/AppStoreControllerTest.java | 97 ++
.../src/test/javascript/controllersSpec.js | 249 ++++
.../src/test/javascript/karma.conf.js | 34 +
.../src/test/resources/configsets.tgz | Bin 0 -> 2996 bytes
.../exampleCollection/conf/lang/stopwords_en.txt | 54 +
.../configsets/exampleCollection/conf/params.json | 20 +
.../exampleCollection/conf/protwords.txt | 20 +
.../configsets/exampleCollection/conf/schema.xml | 115 ++
.../exampleCollection/conf/solrconfig.xml | 36 +
.../exampleCollection/conf/stopwords.txt | 0
.../configsets/exampleCollection/conf/synonyms.txt | 28 +
.../src/test/resources/log4j.properties | 11 +
.../hadoop-yarn-applications-catalog/pom.xml | 37 +
.../pom.xml | 4 -
.../distributedshell/ApplicationMaster.java | 33 +-
.../yarn/applications/distributedshell/Client.java | 29 +-
.../distributedshell/TestDistributedShell.java | 37 +
.../pom.xml | 4 -
.../yarn/service/client/ApiServiceClient.java | 24 +-
.../service/client/SystemServiceManagerImpl.java | 4 +-
.../yarn/service/client/TestApiServiceClient.java | 9 +
.../examples/appcatalog/appcatalog.json | 28 +
.../apache/hadoop/yarn/service/ServiceManager.java | 2 +-
.../hadoop/yarn/service/client/ServiceClient.java | 6 +-
.../hadoop/yarn/service/component/Component.java | 2 +-
.../component/instance/ComponentInstance.java | 10 +-
.../service/containerlaunch/AbstractLauncher.java | 2 +-
.../yarn/service/provider/ProviderUtils.java | 24 +-
.../provider/docker/DockerProviderService.java | 6 +
.../timelineservice/ServiceMetricsSink.java | 8 +-
.../hadoop/yarn/service/utils/ZookeeperUtils.java | 2 +-
.../hadoop/yarn/service/ServiceTestUtils.java | 8 +
.../yarn/service/TestYarnNativeServices.java | 22 +-
.../provider/TestAbstractProviderService.java | 27 +-
.../yarn/service/provider/TestProviderUtils.java | 9 +
.../hadoop-yarn/hadoop-yarn-applications/pom.xml | 1 +
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 4 -
.../apache/hadoop/yarn/client/api/AMRMClient.java | 5 +-
.../yarn/client/api/async/AMRMClientAsync.java | 5 +-
.../api/impl/ContainerManagementProtocolProxy.java | 26 +-
.../yarn/client/api/impl/RemoteRequestsTable.java | 49 +-
.../client/api/impl/SharedCacheClientImpl.java | 4 +-
.../yarn/client/api/impl/YarnClientImpl.java | 6 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 4 -
.../yarn/FileSystemBasedConfigurationProvider.java | 8 +-
.../hadoop/yarn/YarnUncaughtExceptionHandler.java | 14 +-
.../yarn/api/records/impl/pb/ResourcePBImpl.java | 7 +-
.../org/apache/hadoop/yarn/client/AHSProxy.java | 7 +-
.../apache/hadoop/yarn/client/ClientRMProxy.java | 7 +-
.../client/ConfiguredRMFailoverProxyProvider.java | 8 +-
.../org/apache/hadoop/yarn/client/RMProxy.java | 7 +-
.../RequestHedgingRMFailoverProxyProvider.java | 8 +-
.../yarn/client/api/impl/DirectTimelineWriter.java | 8 +-
.../client/api/impl/FileSystemTimelineWriter.java | 98 +-
.../yarn/client/api/impl/TimelineClientImpl.java | 7 +-
.../yarn/client/api/impl/TimelineConnector.java | 7 +-
.../client/api/impl/TimelineReaderClientImpl.java | 8 +-
.../yarn/client/api/impl/TimelineV2ClientImpl.java | 7 +-
.../yarn/client/api/impl/TimelineWriter.java | 23 +-
.../apache/hadoop/yarn/event/AsyncDispatcher.java | 19 +-
.../apache/hadoop/yarn/event/EventDispatcher.java | 13 +-
.../factories/impl/pb/RpcClientFactoryPBImpl.java | 8 +-
.../factories/impl/pb/RpcServerFactoryPBImpl.java | 7 +-
.../apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java | 13 +-
.../java/org/apache/hadoop/yarn/ipc/YarnRPC.java | 9 +-
.../AggregatedLogDeletionService.java | 7 +-
.../LogAggregationFileControllerFactory.java | 6 +-
.../ifile/IndexedFileAggregatedLogsBlock.java | 64 +-
.../ifile/LogAggregationIndexedFileController.java | 6 +-
.../tfile/LogAggregationTFileController.java | 6 +-
.../yarn/nodelabels/CommonNodeLabelsManager.java | 27 +-
.../yarn/nodelabels/FileSystemNodeLabelsStore.java | 8 +-
.../nodelabels/NonAppendableFSNodeLabelStore.java | 18 +-
.../yarn/nodelabels/store/AbstractFSNodeStore.java | 7 +-
.../hadoop/yarn/security/AMRMTokenIdentifier.java | 7 +-
.../hadoop/yarn/security/AMRMTokenSelector.java | 14 +-
.../hadoop/yarn/security/AdminACLsManager.java | 7 +-
.../yarn/security/ConfiguredYarnAuthorizer.java | 4 +-
.../yarn/security/ContainerTokenIdentifier.java | 9 +-
.../yarn/security/ContainerTokenSelector.java | 14 +-
.../hadoop/yarn/security/NMTokenIdentifier.java | 9 +-
.../hadoop/yarn/security/NMTokenSelector.java | 14 +-
.../yarn/security/YarnAuthorizationProvider.java | 9 +-
.../security/client/ClientToAMTokenSelector.java | 14 +-
.../security/client/RMDelegationTokenSelector.java | 14 +-
.../client/TimelineDelegationTokenSelector.java | 18 +-
.../server/security/ApplicationACLsManager.java | 34 +-
.../yarn/util/AbstractLivelinessMonitor.java | 7 +-
.../apache/hadoop/yarn/util/AdHocLogDumper.java | 123 +-
.../yarn/util/DockerClientConfigHandler.java | 6 +-
.../org/apache/hadoop/yarn/util/FSDownload.java | 25 +-
.../hadoop/yarn/util/ProcfsBasedProcessTree.java | 37 +-
.../hadoop/yarn/util/ResourceCalculatorPlugin.java | 8 +-
.../yarn/util/ResourceCalculatorProcessTree.java | 8 +-
.../java/org/apache/hadoop/yarn/util/Times.java | 7 +-
.../hadoop/yarn/util/WindowsBasedProcessTree.java | 14 +-
.../apache/hadoop/yarn/util/YarnVersionInfo.java | 9 +-
.../util/resource/DefaultResourceCalculator.java | 8 +-
.../util/resource/DominantResourceCalculator.java | 7 +-
.../hadoop/yarn/util/resource/Resources.java | 8 +-
.../yarn/webapp/GenericExceptionHandler.java | 8 +-
.../src/main/resources/yarn-default.xml | 38 +-
.../apache/hadoop/yarn/TestContainerLaunchRPC.java | 9 +-
.../yarn/TestContainerResourceIncreaseRPC.java | 8 +-
.../hadoop/yarn/api/BasePBImplRecordsTest.java | 7 +-
.../api/records/timeline/TestTimelineRecords.java | 8 +-
.../TestTimelineServiceRecords.java | 8 +-
.../api/impl/TestTimelineClientForATS1_5.java | 8 +-
.../client/api/impl/TestTimelineClientV2Impl.java | 8 +-
.../apache/hadoop/yarn/event/InlineDispatcher.java | 7 +-
.../logaggregation/TestAggregatedLogFormat.java | 8 +-
.../hadoop/yarn/util/TestAdHocLogDumper.java | 31 +-
.../apache/hadoop/yarn/util/TestFSDownload.java | 7 +-
.../util/TestLog4jWarningErrorMetricsAppender.java | 32 +-
.../yarn/util/TestProcfsBasedProcessTree.java | 10 +-
.../apache/hadoop/yarn/util/TestRackResolver.java | 7 +-
.../yarn/util/TestWindowsBasedProcessTree.java | 8 +-
.../hadoop-yarn/hadoop-yarn-csi/pom.xml | 6 +-
.../yarn/csi/adaptor/DefaultCsiAdaptorImpl.java | 20 +-
.../hadoop/yarn/csi/client/CsiGrpcClient.java | 7 +-
.../hadoop/yarn/csi/client/FakeCsiDriver.java | 5 +-
.../ApplicationHistoryManagerOnTimelineStore.java | 7 +-
.../webapp/AHSWebServices.java | 8 +-
.../webapp/ContextFactory.java | 62 +
.../yarn/server/timeline/LeveldbTimelineStore.java | 32 +-
.../timeline/RollingLevelDBTimelineStore.java | 46 +-
.../timeline/security/TimelineACLsManager.java | 9 +-
...elineV1DelegationTokenSecretManagerService.java | 20 +-
.../webapp/TestAHSWebServices.java | 11 +
.../hadoop-yarn-server-common/pom.xml | 4 -
.../yarn/server/AMHeartbeatRequestHandler.java | 20 +-
.../LocalityMulticastAMRMProxyPolicy.java | 7 +-
.../store/impl/SQLFederationStateStore.java | 18 +-
.../server/scheduler/ResourceRequestSetKey.java | 2 +-
.../security/BaseContainerTokenSecretManager.java | 8 +-
.../server/uam/UnmanagedApplicationManager.java | 8 +-
.../server/util/timeline/TimelineServerUtils.java | 7 +-
.../yarn/server/utils/YarnServerSecurityUtils.java | 2 +-
.../hadoop/yarn/server/webapp/LogWebService.java | 8 +-
.../hadoop-yarn-server-nodemanager/pom.xml | 4 -
.../yarn/server/nodemanager/ContainerExecutor.java | 19 +-
.../nodemanager/DefaultContainerExecutor.java | 6 +-
.../yarn/server/nodemanager/DeletionService.java | 7 +-
.../server/nodemanager/LinuxContainerExecutor.java | 8 +-
.../yarn/server/nodemanager/NodeManager.java | 23 +-
.../server/nodemanager/NodeStatusUpdaterImpl.java | 64 +-
.../WindowsSecureContainerExecutor.java | 32 +-
.../nodemanager/amrmproxy/AMRMProxyService.java | 10 +-
.../amrmproxy/AMRMProxyTokenSecretManager.java | 5 +-
.../amrmproxy/DefaultRequestInterceptor.java | 10 +-
.../amrmproxy/FederationInterceptor.java | 16 +-
.../nodemanager/api/impl/pb/NMProtoUtils.java | 12 +-
.../nodemanager/containermanager/AuxServices.java | 4 +-
.../containermanager/ContainerManagerImpl.java | 29 +-
.../application/ApplicationImpl.java | 7 +-
.../containermanager/container/ContainerImpl.java | 24 +-
.../deletion/task/DockerContainerDeletionTask.java | 5 +-
.../deletion/task/FileDeletionTask.java | 18 +-
.../launcher/ContainerCleanup.java | 34 +-
.../containermanager/launcher/ContainerLaunch.java | 33 +-
.../launcher/RecoverPausedContainerLaunch.java | 6 +-
.../resources/CGroupElasticMemoryController.java | 8 +-
.../linux/resources/CGroupsHandlerImpl.java | 27 +-
.../linux/resources/CGroupsResourceCalculator.java | 18 +-
.../resources/CombinedResourceCalculator.java | 8 +-
.../linux/resources/DefaultOOMHandler.java | 8 +-
.../resources/NetworkPacketTaggingHandlerImpl.java | 4 +-
.../resources/NetworkTagMappingManagerFactory.java | 6 +-
.../linux/resources/ResourceHandlerModule.java | 16 +-
.../TrafficControlBandwidthHandlerImpl.java | 10 +-
.../linux/resources/TrafficController.java | 20 +-
.../resources/fpga/FpgaResourceAllocator.java | 258 ++--
.../resources/fpga/FpgaResourceHandlerImpl.java | 113 +-
.../linux/resources/gpu/GpuResourceAllocator.java | 11 +-
.../resources/gpu/GpuResourceHandlerImpl.java | 8 +-
.../linux/resources/numa/NumaNodeResource.java | 7 +-
.../resources/numa/NumaResourceAllocator.java | 7 +-
.../resources/numa/NumaResourceHandlerImpl.java | 8 +-
.../runtime/DelegatingLinuxContainerRuntime.java | 4 +-
.../linux/runtime/DockerLinuxContainerRuntime.java | 40 +-
.../runtime/docker/DockerCommandExecutor.java | 19 +-
.../localizer/LocalizedResource.java | 17 +-
.../localizer/ResourceLocalizationService.java | 37 +-
.../localizer/security/LocalizerTokenSelector.java | 4 +-
.../logaggregation/AppLogAggregatorImpl.java | 8 +-
.../loghandler/NonAggregatingLogHandler.java | 6 +-
.../monitor/ContainersMonitorImpl.java | 26 +-
.../com/nvidia/NvidiaGPUPluginForRuntimeV2.java | 18 +-
.../deviceframework/DeviceMappingManager.java | 24 +-
.../deviceframework/DevicePluginAdapter.java | 7 +-
.../DeviceResourceDockerRuntimePluginImpl.java | 47 +-
.../deviceframework/DeviceResourceHandlerImpl.java | 20 +-
.../deviceframework/DeviceResourceUpdaterImpl.java | 7 +-
.../fpga/AbstractFpgaVendorPlugin.java | 24 +-
.../fpga/AoclDiagnosticOutputParser.java | 166 +++
.../resourceplugin/fpga/FpgaDiscoverer.java | 167 ++-
.../fpga/FpgaNodeResourceUpdateHandler.java | 24 +-
.../resourceplugin/fpga/FpgaResourcePlugin.java | 19 +-
.../resourceplugin/fpga/IntelFpgaOpenclPlugin.java | 319 ++---
.../AoclOutputBasedDiscoveryStrategy.java | 52 +
.../fpga/discovery/DeviceSpecParser.java | 82 ++
.../fpga/discovery/FPGADiscoveryStrategy.java | 32 +
.../ScriptBasedFPGADiscoveryStrategy.java | 66 +
.../SettingsBasedFPGADiscoveryStrategy.java | 54 +
.../fpga/discovery/package-info.java | 19 +
.../resourceplugin/fpga/package-info.java | 19 +
.../resourceplugin/gpu/GpuDiscoverer.java | 41 +-
.../gpu/NvidiaDockerV1CommandPlugin.java | 21 +-
.../gpu/NvidiaDockerV2CommandPlugin.java | 7 +-
.../AllocationBasedResourceUtilizationTracker.java | 8 +-
.../scheduler/ContainerScheduler.java | 6 +-
.../nodemanager/metrics/NodeManagerMetrics.java | 13 +
.../recovery/NMLeveldbStateStoreService.java | 77 +-
.../scheduler/DistributedScheduler.java | 4 +-
.../security/NMTokenSecretManagerInNM.java | 16 +-
.../timelineservice/NMTimelinePublisher.java | 64 +-
.../util/CgroupsLCEResourcesHandler.java | 14 +-
.../nodemanager/util/NodeManagerHardwareUtils.java | 8 +-
.../nodemanager/util/ProcessIdFileReader.java | 10 +-
.../nodemanager/webapp/ContainerLogsPage.java | 4 +-
.../server/nodemanager/webapp/NMWebServices.java | 10 +-
.../webapp/dao/gpu/GpuDeviceInformationParser.java | 12 +-
.../container-executor/impl/utils/docker-util.c | 11 +-
.../test/utils/test_docker_util.cc | 4 +-
.../server/nodemanager/TestNodeManagerMXBean.java | 8 +-
.../containermanager/container/TestContainer.java | 3 +
.../launcher/TestContainerLaunch.java | 5 +
.../TestCGroupElasticMemoryController.java | 8 +-
.../resources/fpga/TestFpgaResourceHandler.java | 340 +++--
.../logaggregation/TestLogAggregationService.java | 2 +-
.../resourceplugin/fpga/TestAoclOutputParser.java | 105 ++
.../resourceplugin/fpga/TestFpgaDiscoverer.java | 368 +++--
.../resourceplugin/gpu/TestGpuDiscoverer.java | 317 ++++-
.../timelineservice/TestNMTimelinePublisher.java | 20 +-
.../hadoop-yarn-server-resourcemanager/pom.xml | 4 -
.../server/resourcemanager/AMSProcessingChain.java | 7 +-
.../ActiveStandbyElectorBasedElectorService.java | 14 +-
.../yarn/server/resourcemanager/AdminService.java | 7 +-
.../resourcemanager/ApplicationMasterService.java | 7 +-
.../server/resourcemanager/ClientRMService.java | 13 +-
.../CuratorBasedElectorService.java | 8 +-
.../DecommissioningNodesWatcher.java | 18 +-
.../resourcemanager/DefaultAMSProcessor.java | 7 +-
.../server/resourcemanager/NodesListManager.java | 13 +-
.../OpportunisticContainerAllocatorAMService.java | 20 +-
.../resourcemanager/RMActiveServiceContext.java | 8 +-
.../yarn/server/resourcemanager/RMAppManager.java | 133 +-
.../yarn/server/resourcemanager/RMAuditLogger.java | 7 +-
.../yarn/server/resourcemanager/RMContextImpl.java | 7 +-
.../RMCriticalThreadUncaughtExceptionHandler.java | 6 +-
.../yarn/server/resourcemanager/RMNMInfo.java | 7 +-
.../yarn/server/resourcemanager/RMServerUtils.java | 13 +-
.../server/resourcemanager/ResourceManager.java | 31 +-
.../resourcemanager/ResourceTrackerService.java | 42 +-
.../ahs/RMApplicationHistoryWriter.java | 8 +-
.../resourcemanager/amlauncher/AMLauncher.java | 7 +-
.../amlauncher/ApplicationMasterLauncher.java | 6 +-
.../blacklist/SimpleBlacklistManager.java | 15 +-
.../metrics/TimelineServiceV1Publisher.java | 21 +-
.../metrics/TimelineServiceV2Publisher.java | 14 +-
.../resourcemanager/monitor/SchedulingMonitor.java | 7 +-
.../monitor/SchedulingMonitorManager.java | 6 +-
.../monitor/capacity/FifoCandidatesSelector.java | 16 +-
.../capacity/FifoIntraQueuePreemptionPlugin.java | 20 +-
.../capacity/IntraQueueCandidatesSelector.java | 36 +-
.../capacity/PreemptableResourceCalculator.java | 25 +-
.../ProportionalCapacityPreemptionPolicy.java | 21 +-
.../QueuePriorityContainerCandidateSelector.java | 34 +-
.../ReservedContainerCandidatesSelector.java | 23 +-
.../nodelabels/FileSystemNodeAttributeStore.java | 8 +-
.../nodelabels/NodeAttributesManagerImpl.java | 30 +-
.../nodelabels/NodeLabelsUtils.java | 7 +-
.../nodelabels/RMDelegatedNodeLabelsUpdater.java | 14 +-
.../nodelabels/RMNodeLabelsManager.java | 35 +-
.../placement/AppNameMappingPlacementRule.java | 8 +-
.../placement/DefaultPlacementRule.java | 113 ++
.../resourcemanager/placement/FSPlacementRule.java | 177 +++
.../placement/FairQueuePlacementUtils.java | 100 ++
.../placement/PlacementFactory.java | 35 +-
.../placement/PlacementManager.java | 13 +-
.../resourcemanager/placement/PlacementRule.java | 57 +-
.../placement/PrimaryGroupPlacementRule.java | 103 ++
.../placement/QueuePlacementRuleUtils.java | 2 +-
.../placement/RejectPlacementRule.java | 65 +
.../SecondaryGroupExistingPlacementRule.java | 102 ++
.../placement/SpecifiedPlacementRule.java | 77 +
.../placement/UserGroupMappingPlacementRule.java | 8 +-
.../placement/UserPlacementRule.java | 70 +
.../recovery/FileSystemRMStateStore.java | 32 +-
.../recovery/LeveldbRMStateStore.java | 81 +-
.../resourcemanager/recovery/RMStateStore.java | 20 +-
.../recovery/RMStateStoreFactory.java | 7 +-
.../recovery/RMStateStoreUtils.java | 7 +-
.../resourcemanager/recovery/ZKRMStateStore.java | 162 +--
.../impl/pb/ApplicationAttemptStateDataPBImpl.java | 8 +-
.../reservation/AbstractSchedulerPlanFollower.java | 12 +-
.../planning/SimpleCapacityReplanner.java | 8 +-
.../resource/DynamicResourceConfiguration.java | 23 +-
.../resource/ResourceProfilesManagerImpl.java | 8 +-
.../server/resourcemanager/rmapp/RMAppImpl.java | 24 +-
.../rmapp/attempt/RMAppAttemptImpl.java | 54 +-
.../rmapp/attempt/RMAppAttemptMetrics.java | 11 +-
.../rmapp/monitor/RMAppLifetimeMonitor.java | 7 +-
.../rmcontainer/RMContainerImpl.java | 40 +-
.../server/resourcemanager/rmnode/RMNodeImpl.java | 21 +-
.../scheduler/AbstractResourceUsage.java | 14 +-
.../scheduler/AbstractYarnScheduler.java | 89 +-
.../scheduler/ActiveUsersManager.java | 19 +-
.../scheduler/AppSchedulingInfo.java | 45 +-
.../scheduler/ClusterNodeTracker.java | 7 +-
.../ConfigurationMutationACLPolicyFactory.java | 6 +-
.../scheduler/QueueStateManager.java | 7 +-
.../resourcemanager/scheduler/ResourceUsage.java | 4 +-
.../scheduler/SchedulerAppUtils.java | 19 +-
.../scheduler/SchedulerApplicationAttempt.java | 61 +-
.../resourcemanager/scheduler/SchedulerNode.java | 7 +-
.../resourcemanager/scheduler/SchedulerUtils.java | 19 +-
.../scheduler/activities/ActivitiesLogger.java | 7 +-
.../scheduler/activities/ActivitiesManager.java | 7 +-
.../scheduler/activities/AllocationActivity.java | 7 +-
.../scheduler/activities/NodeAllocation.java | 7 +-
.../capacity/AbstractAutoCreatedLeafQueue.java | 6 +-
.../scheduler/capacity/AbstractCSQueue.java | 51 +-
.../capacity/AbstractManagedParentQueue.java | 17 +-
.../AppPriorityACLConfigurationParser.java | 8 +-
.../scheduler/capacity/AutoCreatedLeafQueue.java | 5 +-
.../scheduler/capacity/CapacityScheduler.java | 216 ++-
.../capacity/CapacitySchedulerConfiguration.java | 50 +-
.../capacity/CapacitySchedulerQueueManager.java | 6 +-
.../scheduler/capacity/LeafQueue.java | 141 +-
.../scheduler/capacity/ManagedParentQueue.java | 16 +-
.../scheduler/capacity/ParentQueue.java | 63 +-
.../scheduler/capacity/PlanQueue.java | 2 +-
.../scheduler/capacity/QueueCapacities.java | 12 +-
.../capacity/QueueManagementDynamicEditPolicy.java | 13 +-
.../scheduler/capacity/ReservationQueue.java | 2 +-
.../scheduler/capacity/UsersManager.java | 87 +-
.../allocator/AbstractContainerAllocator.java | 7 +-
.../allocator/RegularContainerAllocator.java | 35 +-
.../conf/FSSchedulerConfigurationStore.java | 6 +-
.../capacity/conf/LeveldbConfigurationStore.java | 8 +-
.../conf/MutableCSConfigurationProvider.java | 8 +-
.../capacity/conf/YarnConfigurationStore.java | 8 +-
.../conf/YarnConfigurationStoreFactory.java | 6 +-
.../capacity/conf/ZKConfigurationStore.java | 8 +-
.../capacity/preemption/PreemptionManager.java | 12 +-
.../GuaranteedOrZeroCapacityOverTimePolicy.java | 35 +-
.../scheduler/common/fica/FiCaSchedulerApp.java | 102 +-
.../scheduler/common/fica/FiCaSchedulerNode.java | 7 +-
.../MemoryPlacementConstraintManager.java | 34 +-
.../PlacementConstraintManagerService.java | 8 +-
.../constraint/PlacementConstraintsUtil.java | 47 +-
.../algorithm/LocalAllocationTagsManager.java | 4 +-
.../distributed/NodeQueueLoadMonitor.java | 38 +-
.../scheduler/fair/AllocationConfiguration.java | 30 +-
.../fair/AllocationFileLoaderService.java | 34 +-
.../scheduler/fair/FSAppAttempt.java | 73 +-
.../scheduler/fair/FSLeafQueue.java | 13 +-
.../scheduler/fair/FSParentQueue.java | 6 +-
.../scheduler/fair/FSPreemptionThread.java | 7 +-
.../resourcemanager/scheduler/fair/FSQueue.java | 16 +-
.../scheduler/fair/FSQueueMetrics.java | 69 +-
.../scheduler/fair/FSSchedulerNode.java | 7 +-
.../scheduler/fair/FairScheduler.java | 185 +--
.../scheduler/fair/FairSchedulerConfiguration.java | 6 +-
.../scheduler/fair/MaxRunningAppsEnforcer.java | 6 +-
.../scheduler/fair/QueueManager.java | 23 +-
.../scheduler/fair/QueuePlacementPolicy.java | 344 +++--
.../scheduler/fair/QueuePlacementRule.java | 366 -----
.../fair/VisitedResourceRequestTracker.java | 8 +-
.../scheduler/fair/allocation/QueueProperties.java | 3 +-
.../scheduler/fair/policies/FairSharePolicy.java | 7 +-
.../scheduler/fair/policies/FifoPolicy.java | 7 +-
.../scheduler/fifo/FifoAppAttempt.java | 11 +-
.../scheduler/fifo/FifoScheduler.java | 18 +-
.../placement/LocalityAppPlacementAllocator.java | 28 +-
.../scheduler/placement/MultiNodeSorter.java | 7 +-
.../placement/MultiNodeSortingManager.java | 8 +-
.../SingleConstraintAppPlacementAllocator.java | 12 +-
.../policy/AbstractComparatorOrderingPolicy.java | 7 +-
.../scheduler/policy/CompoundComparator.java | 3 -
.../security/AMRMTokenSecretManager.java | 12 +-
.../security/AppPriorityACLsManager.java | 8 +-
.../security/DelegationTokenRenewer.java | 13 +-
.../security/NMTokenSecretManagerInRM.java | 20 +-
.../resourcemanager/security/QueueACLsManager.java | 7 +-
.../security/RMContainerTokenSecretManager.java | 8 +-
.../security/RMDelegationTokenSecretManager.java | 8 +-
.../RMTimelineCollectorManager.java | 20 +-
.../volume/csi/VolumeManagerImpl.java | 7 +-
.../volume/csi/lifecycle/VolumeImpl.java | 13 +-
.../resourcemanager/webapp/DeSelectFields.java | 8 +-
.../server/resourcemanager/webapp/NavBlock.java | 8 +-
.../webapp/RMAppLogAggregationStatusBlock.java | 8 +-
.../server/resourcemanager/webapp/RMWebApp.java | 8 +-
.../resourcemanager/webapp/RMWebAppUtil.java | 7 +-
.../resourcemanager/webapp/dao/ActivitiesInfo.java | 7 +-
.../webapp/dao/AppActivitiesInfo.java | 7 +-
.../webapp/dao/AppAllocationInfo.java | 7 +-
.../webapp/dao/NodeAllocationInfo.java | 7 +-
.../yarn/server/resourcemanager/ACLsTestBase.java | 7 +-
.../yarn/server/resourcemanager/Application.java | 42 +-
.../ApplicationMasterServiceTestBase.java | 8 +-
.../hadoop/yarn/server/resourcemanager/MockAM.java | 6 +-
.../hadoop/yarn/server/resourcemanager/MockRM.java | 17 +-
.../yarn/server/resourcemanager/NodeManager.java | 23 +-
.../hadoop/yarn/server/resourcemanager/Task.java | 7 +-
.../resourcemanager/TestAMAuthorization.java | 7 +-
.../server/resourcemanager/TestAppManager.java | 7 +-
.../TestAppManagerWithFairScheduler.java | 289 ++--
.../resourcemanager/TestApplicationACLs.java | 9 +-
.../resourcemanager/TestApplicationCleanup.java | 22 +-
.../TestApplicationMasterLauncher.java | 18 +-
.../TestApplicationMasterServiceInterceptor.java | 8 +-
.../resourcemanager/TestClientRMService.java | 7 +-
.../server/resourcemanager/TestClientRMTokens.java | 7 +-
.../TestContainerResourceUsage.java | 8 +-
.../TestKillApplicationWithRMHA.java | 8 +-
.../resourcemanager/TestLeaderElectorService.java | 7 +-
.../hadoop/yarn/server/resourcemanager/TestRM.java | 20 +-
.../resourcemanager/TestRMEmbeddedElector.java | 8 +-
.../yarn/server/resourcemanager/TestRMHA.java | 6 +-
.../resourcemanager/TestRMHAForNodeLabels.java | 8 +-
.../yarn/server/resourcemanager/TestRMRestart.java | 14 +-
.../resourcemanager/TestResourceManager.java | 7 +-
.../resourcemanager/TestResourceManagerMXBean.java | 8 +-
.../resourcemanager/TestSignalContainer.java | 18 +-
.../TestSubmitApplicationWithRMHA.java | 8 +-
.../TestWorkPreservingRMRestart.java | 7 +-
.../TestWorkPreservingUnmanagedAM.java | 8 +-
.../applicationsmanager/TestAMLaunchFailure.java | 9 +-
.../applicationsmanager/TestASMStateMachine.java | 9 +-
.../TestApplicationMasterExpiry.java | 9 +-
.../metrics/TestSystemMetricsPublisher.java | 10 +-
.../metrics/TestSystemMetricsPublisherForV2.java | 3 +-
...ionalCapacityPreemptionPolicyMockFramework.java | 8 +-
.../placement/TestFairQueuePlacementUtils.java | 137 ++
.../placement/TestPlacementFactory.java | 75 +
.../placement/TestPlacementRuleFS.java | 212 +++
.../recovery/RMStateStoreTestBase.java | 7 +-
.../recovery/TestFSRMStateStore.java | 7 +-
.../recovery/TestZKRMStateStore.java | 7 +-
.../recovery/TestZKRMStateStorePerf.java | 7 +-
.../TestZKRMStateStoreZKClientConnections.java | 8 +-
.../reservation/TestReservationInputValidator.java | 8 +-
.../reservation/TestSchedulerPlanFollowerBase.java | 12 +-
.../resourcetracker/TestNMExpiry.java | 7 +-
.../rmapp/TestApplicationLifetimeMonitor.java | 15 +-
.../rmapp/TestNodesListManager.java | 17 +-
.../rmapp/TestRMAppTransitions.java | 50 +-
.../rmapp/attempt/TestRMAppAttemptTransitions.java | 11 +-
.../scheduler/TestResourceUsage.java | 7 +-
.../scheduler/TestSchedulerUtils.java | 17 +-
.../scheduler/capacity/TestApplicationLimits.java | 7 +-
.../scheduler/capacity/TestCapacityScheduler.java | 7 +-
.../TestCapacitySchedulerAutoCreatedQueueBase.java | 6 +-
.../TestCapacitySchedulerAutoQueueCreation.java | 6 +-
.../TestCapacitySchedulerDynamicBehavior.java | 8 +-
.../capacity/TestCapacitySchedulerMultiNodes.java | 8 +-
.../capacity/TestCapacitySchedulerPerf.java | 12 +-
.../scheduler/capacity/TestChildQueueOrder.java | 7 +-
.../capacity/TestContainerAllocation.java | 8 +-
.../scheduler/capacity/TestContainerResizing.java | 7 +-
.../scheduler/capacity/TestLeafQueue.java | 7 +-
.../scheduler/capacity/TestParentQueue.java | 7 +-
.../scheduler/capacity/TestQueueCapacities.java | 7 +-
.../scheduler/capacity/TestQueueMappings.java | 7 +-
.../scheduler/capacity/TestQueueParsing.java | 7 +-
.../scheduler/capacity/TestReservations.java | 15 +-
.../scheduler/capacity/TestUtils.java | 7 +-
.../conf/TestLeveldbConfigurationStore.java | 8 +-
.../capacity/conf/TestZKConfigurationStore.java | 8 +-
.../constraint/TestPlacementProcessor.java | 8 +-
.../distributed/TestNodeQueueLoadMonitor.java | 50 +
.../scheduler/fair/FairSchedulerTestBase.java | 28 +-
.../fair/TestAllocationFileLoaderService.java | 152 +-
.../scheduler/fair/TestAppRunnability.java | 35 +-
.../fair/TestApplicationMasterServiceWithFS.java | 8 +-
.../scheduler/fair/TestContinuousScheduling.java | 30 +-
.../scheduler/fair/TestFSAppAttempt.java | 5 +-
.../scheduler/fair/TestFSParentQueue.java | 20 +-
.../scheduler/fair/TestFairScheduler.java | 295 ++--
.../fair/TestFairSchedulerConfiguration.java | 15 +-
.../TestFairSchedulerWithMultiResourceTypes.java | 9 +
.../scheduler/fair/TestMaxRunningAppsEnforcer.java | 25 +-
.../scheduler/fair/TestQueueManager.java | 45 +-
.../scheduler/fair/TestQueuePlacementPolicy.java | 515 +++++--
.../scheduler/fair/TestSchedulingPolicy.java | 18 +-
.../scheduler/fifo/TestFifoScheduler.java | 18 +-
.../resourcemanager/security/TestAMRMTokens.java | 7 +-
.../security/TestDelegationTokenRenewer.java | 8 +-
.../security/TestRMDelegationTokens.java | 7 +-
.../webapp/TestRMWebServicesNodeLabels.java | 8 +-
.../TestRMWebServicesSchedulerActivities.java | 6 +-
.../webapp/dao/TestFairSchedulerQueueInfo.java | 24 +-
.../clientrm/FederationClientInterceptor.java | 2 +-
.../metrics/ClientSCMMetrics.java | 2 +-
.../metrics/SharedCacheUploaderMetrics.java | 2 +-
.../hadoop-yarn-server-tests/pom.xml | 4 -
.../server/ContainerTokenIdentifierForTest.java | 2 +-
.../yarn/server/NMTokenIdentifierNewForTest.java | 2 +-
.../security/TestTimelineAuthFilterForV2.java | 4 +
.../pom.xml | 150 ++
.../DocumentStoreCollectionCreator.java | 66 +
.../DocumentStoreTimelineReaderImpl.java | 121 ++
.../DocumentStoreTimelineWriterImpl.java | 285 ++++
.../documentstore/DocumentStoreUtils.java | 489 +++++++
.../documentstore/collection/CollectionType.java | 44 +
.../document/NoDocumentFoundException.java | 39 +
.../collection/document/TimelineDocument.java | 37 +
.../document/entity/TimelineEntityDocument.java | 242 ++++
.../document/entity/TimelineEventSubDoc.java | 96 ++
.../document/entity/TimelineMetricSubDoc.java | 167 +++
.../collection/document/entity/package-info.java | 30 +
.../flowactivity/FlowActivityDocument.java | 131 ++
.../document/flowactivity/FlowActivitySubDoc.java | 73 +
.../document/flowactivity/package-info.java | 29 +
.../document/flowrun/FlowRunDocument.java | 239 ++++
.../collection/document/flowrun/package-info.java | 29 +
.../collection/document/package-info.java | 30 +
.../documentstore/collection/package-info.java | 30 +
.../documentstore/lib/DocumentStoreFactory.java | 96 ++
.../lib/DocumentStoreNotSupportedException.java | 35 +
.../documentstore/lib/DocumentStoreVendor.java | 39 +
.../documentstore/lib/package-info.java | 30 +
.../documentstore/package-info.java | 29 +
.../documentstore/reader/DocumentStoreReader.java | 45 +
.../reader/TimelineCollectionReader.java | 220 +++
.../cosmosdb/CosmosDBDocumentStoreReader.java | 232 +++
.../reader/cosmosdb/package-info.java | 28 +
.../documentstore/reader/package-info.java | 29 +
.../documentstore/writer/DocumentStoreWriter.java | 35 +
.../writer/TimelineCollectionWriter.java | 146 ++
.../cosmosdb/CosmosDBDocumentStoreWriter.java | 235 ++++
.../writer/cosmosdb/package-info.java | 28 +
.../documentstore/writer/package-info.java | 29 +
.../documentstore/DocumentStoreTestUtils.java | 81 ++
.../timelineservice/documentstore/JsonUtils.java | 59 +
.../TestDocumentStoreCollectionCreator.java | 64 +
.../TestDocumentStoreTimelineReaderImpl.java | 407 ++++++
.../TestDocumentStoreTimelineWriterImpl.java | 90 ++
.../collection/TestDocumentOperations.java | 177 +++
.../reader/DummyDocumentStoreReader.java | 118 ++
.../writer/DummyDocumentStoreWriter.java | 46 +
.../test/resources/documents/flowactivity-doc.json | 20 +
.../src/test/resources/documents/flowrun-doc.json | 126 ++
.../documents/test-timeline-entities-doc.json | 185 +++
.../test/resources/documents/timeline-app-doc.json | 203 +++
.../resources/documents/timeline-entities.json | 119 ++
.../storage/HBaseTimelineReaderImpl.java | 6 +-
.../storage/common/ColumnRWHelper.java | 4 +-
.../storage/flow/FlowActivityTableRW.java | 2 +-
.../storage/flow/FlowRunTableRW.java | 2 +-
.../storage/reader/TimelineEntityReader.java | 8 +-
.../timelineservice/storage/flow/FlowScanner.java | 4 +-
.../timelineservice/storage/flow/FlowScanner.java | 12 +-
.../hadoop-yarn-server-timelineservice/pom.xml | 5 -
.../collector/NodeTimelineCollectorManager.java | 16 +-
.../collector/TimelineCollector.java | 16 +-
.../PerNodeAggTimelineCollectorMetrics.java | 2 +-
.../reader/TimelineReaderWebServices.java | 6 +-
.../storage/FileSystemTimelineWriterImpl.java | 30 +-
.../storage/NoOpTimelineReaderImpl.java | 12 +-
.../storage/NoOpTimelineWriterImpl.java | 16 +-
...TimelineReaderWhitelistAuthorizationFilter.java | 58 +-
.../storage/TestFileSystemTimelineWriterImpl.java | 119 ++
.../hadoop-yarn-server-web-proxy/pom.xml | 4 -
.../hadoop/yarn/server/webproxy/ProxyCA.java | 4 +-
.../hadoop/yarn/server/webproxy/ProxyUtils.java | 4 +-
.../yarn/server/webproxy/WebAppProxyServlet.java | 8 +-
.../yarn/server/webproxy/amfilter/AmIpFilter.java | 27 +-
.../hadoop-yarn/hadoop-yarn-server/pom.xml | 1 +
.../src/site/markdown/DockerContainers.md | 15 +-
.../src/site/markdown/TimelineServiceV2.md | 45 +-
.../src/site/markdown/yarn-service/Examples.md | 13 +
.../main/webapp/app/controllers/yarn-flow/runs.js | 2 +-
.../main/webapp/app/models/yarn-flowrun-brief.js | 9 +-
pom.xml | 2 +
1348 files changed, 62171 insertions(+), 14917 deletions(-)
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/InnerNode.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/InnerNodeImpl.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NetConstants.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NetUtils.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NetworkTopology.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NetworkTopologyImpl.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/Node.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NodeImpl.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NodeSchema.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NodeSchemaLoader.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NodeSchemaManager.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/IntegerCodec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/LongCodec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RocksDBCheckpoint.java
create mode 100644 hadoop-hdds/common/src/main/resources/network-topology-default.xml
create mode 100644 hadoop-hdds/common/src/main/resources/network-topology-nodegroup.xml
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/conf/TestOzoneConfiguration.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/conf/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/net/TestNetworkTopologyImpl.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/net/TestNodeSchemaLoader.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/net/TestNodeSchemaManager.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/tracing/TestStringCodec.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/tracing/package-info.java
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/enforce-error.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/good.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/invalid-cost.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/invalid-version.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/multiple-leaf.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/multiple-root.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/multiple-topology.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/no-leaf.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/no-root.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/no-topology.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/path-layers-size-mismatch.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/path-with-id-reference-failure.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/unknown-layer-type.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/wrong-path-order-1.xml
create mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/wrong-path-order-2.xml
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeHttpServer.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerScrubber.java
create mode 100644 hadoop-hdds/container-service/src/main/resources/webapps/hddsDatanode/.gitkeep
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/TestHddsSecureDatanodeInit.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerCheck.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModeHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/BackgroundPipelineCreator.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMContainerMetrics.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/chillmode/TestChillModeHandler.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestReplicationManager.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/MockRatisPipelineProvider.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMContainerMetrics.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/SSLConnectionConfigurator.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_htable.c
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/common/htable.c
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/common/htable.h
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jclasses.c
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jclasses.h
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/AddBlockPoolException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MemoryCacheStats.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MemoryMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/MemoryMappedBlock.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemMappedBlock.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/PmemVolumeManager.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestStripedFileAppend.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestCacheByPmemMappableBlockLoader.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetCache.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/ha/OMProxyInfo.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/ha/OMProxyProvider.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/ha/package-info.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/S3SecretManager.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/S3SecretManagerImpl.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/TokenIdentifierCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/exceptions/NotLeaderException.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/ha/OMFailoverProxyProvider.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/ha/package-info.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OMRatisHelper.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OzoneFileStatus.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/OzoneManagerHAProtocol.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/OzoneManagerServerProtocol.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/AWSV4AuthValidator.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestAWSV4AuthValidator.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestOzoneBlockTokenSecretManager.java
delete mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestOzoneDelegationTokenSecretManager.java
copy hadoop-ozone/dist/src/main/compose/{ozone => ozone-om-ha}/.env (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-om-ha/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-om-ha/docker-config
copy hadoop-ozone/dist/src/main/compose/{ozone => ozone-recon}/.env (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-recon/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-recon/docker-config
copy hadoop-ozone/dist/src/main/compose/{ozonesecure => ozonesecure-mr}/.env (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/README.md
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/docker-image/docker-krb5/Dockerfile-krb5
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/docker-image/docker-krb5/README.md
copy hadoop-ozone/dist/src/main/compose/{ozonesecure => ozonesecure-mr}/docker-image/docker-krb5/kadm5.acl (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonesecure => ozonesecure-mr}/docker-image/docker-krb5/krb5.conf (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonesecure => ozonesecure-mr}/docker-image/docker-krb5/launcher.sh (100%)
create mode 100644 hadoop-ozone/dist/src/main/smoketest/__init__.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/bucketcreate.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-fs.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-s3.robot
delete mode 100644 hadoop-ozone/dist/src/main/smoketest/security/ozone-secure.robot
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneChaosCluster.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneLoadGenerator.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestMiniChaosOzoneCluster.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestBlockOutputStream.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestBlockOutputStreamWithFailures.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneClientRetriesOnException.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMDbCheckpointServlet.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestSecureOzoneManager.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/ozShell/TestS3Shell.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3SecretManager.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3SecretManagerImpl.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ScmClient.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/fs/OzoneManagerFS.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/fs/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OMRatisHelper.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/RequestHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/OzoneShell.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/S3Commands.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/S3Shell.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerStateMachine.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/TestOzoneBlockTokenSecretManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/TestOzoneDelegationTokenSecretManager.java
create mode 100644 hadoop-ozone/ozone-recon/pom.xml
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/OzoneConfigurationProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconConstants.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconControllerModule.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconGuiceServletContextListener.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconHttpServer.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconRestServletModule.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconServer.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconServerConfigKeys.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/ReconUtils.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/ContainerKeyService.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/ContainerKeyPrefix.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/ContainerMetadata.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/IsoDateAdapter.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/KeyMetadata.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOMMetadataManager.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/ReconOmMetadataManagerImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/recovery/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ContainerDBServiceProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/HddsDatanodeServiceProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/OzoneManagerServiceProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/StorageContainerServiceProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ContainerDBServiceProviderImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ContainerKeyPrefixCodec.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerDBProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/spi/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperTask.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/main/resources/webapps.recon.WEB-INF/web.xml
create mode 100644 hadoop-ozone/ozone-recon/src/main/resources/webapps/recon/index.html
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/AbstractOMMetadataManagerTest.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/TestReconCodecs.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/TestReconUtils.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestContainerKeyService.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/api/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/recovery/TestReconOmMetadataManagerImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/recovery/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestContainerDBServiceProviderImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestOzoneManagerServiceProviderImpl.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerDBProvider.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/package-info.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestContainerKeyMapperTask.java
create mode 100644 hadoop-ozone/ozone-recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/package-info.java
create mode 100644 hadoop-ozone/ozonefs-lib-current/pom.xml
delete mode 100644 hadoop-ozone/ozonefs-lib/pom.xml
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/O3fsDtFetcher.java
create mode 100644 hadoop-ozone/ozonefs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.DtFetcher
create mode 100644 hadoop-ozone/ozonefs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier
create mode 100644 hadoop-ozone/ozonefs/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenRenewer
create mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithMocks.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/AWSAuthParser.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/AWSV4AuthParser.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneServiceProvider.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/PlainTextMultipartUploadReader.java
delete mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AWSConstants.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/OzoneS3Util.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestOzoneClientProducer.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkBlockManager.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOMClient.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOzoneManager.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkSCM.java
delete mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/scm/TestContainerSQLCli.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/NoVersionAttributeException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/RemoteFileChangedException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ChangeDetectionPolicy.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ChangeTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/LogExactlyOnce.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/package-info.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARemoteFileChanged.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardOutOfBandOperations.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestStreamChangeTracker.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/AbstractYarnClusterITest.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/magic/ITMagicCommitMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/magic/ITestMagicCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITDirectoryCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITPartitionCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITStagingCommitMRJob.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITStagingCommitMRJobBadDest.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITestDirectoryCommitMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITestPartitionCommitMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITestStagingCommitMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITestStagingCommitMRJobBadDest.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/terasort/AbstractCommitTerasortIT.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/terasort/ITestTerasortDirectoryCommitter.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/terasort/ITestTerasortMagicCommitter.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/mapreduce/filecache/TestS3AResourceScope.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/resources/org/apache/hadoop/yarn/api/records/timeline/jaxb.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/README.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/Dockerfile
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/README.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/resources/jaas.config
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/resources/samples.xml
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/scripts/entrypoint.sh
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker/src/main/scripts/setup-image.sh
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/.gitignore
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/.yarnrc
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/README.md
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/{hadoop-yarn-services/hadoop-yarn-services-api => hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp}/dev-support/findbugs-exclude.xml (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/package.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/AppCatalog.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/AppCatalogSolrClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/YarnServiceClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/application/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/AppDetailsController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/AppListController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/AppStoreController.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/controller/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/AppDetails.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/AppEntry.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/AppStoreEntry.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/Application.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/model/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/utils/RandomWord.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/utils/WordLengthException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/java/org/apache/hadoop/yarn/appcatalog/utils/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/app.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/controllers.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/filters.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/routes.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/javascript/services.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/resources/appcatalog.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/resources/log4j.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/.gitignore
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/WEB-INF/beans.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/WEB-INF/web.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/bootstrap-hadoop.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/bootstrap-hadoop.min.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/feather.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/jenkins.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/loading.svg
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/img/pattern-f61c2e99f82389a67432f54155c5f483.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/specific.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/css/theme.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/index.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/js/bootstrap-hadoop.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/js/bootstrap-hadoop.min.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/deploy.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/details.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/home.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/partials/new.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/main/webapp/theme.html
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/application/EmbeddedSolrServerFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/application/TestAppCatalogSolrClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/controller/AppDetailsControllerTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/controller/AppListControllerTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/java/org/apache/hadoop/yarn/appcatalog/controller/AppStoreControllerTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/javascript/controllersSpec.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/javascript/karma.conf.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets.tgz
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/lang/stopwords_en.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/params.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/protwords.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/schema.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/solrconfig.xml
copy hadoop-tools/hadoop-resourceestimator/src/main/resources/webapps/ResourceEstimatorServer/.gitignore => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/stopwords.txt (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/configsets/exampleCollection/conf/synonyms.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/src/test/resources/log4j.properties
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/pom.xml
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/examples/appcatalog/appcatalog.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/ContextFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/AoclDiagnosticOutputParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/AoclOutputBasedDiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/DeviceSpecParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/FPGADiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/ScriptBasedFPGADiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/SettingsBasedFPGADiscoveryStrategy.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/TestAoclOutputParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/DefaultPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/FSPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/FairQueuePlacementUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/PrimaryGroupPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/RejectPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/SecondaryGroupExistingPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/SpecifiedPlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/UserPlacementRule.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestFairQueuePlacementUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestPlacementFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestPlacementRuleFS.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/pom.xml
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreCollectionCreator.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreTimelineReaderImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreTimelineWriterImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreUtils.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/CollectionType.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/NoDocumentFoundException.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/TimelineDocument.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/TimelineEntityDocument.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/TimelineEventSubDoc.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/TimelineMetricSubDoc.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/entity/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowactivity/FlowActivityDocument.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowactivity/FlowActivitySubDoc.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowactivity/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowrun/FlowRunDocument.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/flowrun/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/document/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/DocumentStoreFactory.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/DocumentStoreNotSupportedException.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/DocumentStoreVendor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/lib/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/DocumentStoreReader.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/TimelineCollectionReader.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/cosmosdb/CosmosDBDocumentStoreReader.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/cosmosdb/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/DocumentStoreWriter.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/TimelineCollectionWriter.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/cosmosdb/CosmosDBDocumentStoreWriter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/cosmosdb/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/main/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/package-info.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/DocumentStoreTestUtils.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/JsonUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/TestDocumentStoreCollectionCreator.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/TestDocumentStoreTimelineReaderImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/TestDocumentStoreTimelineWriterImpl.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/collection/TestDocumentOperations.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/reader/DummyDocumentStoreReader.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/java/org/apache/hadoop/yarn/server/timelineservice/documentstore/writer/DummyDocumentStoreWriter.java
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/flowactivity-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/flowrun-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/test-timeline-entities-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/timeline-app-doc.json
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore/src/test/resources/documents/timeline-entities.json
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org