You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by si...@apache.org on 2021/07/09 21:35:01 UTC
[ozone] branch HDDS-4944 updated (54ea4c7 -> 77c83c0)
This is an automated email from the ASF dual-hosted git repository.
siyao pushed a change to branch HDDS-4944
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from 54ea4c7 HDDS-5080. enable s3 test suite for secure-ha (#2147)
add d6f4d07 HDDS-2949: mkdir : store directory entries in a separate table (#1404)
add 8e17992 HDDS-4266: CreateFile : store parent dir entries into DirTable and file entry into separate FileTable (#1473)
add 60dd692 HDDS-4332: ListFileStatus - do lookup in directory and file tables (#1503)
add 172ab51 HDDS-4357: Rename : make rename an atomic ops by updating key path entry in dir/file table (#1557)
add 995b3bd HDDS-4358: Delete : make delete an atomic operation (#1607)
add d810193 HDDS-4321. Fix compilation errors : merge HDDS-4308 and HDDS-4473 changes into the branch (#1668)
add b9771e4 HDDS-4596. Directory table, fileTable and openFile Table is missing from the OM DB Definition. (#1724)
add c10abe6 HDDS-4514. AllocateBlock : lookup and update open file table for the given path (#1679)
add 0bdbe41 HDDS-2942. Putkey : create key table entries for intermediate directories in the key path (#1764)
add f132f1e HDDS-4658. LookupKey: do lookup in dir and file tables (#1775)
add 9003eca HDDS-4717. Fix TestOzoneFileSystemV1 and TestObjectStoreV1 cases (#1815)
add f5e4167 HDDS-4720. RenameKey : add unit test to verify bucket#renameKey (#1847)
add b4d75d6 HDDS-4486. Feature Config: Make proper enableFSPaths and OFS optimized flag combinations (#1848)
add 5b86a07 HDDS-4771. [FSO]S3MultiPart: Implement InitiateMultiPartUpload (#1877)
add 797cf96 HDDS-4805. [FSO]Fix findbugs issues after HDDS-2195 (#1906)
add 85f07a2 HDDS-4781. [FSO]S3MultiPart: Implement create and commit upload part file (#1897)
add b3327bc HDDS-4742. Make trash work with FS Optimised Buckets. (#1915)
add 30d9fa4 HDDS-4813. [FSO]S3Multipart: Implement UploadCompleteRequest (#1923)
add f28699c HDDS-4513.[FSO]OzoneContract unit test case fixes (#1945)
add 0dfad20 HDDS-4683. [FSO]ListKeys: do lookup in dir and file tables (#1954)
add 7be14c8 HDDS-4490.[FSO]RenameAndDelete : make ofs#rename and ofs#delete an atomic operation. (#1965)
add 46d9192 HDDS-4835. [FSO]S3Multipart: Implement UploadAbortRequest (#1997)
add cdc7855 HDDS-4924. [FSO]S3Multipart: Implement OzoneBucket#listParts (#2016)
add 7ca2c5f HDDS-4973. [FSO] Missed to cleanup new FileTables in OMRequests (#2035)
add 2f7606b HDDS-4790. Add a tool to parse entries in the prefix format (#1891)
add 81361d6 HDDS-4917.[FSO]Implement ACL requests for new layout (#2024)
add 5ed34df HDDS-4932. [FSO] Provide list subpaths function to perform recursive ACL check during delete and rename op (#2008)
add 459e990 HDDS-4495. [FSO]Delete : Implement async cleanup of garbage and orphan sub-dirs/files (#2093)
add 9ecfafe HDDS-5018. [FSO] Add robot tests for new Filesystem layout (#2071)
add bd22405 HDDS-4727. [FSO]OMConfig: Modify 'ozone.om.layout.version' config (#2118)
add c326199 HDDS-4691. [FSO]Authorizer: OM can do recursive ACL check for subpaths (#2011)
add f9910c6 HDDS-5072. [FSO] Addendum patch to fix compilation error (#2130)
add 45c62e8 HDDS-5067. [FSO] Add documentation (#2117)
add aa0abab HDDS-5042. [FSO] Improve KeyDeletingService to cleanup FSO files (#2128)
add 4449cfda HDDS-5079. [FSO] Rename om metadata layout configuration value LEGACY (#2135)
add ec38f01 HDDS-5092. [FSO] Update statis links in the design doc section of the Ozone docs (#2150)
add e63d80a HDDS-5106. [FSO] Avoid using V1 postfixes for prefix related classes (#2174)
add e8d001f HDDS-4888. Merge basic CI checks (#1984)
add 8a80c80 HDDS-5090. make Decommission work under SCM HA. (#2148)
add 6da41ca HDDS-5110. TestStorageContainerManagerHttpServer fails in CI (#2159)
add 014d0a2 HDDS-5075. [SCM HA Security] Remove code of not starting ozone services when Security is enabled on SCM HA cluster (#2132)
add 8badca9 HDDS-5084. Include HISTORY.md/SECURITY.md/CONTRIBUTING.md in the release artifacts. (#2140)
add 7afb1f8 HDDS-5091. Add project separation and first stable release to the HISTORY.md (#2149)
add 724d96c HDDS-5026. Cancel failing PR workflow runs (#2129)
add cda430a HDDS-5087. Ozone RPC client leaks KeyProvider instances. (#2144)
add 8036536 HDDS-5117. CRLInfo should include CRL Sequence ID (#2161)
add 664837b HDDS-5121. Releasing 1.1.0 from "ozone-1.1" branch. Updating version on Master.
add 24ecd22 HDDS-5121. Remove duplicate SNAPSHOT from version (#2163)
add 8cdabec HDDS-5116. Secure datanode/OM may exit if it cannot connect to SCM. (#2162)
add 1761426 HDDS-4986. Read failure because of unhealthy container. (#2152)
add 057f0a0 HDDS-5061. Improve block commit (#2108)
add c89f08e HDDS-5068. build errors due to missing JAXB-runtime jar dependency for openjdk 11 se standalone (#2153)
add 9912b69 HDDS-4195. Fix Recon after HDDS-4133. (#2103)
add f2be003 HDDS-5125. Only test ozonesecure with SCM Ratis disabled (#2168)
add 4e45726 HDDS-5122. SCM Reinitialization can end up leaking Ratis Segmented RaftLogWorker threads (#2167)
add f2ff682 HDDS-5083. Bump version of common-compress (#2139)
add fb3dee8 HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (#2096)
add 2c183b2 HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (#2166)
add bef180e HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (#2155)
add 4c313b8 HDDS-5131. Use timeout in github actions (#2176)
add f1c6757 Merge remote-tracking branch 'origin/master' into HDDS-2939
add a4a9971 HDDS-5093. [FSO] Reducing time of ozonefs acceptance testmatrix (#2164)
add b6908b0 HDDS-5097. [FSO] Cleanup integration tests and reduce the execution time (#2185)
add 3c82503 HDDS-5094. [FSO] Fail OM startup when turn on prefix layout with old buckets (#2151)
add 72d3106 HDDS-5201. [FSO] S3MultiPart: Use existing ozone key format for MPU Info in DB (#2228)
add 974136d HDDS-4923. [FSO]S3Multipart: Run S3 acceptance test with prefix layout (#2208)
add 6af26ca HDDS-4889. Add simple CI check for docs (#2156)
add 7f2dc04 HDDS-5127. Fix getServiceList when SCM HA is enabled (#2173)
add 1e1eefa HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (#2141)
add 6f9c3f2 HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (#2187)
add bddc873 HDDS-4515. Datanodes should be able to persist and load CRL (#2181)
add ce29843 HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (#2184)
add bfb6be8 HDDS-5152. Fix Suggested leader in Client. (#2189)
add bccc439 HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (#2188)
add a920f25 HDDS-5153. Decommissioning a dead node should complete immediately (#2190)
add 66a411b HDDS-4585. Support bucket acl operation in S3g (#1701)
add 5dd0943 HDDS-5144. Create github check to alert when dependency tree is changed (#2177)
add 10312fb HDDS-4983. Display key offset for each block in command key info (#2051)
add 799c38e Revert "HDDS-5144. Create github check to alert when dependency tree is changed (#2177)"
add dcac0f1 HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (#2195)
add 48b84c5 HDDS-2212. Genconf tool should generate config files for secure clust… (#1788)
add a3e7f04 HDDS-5182. Acceptance test may exit with 0 in case of error (#2212)
add 2e07c9f HDDS-5178. Update project information of Contribution guideline (#2210)
add 8bbbf3f HDDS-5177. Update link of weekly calls in README (#2209)
add 00eba3e HDDS-5144. Create github check to alert when dependency tree is changed (#2199)
add 142a2dd HDDS-5185. Update commons-io to 2.8.0 (#2215)
add d430ff3 HDDS-5159. Make periodic disk check interval configurable. (#2193)
add 74496a7 HDDS-5196. Fix typo "vaild" in HeartbeatEndpointTask.java and RegisterEndpointTask (#2222).
add 2061d7d HDDS-5192. Intermittent failure in TestOzoneRpcClient due to volume name conflict (#2220)
add 5e8ae99 HDDS-5186. If primordial SCM id is set, a non-HA cluster can not be initialized. (#2218)
add a16e648 HDDS-5126.Recon should check new containers of a container report with batch (#2172)
add 2c37a15 HDDS-5183. For AccessControlException do not perform failover. (#2213)
add 75c5c10 HDDS-5099. Error with unit test for hdds.container-service TestSchemaOneBackwardsCompatibility (#2183)
add 8e8a01d HDDS-5197. Pass option variables to OZONE_OPTS before adding default GC opts (#2225)
add 7ba1499 HDDS-5198. Remove JBoss repo definition (#2226)
add 2c4a4de HDDS-5173. Divide snapshot related work into notifyInstallSnapshotFromLeader and reinitialize for SCMStateMachine. (#2224)
add 49538aa HDDS-5207. Intermittent failure in TestRatisPipelineProvider#testCreatePipelineWithFactorThree (#2233)
add 3e66778 HDDS-5200. Fix scm roles command if one of the host is unresolvable. (#2227)
add abe136f HDDS-5203. Allow suppressing deprecation warning for HADOOP_ variables (#2231)
add 248d72d Bump logical release name of Ozone 1.2 (#2238)
add b5cbb65 HDDS-5164. Improve client and server logging. (#2234)
add 7262004 HDDS-5089. On-demand disk checker for hdds volume (#2158)
add 842e6f6 HDDS-4729. Add token support for container admin operations (#2186)
add 03620da HDDS-5059. Increase number of client retries/ failovers to OMs (#2171)
add f8a06e0 HDDS-5202. Use scm#checkLeader before processing client requests . (#2229)
add 6075acb HDDS-5205. Make admin check work for SCM HA cluster (#2236)
add 0f6bb36 HDDS-5040. Append om1,om2,om3 to ozone.administrators config for secure OM HA docker config (#2245)
add a7c8695 HDDS-5160. Remove some Freon integration tests (#2194)
add 6b24c18 HDDS-4976. Add container replica related commands to debug (#2070)
add fd71f09 The modificationTime should modify when setAcl success (#2242)
add 4189088 HDDS-5165. OM DB checkpoint servlet not accessible in a secure cluster (#2200)
add eff4914 HDDS-5208. bump rocksdb version to 6.20.3 (#2235)
add 36778af HDDS-5128. Return specific error messages for S3-SDK (#2169)
add 19f989f HDDS-5176. Increase default block cache capacity for Datanodes (#2207)
add 9cab560 HDDS-5155. Add revokeCertificate to SCMSecurityProtocolServer. (#2191)
add 727b258 HDDS-5189. fix markdown files to make them hugo83 compatible (and valid) (#2232)
add 7a4a93e HDDS-5229. Update copyright year in NOTICE (#2248)
add ac7fc7e HDDS-5082. Create unit (!) test for OzoneClient (#2138)
add 2254abf HDDS-5216. Fix race condition causing SCM failOverProxy which is causing failover wrongly. (#2247)
add fcc8e92 HDDS-5187. Avoid Maven connection errors in CI (#2237)
add 10a5b39 HDDS-5113. Disable flaky TestBlockDeletingService#testContainerThrottle
add 91993c1 HDDS-5234.Change default grpc and ratis ports for scm ha (#2251)
add ea1ed7c HDDS-5238. Disable animal-sniffer maven plugin (#2253)
add 9f6d7f2 HDDS-5167. Add clean option for Freon randomkeys to clean generated objects (#2198)
add d63dd6b HDDS-5239. Create http Basedir if not exist (#2255)
add 5a50866 HDDS-5209. Datanode hasEnoughSpace check should apply on volume instead of global DN (#2246)
add 30fe6ea HDDS-5245. Fix OzoneContainer TLS configuration. (#2264)
add 4fd8187 HDDS-5206. Support revoking S3 secret (#2239)
add 08375d7 HDDS-5246. Wait for ever to obtain CA list which is needed during OM/DN startup (#2266)
add 70ef475 HDDS-5233. SCM subsequent init failed when previous scm init failed. (#2249)
add 09c2278 HDDS-5257. Avoid SCM call to get CA certs in non-HA from OM. (#2273)
add e1acbb0 HDDS-5256. Fix fall back of config in SCM HA Cluster (#2272)
add 61a237a HDDS-5261. Delete Useless ozone subcommand (#2275)
add 723cdaa Merge remote-tracking branch 'origin/master' into HDDS-2939
add e50fe52 HDDS-5041. Use getShortUserName in getTrashRoot(s) (#2244)
add fc7b0bd HDDS-5260. Bump node to v16.2.0 for Recon (#2276)
add 9080fc3 HDDS-5248. SCM HA: Continuous PipelineNotFoundException seen in SCM log. (#2267)
add e00fa80 HDDS-5206. Addendum: Support revoking S3 secret (#2270)
add 8d29be1 HDDS-5142. Make generic streaming client/service for container re-replication, data read, scm/om snapshot download (#2256)
add ab8f07d HDDS-5249. Race Condition between Full and Incremental Container Reports (#2268)
add 931a041 HDDS-5266 Misspelt words in S3MultipartUploadCommitPartRequest.java line 202 (#2279)
add 355096b HDDS-5236. Require block token for more operations (#2254)
add 19f5bb2 HDDS-5250. Build integration tests with Maven cache (#2269)
add 0b4779c HDDS-5073. Use ReplicationConfig on client side (#2136)
add 2dbf04e HDDS-5191. Increase default pvc storage size (#2219)
add 000e7c5 HDDS-5252. PipelinePlacementPolicy filter out datanodes with not enough space. (#2271)
add 2671b48 HDDS-5219. Limit number of bad volumes by dfs.datanode.failed.volumes.tolerated. (#2243)
add d9ae1b3 HDDS-5241. SCM UI should have leader/follower and Primordial SCM information (#2260)
add 53bbabf HDDS-5275. Datanode Report Publisher publishes one extra report after DN shutdown (#2283)
add 4301315 HDDS-5158. Add documentation for SCM HA Security. (#2205)
add d567360 HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. (#2281)
add 00b675d HDDS-4927. Determine over and under utilized datanodes in Container Balancer. (#2230)
add 8586815 HDDS-4043. allow deletion from Trash directory without -skipTrash option (#2110)
add fa4ddbc HDDS-4936. Change ozone groupId from org.apache.hadoop to org.apache.ozone (#2018)
add 3ffbdd0 HDDS-4993. Add guardrail for reserved buffer size when DN reads a chunk (#2058)
add e5a774f HDDS-5123. Use the pre-created apache/ozone-testkrb5 image during secure acceptance tests (#2165)
add 5fd9780 HDDS-5272. Make ozonefs.robot execution repeatable (#2280)
add 12e2918 HDDS-5280. Make XceiverClientManager creation when necessary in ContainerOperationClient (#2289)
add 6b09219 HDDS-5264. SCM should send token for CloseContainer command (#2277)
add e35dde9 HDDS-5151. Support ByteBuffer read in OzoneInputStream (#2203)
add 96d9ed7 HDDS-5240. Fix out of bound exception when loading auditparser (#2258)
add 69b92a0 HDDS-5281. Add reinitialize() for SequenceIdGenerator. (#2292)
add 183f4b4 HDDS-5242. Skip `failing` acceptance suite by default (#2263)
add 18f0fe1 HDDS-4733. Remove leveldb from codebase (#2274)
add 656339e HDDS-5204. Missing type-level Javadoc comments (#2262)
add 90a8d7c HDDS-5276. Use built-in cancel support for duplicates (#2284)
add f3f258a HDDS-5277. Recon shows operational status as "DECOMMISSIONING" for "DECOMMISSIONED" DNs (#2286)
add b6a035a HDDS-5288. Recon UI cannot sort DN by operational state (#2298)
add 3030c9f HDDS-5282. Relocate classes copied from Hadoop (#2293)
add 095d09b HDDS-4483. Datanodes should send last processed CRL sequence ID in heartbeats (#2285)
add 25d66d7 HDDS-5154. Add SCM GRPC server to publish CRL update. (#2216)
add c2ae90b HDDS-5295. testCRLStatusReportPublisher fails to create CRLInfo (#2302)
add 8c1de61 HDDS-5274. Revert "HDDS-5153. Decommissioning a dead node should complete immediately (#2190)" (#2282)
add bb8eb39 HDDS-5286. Document SCM HA classes (#2297)
add 405925f HDDS-5168. Use ReplicationConfig in OmKeyArgs (#2287). Contributed by elek.
add 7bc1a25 HDDS-5292. Introduce the WritableContainerInterface to SCM (#2300)
add 9ad2e98 HDDS-5298. Use netty-bom to ensure consistent Netty version (#2304)
add 86bcebb Merge remote-tracking branch 'origin/master' into HDDS-2939
add 0323530 HDDS-5311. [FSO] Fix checkstyle warnings after branch rebase (#2307)
add 39954ad HDDS-5111. DataNode should not always report full information in heartbeat (#2182)
add 3d3a87f HDDS-5235. Add precheck to avoid ContainerHealthTask setting Deleted containers to Missing state (#2250)
add 6e278ba HDDS-5243. Return latest key location for clients (#2261)
add 94fb6f3 Merge remote-tracking branch 'origin/master' into HDDS-2939
add 23352f8 HDDS-5320. [FSO] Fix minor checkstyle warnings (#2313)
add fe9ca6e HDDS-5284. [SCM-HA] SCM start failed with PipelineNotFoundException. (#2295)
add 2bd7f12 HDDS-5313. ContainerInfo should use ReplicationConfig (#2309)
add d1dc91d HDDS-5318. Intermittent failure in TestOzoneManagerDoubleBufferWithOMResponse (#2314)
add 2e385fc HDDS-5312. Intermittent failure in SCM Ratis integration test (#2308)
add fb4436f HDDS-5285. Document block deleting service classes (#2296)
add baf8272 HDDS-5323. Avoid unncessary report processing log messages in follower. (#2317)
add 6752a71 HDDS-5317. BootStrapped SCM fails to bootstrap if it connects to another bootstrapped SCM first. (#2312)
add 159fb24 HDDS-5304. InterSCM protocol should be server-only (#2305)
add f7ac8c1 HDDS-5307. Remove checkAclRight method, duplicates checkAclRights (#2323)
add 6e6aefe HDDS-5315 Skip storing unwanted block tokens on OM DB (#2311)
add ac7166b HDDS-5263. SCM may stay in safe mode forever after a unclean shutdown of SCM. (#2294)
add 2262395 HDDS-4913. Refine the native authorizer parent context right check. (#2012)
add 7e59cb8 HDDS-5334. Remove unncessary log added durig HDDS-5263. (#2326)
add 838d5d8 HDDS-5324. Shouldn't dectNumKeys when delete keys fail (#2319)
add d45819f HDDS-5329. Remove lockmanager and synchronize on ContainerInfo in Replication Manager (#2325)
add ad02d6c HDDS-5316. Fix datanode reserved space calculation. (#2310)
add 9cf7d31 HDDS-5268. Ensure disk checker also scans the ratis log disks periodically (#2290)
add 623e1d9 HDDS-5344. Adding debug log for block token verification. (#2333)
add 7bfd76b HDDS-5348. Pipeline creator may miss one-shot run (#2339)
add c0eb347 HDDS-5237. Add SSL support to the Ozone streaming API (#2315)
add 698456b HDDS-5363. Datanode shutdown due to too many bad volumes in CI (#2352)
add b545e69 HDDS-4692. Handle CRLStatusReport got from DN heartbeats and persist them (#2334)
add 51df153 HDDS-5356. [SCM-HA] SCM start failed with PipelineNotFoundException (#2346)
add b9caec3 HDDS-5342. HTML report missing from acceptance results (#2332)
add bb4a5b2 HDDS-5361. Suppress logging of ServerNotLeaderException. (#2351)
add 76d9a10 HDDS-5347. Wrong cache key for integration tests (#2336)
add eb6c1b6 HDDS-5314. Show number of Open containers per Node in Recon UI (#2321)
add 5c3f9aa HDDS-5355. In ContainerStateMachine, share the executor threads between the containers. (#2350)
add 5a9257e HDDS-5357. Move SCMUpdateProtocol to hdds interface-server package. (#2348)
add fa9616d HDDS-5352. java.lang.ClassNotFoundException: org/eclipse/jetty/alpn/ALPN (#2347)
add 5e1308a HDDS-5367. Update modification time when updating quota/storageType/versioning (#2355)
add 2cd55d7 HDDS-5269. Datandoe with low ratis log volume space should not be considered for new pipeline allocation. (#2344)
add dc02775 HDDS-4926. Support start/stop for container balancer via command line (#2278)
add b563669 HDDS-5350 : Add allocate block support in MockOmTransport (#2341). Contributed by Uma Maheswara Rao G.
add 93fb714 HDDS-5353. Avoid unnecessary executeBatch call in insertAudits (#2342)
add 0766d2c HDDS-5381. SCM terminated with exit status 1: null. (#2362)
add bf42bea HDDS-5369. Cleanup unused configuration related to SCM HA (#2359)
add e251bab HDDS-5289. Update container's deleteTransactionId on creation of the transaction in SCM. (#2361)
add 3232fb0 HDDS-5387. ProfileServlet to move the default output location to an ozone specific directory (#2368)
add 450c375 HDDS-5341. Container report processing is single threaded (#2338)
add be7264d HDDS-5380. Get more accurate space info for DedicatedDiskSpaceUsage. (#2365)
add b7ecf8b HDDS-5383. Eliminate expensive string creation in debug log messages (#2372)
add 474457c HDDS-5389. Include ozoneserviceid in fs.defaultFS when configuring o3fs (#2370)
add 1e5c708 HDDS-5395. Avoid unnecessary numKeyOps.incr() call in OMMetrics (#2374)
add 88383d1 HDDS-5396. Fix negligence issue conditional expressions in MockCRLStore.java (#2380)
add 1d8f972 HDDS-5398. Avoid object creation in ReplicationManger debug log statements (#2379)
add 4e8aec5 HDDS-5406. add proto version to all the proto files. (#2385)
add 9948632 HDDS-5405. Refactor pom files for HadoopRpc and Grpc/Ratis compilation properties. (#2386)
add b9a86c3 HDDS-5332. Add a new column family and a service provider in Recon DB for Namespace Summaries (#2366)
add 19ed794 HDDS-3292. Support Hadoop 3.3 (#2356)
add 4859acc HDDS-5417. Fix json parse stackoverflow for cmd ozone debug container list. (#2391)
add eb753e1 HDDS-5416. Intermittent test failure in TestSCMPipelineManager#testPipelineReload (#2394)
add a2739e2 HDDS-5424. Pipeline list command's output should be in local timezone. (#2400)
add 77c83c0 HDDS-5400. Remove inflight actions in Replication Manager if node switches to not IN_SERVICE (#2381)
No new revisions were added by this update.
Summary of changes:
.github/buildenv/Dockerfile | 20 -
.github/buildenv/entrypoint.sh | 18 -
.github/workflows/cancel-ci.yaml | 9 +-
.github/workflows/post-commit.yml | 260 +++--
.gitignore | 3 +
CONTRIBUTING.md | 10 +-
HISTORY.md | 4 +-
NOTICE.txt | 2 +-
README.md | 2 +-
hadoop-hdds/client/pom.xml | 34 +-
.../hadoop/hdds/scm/XceiverClientFactory.java | 3 +-
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 7 +
.../hadoop/hdds/scm/storage/BlockInputStream.java | 36 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 14 +-
.../hadoop/hdds/scm/storage/ByteArrayReader.java | 67 ++
.../hadoop/hdds/scm/storage/ByteBufferReader.java | 71 ++
.../hdds/scm/storage/ByteReaderStrategy.java | 41 +
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 38 +-
.../storage/DummyBlockInputStreamWithRetry.java | 7 +-
.../hdds/scm/storage/TestBlockInputStream.java | 37 +-
.../hdds/scm/storage/TestChunkInputStream.java | 2 +-
hadoop-hdds/common/pom.xml | 50 +-
.../apache/hadoop/hdds/DFSConfigKeysLegacy.java | 17 -
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 11 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 216 +++--
.../hadoop/hdds/client/RatisReplicationConfig.java | 22 +-
.../hadoop/hdds/client/ReplicationConfig.java | 55 ++
.../hdds/client/StandaloneReplicationConfig.java | 24 +-
.../hadoop/hdds/fs/AbstractSpaceUsageSource.java | 7 +-
.../hadoop/hdds/fs/DedicatedDiskSpaceUsage.java | 8 +-
.../hdds/ratis/ServerNotLeaderException.java | 113 +++
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 102 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 21 +
.../hdds/scm/container/ContainerException.java | 14 +-
.../hadoop/hdds/scm/container/ContainerInfo.java | 55 +-
.../scm/container/ContainerNotFoundException.java | 4 +-
.../ContainerReplicaNotFoundException.java | 4 +-
.../hadoop/hdds/scm/exceptions/SCMException.java | 9 +-
.../hadoop/hdds/scm/ha/NonRetriableException.java | 30 +
.../scm/ha/RetriableWithFailOverException.java | 31 +
.../scm/ha/RetriableWithNoFailoverException.java | 4 +
.../hadoop/hdds/scm/ha/SCMHAConfiguration.java | 8 +-
.../org/apache/hadoop/hdds/scm/ha/SCMHAUtils.java | 122 ++-
.../org/apache/hadoop/hdds/scm/ha/SCMNodeInfo.java | 5 +-
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 71 +-
.../scm/pipeline/PipelineNotFoundException.java | 8 +-
.../pipeline/UnknownPipelineStateException.java | 8 +-
.../protocol/StorageContainerLocationProtocol.java | 39 +
.../hadoop/hdds/scm/utils/ClientCommandsUtils.java | 3 +
.../security/token/ContainerTokenIdentifier.java | 103 ++
.../security/token/OzoneBlockTokenIdentifier.java | 146 ++-
.../security/token/ShortLivedTokenIdentifier.java | 112 +++
.../hadoop/hdds/security/x509/SecurityConfig.java | 16 +
.../hadoop/hdds/security/x509/crl/CRLStatus.java | 87 ++
.../hdds/security/x509/crl/package-info.java | 24 +
.../apache/hadoop/hdds/utils/EntryConsumer.java | 38 -
.../apache/hadoop/hdds/utils/HddsVersionInfo.java | 3 +-
.../org/apache/hadoop/hdds/utils/VersionInfo.java | 8 +-
.../org/apache/hadoop/hdds/utils/db/Codec.java | 0
.../apache/hadoop/hdds/utils/db/package-info.java | 22 +
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 22 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 15 +-
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +
.../ozone/common/ChecksumByteBufferFactory.java | 3 +
.../ozone/common/ChecksumByteBufferImpl.java | 5 +-
.../org/apache/hadoop/ozone/common/Storage.java | 12 +
.../hadoop/ozone/common/utils/BufferUtils.java | 3 +
.../ozone/container/common/helpers/BlockData.java | 2 +-
.../java/org/apache/hadoop/ozone/ha/ConfUtils.java | 3 +
.../main/resources/hdds-version-info.properties | 3 +-
.../common/src/main/resources/ozone-default.xml | 366 +++----
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 24 +-
.../hadoop/hdds/client/TestReplicationConfig.java | 147 +++
.../hdds/fs/TestCachingSpaceUsageSource.java | 2 +-
.../java/org/apache/hadoop/hdds/fs/TestDU.java | 2 +-
.../org/apache/hadoop/hdds/fs/TestDUFactory.java | 2 +-
.../hdds/fs/TestDedicatedDiskSpaceUsage.java | 2 +-
.../fs/TestDedicatedDiskSpaceUsageFactory.java | 2 +-
.../hadoop/hdds/fs/TestSaveSpaceUsageToFile.java | 4 +-
.../hadoop/hdds/fs/TestSpaceUsageFactory.java | 2 +-
.../hdds/ratis/TestServerNotLeaderException.java | 82 ++
.../apache/hadoop/hdds/scm/ha/TestSCMNodeInfo.java | 3 +
.../hadoop/hdds/scm/pipeline/MockPipeline.java | 12 +-
.../hadoop/hdds/tracing/TestStringCodec.java | 2 +-
.../hadoop/hdds/utils/TestResourceLimitCache.java | 2 +-
.../common/TestChecksumImplsComputeSameValues.java | 4 +
.../ozone/container/ContainerTestHelper.java | 628 ++++++++++++
.../apache/hadoop/ozone/lock/TestLockManager.java | 2 +-
hadoop-hdds/config/pom.xml | 14 +-
hadoop-hdds/container-service/pom.xml | 66 +-
.../hdds/datanode/metadata/CRLDBDefinition.java | 99 ++
.../hdds/datanode/metadata/DatanodeCRLStore.java | 84 ++
.../datanode/metadata/DatanodeCRLStoreImpl.java | 127 +++
.../hdds/datanode/metadata/package-info.java | 22 +
.../apache/hadoop/ozone/HddsDatanodeService.java | 32 +-
.../ozone/container/common/impl/ContainerData.java | 2 +
.../container/common/impl/HddsDispatcher.java | 25 +-
.../common/impl/StorageLocationReport.java | 55 +-
.../common/report/CRLStatusReportPublisher.java | 82 ++
.../container/common/report/ReportPublisher.java | 12 +-
.../common/report/ReportPublisherFactory.java | 2 +
.../common/statemachine/DatanodeConfiguration.java | 149 +++
.../common/statemachine/DatanodeStateMachine.java | 31 +-
.../common/statemachine/StateContext.java | 140 ++-
.../CloseContainerCommandHandler.java | 9 +-
.../common/states/datanode/InitDatanodeState.java | 4 +-
.../states/endpoint/HeartbeatEndpointTask.java | 11 +-
.../states/endpoint/RegisterEndpointTask.java | 2 +-
.../states/endpoint/VersionEndpointTask.java | 14 +-
.../server/ratis/ContainerStateMachine.java | 74 +-
.../transport/server/ratis/XceiverServerRatis.java | 54 +-
.../container/common/utils/StorageVolumeUtil.java | 51 +
.../ozone/container/common/volume/HddsVolume.java | 162 +---
.../container/common/volume/HddsVolumeChecker.java | 425 --------
.../container/common/volume/HddsVolumeFactory.java | 93 ++
.../common/volume/ImmutableVolumeSet.java | 8 +-
.../container/common/volume/MetadataVolume.java | 52 +
.../common/volume/MetadataVolumeFactory.java | 56 ++
.../container/common/volume/MutableVolumeSet.java | 366 ++++---
.../container/common/volume/StorageVolume.java | 214 ++++
.../common/volume/StorageVolumeChecker.java | 424 ++++++++
.../common/volume/StorageVolumeFactory.java | 60 ++
.../ozone/container/common/volume/VolumeInfo.java | 56 +-
.../ozone/container/common/volume/VolumeSet.java | 4 +-
.../ozone/container/common/volume/VolumeUsage.java | 30 +-
.../container/keyvalue/KeyValueContainer.java | 12 +-
.../container/keyvalue/KeyValueContainerCheck.java | 4 +-
.../container/keyvalue/KeyValueContainerData.java | 2 +
.../ozone/container/keyvalue/KeyValueHandler.java | 67 +-
.../container/keyvalue/helpers/BlockUtils.java | 33 +-
.../container/keyvalue/helpers/ChunkUtils.java | 54 +-
.../container/keyvalue/impl/BlockManagerImpl.java | 16 +-
.../keyvalue/impl/ChunkManagerDispatcher.java | 10 +-
.../keyvalue/impl/ChunkManagerDummyImpl.java | 6 +-
.../keyvalue/impl/ChunkManagerFactory.java | 5 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 37 +-
.../keyvalue/impl/FilePerChunkStrategy.java | 17 +-
.../background/BlockDeletingService.java | 3 +
.../ozone/container/ozoneimpl/OzoneContainer.java | 75 +-
.../container/replication/ReplicationServer.java | 3 +
.../replication/ReplicationSupervisor.java | 3 +
.../stream/DirectoryServerDestination.java | 38 +
.../container/stream/DirectoryServerSource.java | 59 ++
.../container/stream/DirstreamClientHandler.java | 148 +++
.../container/stream/DirstreamServerHandler.java | 144 +++
.../ozone/container/stream/StreamingClient.java | 111 +++
.../container/stream/StreamingDestination.java | 32 +
.../ozone/container/stream/StreamingException.java | 38 +
.../ozone/container/stream/StreamingServer.java | 114 +++
.../ozone/container/stream/StreamingSource.java | 36 +
.../ozone/container/stream/package-info.java | 22 +
.../hadoop/ozone/protocol/commands/SCMCommand.java | 10 +
.../metadata/TestDatanodeCRLStoreImpl.java | 127 +++
.../hdds/datanode/metadata/package-info.java | 22 +
.../hadoop/ozone/TestHddsDatanodeService.java | 8 +-
.../hadoop/ozone/TestHddsSecureDatanodeInit.java | 4 +-
.../ozone/container/ContainerTestHelper.java | 612 ------------
.../ozone/container/common/SCMTestUtils.java | 2 +-
.../container/common/TestBlockDeletingService.java | 60 +-
.../container/common/TestDatanodeStateMachine.java | 15 +-
.../TestSchemaOneBackwardsCompatibility.java | 35 +-
.../container/common/helpers/TestBlockData.java | 2 +-
.../common/helpers/TestDatanodeVersionFile.java | 2 +-
.../common/impl/TestContainerDataYaml.java | 15 +-
.../impl/TestContainerDeletionChoosingPolicy.java | 33 +-
.../common/impl/TestContainerPersistence.java | 11 +-
.../container/common/impl/TestContainerSet.java | 2 +-
.../container/common/impl/TestHddsDispatcher.java | 14 +-
.../common/report/TestReportPublisher.java | 54 +-
.../common/report/TestReportPublisherFactory.java | 13 +-
.../statemachine/TestDatanodeConfiguration.java | 70 ++
.../common/statemachine/TestStateContext.java | 138 +--
.../states/endpoint/TestHeartbeatEndpointTask.java | 14 +-
.../container/common/volume/TestHddsVolume.java | 116 +++
.../common/volume/TestHddsVolumeChecker.java | 319 ------
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 2 +-
.../common/volume/TestStorageVolumeChecker.java | 323 ++++++
.../container/common/volume/TestVolumeSet.java | 53 +-
.../common/volume/TestVolumeSetDiskChecks.java | 75 +-
.../container/keyvalue/ChunkLayoutTestInfo.java | 4 +-
.../keyvalue/TestKeyValueBlockIterator.java | 8 +-
.../container/keyvalue/TestKeyValueContainer.java | 11 +-
.../keyvalue/TestKeyValueContainerCheck.java | 8 +-
.../TestKeyValueContainerMarkUnhealthy.java | 2 +-
.../container/keyvalue/TestKeyValueHandler.java | 8 +-
.../TestKeyValueHandlerWithUnhealthyContainer.java | 18 +-
.../container/keyvalue/TestTarContainerPacker.java | 2 +-
.../container/keyvalue/helpers/TestChunkUtils.java | 18 +-
.../keyvalue/impl/CommonChunkManagerTestCases.java | 35 +-
.../keyvalue/impl/TestBlockManagerImpl.java | 2 +-
.../keyvalue/impl/TestFilePerChunkStrategy.java | 3 +-
.../container/ozoneimpl/TestContainerReader.java | 12 +-
.../container/ozoneimpl/TestOzoneContainer.java | 19 +-
.../replication/TestReplicationSupervisor.java | 2 +-
.../replication/TestSimpleContainerDownloader.java | 4 +-
.../stream/TestDirstreamClientHandler.java | 139 +++
.../container/stream/TestStreamingServer.java | 175 ++++
.../ozone/container/stream/package-info.java | 20 +
hadoop-hdds/dev-support/checkstyle/checkstyle.xml | 3 +
.../docs/content/design/namespace-support.md | 13 +-
.../docs/content/feature/PrefixFSO-Delete.png | Bin 0 -> 253792 bytes
.../docs/content/feature/PrefixFSO-Format.png | Bin 0 -> 270550 bytes
.../docs/content/feature/PrefixFSO-Rename.png | Bin 0 -> 227851 bytes
hadoop-hdds/docs/content/feature/PrefixFSO.md | 75 ++
hadoop-hdds/docs/content/feature/SCM-HA.md | 68 +-
hadoop-hdds/docs/content/feature/scm-secure-ha.png | Bin 0 -> 46757 bytes
hadoop-hdds/docs/content/interface/O3fs.md | 12 +-
hadoop-hdds/docs/content/interface/O3fs.zh.md | 16 +-
hadoop-hdds/docs/content/interface/Ofs.md | 4 +-
hadoop-hdds/docs/content/interface/ReconApi.md | 390 ++++----
hadoop-hdds/docs/content/interface/ReconApi.zh.md | 356 +++----
hadoop-hdds/docs/content/interface/S3.md | 21 +
hadoop-hdds/docs/content/recipe/SparkOzoneFSK8S.md | 6 +-
.../docs/content/recipe/SparkOzoneFSK8S.zh.md | 6 +-
hadoop-hdds/docs/pom.xml | 10 +-
hadoop-hdds/framework/pom.xml | 40 +-
.../java/org/apache/hadoop/hdds/NodeDetails.java | 5 +-
.../hadoop/hdds/protocol/SCMSecurityProtocol.java | 11 +
.../SCMSecurityProtocolClientSideTranslatorPB.java | 14 +
.../scm/protocol/ScmBlockLocationProtocol.java | 25 +-
.../hadoop/hdds/scm/protocolPB/OzonePBHelper.java | 82 ++
...lockLocationProtocolClientSideTranslatorPB.java | 53 +-
...inerLocationProtocolClientSideTranslatorPB.java | 138 ++-
.../SCMBlockLocationFailoverProxyProvider.java | 152 +--
.../hadoop/hdds/scm/proxy/SCMClientConfig.java | 27 +-
.../SCMContainerLocationFailoverProxyProvider.java | 185 ++--
.../SCMSecurityProtocolFailoverProxyProvider.java | 99 +-
.../scm/update/client/CRLClientUpdateHandler.java | 201 ++++
.../hadoop/hdds/scm/update/client/CRLStore.java | 34 +
.../hdds/scm/update/client/ClientCRLStore.java | 98 ++
.../scm/update/client/ClientUpdateHandler.java | 29 +
.../client/SCMUpdateClientConfiguration.java | 53 +
.../update/client/SCMUpdateServiceGrpcClient.java | 218 +++++
.../scm/update/client/UpdateServiceConfig.java | 44 +
.../hdds/scm/update/client/package-info.java | 22 +
.../hdds/scm/update/server/CRLClientInfo.java | 56 ++
.../scm/update/server/SCMUpdateClientInfo.java | 67 ++
.../hdds/scm/update/server/package-info.java | 22 +
.../hadoop/hdds/security/OzoneSecretKey.java | 176 ++++
.../hadoop/hdds/security/OzoneSecretManager.java | 258 +++++
.../hdds/security/OzoneSecurityException.java | 105 ++
.../apache/hadoop/hdds/security/package-info.java | 22 +
.../hdds/security/token/BlockTokenVerifier.java | 164 +---
.../security/token/CompositeTokenVerifier.java | 47 +
.../security/token/ContainerTokenGenerator.java | 58 ++
.../token/ContainerTokenSecretManager.java | 76 ++
.../security/token/ContainerTokenVerifier.java | 51 +
.../hdds/security/token/NoopTokenVerifier.java | 37 +
.../token/ShortLivedTokenSecretManager.java | 107 ++
.../security/token/ShortLivedTokenVerifier.java | 135 +++
.../hadoop/hdds/security/token/TokenVerifier.java | 64 +-
.../certificate/authority/CertificateServer.java | 4 +-
.../certificate/authority/CertificateStore.java | 6 +
.../certificate/authority/DefaultCAServer.java | 3 +-
.../certificate/authority/DefaultCRLApprover.java | 2 +-
.../authority/PKIProfiles/DefaultProfile.java | 2 +-
.../x509/certificate/client/CertificateClient.java | 41 +-
.../client/DefaultCertificateClient.java | 71 +-
.../certificate/client/OMCertificateClient.java | 11 +-
.../security/x509/certificate/utils/CRLCodec.java | 206 ----
.../hadoop/hdds/security/x509/crl/CRLCodec.java | 206 ++++
.../hadoop/hdds/security/x509/crl/CRLInfo.java | 64 +-
.../hdds/security/x509/crl/CRLInfoCodec.java | 57 ++
.../hadoop/hdds/server/events/EventExecutor.java | 5 +
.../hadoop/hdds/server/events/EventQueue.java | 44 +-
.../server/events/FixedThreadPoolExecutor.java | 134 +++
.../hdds/server/events/SingleThreadExecutor.java | 9 +
.../hadoop/hdds/server/http/BaseHttpServer.java | 2 +
.../hadoop/hdds/server/http/ProfileServlet.java | 2 +-
.../hadoop/hdds/utils/DBCheckpointServlet.java | 30 +-
.../java/org/apache/hadoop/hdds/utils/HAUtils.java | 178 ++--
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 127 +--
.../org/apache/hadoop/hdds/utils/LevelDBStore.java | 400 --------
.../hadoop/hdds/utils/LevelDBStoreIterator.java | 64 --
.../apache/hadoop/hdds/utils/MetadataStore.java | 233 -----
.../hadoop/hdds/utils/MetadataStoreBuilder.java | 151 ---
.../org/apache/hadoop/hdds/utils/RocksDBStore.java | 397 --------
.../hadoop/hdds/utils/RocksDBStoreIterator.java | 66 --
.../apache/hadoop/hdds/utils/db/ShortCodec.java | 43 +
.../hadoop/hdds/utils/db/cache/TableCache.java | 3 +
.../hdds/scm/protocol/TestReplicationConfig.java | 66 --
.../security/token/TestBlockTokenVerifier.java | 75 ++
.../security/token/TestContainerTokenVerifier.java | 75 ++
.../token/TestOzoneBlockTokenIdentifier.java | 4 +-
.../hdds/security/token/TokenVerifierTests.java | 242 +++++
.../x509/certificate/authority/MockCAStore.java | 12 +
.../certificate/authority/TestDefaultCAServer.java | 8 +-
.../client/TestCertificateClientInit.java | 2 +-
.../client/TestDefaultCertificateClient.java | 4 +-
.../x509/certificate/utils/TestCRLCodec.java | 1 +
.../security/x509/keys/TestHDDSKeyGenerator.java | 2 +-
.../hdds/security/x509/keys/TestKeyCodec.java | 2 +-
.../hadoop/hdds/server/events/TestEventQueue.java | 62 +-
.../hadoop/hdds/utils/TestMetadataStore.java | 592 -----------
.../hadoop/hdds/utils/TestRocksDBStoreMBean.java | 230 -----
.../hdds/utils/db/TestTypedRDBTableStore.java | 2 +-
.../hadoop/hdds/utils/db/cache/TestTableCache.java | 2 +-
hadoop-hdds/hadoop-dependency-client/pom.xml | 14 +-
hadoop-hdds/hadoop-dependency-server/pom.xml | 14 +-
hadoop-hdds/hadoop-dependency-test/pom.xml | 19 +-
hadoop-hdds/interface-admin/pom.xml | 16 +-
.../src/main/proto/ScmAdminProtocol.proto | 69 +-
hadoop-hdds/interface-client/pom.xml | 28 +-
.../interface-client/src/main/proto/hdds.proto | 19 +-
.../dev-support/findbugsExcludeFile.xml | 3 +
hadoop-hdds/interface-server/pom.xml | 72 +-
.../src/main/proto/InterSCMProtocol.proto | 0
.../src/main/proto/SCMRatisProtocol.proto | 2 +-
.../src/main/proto/SCMUpdateProtocol.proto | 98 ++
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 18 +
.../src/main/proto/ScmServerProtocol.proto | 9 +-
.../src/main/proto/ScmServerSecurityProtocol.proto | 29 +
hadoop-hdds/pom.xml | 82 +-
hadoop-hdds/server-scm/pom.xml | 84 +-
.../hadoop/hdds/scm/SCMCommonPlacementPolicy.java | 43 +-
.../java/org/apache/hadoop/hdds/scm/ScmUtils.java | 51 +-
.../apache/hadoop/hdds/scm/block/BlockManager.java | 9 +-
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 102 +-
.../block/DatanodeDeletedBlockTransactions.java | 6 -
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 2 +-
.../hdds/scm/block/DeletedBlockLogImplV2.java | 3 +-
.../scm/block/DeletedBlockLogStateManager.java | 4 +
.../scm/block/DeletedBlockLogStateManagerImpl.java | 30 +-
.../hdds/scm/block/PendingDeleteHandler.java | 41 -
.../hdds/scm/block/PendingDeleteStatusList.java | 85 --
.../hdds/scm/block/SCMBlockDeletingService.java | 26 +-
.../scm/container/CloseContainerEventHandler.java | 9 +
.../hdds/scm/container/ContainerManagerImpl.java | 18 +-
.../hdds/scm/container/ContainerManagerV2.java | 9 +-
.../hdds/scm/container/ContainerReportHandler.java | 89 +-
.../hdds/scm/container/ContainerStateManager.java | 31 +-
.../scm/container/ContainerStateManagerImpl.java | 13 +-
.../IncrementalContainerReportHandler.java | 54 +-
.../hdds/scm/container/ReplicationManager.java | 264 ++---
.../hdds/scm/container/SCMContainerManager.java | 6 +-
.../scm/container/balancer/ContainerBalancer.java | 406 +++++++-
.../balancer/ContainerBalancerConfiguration.java | 80 +-
.../balancer/ContainerBalancerMetrics.java | 130 +++
.../scm/container/states/ContainerStateMap.java | 10 +-
.../hdds/scm/crl/CRLStatusReportHandler.java | 87 ++
.../apache/hadoop/hdds/scm/crl/package-info.java | 22 +
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 19 +-
.../apache/hadoop/hdds/scm/ha/HASecurityUtils.java | 9 +-
.../hadoop/hdds/scm/ha/InterSCMGrpcClient.java | 30 +-
.../hdds/scm/ha/InterSCMGrpcProtocolService.java | 23 +
.../hdds/scm/ha/MockSCMHADBTransactionBuffer.java | 4 +
.../hadoop/hdds/scm/ha/MockSCMHAManager.java | 16 +-
.../org/apache/hadoop/hdds/scm/ha/RatisUtil.java | 40 +
.../org/apache/hadoop/hdds/scm/ha/SCMContext.java | 76 +-
.../hdds/scm/ha/SCMDBCheckpointProvider.java | 4 +-
.../apache/hadoop/hdds/scm/ha/SCMHAManager.java | 29 +-
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 207 ++--
.../hadoop/hdds/scm/ha/SCMHANodeDetails.java | 15 +-
.../apache/hadoop/hdds/scm/ha/SCMNodeDetails.java | 4 +
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 91 +-
.../hadoop/hdds/scm/ha/SCMSnapshotDownloader.java | 5 +-
.../hadoop/hdds/scm/ha/SCMSnapshotProvider.java | 37 +-
.../apache/hadoop/hdds/scm/ha/SCMStateMachine.java | 182 +++-
.../hadoop/hdds/scm/ha/SequenceIdGenerator.java | 29 +-
.../apache/hadoop/hdds/scm/ha/io/BooleanCodec.java | 3 +
.../org/apache/hadoop/hdds/scm/ha/io/Codec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/CodecFactory.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/EnumCodec.java | 3 +
.../hdds/scm/ha/io/GeneratedMessageCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/ListCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/LongCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/StringCodec.java | 3 +
.../hadoop/hdds/scm/metadata/CRLInfoCodec.java | 58 --
.../hadoop/hdds/scm/metadata/SCMDBDefinition.java | 1 +
.../apache/hadoop/hdds/scm/node/DatanodeInfo.java | 14 +
.../hadoop/hdds/scm/node/DatanodeUsageInfo.java | 3 +
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 3 +
.../hdds/scm/node/NodeDecommissionManager.java | 26 +-
.../apache/hadoop/hdds/scm/node/NodeStatus.java | 2 +-
.../hadoop/hdds/scm/node/SCMNodeManager.java | 62 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 41 +-
.../scm/pipeline/BackgroundPipelineCreatorV2.java | 64 +-
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 24 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 40 +-
.../hdds/scm/pipeline/PipelineManagerV2Impl.java | 74 +-
.../hdds/scm/pipeline/PipelinePlacementPolicy.java | 56 +-
.../hadoop/hdds/scm/pipeline/PipelineProvider.java | 29 +-
.../hdds/scm/pipeline/PipelineReportHandler.java | 29 +-
.../hdds/scm/pipeline/PipelineStateManager.java | 34 +-
.../scm/pipeline/PipelineStateManagerV2Impl.java | 45 +-
.../hadoop/hdds/scm/pipeline/PipelineStateMap.java | 163 +---
.../hdds/scm/pipeline/RatisPipelineProvider.java | 54 +-
.../hdds/scm/pipeline/RatisPipelineUtils.java | 14 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 80 +-
.../hdds/scm/pipeline/SCMPipelineMetrics.java | 4 +-
.../hdds/scm/pipeline/SimplePipelineProvider.java | 27 +-
.../hadoop/hdds/scm/pipeline/StateManager.java | 42 +-
.../scm/pipeline/WritableContainerFactory.java | 58 ++
.../scm/pipeline/WritableContainerProvider.java | 57 ++
.../pipeline/WritableRatisContainerProvider.java | 158 +++
.../SCMSecurityProtocolServerSideTranslatorPB.java | 33 +-
...lockLocationProtocolServerSideTranslatorPB.java | 30 +-
...inerLocationProtocolServerSideTranslatorPB.java | 128 ++-
.../hdds/scm/safemode/ContainerSafeModeRule.java | 56 +-
.../hdds/scm/safemode/DataNodeSafeModeRule.java | 8 +
.../scm/safemode/HealthyPipelineSafeModeRule.java | 89 +-
.../safemode/OneReplicaPipelineSafeModeRule.java | 86 +-
.../hdds/scm/safemode/SCMSafeModeManager.java | 41 +-
.../hadoop/hdds/scm/safemode/SafeModeExitRule.java | 9 +
.../scm/server/OzoneStorageContainerManager.java | 3 +
.../hdds/scm/server/SCMBlockProtocolServer.java | 17 +-
.../hadoop/hdds/scm/server/SCMCertStore.java | 24 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 159 ++-
.../hadoop/hdds/scm/server/SCMConfigurator.java | 20 +
.../hdds/scm/server/SCMDBCheckpointServlet.java | 5 +-
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 15 +-
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 7 +-
.../apache/hadoop/hdds/scm/server/SCMMXBean.java | 10 +
.../hdds/scm/server/SCMSecurityProtocolServer.java | 56 +-
.../hadoop/hdds/scm/server/SCMStorageConfig.java | 19 +-
.../hdds/scm/server/StorageContainerManager.java | 340 +++++--
.../hadoop/hdds/scm/update/server/SCMCRLStore.java | 51 +
.../scm/update/server/SCMCRLUpdateHandler.java | 151 +++
.../scm/update/server/SCMUpdateClientManager.java | 149 +++
.../hdds/scm/update/server/SCMUpdateHandler.java | 53 +
.../update/server/SCMUpdateServiceGrpcServer.java | 99 ++
.../scm/update/server/SCMUpdateServiceImpl.java | 119 +++
.../hdds/scm/update/server/package-info.java | 22 +
.../main/resources/webapps/scm/scm-overview.html | 8 +
.../apache/hadoop/hdds/scm/TestHddsServerUtil.java | 31 +-
.../hadoop/hdds/scm/TestHddsServerUtils.java | 21 +-
.../scm/TestStorageContainerManagerHttpServer.java | 4 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 138 ++-
.../hadoop/hdds/scm/block/TestBlockManager.java | 99 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 113 ++-
.../command/TestCommandStatusReportHandler.java | 2 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 115 ++-
.../container/TestCloseContainerEventHandler.java | 12 +-
.../scm/container/TestContainerManagerImpl.java | 20 +-
.../scm/container/TestContainerReportHandler.java | 8 +-
.../scm/container/TestContainerStateManager.java | 17 +-
.../TestIncrementalContainerReportHandler.java | 98 +-
.../hdds/scm/container/TestReplicationManager.java | 51 +-
.../scm/container/TestSCMContainerManager.java | 2 +-
.../container/balancer/TestContainerBalancer.java | 236 +++++
.../algorithms/TestContainerPlacementFactory.java | 62 +-
.../TestSCMContainerPlacementCapacity.java | 53 +-
.../TestSCMContainerPlacementRackAware.java | 127 ++-
.../TestSCMContainerPlacementRandom.java | 51 +-
.../hdds/scm/crl/TestCRLStatusReportHandler.java | 137 +++
.../apache/hadoop/hdds/scm/ha/TestSCMContext.java | 9 +-
.../hadoop/hdds/scm/ha/TestSCMHAConfiguration.java | 79 ++
.../hadoop/hdds/scm/ha/TestSCMServiceManager.java | 7 +-
.../hdds/scm/ha/TestSequenceIDGenerator.java | 3 +
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 64 +-
.../hdds/scm/node/TestNodeDecommissionManager.java | 36 +-
.../hdds/scm/node/TestNodeReportHandler.java | 23 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 45 +-
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 13 +-
.../hadoop/hdds/scm/node/TestStatisticsUpdate.java | 16 +-
.../hdds/scm/pipeline/MockPipelineManager.java | 53 +-
.../scm/pipeline/MockRatisPipelineProvider.java | 21 +-
.../TestPipelineDatanodesIntersection.java | 13 +-
.../hdds/scm/pipeline/TestPipelineManagerImpl.java | 90 +-
.../scm/pipeline/TestPipelinePlacementPolicy.java | 66 +-
.../scm/pipeline/TestPipelineStateManager.java | 101 +-
.../scm/pipeline/TestRatisPipelineProvider.java | 104 +-
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 238 +++--
.../scm/pipeline/TestSimplePipelineProvider.java | 28 +-
.../safemode/TestHealthyPipelineSafeModeRule.java | 34 +-
.../TestOneReplicaPipelineSafeModeRule.java | 17 +-
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 55 +-
.../scm/server/TestSCMBlockProtocolServer.java | 2 +-
.../hadoop/hdds/scm/server/TestSCMCertStore.java | 6 +-
.../scm/server/TestSCMSecurityProtocolServer.java | 4 +-
.../hdds/scm/update/server/MockCRLStore.java | 138 +++
.../server/TestSCMUpdateServiceGrpcServer.java | 301 ++++++
.../ozone/container/common/TestEndPoint.java | 34 +-
hadoop-hdds/test-utils/pom.xml | 10 +-
.../org/apache/hadoop/test/GenericTestUtils.java | 365 -------
.../org/apache/hadoop/test/LambdaTestUtils.java | 814 ----------------
.../apache/hadoop/test/TimedOutTestsListener.java | 183 ----
.../org/apache/ozone/test/GenericTestUtils.java | 365 +++++++
.../org/apache/ozone/test/LambdaTestUtils.java | 814 ++++++++++++++++
.../apache/ozone/test/TimedOutTestsListener.java | 183 ++++
.../java/org/apache/ozone/test/package-info.java | 22 +
hadoop-hdds/tools/pom.xml | 42 +-
.../hdds/scm/cli/ContainerBalancerCommands.java | 108 +++
.../scm/cli/ContainerBalancerStartSubcommand.java | 66 ++
.../scm/cli/ContainerBalancerStatusSubcommand.java | 45 +
.../scm/cli/ContainerBalancerStopSubcommand.java | 40 +
.../hdds/scm/cli/ContainerOperationClient.java | 90 +-
.../org/apache/hadoop/hdds/scm/cli/ScmOption.java | 14 +-
.../hdds/scm/cli/datanode/ListInfoSubcommand.java | 6 +-
.../scm/cli/pipeline/CreatePipelineSubcommand.java | 3 +-
.../scm/cli/pipeline/ListPipelinesSubcommand.java | 4 +-
.../datanode/TestContainerBalancerSubCommand.java | 141 +++
.../scm/cli/datanode/TestListInfoSubcommand.java | 4 +-
hadoop-ozone/client/pom.xml | 18 +-
.../apache/hadoop/ozone/client/ObjectStore.java | 4 +
.../apache/hadoop/ozone/client/OzoneBucket.java | 434 ++++++++-
.../org/apache/hadoop/ozone/client/OzoneKey.java | 41 +-
.../hadoop/ozone/client/OzoneKeyDetails.java | 20 +
.../hadoop/ozone/client/OzoneKeyLocation.java | 15 +-
.../hadoop/ozone/client/OzoneMultipartUpload.java | 34 +-
.../client/OzoneMultipartUploadPartListParts.java | 34 +-
.../apache/hadoop/ozone/client/OzoneVolume.java | 57 +-
.../client/io/BlockOutputStreamEntryPool.java | 9 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 51 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 43 +-
.../client/io/MultipartCryptoKeyInputStream.java | 3 +
.../hadoop/ozone/client/io/OzoneInputStream.java | 15 +-
.../ozone/client/protocol/ClientProtocol.java | 70 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 211 +++-
.../hadoop/ozone/client/MockBlockAllocator.java | 33 +
.../hadoop/ozone/client/MockDatanodeStorage.java | 73 ++
.../hadoop/ozone/client/MockOmTransport.java | 288 ++++++
.../ozone/client/MockXceiverClientFactory.java | 72 ++
.../hadoop/ozone/client/MockXceiverClientSpi.java | 180 ++++
.../ozone/client/SinglePipelineBlockAllocator.java | 90 ++
.../hadoop/ozone/client/TestHddsClientUtils.java | 150 +--
.../hadoop/ozone/client/TestOzoneClient.java | 194 ++++
hadoop-ozone/common/pom.xml | 30 +-
.../main/java/org/apache/hadoop/ozone/OFSPath.java | 3 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 2 +
.../org/apache/hadoop/ozone/audit/OMAction.java | 3 +-
.../hadoop/ozone/client/io/LengthInputStream.java | 17 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 25 +
.../hadoop/ozone/om/exceptions/OMException.java | 4 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 39 +-
.../hadoop/ozone/om/helpers/OmDirectoryInfo.java | 264 +++++
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 76 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 243 +++--
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 6 +-
.../ozone/om/helpers/OmMultipartKeyInfo.java | 110 ++-
.../hadoop/ozone/om/helpers/OmMultipartUpload.java | 32 +-
.../om/helpers/OmMultipartUploadListParts.java | 25 +-
.../hadoop/ozone/om/helpers/OzoneAclUtil.java | 33 +-
.../hadoop/ozone/om/helpers/OzoneFSUtils.java | 122 +++
.../ozone/om/helpers/WithParentObjectId.java | 55 ++
.../ozone/om/protocol/OzoneManagerProtocol.java | 8 +
...OzoneManagerProtocolClientSideTranslatorPB.java | 89 +-
.../ozone/security/OzoneSecurityException.java | 105 --
.../ozone/security/acl/IAccessAuthorizer.java | 1 -
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 2 +
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 20 +-
.../hadoop/ozone/security/acl/OzonePrefixPath.java | 67 ++
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 3 +-
.../main/resources/ozone-version-info.properties | 3 +-
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 2 +-
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 17 +-
.../ozone/om/helpers/TestOmMultipartKeyInfo.java | 5 +-
hadoop-ozone/csi/pom.xml | 26 +-
hadoop-ozone/datanode/pom.xml | 30 +-
hadoop-ozone/dev-support/checks/_lib.sh | 36 +
hadoop-ozone/dev-support/checks/build.sh | 2 +-
hadoop-ozone/dev-support/checks/coverage.sh | 3 +-
hadoop-ozone/dev-support/checks/dependency.sh | 50 +
hadoop-ozone/dev-support/checks/docs.sh | 40 +
hadoop-ozone/dev-support/checks/findbugs.sh | 12 +-
hadoop-ozone/dev-support/checks/integration.sh | 2 +-
hadoop-ozone/dev-support/checks/junit.sh | 2 +-
hadoop-ozone/dev-support/checks/unit.sh | 2 +-
hadoop-ozone/dev-support/docker/Dockerfile | 2 +-
.../intellij/runConfigurations/CsiServer.xml | 4 +-
.../intellij/runConfigurations/Datanode1.xml | 2 +-
.../intellij/runConfigurations/Datanode2.xml | 2 +-
.../intellij/runConfigurations/Datanode3.xml | 2 +-
.../intellij/runConfigurations/FreonStandalone.xml | 4 +-
.../intellij/runConfigurations/OzoneFsShell.xml | 2 +-
.../intellij/runConfigurations/OzoneManager.xml | 2 +-
.../runConfigurations/OzoneManagerInit.xml | 2 +-
.../intellij/runConfigurations/OzoneShell.xml | 2 +-
.../intellij/runConfigurations/Recon.xml | 4 +-
.../intellij/runConfigurations/S3Gateway.xml | 4 +-
.../runConfigurations/StorageContainerManager.xml | 2 +-
.../StorageContainerManagerInit.xml | 2 +-
.../dist/dev-support/bin/dist-layout-stitching | 6 +
.../dist/dev-support/bin/dist-tar-stitching | 6 +-
hadoop-ozone/dist/pom.xml | 79 +-
.../dist/src/main/assemblies/ozone-src.xml | 3 +
.../docker-image/docker-krb5/Dockerfile-krb5 | 35 -
.../common/docker-image/docker-krb5/README.md | 34 -
.../common/docker-image/docker-krb5/kadm5.acl | 20 -
.../common/docker-image/docker-krb5/krb5.conf | 41 -
.../common/docker-image/docker-krb5/launcher.sh | 25 -
.../src/main/compose/compatibility/docker-config | 2 +
.../dist/src/main/compose/ozone-csi/docker-config | 2 +
.../src/main/compose/ozone-ha/docker-compose.yaml | 10 +-
.../dist/src/main/compose/ozone-ha/docker-config | 3 +
.../dist/src/main/compose/ozone-mr/common-config | 1 +
.../main/compose/ozone-mr/hadoop27/docker-config | 4 +-
.../main/compose/ozone-mr/hadoop31/docker-config | 4 +-
.../main/compose/ozone-mr/hadoop32/docker-config | 4 +-
.../dist/src/main/compose/ozone-mr/test.sh | 6 +-
.../src/main/compose/ozone-om-ha/docker-config | 2 +
.../src/main/compose/ozone-topology/docker-config | 2 +
.../src/main/compose/ozone/docker-compose.yaml | 10 +
.../dist/src/main/compose/ozone/docker-config | 2 +
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 29 +-
.../src/main/compose/ozoneblockade/docker-config | 1 +
.../src/main/compose/ozones3-haproxy/docker-config | 2 +
.../src/main/compose/ozonescripts/docker-config | 1 +
.../dist/src/main/compose/ozonesecure-ha/.env | 1 +
.../compose/ozonesecure-ha/docker-compose.yaml | 84 +-
.../src/main/compose/ozonesecure-ha/docker-config | 30 +-
.../compose/ozonesecure-ha/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../main/compose/ozonesecure-ha/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure-ha/keytabs/om.keytab | Bin 0 -> 278 bytes
.../compose/ozonesecure-ha/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure-ha/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure-ha/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure-ha/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../ozonesecure-ha/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure-ha/krb5.conf | 41 +
.../dist/src/main/compose/ozonesecure-ha/test.sh | 5 +
.../dist/src/main/compose/ozonesecure-mr/.env | 1 +
.../dist/src/main/compose/ozonesecure-mr/README.md | 2 +-
.../compose/ozonesecure-mr/docker-compose.yaml | 30 +-
.../src/main/compose/ozonesecure-mr/docker-config | 29 +-
.../compose/ozonesecure-mr/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../main/compose/ozonesecure-mr/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure-mr/keytabs/om.keytab | Bin 0 -> 278 bytes
.../compose/ozonesecure-mr/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure-mr/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure-mr/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure-mr/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../ozonesecure-mr/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure-mr/krb5.conf | 39 +
.../dist/src/main/compose/ozonesecure/.env | 1 +
.../main/compose/ozonesecure/docker-compose.yaml | 24 +-
.../src/main/compose/ozonesecure/docker-config | 26 +-
.../main/compose/ozonesecure/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../src/main/compose/ozonesecure/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../src/main/compose/ozonesecure/keytabs/om.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../compose/ozonesecure/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure/krb5.conf | 41 +
.../dist/src/main/compose/ozonesecure/test.sh | 51 +-
.../dist/src/main/compose/restart/docker-config | 2 +
hadoop-ozone/dist/src/main/compose/test-all.sh | 8 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 25 +-
.../dist/src/main/compose/upgrade/docker-config | 2 +
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 4 +-
.../dist/src/main/compose/xcompat/docker-config | 2 +
.../definitions/ozone/definitions/persistence.yaml | 4 +-
.../examples/ozone-ha/datanode-statefulset.yaml | 2 +-
.../k8s/examples/ozone/datanode-statefulset.yaml | 2 +-
hadoop-ozone/dist/src/main/keytabs/HTTP.keytab | Bin 0 -> 432 bytes
hadoop-ozone/dist/src/main/keytabs/dn.keytab | Bin 0 -> 278 bytes
hadoop-ozone/dist/src/main/keytabs/hadoop.keytab | Bin 0 -> 146 bytes
hadoop-ozone/dist/src/main/keytabs/jhs.keytab | Bin 0 -> 142 bytes
hadoop-ozone/dist/src/main/keytabs/nm.keytab | Bin 0 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/om.keytab | Bin 0 -> 278 bytes
hadoop-ozone/dist/src/main/keytabs/recon.keytab | Bin 0 -> 296 bytes
hadoop-ozone/dist/src/main/keytabs/rm.keytab | Bin 0 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/s3g.keytab | Bin 0 -> 434 bytes
hadoop-ozone/dist/src/main/keytabs/scm.keytab | Bin 0 -> 586 bytes
hadoop-ozone/dist/src/main/keytabs/testuser.keytab | Bin 0 -> 450 bytes
.../dist/src/main/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/keytabs/update-keytabs.sh | 77 ++
hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 13 +-
hadoop-ozone/dist/src/main/license/bin/NOTICE.txt | 2 +-
.../src/main/license/bin/licenses/LICENSE-CDDL.txt | 133 +++
hadoop-ozone/dist/src/main/license/jar-report.txt | 236 +++++
.../dist/src/main/license/update-jar-report.sh | 33 +
.../src/main/smoketest/admincli/container.robot | 25 +-
.../src/main/smoketest/admincli/datanode.robot | 6 +-
.../src/main/smoketest/admincli/pipeline.robot | 14 +-
.../smoketest/admincli/replicationmanager.robot | 7 +-
.../src/main/smoketest/admincli/safemode.robot | 6 +-
.../dist/src/main/smoketest/admincli/scmha.robot | 2 +-
.../dist/src/main/smoketest/basic/links.robot | 10 +-
.../src/main/smoketest/basic/ozone-shell-lib.robot | 6 +-
.../dist/src/main/smoketest/cli/classpath.robot | 14 +-
.../dist/src/main/smoketest/cli/envvars.robot | 15 +
.../dist/src/main/smoketest/commonlib.robot | 19 +-
.../dist/src/main/smoketest/freon/remove.robot | 43 +
.../src/main/smoketest/ozonefs/hadoopo3fs.robot | 2 +
.../dist/src/main/smoketest/ozonefs/setup.robot | 10 +-
.../main/smoketest/security/ozone-secure-fs.robot | 18 +-
.../dist/src/main/smoketest/spnego/web.robot | 2 +-
hadoop-ozone/dist/src/shell/ozone/ozone | 49 +-
.../dist/src/shell/ozone/ozone-functions.sh | 3 +-
.../dist/src/test/shell/compose_testlib.bats | 13 +
hadoop-ozone/dist/src/test/shell/failing1/test.sh | 17 +
.../fault-injection-test/mini-chaos-tests/pom.xml | 28 +-
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 5 +-
.../fault-injection-test/network-tests/pom.xml | 8 +-
.../network-tests/src/test/compose/docker-config | 1 +
hadoop-ozone/fault-injection-test/pom.xml | 10 +-
hadoop-ozone/insight/pom.xml | 54 +-
.../hadoop/ozone/insight/BaseInsightPoint.java | 4 +-
hadoop-ozone/integration-test/pom.xml | 77 +-
.../ozone/TestDirectoryDeletingServiceWithFSO.java | 402 ++++++++
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 18 +-
.../fs/ozone/TestOzoneFileInterfacesWithFSO.java | 96 ++
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 511 +++++++++-
.../fs/ozone/TestOzoneFileSystemMissingParent.java | 6 +-
.../fs/ozone/TestOzoneFileSystemPrefixParser.java | 168 ++++
.../fs/ozone/TestOzoneFileSystemWithFSO.java | 479 +++++++++
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 2 +-
.../hadoop/fs/ozone/TestRootedOzoneFileSystem.java | 234 ++++-
.../fs/ozone/TestRootedOzoneFileSystemWithFSO.java | 160 +++
.../ozone/contract/ITestOzoneContractCreate.java | 26 +-
.../ozone/contract/ITestOzoneContractDelete.java | 26 +-
.../contract/ITestOzoneContractGetFileStatus.java | 27 +-
.../fs/ozone/contract/ITestOzoneContractMkdir.java | 26 +-
.../fs/ozone/contract/ITestOzoneContractOpen.java | 27 +-
.../ozone/contract/ITestOzoneContractRename.java | 26 +-
.../ozone/contract/ITestOzoneContractRootDir.java | 26 +-
.../fs/ozone/contract/ITestOzoneContractSeek.java | 1 +
.../fs/ozone/contract/ITestOzoneContractUtils.java | 61 ++
.../hadoop/fs/ozone/contract/OzoneContract.java | 13 +
.../hadoop/hdds/scm/TestRatisPipelineLeader.java | 13 +-
.../hdds/scm/TestSCMDbCheckpointServlet.java | 9 +-
.../hadoop/hdds/scm/TestSCMInstallSnapshot.java | 25 +-
.../apache/hadoop/hdds/scm/TestSCMSnapshot.java | 10 +-
.../TestContainerStateManagerIntegration.java | 7 +-
.../metrics/TestSCMContainerManagerMetrics.java | 11 +-
.../hdds/scm/pipeline/TestLeaderChoosePolicy.java | 29 +-
.../hdds/scm/pipeline/TestNode2PipelineMap.java | 9 +-
.../hadoop/hdds/scm/pipeline/TestNodeFailure.java | 9 +-
.../hdds/scm/pipeline/TestPipelineClose.java | 12 +-
.../TestRatisPipelineCreateAndDestroy.java | 32 +-
.../hadoop/hdds/scm/pipeline/TestSCMRestart.java | 16 +-
.../safemode/TestSCMSafeModeWithPipelineRules.java | 17 +-
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 2 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 28 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 12 +-
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 2 +-
.../org/apache/hadoop/ozone/RatisTestHelper.java | 33 +
.../ozone/TestContainerBalancerOperations.java | 112 +++
.../hadoop/ozone/TestContainerOperations.java | 21 +-
.../apache/hadoop/ozone/TestDelegationToken.java | 6 +-
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 14 +-
.../hadoop/ozone/TestMiniOzoneOMHACluster.java | 2 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 8 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 84 +-
.../hadoop/ozone/TestStorageContainerManager.java | 49 +-
.../ozone/client/CertificateClientTestImpl.java | 14 +
.../ozone/client/rpc/Test2WayCommitInRatis.java | 5 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 9 +-
.../rpc/TestCloseContainerHandlingByClient.java | 39 +-
.../rpc/TestContainerReplicationEndToEnd.java | 2 +-
.../client/rpc/TestContainerStateMachine.java | 2 +-
.../TestContainerStateMachineFailureOnRead.java | 15 +-
.../rpc/TestContainerStateMachineFailures.java | 2 +-
.../rpc/TestContainerStateMachineFlushDelay.java | 2 +-
.../client/rpc/TestDeleteWithSlowFollower.java | 25 +-
.../client/rpc/TestFailureHandlingByClient.java | 28 +-
.../rpc/TestFailureHandlingByClientFlushDelay.java | 7 +-
.../client/rpc/TestHybridPipelineOnDatanode.java | 2 +-
.../rpc/TestMultiBlockWritesWithDnFailures.java | 13 +-
.../client/rpc/TestOzoneAtRestEncryption.java | 31 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 1024 ++++++++++++++++++++
.../client/rpc/TestOzoneRpcClientAbstract.java | 140 ++-
.../rpc/TestOzoneRpcClientForAclAuditLog.java | 2 +-
.../hadoop/ozone/client/rpc/TestReadRetries.java | 60 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 25 +-
.../ozone/client/rpc/TestWatchForCommit.java | 5 +-
.../ozone/client/rpc/read/TestInputStreamBase.java | 5 +-
.../ozone/client/rpc/read/TestKeyInputStream.java | 71 +-
.../ozone/container/TestContainerReplication.java | 6 +-
.../apache/hadoop/ozone/container/TestHelper.java | 2 +-
.../commandhandler/TestBlockDeletion.java | 59 +-
.../TestCloseContainerByPipeline.java | 23 +-
.../commandhandler/TestCloseContainerHandler.java | 13 +-
.../commandhandler/TestDeleteContainerHandler.java | 12 +-
.../transport/server/ratis/TestCSMMetrics.java | 2 +-
.../container/metrics/TestContainerMetrics.java | 13 +-
.../ozoneimpl/TestOzoneContainerWithTLS.java | 13 +-
.../ozoneimpl/TestSecureOzoneContainer.java | 153 +--
.../hadoop/ozone/container/package-info.java | 22 +
.../container/server/TestContainerServer.java | 2 +-
.../server/TestSecureContainerServer.java | 242 +++--
.../apache/hadoop/ozone/dn/DatanodeTestUtils.java | 261 +++++
.../ozone/dn/TestDatanodeLayoutUpgradeTool.java | 4 +-
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +-
.../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 4 +-
.../TestDatanodeHddsVolumeFailureDetection.java | 299 ++++++
.../TestDatanodeHddsVolumeFailureToleration.java | 135 +++
.../ozone/freon/TestFreonWithDatanodeRestart.java | 126 ---
.../ozone/freon/TestHadoopDirTreeGenerator.java | 26 +-
.../freon/TestHadoopDirTreeGeneratorWithFSO.java | 36 +
.../ozone/freon/TestHadoopNestedDirGenerator.java | 7 +-
.../ozone/freon/TestOzoneClientKeyGenerator.java | 94 --
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 19 +
.../hadoop/ozone/fsck/TestContainerMapper.java | 7 +-
.../ozone/om/TestContainerReportWithKeys.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 149 ++-
.../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +-
.../hadoop/ozone/om/TestOMDbCheckpointServlet.java | 274 ++++--
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 2 +-
.../hadoop/ozone/om/TestOMStartupWithLayout.java | 197 ++++
.../hadoop/ozone/om/TestObjectStoreWithFSO.java | 722 ++++++++++++++
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 2 +-
.../hadoop/ozone/om/TestOmBlockVersioning.java | 4 +
.../ozone/om/TestOzoneManagerConfiguration.java | 2 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 2 +-
.../ozone/om/TestOzoneManagerHAKeyDeletion.java | 5 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 2 +-
.../ozone/om/TestOzoneManagerRestInterface.java | 3 +-
.../hadoop/ozone/om/TestOzoneManagerRestart.java | 2 +-
.../ozone/om/TestOzoneManagerRocksDBLogging.java | 4 +-
.../hadoop/ozone/om/TestRecursiveAclWithFSO.java | 338 +++++++
.../apache/hadoop/ozone/om/TestScmSafeMode.java | 4 +-
.../hadoop/ozone/om/TestSecureOzoneManager.java | 4 +-
.../ozone/om/parser/TestOMRatisLogParser.java | 2 +-
.../hadoop/ozone/recon/TestReconAsPassiveScm.java | 17 +-
.../apache/hadoop/ozone/recon/TestReconTasks.java | 7 +-
.../ozone/recon/TestReconWithOzoneManager.java | 5 +-
.../ozone/recon/TestReconWithOzoneManagerHA.java | 10 +-
.../hadoop/ozone/scm/TestCloseContainer.java | 2 +-
.../hadoop/ozone/scm/TestFailoverWithSCMHA.java | 147 +++
.../TestSCMContainerPlacementPolicyMetrics.java | 7 +-
.../ozone/scm/TestSCMInstallSnapshotWithHA.java | 137 +--
.../org/apache/hadoop/ozone/scm/TestSCMMXBean.java | 9 +-
.../ozone/scm/TestStorageContainerManagerHA.java | 10 +-
.../hadoop/ozone/scm/TestXceiverClientGrpc.java | 7 +-
.../hadoop/ozone/scm/TestXceiverClientManager.java | 2 +-
.../hadoop/ozone/scm/TestXceiverClientMetrics.java | 2 +-
.../scm/node/TestDecommissionAndMaintenance.java | 2 +-
.../hadoop/ozone/scm/node/TestQueryNode.java | 2 +-
.../TestSCMPipelineBytesWrittenMetrics.java | 2 +-
.../ozone/scm/pipeline/TestSCMPipelineMetrics.java | 11 +-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 104 +-
hadoop-ozone/interface-client/pom.xml | 16 +-
.../src/main/proto/OmClientProtocol.proto | 42 +
.../interface-client/src/main/proto/Security.proto | 1 +
hadoop-ozone/interface-storage/pom.xml | 36 +-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 46 +
.../ozone/om/codec/OmDirectoryInfoCodec.java | 60 ++
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 8 +
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 5 +-
.../om/codec/TestOmMultipartKeyInfoCodec.java | 7 +-
.../ozone/om/codec/TestOmPrefixInfoCodec.java | 2 +-
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 5 +-
.../ozone/om/codec/TestS3SecretValueCodec.java | 2 +-
.../ozone/om/codec/TestTransactionInfoCodec.java | 2 +-
hadoop-ozone/native-client/README.md | 4 +-
hadoop-ozone/ozone-manager/pom.xml | 64 +-
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 7 +-
.../hadoop/ozone/om/DirectoryDeletingService.java | 286 ++++++
.../org/apache/hadoop/ozone/om/KeyManager.java | 56 ++
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 908 +++++++++++++++--
.../hadoop/ozone/om/OMDBCheckpointServlet.java | 27 +-
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 18 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 111 ++-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 245 ++++-
.../hadoop/ozone/om/OzoneManagerStarter.java | 4 -
.../hadoop/ozone/om/OzonePrefixPathImpl.java | 164 ++++
.../hadoop/ozone/om/S3SecretManagerImpl.java | 4 +-
.../java/org/apache/hadoop/ozone/om/ScmClient.java | 11 +
.../hadoop/ozone/om/TrashOzoneFileSystem.java | 146 ++-
.../apache/hadoop/ozone/om/TrashPolicyOzone.java | 40 +-
.../hadoop/ozone/om/codec/OMDBDefinition.java | 37 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 88 ++
.../hadoop/ozone/om/request/OMClientRequest.java | 48 +
.../om/request/bucket/OMBucketCreateRequest.java | 34 +
.../om/request/bucket/OMBucketDeleteRequest.java | 2 +-
.../request/bucket/OMBucketSetPropertyRequest.java | 19 +-
.../om/request/file/OMDirectoryCreateRequest.java | 9 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 314 ++++++
.../ozone/om/request/file/OMFileCreateRequest.java | 55 +-
.../request/file/OMFileCreateRequestWithFSO.java | 261 +++++
.../ozone/om/request/file/OMFileRequest.java | 745 ++++++++++++++
.../om/request/key/OMAllocateBlockRequest.java | 2 +-
.../request/key/OMAllocateBlockRequestWithFSO.java | 226 +++++
.../ozone/om/request/key/OMKeyCommitRequest.java | 36 +-
.../om/request/key/OMKeyCommitRequestWithFSO.java | 190 ++++
.../ozone/om/request/key/OMKeyCreateRequest.java | 24 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 232 +++++
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 205 ++++
.../om/request/key/OMKeyRenameRequestWithFSO.java | 301 ++++++
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 294 +++---
.../ozone/om/request/key/OMKeysDeleteRequest.java | 1 -
.../om/request/key/OMPathsPurgeRequestWithFSO.java | 64 ++
.../ozone/om/request/key/acl/OMKeyAclRequest.java | 2 +-
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 167 ++++
.../request/key/acl/OMKeyAddAclRequestWithFSO.java | 151 +++
.../key/acl/OMKeyRemoveAclRequestWithFSO.java | 161 +++
.../request/key/acl/OMKeySetAclRequestWithFSO.java | 155 +++
.../S3InitiateMultipartUploadRequest.java | 24 +-
.../S3InitiateMultipartUploadRequestWithFSO.java | 251 +++++
.../multipart/S3MultipartUploadAbortRequest.java | 58 +-
.../S3MultipartUploadAbortRequestWithFSO.java | 89 ++
.../S3MultipartUploadCommitPartRequest.java | 78 +-
.../S3MultipartUploadCommitPartRequestWithFSO.java | 87 ++
.../S3MultipartUploadCompleteRequest.java | 404 +++++---
.../S3MultipartUploadCompleteRequestWithFSO.java | 169 ++++
.../om/request/s3/security/S3GetSecretRequest.java | 14 +-
.../request/s3/security/S3RevokeSecretRequest.java | 155 +++
.../file/OMDirectoryCreateResponseWithFSO.java | 103 ++
.../response/file/OMFileCreateResponseWithFSO.java | 94 ++
.../response/key/AbstractOMKeyDeleteResponse.java | 44 +
.../om/response/key/OMAllocateBlockResponse.java | 12 +
.../key/OMAllocateBlockResponseWithFSO.java | 67 ++
.../ozone/om/response/key/OMKeyCommitResponse.java | 15 +
.../response/key/OMKeyCommitResponseWithFSO.java | 75 ++
.../ozone/om/response/key/OMKeyCreateResponse.java | 14 +-
.../response/key/OMKeyCreateResponseWithFSO.java | 56 ++
.../ozone/om/response/key/OMKeyDeleteResponse.java | 8 +
.../response/key/OMKeyDeleteResponseWithFSO.java | 100 ++
.../ozone/om/response/key/OMKeyRenameResponse.java | 11 +
.../response/key/OMKeyRenameResponseWithFSO.java | 77 ++
.../response/key/OMPathsPurgeResponseWithFSO.java | 125 +++
.../om/response/key/acl/OMKeyAclResponse.java | 3 +
.../response/key/acl/OMKeyAclResponseWithFSO.java | 75 ++
.../S3InitiateMultipartUploadResponseWithFSO.java | 90 ++
.../multipart/S3MultipartUploadAbortResponse.java | 9 +-
.../S3MultipartUploadAbortResponseWithFSO.java | 57 ++
.../S3MultipartUploadCommitPartResponse.java | 3 +
...S3MultipartUploadCommitPartResponseWithFSO.java | 69 ++
.../S3MultipartUploadCompleteResponse.java | 41 +-
.../S3MultipartUploadCompleteResponseWithFSO.java | 85 ++
.../s3/security/S3RevokeSecretResponse.java | 57 ++
...OzoneManagerProtocolServerSideTranslatorPB.java | 27 +-
.../protocolPB/OzoneManagerRequestHandler.java | 18 +-
.../security/OzoneBlockTokenSecretManager.java | 86 +-
.../OzoneDelegationTokenSecretManager.java | 1 +
.../hadoop/ozone/security/OzoneSecretKey.java | 176 ----
.../hadoop/ozone/security/OzoneSecretManager.java | 258 -----
.../ozone/security/acl/OzoneNativeAuthorizer.java | 35 +-
.../ozone/om/ScmBlockLocationTestingClient.java | 12 +-
.../hadoop/ozone/om/TestBucketManagerImpl.java | 10 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 30 +-
.../org/apache/hadoop/ozone/om/TestOMStorage.java | 2 +-
.../ozone/om/TestOzoneManagerHttpServer.java | 4 +-
.../apache/hadoop/ozone/om/TestTrashService.java | 4 +
.../hadoop/ozone/om/failover/TestOMFailovers.java | 2 +-
...tOzoneManagerDoubleBufferWithDummyResponse.java | 2 +-
...TestOzoneManagerDoubleBufferWithOMResponse.java | 7 +-
.../om/ratis/TestOzoneManagerRatisServer.java | 2 +-
.../ozone/om/request/TestOMRequestUtils.java | 250 ++++-
.../ozone/om/request/bucket/TestBucketRequest.java | 1 +
.../request/bucket/TestOMBucketCreateRequest.java | 4 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 115 +++
.../bucket/TestOMBucketSetPropertyRequest.java | 12 +-
.../file/TestOMDirectoryCreateRequestWithFSO.java | 657 +++++++++++++
.../om/request/file/TestOMFileCreateRequest.java | 69 +-
.../file/TestOMFileCreateRequestWithFSO.java | 197 ++++
.../om/request/key/TestOMAllocateBlockRequest.java | 44 +-
.../key/TestOMAllocateBlockRequestWithFSO.java | 120 +++
.../ozone/om/request/key/TestOMKeyAclRequest.java | 81 +-
.../om/request/key/TestOMKeyAclRequestWithFSO.java | 81 ++
.../om/request/key/TestOMKeyCommitRequest.java | 142 ++-
.../request/key/TestOMKeyCommitRequestWithFSO.java | 114 +++
.../om/request/key/TestOMKeyCreateRequest.java | 53 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 135 +++
.../om/request/key/TestOMKeyDeleteRequest.java | 40 +-
.../request/key/TestOMKeyDeleteRequestWithFSO.java | 139 +++
.../ozone/om/request/key/TestOMKeyRequest.java | 40 +-
.../TestS3InitiateMultipartUploadRequest.java | 24 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 145 +++
.../s3/multipart/TestS3MultipartRequest.java | 60 +-
.../TestS3MultipartUploadAbortRequest.java | 32 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 69 ++
.../TestS3MultipartUploadCommitPartRequest.java | 70 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 118 +++
.../TestS3MultipartUploadCompleteRequest.java | 118 ++-
...estS3MultipartUploadCompleteRequestWithFSO.java | 139 +++
.../request/volume/TestOMVolumeCreateRequest.java | 4 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 2 +-
.../file/TestOMDirectoryCreateResponseWithFSO.java | 87 ++
.../file/TestOMFileCreateResponseWithFSO.java | 78 ++
.../response/key/TestOMAllocateBlockResponse.java | 37 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 85 ++
.../om/response/key/TestOMKeyCommitResponse.java | 64 +-
.../key/TestOMKeyCommitResponseWithFSO.java | 102 ++
.../om/response/key/TestOMKeyCreateResponse.java | 38 +-
.../key/TestOMKeyCreateResponseWithFSO.java | 76 ++
.../om/response/key/TestOMKeyDeleteResponse.java | 92 +-
.../key/TestOMKeyDeleteResponseWithFSO.java | 84 ++
.../ozone/om/response/key/TestOMKeyResponse.java | 25 +-
.../TestS3InitiateMultipartUploadResponse.java | 2 +-
...stS3InitiateMultipartUploadResponseWithFSO.java | 85 ++
.../s3/multipart/TestS3MultipartResponse.java | 191 +++-
.../TestS3MultipartUploadAbortResponse.java | 23 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 85 ++
...S3MultipartUploadCommitPartResponseWithFSO.java | 226 +++++
...stS3MultipartUploadCompleteResponseWithFSO.java | 279 ++++++
.../security/TestOzoneBlockTokenSecretManager.java | 289 +++---
.../TestOzoneDelegationTokenSecretManager.java | 2 +-
.../ozone/security/TestOzoneManagerBlockToken.java | 2 +-
.../ozone/security/TestOzoneTokenIdentifier.java | 2 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 7 +-
.../hadoop/ozone/security/acl/TestOzoneObj.java | 79 ++
.../hadoop/ozone/security/acl/TestParentAcl.java | 411 ++++++++
.../ozone/security/acl/TestRequestContext.java | 7 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 10 +-
hadoop-ozone/ozonefs-common/pom.xml | 46 +-
.../fs/ozone/BasicOzoneClientAdapterImpl.java | 66 +-
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 37 +
.../ozone/BasicRootedOzoneClientAdapterImpl.java | 60 +-
.../fs/ozone/BasicRootedOzoneFileSystem.java | 37 +
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 5 +-
.../apache/hadoop/fs/ozone/OzoneFSInputStream.java | 30 +-
hadoop-ozone/ozonefs-hadoop2/pom.xml | 30 +-
hadoop-ozone/ozonefs-hadoop3/pom.xml | 28 +-
hadoop-ozone/ozonefs-shaded/pom.xml | 18 +-
hadoop-ozone/ozonefs/pom.xml | 28 +-
hadoop-ozone/pom.xml | 146 +--
hadoop-ozone/recon-codegen/pom.xml | 12 +-
hadoop-ozone/recon/pom.xml | 42 +-
.../apache/hadoop/ozone/recon/ReconConstants.java | 7 +
.../hadoop/ozone/recon/ReconControllerModule.java | 18 +-
.../org/apache/hadoop/ozone/recon/ReconServer.java | 25 +-
.../hadoop/ozone/recon/api/ContainerEndpoint.java | 17 +-
.../hadoop/ozone/recon/api/NodeEndpoint.java | 23 +-
.../hadoop/ozone/recon/api/PipelineEndpoint.java | 11 +-
.../ozone/recon/api/types/DatanodeMetadata.java | 15 +
.../hadoop/ozone/recon/api/types/NSSummary.java | 70 ++
.../hadoop/ozone/recon/codec/NSSummaryCodec.java | 81 ++
.../ozone/recon/fsck/ContainerHealthStatus.java | 10 +-
.../ozone/recon/fsck/ContainerHealthTask.java | 51 +-
.../ozone/recon/persistence/ContainerHistory.java | 3 +
.../hadoop/ozone/recon/scm/PipelineSyncTask.java | 46 +-
.../ozone/recon/scm/ReconContainerManager.java | 133 ++-
.../recon/scm/ReconContainerReportHandler.java | 24 +-
.../ozone/recon/scm/ReconDeadNodeHandler.java | 79 ++
.../ReconIncrementalContainerReportHandler.java | 2 +-
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 20 +
.../ozone/recon/scm/ReconPipelineFactory.java | 9 +-
.../recon/scm/ReconPipelineReportHandler.java | 2 +-
.../scm/ReconStorageContainerManagerFacade.java | 37 +-
.../recon/spi/ContainerDBServiceProvider.java | 198 ----
.../recon/spi/ReconContainerMetadataManager.java | 193 ++++
.../recon/spi/ReconNamespaceSummaryManager.java | 37 +
.../recon/spi/StorageContainerServiceProvider.java | 14 +
.../spi/impl/ContainerDBServiceProviderImpl.java | 488 ----------
.../recon/spi/impl/ReconContainerDBProvider.java | 95 --
.../impl/ReconContainerMetadataManagerImpl.java | 458 +++++++++
.../ozone/recon/spi/impl/ReconDBDefinition.java | 13 +-
.../ozone/recon/spi/impl/ReconDBProvider.java | 117 +++
.../spi/impl/ReconNamespaceSummaryManagerImpl.java | 67 ++
.../impl/StorageContainerServiceProviderImpl.java | 14 +
.../ozone/recon/tasks/ContainerKeyMapperTask.java | 35 +-
.../ozone/recon/tasks/FileSizeCountTask.java | 5 +-
.../hadoop/ozone/recon/tasks/ReconTaskConfig.java | 4 +-
.../webapps/recon/ozone-recon-web/api/db.json | 20 +
.../src/views/datanodes/datanodes.tsx | 13 +-
.../ozone/recon/OMMetadataManagerTestUtils.java | 12 +-
.../hadoop/ozone/recon/ReconTestInjector.java | 20 +-
.../ozone/recon/api/TestContainerEndpoint.java | 17 +-
.../hadoop/ozone/recon/api/TestEndpoints.java | 22 +-
.../ozone/recon/api/TestOpenContainerCount.java | 424 ++++++++
.../recon/fsck/TestContainerHealthStatus.java | 6 +-
.../ozone/recon/fsck/TestContainerHealthTask.java | 97 +-
.../TestContainerHealthTaskRecordGenerator.java | 6 +-
.../recovery/TestReconOmMetadataManagerImpl.java | 9 +-
.../scm/AbstractReconContainerManagerTest.java | 44 +-
.../ozone/recon/scm/TestReconContainerManager.java | 55 +-
...TestReconIncrementalContainerReportHandler.java | 5 +-
.../ozone/recon/scm/TestReconNodeManager.java | 41 +-
.../ozone/recon/scm/TestReconPipelineManager.java | 12 +-
.../impl/TestContainerDBServiceProviderImpl.java | 386 --------
.../spi/impl/TestReconContainerDBProvider.java | 74 --
.../TestReconContainerMetadataManagerImpl.java | 388 ++++++++
.../ozone/recon/spi/impl/TestReconDBProvider.java | 71 ++
.../impl/TestReconNamespaceSummaryManagerImpl.java | 96 ++
.../recon/tasks/TestContainerKeyMapperTask.java | 63 +-
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 5 +-
hadoop-ozone/s3gateway/pom.xml | 58 +-
.../hadoop/ozone/s3/OzoneClientProducer.java | 17 +-
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 283 +++++-
.../s3/endpoint/ListMultipartUploadsResult.java | 53 +-
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 8 +-
.../endpoint/PutBucketAclRequestUnmarshaller.java | 85 ++
.../org/apache/hadoop/ozone/s3/endpoint/S3Acl.java | 348 +++++++
.../hadoop/ozone/s3/endpoint/S3BucketAcl.java | 241 +++++
.../apache/hadoop/ozone/s3/endpoint/S3Owner.java | 75 ++
.../hadoop/ozone/s3/exception/S3ErrorTable.java | 5 +
.../hadoop/ozone/s3/signature/SignatureInfo.java | 3 +
.../apache/hadoop/ozone/s3/util/RFC1123Util.java | 3 +-
.../hadoop/ozone/client/OzoneBucketStub.java | 28 +-
.../hadoop/ozone/client/OzoneVolumeStub.java | 24 +
.../ozone/s3/TestEmptyContentTypeFilter.java | 5 +-
.../hadoop/ozone/s3/TestOzoneClientProducer.java | 4 +-
.../ozone/s3/TestVirtualHostStyleFilter.java | 2 +-
.../hadoop/ozone/s3/endpoint/TestBucketAcl.java | 266 +++++
.../hadoop/ozone/s3/endpoint/TestBucketGet.java | 376 -------
.../hadoop/ozone/s3/endpoint/TestBucketList.java | 376 +++++++
.../hadoop/ozone/s3/endpoint/TestBucketPut.java | 6 +-
.../ozone/s3/endpoint/TestPermissionCheck.java | 56 +-
.../signature/TestAuthorizationV4HeaderParser.java | 2 +-
.../signature/TestAuthorizationV4QueryParser.java | 5 +-
.../hadoop/ozone/s3/util/TestOzoneS3Util.java | 2 +-
.../src/test/resources/groupAccessControlList.xml | 57 ++
.../src/test/resources/userAccessControlList.xml | 42 +
hadoop-ozone/tools/pom.xml | 62 +-
.../ozone/audit/parser/common/DatabaseHelper.java | 7 +-
.../StorageContainerManagersCommandHandler.java | 2 +-
.../apache/hadoop/ozone/debug/ChunkKeyHandler.java | 5 +-
.../apache/hadoop/ozone/debug/DatanodeLayout.java | 8 +-
.../apache/hadoop/ozone/debug/ExportContainer.java | 191 ----
.../org/apache/hadoop/ozone/debug/OzoneDebug.java | 10 +
.../apache/hadoop/ozone/debug/PrefixParser.java | 236 +++++
.../ozone/debug/container/ContainerCommands.java | 209 ++++
.../ozone/debug/container/ExportSubcommand.java | 78 ++
.../ozone/debug/container/InfoSubcommand.java | 56 ++
.../ozone/debug/container/ListSubcommand.java | 53 +
.../hadoop/ozone/debug/container/package-info.java | 22 +
.../hadoop/ozone/freon/BaseFreonGenerator.java | 10 +-
.../hadoop/ozone/freon/ChunkManagerDiskWrite.java | 6 +-
.../ozone/freon/ClosedContainerReplicator.java | 7 +-
.../hadoop/ozone/freon/DatanodeChunkGenerator.java | 3 +-
.../hadoop/ozone/freon/DatanodeChunkValidator.java | 5 +-
.../java/org/apache/hadoop/ozone/freon/Freon.java | 5 +-
.../ozone/freon/LeaderAppendLogEntryGenerator.java | 6 +-
.../hadoop/ozone/freon/OmBucketGenerator.java | 2 +-
.../apache/hadoop/ozone/freon/OmBucketRemover.java | 90 ++
.../apache/hadoop/ozone/freon/OmKeyGenerator.java | 5 +-
.../hadoop/ozone/freon/OzoneClientKeyRemover.java | 92 ++
.../hadoop/ozone/freon/RandomKeyGenerator.java | 137 +++
.../apache/hadoop/ozone/freon/S3KeyGenerator.java | 1 +
.../hadoop/ozone/freon/StreamingGenerator.java | 142 +++
.../freon/containergenerator/BaseGenerator.java | 3 +
.../containergenerator/GeneratorDatanode.java | 9 +-
.../freon/containergenerator/GeneratorOm.java | 11 +-
.../freon/containergenerator/GeneratorScm.java | 6 +-
.../GenerateOzoneRequiredConfigurations.java | 44 +-
.../hadoop/ozone/genesis/BenchMarkCRCBatch.java | 3 +
.../ozone/genesis/BenchMarkCRCStreaming.java | 3 +
.../ozone/genesis/BenchMarkContainerStateMap.java | 88 +-
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 4 +-
.../ozone/genesis/BenchMarkMetadataStoreReads.java | 70 --
.../genesis/BenchMarkMetadataStoreWrites.java | 62 --
.../ozone/genesis/BenchMarkOMKeyAllocation.java | 6 +-
.../ozone/genesis/BenchMarkOzoneManager.java | 18 +-
.../ozone/genesis/BenchMarkRocksDbStore.java | 119 ---
.../apache/hadoop/ozone/genesis/BenchMarkSCM.java | 7 +-
.../ozone/genesis/BenchmarkChunkManager.java | 4 +-
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 36 +-
.../org/apache/hadoop/ozone/scm/cli/SQLCLI.java | 562 -----------
.../hadoop/ozone/shell/keys/CopyKeyHandler.java | 28 +-
.../hadoop/ozone/shell/keys/PutKeyHandler.java | 27 +-
.../hadoop/ozone/shell/s3/GetS3SecretHandler.java | 11 +-
.../ozone/shell/s3/RevokeS3SecretHandler.java | 76 ++
.../org/apache/hadoop/ozone/shell/s3/S3Shell.java | 3 +-
.../hadoop/ozone/audit/parser/TestAuditParser.java | 31 +-
.../TestGenerateOzoneRequiredConfigurations.java | 54 +-
.../tools/src/test/resources/testaudit.log | 3 +
.../tools/src/test/resources/testloadaudit.log | 21 +
pom.xml | 96 +-
1145 files changed, 51852 insertions(+), 17927 deletions(-)
delete mode 100644 .github/buildenv/Dockerfile
delete mode 100755 .github/buildenv/entrypoint.sh
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteArrayReader.java
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteBufferReader.java
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteReaderStrategy.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/ServerNotLeaderException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ha/NonRetriableException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ha/RetriableWithFailOverException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/ContainerTokenIdentifier.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/ShortLivedTokenIdentifier.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLStatus.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/crl/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/EntryConsumer.java
rename hadoop-hdds/{framework => common}/src/main/java/org/apache/hadoop/hdds/utils/db/Codec.java (100%)
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/client/TestReplicationConfig.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/ratis/TestServerNotLeaderException.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/datanode/metadata/CRLDBDefinition.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/datanode/metadata/DatanodeCRLStore.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/datanode/metadata/DatanodeCRLStoreImpl.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/hdds/datanode/metadata/package-info.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/CRLStatusReportPublisher.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/StorageVolumeUtil.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeFactory.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolume.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolumeFactory.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolume.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolumeFactory.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/DirectoryServerDestination.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/DirectoryServerSource.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/DirstreamClientHandler.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/DirstreamServerHandler.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingClient.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingDestination.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingException.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingServer.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingSource.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/package-info.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/hdds/datanode/metadata/TestDatanodeCRLStoreImpl.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/hdds/datanode/metadata/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestHddsVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestStorageVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/TestDirstreamClientHandler.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/TestStreamingServer.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/package-info.java
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Delete.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Format.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Rename.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO.md
create mode 100644 hadoop-hdds/docs/content/feature/scm-secure-ha.png
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/OzonePBHelper.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/CRLClientUpdateHandler.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/CRLStore.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/ClientCRLStore.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/ClientUpdateHandler.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/SCMUpdateClientConfiguration.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/SCMUpdateServiceGrpcClient.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/UpdateServiceConfig.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/package-info.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/CRLClientInfo.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateClientInfo.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/package-info.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/OzoneSecretKey.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/OzoneSecretManager.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/OzoneSecurityException.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/package-info.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/CompositeTokenVerifier.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ContainerTokenGenerator.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ContainerTokenSecretManager.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ContainerTokenVerifier.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/NoopTokenVerifier.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ShortLivedTokenSecretManager.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ShortLivedTokenVerifier.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/CRLCodec.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLCodec.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfoCodec.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/FixedThreadPoolExecutor.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/LevelDBStore.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/LevelDBStoreIterator.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/MetadataStore.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/MetadataStoreBuilder.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStore.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreIterator.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/ShortCodec.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/scm/protocol/TestReplicationConfig.java
create mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/security/token/TestBlockTokenVerifier.java
create mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/security/token/TestContainerTokenVerifier.java
create mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/security/token/TokenVerifierTests.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestMetadataStore.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestRocksDBStoreMBean.java
rename hadoop-hdds/{interface-client => interface-server}/src/main/proto/InterSCMProtocol.proto (100%)
create mode 100644 hadoop-hdds/interface-server/src/main/proto/SCMUpdateProtocol.proto
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteStatusList.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/crl/CRLStatusReportHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/crl/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/CRLInfoCodec.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableContainerFactory.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableContainerProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMCRLStore.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMCRLUpdateHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateClientManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateServiceGrpcServer.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateServiceImpl.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestContainerBalancer.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/crl/TestCRLStatusReportHandler.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/update/server/MockCRLStore.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/update/server/TestSCMUpdateServiceGrpcServer.java
delete mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/GenericTestUtils.java
delete mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/LambdaTestUtils.java
delete mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/TimedOutTestsListener.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/GenericTestUtils.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/LambdaTestUtils.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/TimedOutTestsListener.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/package-info.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerCommands.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStartSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStopSubcommand.java
create mode 100644 hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestContainerBalancerSubCommand.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/MockBlockAllocator.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/MockDatanodeStorage.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/MockOmTransport.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/MockXceiverClientFactory.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/MockXceiverClientSpi.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/SinglePipelineBlockAllocator.java
create mode 100644 hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/TestOzoneClient.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/WithParentObjectId.java
delete mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecurityException.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzonePrefixPath.java
create mode 100755 hadoop-ozone/dev-support/checks/dependency.sh
create mode 100755 hadoop-ozone/dev-support/checks/docs.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/Dockerfile-krb5
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/kadm5.acl
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/krb5.conf
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/launcher.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/krb5.conf
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/krb5.conf
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/krb5.conf
create mode 100755 hadoop-ozone/dist/src/main/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/hadoop.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/jhs.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/nm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/rm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/testuser2.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/update-keytabs.sh
create mode 100644 hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-CDDL.txt
create mode 100644 hadoop-ozone/dist/src/main/license/jar-report.txt
create mode 100755 hadoop-ozone/dist/src/main/license/update-jar-report.sh
create mode 100644 hadoop-ozone/dist/src/main/smoketest/freon/remove.robot
create mode 100644 hadoop-ozone/dist/src/test/shell/failing1/test.sh
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestDirectoryDeletingServiceWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfacesWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemPrefixParser.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystemWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractUtils.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestContainerBalancerOperations.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneClientMultipartUploadWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/DatanodeTestUtils.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/volume/TestDatanodeHddsVolumeFailureDetection.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/volume/TestDatanodeHddsVolumeFailureToleration.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithDatanodeRestart.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestHadoopDirTreeGeneratorWithFSO.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestOzoneClientKeyGenerator.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMStartupWithLayout.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStoreWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestRecursiveAclWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestFailoverWithSCMHA.java
create mode 100644 hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/codec/OmDirectoryInfoCodec.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzonePrefixPathImpl.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMAllocateBlockRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyDeleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRenameRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMPathsPurgeRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAddAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyRemoveAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeySetAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3InitiateMultipartUploadRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadAbortRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCommitPartRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCompleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/security/S3RevokeSecretRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/file/OMDirectoryCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/file/OMFileCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMAllocateBlockResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyCommitResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyDeleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyRenameResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMPathsPurgeResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/acl/OMKeyAclResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3InitiateMultipartUploadResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadAbortResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCommitPartResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCompleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/security/S3RevokeSecretResponse.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretKey.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/TestOMBucketCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMAllocateBlockRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCommitRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyDeleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3InitiateMultipartUploadRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadAbortRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCommitPartRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCompleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/file/TestOMDirectoryCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/file/TestOMFileCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyCommitResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyDeleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3InitiateMultipartUploadResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadAbortResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadCommitPartResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadCompleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestOzoneObj.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestParentAcl.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/NSSummary.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/codec/NSSummaryCodec.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconDeadNodeHandler.java
delete mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ContainerDBServiceProvider.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ReconContainerMetadataManager.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ReconNamespaceSummaryManager.java
delete mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ContainerDBServiceProviderImpl.java
delete mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerDBProvider.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerMetadataManagerImpl.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconDBProvider.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconNamespaceSummaryManagerImpl.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestOpenContainerCount.java
delete mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestContainerDBServiceProviderImpl.java
delete mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerDBProvider.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerMetadataManagerImpl.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconDBProvider.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconNamespaceSummaryManagerImpl.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/PutBucketAclRequestUnmarshaller.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/S3Acl.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/S3BucketAcl.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/S3Owner.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketAcl.java
delete mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketGet.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketList.java
create mode 100644 hadoop-ozone/s3gateway/src/test/resources/groupAccessControlList.xml
create mode 100644 hadoop-ozone/s3gateway/src/test/resources/userAccessControlList.xml
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/ExportContainer.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/PrefixParser.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/container/ContainerCommands.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/container/ExportSubcommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/container/InfoSubcommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/container/ListSubcommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/container/package-info.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OmBucketRemover.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OzoneClientKeyRemover.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/StreamingGenerator.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreReads.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreWrites.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkRocksDbStore.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/scm/cli/SQLCLI.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/s3/RevokeS3SecretHandler.java
create mode 100644 hadoop-ozone/tools/src/test/resources/testloadaudit.log
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org