You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by ad...@apache.org on 2021/07/12 07:21:41 UTC
[ozone] branch HDDS-4440-s3-performance updated (0b4779c -> 5af5a33)
This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a change to branch HDDS-4440-s3-performance
in repository https://gitbox.apache.org/repos/asf/ozone.git.
from 0b4779c HDDS-5073. Use ReplicationConfig on client side (#2136)
add 3e21d75 HDDS-3829. Introduce Layout Feature interface in Ozone. (#1322)
add 3fe83ce HDDS-4173. Implement HDDS Version management using the LayoutVersionManager interface. (#1392)
add 9b55f69 HDDS-4141. Implement Finalize command in Ozone Manager client. (#1400)
add 95b8712 HDDS-4174. Add current HDDS layout version to Datanode heartbeat/registration (#1421)
add 6fa5fa1 HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version. (#1405)
add b96834f HDDS-4252. Add the current layout versions to DN - SCM proto payload. (#1432)
add 08d8d36 HDDS-4227. Implement a 'Prepare For Upgrade' step in OM that applies all committed Ratis transactions. (#1430)
add 8fe8a1f HDDS-4253. Add LayoutVersion request/response for DN registration. (#1457)
add 515c619 HDDS-4319. Compile error with Java 11 (#1483)
add bfce8b9 HDDS-4172. Implement Finalize command in Ozone Manager server. (#1456)
add 0d014c0 HDDS-4314. OM Layout Version Manager init throws silent CNF error in integration tests. (#1492)
add b8dcf6b Merge remote-tracking branch 'upstream/master' into HDDS-3698-upgrade
add d4212a7 Fix copilation issue after mergee
add 5186dec HDDS-4401. Fix compilation issue in HDDS-3698-upgrade branch. (#1529)
add 8e29229 HDDS-4296. SCM changes to process Layout Info in heartbeat request/response (#1486)
add 18bef92 HDDS-4142. Expose upgrade related state through JMX. (#1534)
add 28ecd8d Merge remote-tracking branch 'upstream/master' into HDDS-3698-upgrade
add 7a26250 HDDS-4342. Add DataNode state and transitions for a node going through upgrade. (#1508)
add 4384789 Merge remote-tracking branch 'upstream/master' into HDDS-3698-upgrade
add bf7624b Merge remote-tracking branch 'upstream/master' into HDDS-3698-upgrade
add 6b9ed19 Merge remote-tracking branch 'upstream/master' into HDDS-3698-upgrade
add ad9d019 HDDS-4536. Add more unit tests for OM layout version manager. (#1646)
add 9c4f2b4 HDDS-4463. Verify that OM/SCM start fails when Software Layout Version < Metadata Layout Version. (#1645)
add c3e3b35 HDDS-4179. Implement post-finalize SCM logic. (#1611)
add a7797aa HDDS-4480. Implement OM Prepare Request/Response. (#1613)
add 2489968 HDDS-4540. Add a new OM admin operation to submit the OMPrepareRequest. (#1664)
add 6519142 HDDS-4178. SCM Finalize client command implementation (#1651)
add bdd00a7 HDDS-4564. Prepare client should check every OM individually for the prepared check based on Txn ID. (#1692)
add 17a126d HDDS-4569. Add pre append gate and marker file to OM prepare state. (#1705)
add 72c215c HDDS-4612. Create OMCancelPrepareRequest and Response to cancel the prepared state of an OM. (#1765)
add d36b87d HDDS-4175. Implement Datanode Finalization. (#1720)
add 81c1dee HDDS-4219. Revisit 'static' nature of OM Layout Version Manager. (#1773)
add 1de63f4 HDDS-4645. Starting OM with the --upgrade flag should delete the prepare marker file. (#1798)
add a81b36e HDDS-4614. Add Integration test for HDDS upgrade (happy path cases). (#1726)
add ee8d276 HDDS-4611. Add an admin command to cancel "preparation" of an OM quorum. (#1830)
add 1e187f7 Merge remote-tracking branch 'upstream/master' into upgrade-branch-merge-candidate
add 7ae013e Fix spotbugs issues.
add ba4aeca trigger new CI check
add dd14670 trigger new CI check
add 5720ebf HDDS-4182. Onboard HDDS-3869 into Layout version management. (#1913)
add 277cc0b HDDS-4610. Fix issues in 'prepare' operation with one OM down. (#1836)
add 9a078c4 Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonrolling-upgrade
add 8a9be0d trigger new CI check
add 447024b HDDS-4817. Fresh deploy of Ozone must use the highest layout version by default. (#1933)
add 485ec25 HDDS-4902 Add Layout version information to Recon datanode info API. (#1993)
add df57ebc HDDS-4946. Do not wait one heartbeat to move newly registered datanodes that match SCM's MLV from HEALTHY_READONLY to HEALTHY (#2015)
add 7eefee7 Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonrolling-upgrade-merge-candidate
add f6a302a HDDS-4908. Layout version should be available in DB for an un-finalized OM to be finalized through a Ratis snapshot. (#2017)
add f6ecb1d Merge remote-tracking branch 'ssh-upstream/master' into HDDS-3698-nonrolling-upgrade
add d55a714 HDDS-5023. Datanodes should always use MLV 0 when no VERSION file is present. (#2080)
add ed00db7 HDDS-4990. Introduce First upgrade startup action and Pre-finalized state validation in Layout Feature. (#2040)
add 2f96d94 Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonrolling-upgrade
add 1730b0d Allow layout version to be specified through config on --init (for MiniOzone cluster testing).
add de3d735 Fix SCM admin command issues after merge.
add 666f33b Ignore failing upgrade integration test.
add 8f3b5ae Fix rat failure.
add dfc4d99 trigger new CI check
add f16092e Intialize DN layout version before security init.
add 805d93d Checkstyle fix.
add 32d0834 Merge remote-tracking branch 'upstream/master' into HDDS-5046-2
add 9e2e43f Merge remote-tracking branch 'upstream/master' into HDDS-5046-2
add 9363434 Fix ozone-ha acceptance test.
add 3ea88e1 HDDS-5014. Add non-rolling upgrade design docs.
add a55f657 Merge remote-tracking branch 'upstream/master' into ssh-upstream-upgrade-branch
add baeb46d Merge remote-tracking branch 'ssh-upstream/master' into ssh-upstream-upgrade-branch
add c4b7e7b HDDS-4828. SCM should go into "safe mode" until there is at least 1 pipeline to work with after finalization. (#2101)
add 5c6c858 HDDS-4181. Add acceptance tests for upgrade, finalization and downgrade. (#2056)
add 6e027a1 HDDS-5014. Upgrade usage primer documentation. (#2133)
add e8e7ced HDDS-5014. Move upgrade user flow to 'feature' folder.
add 7266f32 HDDS-4914. Failure injection and validating HDDS upgrade. (#1998)
add 58a87fb HDDS-5086. Add pre-finalize validation action for SCM HA. (#2143)
add d2da5a4 HDDS-4992. SCM should not use pipelines with HEALTHY_READONLY datanodes (#2142)
add 3221362 Update upgrade-dev-primer.md
add 2823320 HDDS-5108. Attempt to remove state from *UpgradeFinalizer classes. (#2160)
add 40223ec HDDS-4818. Add command to get status of Finalize in OM & SCM (#2083)
add 5927bd2 HDDS-5118. Recover from failure during upgrade action. (#2179)
add 9168e07 HDDS-4959. NoSuchMethodException when wrapping RpcException on downgrade (#2178)
add bef4aed HDDS-5137. Update licenses for non-rolling upgrade new dependencies. (#2201)
add cdb1471 HDDS-5170. Race condition in NodestateManager#addNode allows datanodes with lower MLV to be used in pipelines. (#2204)
add 253cc55 Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonrolling-upgrade-master-merge2
add 1686e56 Fix compilation errors afte merge Update javassist in recon pom Fix changes introduced in merge that failed TestSCMNodeManager upgrade tests Fix checkstyle Fix intermittent test failure TestSCMNodeManager#testSetNodeOpStateAndCommandFired after merge Skip scm init default layout version in TestOzoneConfigurationFields
add ea239ee HDDS-5226. Do not fail SCM HA pre-finalize validation if SCM HA was already being used. (#2257)
add 5f046fa HDDS-5138. Upgrade related RPC calls should be allowed only for admins. (#2217)
add 92b5106 HDDS-5244. Allow multiple OM request versions to be supported at same layout version. (#2265)
add 2dbf04e HDDS-5191. Increase default pvc storage size (#2219)
add 000e7c5 HDDS-5252. PipelinePlacementPolicy filter out datanodes with not enough space. (#2271)
add 2671b48 HDDS-5219. Limit number of bad volumes by dfs.datanode.failed.volumes.tolerated. (#2243)
add d9ae1b3 HDDS-5241. SCM UI should have leader/follower and Primordial SCM information (#2260)
add 53bbabf HDDS-5275. Datanode Report Publisher publishes one extra report after DN shutdown (#2283)
add 4301315 HDDS-5158. Add documentation for SCM HA Security. (#2205)
add d567360 HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. (#2281)
add 00b675d HDDS-4927. Determine over and under utilized datanodes in Container Balancer. (#2230)
add 8586815 HDDS-4043. allow deletion from Trash directory without -skipTrash option (#2110)
add fa4ddbc HDDS-4936. Change ozone groupId from org.apache.hadoop to org.apache.ozone (#2018)
add 3ffbdd0 HDDS-4993. Add guardrail for reserved buffer size when DN reads a chunk (#2058)
add e5a774f HDDS-5123. Use the pre-created apache/ozone-testkrb5 image during secure acceptance tests (#2165)
add 5fd9780 HDDS-5272. Make ozonefs.robot execution repeatable (#2280)
add 12e2918 HDDS-5280. Make XceiverClientManager creation when necessary in ContainerOperationClient (#2289)
add 1d18f47 Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonrolling-upgrade-master-merge
add a1b50ad HDDS-5321. Merge master branch at 12e2918 into upgrade branch (#2316)
add 2c0adac HDDS-5109. Track OM prepare intermittent integration test failure. (#2288)
add d6f4d07 HDDS-2949: mkdir : store directory entries in a separate table (#1404)
add 8e17992 HDDS-4266: CreateFile : store parent dir entries into DirTable and file entry into separate FileTable (#1473)
add 60dd692 HDDS-4332: ListFileStatus - do lookup in directory and file tables (#1503)
add 172ab51 HDDS-4357: Rename : make rename an atomic ops by updating key path entry in dir/file table (#1557)
add 995b3bd HDDS-4358: Delete : make delete an atomic operation (#1607)
add d810193 HDDS-4321. Fix compilation errors : merge HDDS-4308 and HDDS-4473 changes into the branch (#1668)
add b9771e4 HDDS-4596. Directory table, fileTable and openFile Table is missing from the OM DB Definition. (#1724)
add c10abe6 HDDS-4514. AllocateBlock : lookup and update open file table for the given path (#1679)
add 0bdbe41 HDDS-2942. Putkey : create key table entries for intermediate directories in the key path (#1764)
add f132f1e HDDS-4658. LookupKey: do lookup in dir and file tables (#1775)
add 9003eca HDDS-4717. Fix TestOzoneFileSystemV1 and TestObjectStoreV1 cases (#1815)
add f5e4167 HDDS-4720. RenameKey : add unit test to verify bucket#renameKey (#1847)
add b4d75d6 HDDS-4486. Feature Config: Make proper enableFSPaths and OFS optimized flag combinations (#1848)
add 5b86a07 HDDS-4771. [FSO]S3MultiPart: Implement InitiateMultiPartUpload (#1877)
add 797cf96 HDDS-4805. [FSO]Fix findbugs issues after HDDS-2195 (#1906)
add 85f07a2 HDDS-4781. [FSO]S3MultiPart: Implement create and commit upload part file (#1897)
add b3327bc HDDS-4742. Make trash work with FS Optimised Buckets. (#1915)
add 30d9fa4 HDDS-4813. [FSO]S3Multipart: Implement UploadCompleteRequest (#1923)
add f28699c HDDS-4513.[FSO]OzoneContract unit test case fixes (#1945)
add 0dfad20 HDDS-4683. [FSO]ListKeys: do lookup in dir and file tables (#1954)
add 7be14c8 HDDS-4490.[FSO]RenameAndDelete : make ofs#rename and ofs#delete an atomic operation. (#1965)
add 46d9192 HDDS-4835. [FSO]S3Multipart: Implement UploadAbortRequest (#1997)
add cdc7855 HDDS-4924. [FSO]S3Multipart: Implement OzoneBucket#listParts (#2016)
add 7ca2c5f HDDS-4973. [FSO] Missed to cleanup new FileTables in OMRequests (#2035)
add 2f7606b HDDS-4790. Add a tool to parse entries in the prefix format (#1891)
add 81361d6 HDDS-4917.[FSO]Implement ACL requests for new layout (#2024)
add 5ed34df HDDS-4932. [FSO] Provide list subpaths function to perform recursive ACL check during delete and rename op (#2008)
add 459e990 HDDS-4495. [FSO]Delete : Implement async cleanup of garbage and orphan sub-dirs/files (#2093)
add 9ecfafe HDDS-5018. [FSO] Add robot tests for new Filesystem layout (#2071)
add bd22405 HDDS-4727. [FSO]OMConfig: Modify 'ozone.om.layout.version' config (#2118)
add c326199 HDDS-4691. [FSO]Authorizer: OM can do recursive ACL check for subpaths (#2011)
add f9910c6 HDDS-5072. [FSO] Addendum patch to fix compilation error (#2130)
add 45c62e8 HDDS-5067. [FSO] Add documentation (#2117)
add aa0abab HDDS-5042. [FSO] Improve KeyDeletingService to cleanup FSO files (#2128)
add 4449cfda HDDS-5079. [FSO] Rename om metadata layout configuration value LEGACY (#2135)
add ec38f01 HDDS-5092. [FSO] Update statis links in the design doc section of the Ozone docs (#2150)
add e63d80a HDDS-5106. [FSO] Avoid using V1 postfixes for prefix related classes (#2174)
add f1c6757 Merge remote-tracking branch 'origin/master' into HDDS-2939
add a4a9971 HDDS-5093. [FSO] Reducing time of ozonefs acceptance testmatrix (#2164)
add b6908b0 HDDS-5097. [FSO] Cleanup integration tests and reduce the execution time (#2185)
add 3c82503 HDDS-5094. [FSO] Fail OM startup when turn on prefix layout with old buckets (#2151)
add 72d3106 HDDS-5201. [FSO] S3MultiPart: Use existing ozone key format for MPU Info in DB (#2228)
add 974136d HDDS-4923. [FSO]S3Multipart: Run S3 acceptance test with prefix layout (#2208)
add 723cdaa Merge remote-tracking branch 'origin/master' into HDDS-2939
add 6b09219 HDDS-5264. SCM should send token for CloseContainer command (#2277)
add e35dde9 HDDS-5151. Support ByteBuffer read in OzoneInputStream (#2203)
add 96d9ed7 HDDS-5240. Fix out of bound exception when loading auditparser (#2258)
add 69b92a0 HDDS-5281. Add reinitialize() for SequenceIdGenerator. (#2292)
add 183f4b4 HDDS-5242. Skip `failing` acceptance suite by default (#2263)
add 18f0fe1 HDDS-4733. Remove leveldb from codebase (#2274)
add 656339e HDDS-5204. Missing type-level Javadoc comments (#2262)
add 90a8d7c HDDS-5276. Use built-in cancel support for duplicates (#2284)
add f3f258a HDDS-5277. Recon shows operational status as "DECOMMISSIONING" for "DECOMMISSIONED" DNs (#2286)
add b6a035a HDDS-5288. Recon UI cannot sort DN by operational state (#2298)
add 3030c9f HDDS-5282. Relocate classes copied from Hadoop (#2293)
add 095d09b HDDS-4483. Datanodes should send last processed CRL sequence ID in heartbeats (#2285)
add 25d66d7 HDDS-5154. Add SCM GRPC server to publish CRL update. (#2216)
add c2ae90b HDDS-5295. testCRLStatusReportPublisher fails to create CRLInfo (#2302)
add 8c1de61 HDDS-5274. Revert "HDDS-5153. Decommissioning a dead node should complete immediately (#2190)" (#2282)
add bb8eb39 HDDS-5286. Document SCM HA classes (#2297)
add 405925f HDDS-5168. Use ReplicationConfig in OmKeyArgs (#2287). Contributed by elek.
add 7bc1a25 HDDS-5292. Introduce the WritableContainerInterface to SCM (#2300)
add 9ad2e98 HDDS-5298. Use netty-bom to ensure consistent Netty version (#2304)
add 86bcebb Merge remote-tracking branch 'origin/master' into HDDS-2939
add 0323530 HDDS-5311. [FSO] Fix checkstyle warnings after branch rebase (#2307)
add 39954ad HDDS-5111. DataNode should not always report full information in heartbeat (#2182)
add 3d3a87f HDDS-5235. Add precheck to avoid ContainerHealthTask setting Deleted containers to Missing state (#2250)
add 6e278ba HDDS-5243. Return latest key location for clients (#2261)
add 94fb6f3 Merge remote-tracking branch 'origin/master' into HDDS-2939
add 23352f8 HDDS-5320. [FSO] Fix minor checkstyle warnings (#2313)
add fe9ca6e HDDS-5284. [SCM-HA] SCM start failed with PipelineNotFoundException. (#2295)
add 2bd7f12 HDDS-5313. ContainerInfo should use ReplicationConfig (#2309)
add d1dc91d HDDS-5318. Intermittent failure in TestOzoneManagerDoubleBufferWithOMResponse (#2314)
add 2e385fc HDDS-5312. Intermittent failure in SCM Ratis integration test (#2308)
add fb4436f HDDS-5285. Document block deleting service classes (#2296)
add baf8272 HDDS-5323. Avoid unncessary report processing log messages in follower. (#2317)
add 6752a71 HDDS-5317. BootStrapped SCM fails to bootstrap if it connects to another bootstrapped SCM first. (#2312)
add 159fb24 HDDS-5304. InterSCM protocol should be server-only (#2305)
add 1290b87 Merge remote-tracking branch 'ssh-upstream/master' into ssh-upstream-upgrade-branch
add 4c8b62f HDDS-5333. Remove getRequestType method from OM request classes.
add b335431 HDDS-5336. Fix datanode capacity related race condition. (#2327)
add f7ac8c1 HDDS-5307. Remove checkAclRight method, duplicates checkAclRights (#2323)
add 6e6aefe HDDS-5315 Skip storing unwanted block tokens on OM DB (#2311)
add ac7166b HDDS-5263. SCM may stay in safe mode forever after a unclean shutdown of SCM. (#2294)
add 2262395 HDDS-4913. Refine the native authorizer parent context right check. (#2012)
add 7e59cb8 HDDS-5334. Remove unncessary log added durig HDDS-5263. (#2326)
add 838d5d8 HDDS-5324. Shouldn't dectNumKeys when delete keys fail (#2319)
add d45819f HDDS-5329. Remove lockmanager and synchronize on ContainerInfo in Replication Manager (#2325)
add ad02d6c HDDS-5316. Fix datanode reserved space calculation. (#2310)
add 9cf7d31 HDDS-5268. Ensure disk checker also scans the ratis log disks periodically (#2290)
add 623e1d9 HDDS-5344. Adding debug log for block token verification. (#2333)
add 92eb6a2 Merge remote-tracking branch 'ssh-upstream/master' into ssh-upstream-upgrade-branch
add 4b4e475 Fix checkstyle after merge.
add 7bfd76b HDDS-5348. Pipeline creator may miss one-shot run (#2339)
add 4506883 Merge remote-tracking branch 'ssh-upstream/master' into ssh-upstream-upgrade-branch
add c0eb347 HDDS-5237. Add SSL support to the Ozone streaming API (#2315)
add 698456b HDDS-5363. Datanode shutdown due to too many bad volumes in CI (#2352)
add b545e69 HDDS-4692. Handle CRLStatusReport got from DN heartbeats and persist them (#2334)
add 51df153 HDDS-5356. [SCM-HA] SCM start failed with PipelineNotFoundException (#2346)
add b9caec3 HDDS-5342. HTML report missing from acceptance results (#2332)
add bb4a5b2 HDDS-5361. Suppress logging of ServerNotLeaderException. (#2351)
add 76d9a10 HDDS-5347. Wrong cache key for integration tests (#2336)
add eb6c1b6 HDDS-5314. Show number of Open containers per Node in Recon UI (#2321)
add 5c3f9aa HDDS-5355. In ContainerStateMachine, share the executor threads between the containers. (#2350)
add 5a9257e HDDS-5357. Move SCMUpdateProtocol to hdds interface-server package. (#2348)
add fa9616d HDDS-5352. java.lang.ClassNotFoundException: org/eclipse/jetty/alpn/ALPN (#2347)
add 5e1308a HDDS-5367. Update modification time when updating quota/storageType/versioning (#2355)
add 2cd55d7 HDDS-5269. Datandoe with low ratis log volume space should not be considered for new pipeline allocation. (#2344)
add dc02775 HDDS-4926. Support start/stop for container balancer via command line (#2278)
add b563669 HDDS-5350 : Add allocate block support in MockOmTransport (#2341). Contributed by Uma Maheswara Rao G.
add 93fb714 HDDS-5353. Avoid unnecessary executeBatch call in insertAudits (#2342)
add 0766d2c HDDS-5381. SCM terminated with exit status 1: null. (#2362)
add bf42bea HDDS-5369. Cleanup unused configuration related to SCM HA (#2359)
add e251bab HDDS-5289. Update container's deleteTransactionId on creation of the transaction in SCM. (#2361)
add 3232fb0 HDDS-5387. ProfileServlet to move the default output location to an ozone specific directory (#2368)
add 450c375 HDDS-5341. Container report processing is single threaded (#2338)
add be7264d HDDS-5380. Get more accurate space info for DedicatedDiskSpaceUsage. (#2365)
add b7ecf8b HDDS-5383. Eliminate expensive string creation in debug log messages (#2372)
add 474457c HDDS-5389. Include ozoneserviceid in fs.defaultFS when configuring o3fs (#2370)
add 1e5c708 HDDS-5395. Avoid unnecessary numKeyOps.incr() call in OMMetrics (#2374)
add 88383d1 HDDS-5396. Fix negligence issue conditional expressions in MockCRLStore.java (#2380)
add 1d8f972 HDDS-5398. Avoid object creation in ReplicationManger debug log statements (#2379)
add 039afb3 Merge remote-tracking branch 'ssh-upstream/master' into ssh-upstream-upgrade-branch
add 4e8aec5 HDDS-5406. add proto version to all the proto files. (#2385)
add 9948632 HDDS-5405. Refactor pom files for HadoopRpc and Grpc/Ratis compilation properties. (#2386)
add b9a86c3 HDDS-5332. Add a new column family and a service provider in Recon DB for Namespace Summaries (#2366)
add 01e8811 Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonrolling-upgrade
add aaeae47 HDDS-5415. Fix TestSCMNodeManager after merge of master at 1d8f972 into upgrade branch (#2388)
add 19ed794 HDDS-3292. Support Hadoop 3.3 (#2356)
add 240af4d Merge branch 'master' into HDDS-3698-nonrolling-upgrade
add 4859acc HDDS-5417. Fix json parse stackoverflow for cmd ozone debug container list. (#2391)
add eb753e1 HDDS-5416. Intermittent test failure in TestSCMPipelineManager#testPipelineReload (#2394)
add a2739e2 HDDS-5424. Pipeline list command's output should be in local timezone. (#2400)
add 77c83c0 HDDS-5400. Remove inflight actions in Replication Manager if node switches to not IN_SERVICE (#2381)
add dba2d54 HDDS-5421. SCM throws NPE during JMX call. (#2398)
add 5af5a33 Merge remote-tracking branch 'ssh-upstream/master' into ssh-upstream-upgrade-branch
No new revisions were added by this update.
Summary of changes:
.github/workflows/cancel-ci.yaml | 8 -
.github/workflows/post-commit.yml | 41 +-
hadoop-hdds/client/pom.xml | 30 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 29 +-
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 6 +-
.../hadoop/hdds/scm/storage/ByteArrayReader.java | 67 ++
.../hadoop/hdds/scm/storage/ByteBufferReader.java | 71 ++
.../hdds/scm/storage/ByteReaderStrategy.java | 41 +
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 38 +-
.../hdds/scm/storage/TestBlockInputStream.java | 37 +-
.../hdds/scm/storage/TestChunkInputStream.java | 2 +-
hadoop-hdds/common/pom.xml | 48 +-
.../apache/hadoop/hdds/DFSConfigKeysLegacy.java | 17 -
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 8 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 17 +
.../hadoop/hdds/client/RatisReplicationConfig.java | 7 +-
.../hdds/client/StandaloneReplicationConfig.java | 15 +-
.../hadoop/hdds/fs/AbstractSpaceUsageSource.java | 7 +-
.../hadoop/hdds/fs/DedicatedDiskSpaceUsage.java | 8 +-
.../java/org/apache/hadoop/hdds/scm/ScmConfig.java | 17 +
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 86 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 28 +
.../hadoop/hdds/scm/container/ContainerInfo.java | 55 +-
.../hadoop/hdds/scm/exceptions/SCMException.java | 4 +-
.../scm/ha/RetriableWithFailOverException.java | 31 +
.../scm/ha/RetriableWithNoFailoverException.java | 4 +
.../hadoop/hdds/scm/ha/SCMHAConfiguration.java | 4 +-
.../org/apache/hadoop/hdds/scm/ha/SCMHAUtils.java | 2 +
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 4 +-
.../protocol/StorageContainerLocationProtocol.java | 29 +
.../hadoop/hdds/scm/utils/ClientCommandsUtils.java | 3 +
.../hadoop/hdds/security/x509/crl/CRLInfo.java | 183 ----
.../hadoop/hdds/security/x509/crl/CRLStatus.java | 87 ++
.../hadoop/hdds/upgrade/HDDSLayoutFeature.java | 76 ++
.../hdds/upgrade/HDDSLayoutVersionManager.java | 97 ++
.../hadoop/hdds/upgrade/HDDSUpgradeAction.java | 27 +
.../apache/hadoop/hdds/upgrade/package-info.java | 23 +
.../apache/hadoop/hdds/utils/EntryConsumer.java | 38 -
.../apache/hadoop/hdds/utils/HddsVersionInfo.java | 3 +-
.../org/apache/hadoop/hdds/utils/VersionInfo.java | 8 +-
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 6 +
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 12 +-
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +
.../ozone/common/ChecksumByteBufferFactory.java | 3 +
.../ozone/common/ChecksumByteBufferImpl.java | 5 +-
.../org/apache/hadoop/ozone/common/Storage.java | 41 +-
.../apache/hadoop/ozone/common/StorageInfo.java | 33 +-
.../hadoop/ozone/common/utils/BufferUtils.java | 3 +
.../java/org/apache/hadoop/ozone/ha/ConfUtils.java | 3 +
.../upgrade/AbstractLayoutVersionManager.java | 162 +++
.../ozone/upgrade/BasicUpgradeFinalizer.java | 367 +++++++
.../DefaultUpgradeFinalizationExecutor.java | 66 ++
.../apache/hadoop/ozone/upgrade/LayoutFeature.java | 79 ++
.../upgrade/LayoutVersionInstanceFactory.java | 253 +++++
.../hadoop/ozone/upgrade/LayoutVersionManager.java | 76 ++
.../ozone/upgrade/LayoutVersionManagerMXBean.java | 28 +
.../hadoop/ozone/upgrade/UpgradeActionHdds.java | 51 +
.../hadoop/ozone/upgrade/UpgradeException.java | 114 +++
.../hadoop/ozone/upgrade/UpgradeFinalizer.java | 195 ++++
.../hadoop/ozone/upgrade/VersionFactoryKey.java | 70 ++
.../apache/hadoop/ozone/upgrade/package-info.java | 24 +
.../main/resources/hdds-version-info.properties | 3 +-
.../common/src/main/resources/ozone-default.xml | 189 ++--
.../java/org/apache/hadoop/hdds/TestHddsUtils.java | 2 +-
.../hdds/fs/TestCachingSpaceUsageSource.java | 2 +-
.../java/org/apache/hadoop/hdds/fs/TestDU.java | 2 +-
.../org/apache/hadoop/hdds/fs/TestDUFactory.java | 2 +-
.../hdds/fs/TestDedicatedDiskSpaceUsage.java | 2 +-
.../fs/TestDedicatedDiskSpaceUsageFactory.java | 2 +-
.../hadoop/hdds/fs/TestSaveSpaceUsageToFile.java | 4 +-
.../hadoop/hdds/fs/TestSpaceUsageFactory.java | 2 +-
.../apache/hadoop/hdds/scm/ha/TestSCMNodeInfo.java | 3 +
.../hadoop/hdds/tracing/TestStringCodec.java | 2 +-
.../hdds/upgrade/TestHDDSLayoutVersionManager.java | 102 ++
.../hadoop/hdds/upgrade/test/MockComponent.java | 67 ++
.../hadoop/hdds/utils/TestResourceLimitCache.java | 2 +-
.../common/TestChecksumImplsComputeSameValues.java | 4 +
.../apache/hadoop/ozone/lock/TestLockManager.java | 2 +-
.../InjectedUpgradeFinalizationExecutor.java | 128 +++
.../upgrade/TestAbstractLayoutVersionManager.java | 203 ++++
.../ozone/upgrade/TestBasicUpgradeFinalizer.java | 126 +++
.../TestDefaultUpgradeFinalizationExecutor.java | 99 ++
.../upgrade/TestLayoutVersionInstanceFactory.java | 198 ++++
.../ozone/upgrade/TestUpgradeFinalizerActions.java | 210 ++++
.../hadoop/ozone/upgrade/TestUpgradeUtils.java | 54 +
hadoop-hdds/config/pom.xml | 10 +-
.../hadoop/hdds/conf/ConfigFileGenerator.java | 17 +-
.../org/apache/hadoop/hdds/conf/ConfigTag.java | 3 +-
hadoop-hdds/container-service/pom.xml | 42 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 17 +-
.../container/common/DataNodeLayoutVersion.java | 80 --
.../container/common/DatanodeLayoutStorage.java | 98 ++
.../container/common/HDDSVolumeLayoutVersion.java | 80 ++
.../ozone/container/common/impl/ContainerData.java | 2 +
.../common/impl/StorageLocationReport.java | 55 +-
.../common/report/CRLStatusReportPublisher.java | 82 ++
.../container/common/report/ReportPublisher.java | 12 +-
.../common/report/ReportPublisherFactory.java | 2 +
.../common/statemachine/DatanodeConfiguration.java | 119 +++
.../common/statemachine/DatanodeStateMachine.java | 78 +-
.../common/statemachine/StateContext.java | 143 ++-
.../CloseContainerCommandHandler.java | 9 +-
.../FinalizeNewLayoutVersionCommandHandler.java | 121 +++
.../states/endpoint/HeartbeatEndpointTask.java | 74 +-
.../states/endpoint/RegisterEndpointTask.java | 60 +-
.../states/endpoint/VersionEndpointTask.java | 14 +-
.../server/ratis/ContainerStateMachine.java | 78 +-
.../transport/server/ratis/XceiverServerRatis.java | 54 +-
.../container/common/utils/HddsVolumeUtil.java | 17 +-
.../container/common/utils/StorageVolumeUtil.java | 51 +
.../ozone/container/common/volume/HddsVolume.java | 182 +---
.../container/common/volume/HddsVolumeChecker.java | 425 --------
.../container/common/volume/HddsVolumeFactory.java | 93 ++
.../common/volume/ImmutableVolumeSet.java | 8 +-
.../container/common/volume/MetadataVolume.java | 52 +
.../common/volume/MetadataVolumeFactory.java | 56 +
.../container/common/volume/MutableVolumeSet.java | 362 +++----
.../container/common/volume/StorageVolume.java | 214 ++++
.../common/volume/StorageVolumeChecker.java | 424 ++++++++
.../common/volume/StorageVolumeFactory.java | 60 ++
.../ozone/container/common/volume/VolumeInfo.java | 56 +-
.../ozone/container/common/volume/VolumeSet.java | 4 +-
.../ozone/container/common/volume/VolumeUsage.java | 30 +-
.../container/keyvalue/KeyValueContainer.java | 12 +-
.../container/keyvalue/KeyValueContainerCheck.java | 4 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 6 +-
.../container/keyvalue/helpers/BlockUtils.java | 4 +-
.../container/keyvalue/helpers/ChunkUtils.java | 14 +-
.../keyvalue/impl/ChunkManagerDummyImpl.java | 6 +-
.../keyvalue/impl/FilePerBlockStrategy.java | 7 +-
.../keyvalue/impl/FilePerChunkStrategy.java | 2 +
.../background/BlockDeletingService.java | 3 +
.../ozone/container/ozoneimpl/OzoneContainer.java | 54 +-
.../container/replication/ReplicationServer.java | 3 +
.../replication/ReplicationSupervisor.java | 3 +
.../container/stream/DirectoryServerSource.java | 6 +-
.../container/stream/DirstreamClientHandler.java | 5 +
.../container/stream/DirstreamServerHandler.java | 43 +-
.../ozone/container/stream/StreamingClient.java | 28 +-
.../ozone/container/stream/StreamingException.java | 38 +
.../ozone/container/stream/StreamingServer.java | 68 +-
.../upgrade/DataNodeUpgradeFinalizer.java | 95 ++
.../upgrade/DatanodeMetadataFeatures.java | 53 +
.../upgrade/DatanodeSchemaV2FinalizeAction.java | 48 +
.../ScmHAFirstUpgradeLayoutChangeAction.java | 45 +
.../ozone/container/upgrade/UpgradeUtils.java | 44 +
.../ozone/container/upgrade/package-info.java | 21 +
.../protocol/StorageContainerDatanodeProtocol.java | 6 +-
.../protocol/StorageContainerNodeProtocol.java | 12 +-
.../commands/FinalizeNewLayoutVersionCommand.java | 80 ++
.../hadoop/ozone/protocol/commands/SCMCommand.java | 10 +
...inerDatanodeProtocolClientSideTranslatorPB.java | 11 +-
...inerDatanodeProtocolServerSideTranslatorPB.java | 16 +-
.../metadata/TestDatanodeCRLStoreImpl.java | 2 +-
.../hadoop/ozone/TestHddsDatanodeService.java | 2 +-
.../hadoop/ozone/TestHddsSecureDatanodeInit.java | 4 +-
.../ozone/container/common/SCMTestUtils.java | 2 +-
.../hadoop/ozone/container/common/ScmTestMock.java | 11 +-
.../container/common/TestBlockDeletingService.java | 13 +-
.../common/TestDatanodeLayOutVersion.java | 10 +-
.../container/common/TestDatanodeStateMachine.java | 15 +-
.../TestSchemaOneBackwardsCompatibility.java | 17 +-
.../common/helpers/TestDatanodeVersionFile.java | 6 +-
.../common/impl/TestContainerDataYaml.java | 2 +-
.../impl/TestContainerDeletionChoosingPolicy.java | 2 +-
.../common/impl/TestContainerPersistence.java | 8 +-
.../container/common/impl/TestContainerSet.java | 2 +-
.../container/common/impl/TestHddsDispatcher.java | 14 +-
.../common/report/TestReportPublisher.java | 54 +-
.../common/report/TestReportPublisherFactory.java | 13 +-
.../statemachine/TestDatanodeConfiguration.java | 55 +
.../common/statemachine/TestStateContext.java | 138 +--
.../states/endpoint/TestHeartbeatEndpointTask.java | 26 +-
.../container/common/volume/TestHddsVolume.java | 116 +++
.../common/volume/TestHddsVolumeChecker.java | 319 ------
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 2 +-
.../common/volume/TestStorageVolumeChecker.java | 323 ++++++
.../container/common/volume/TestVolumeSet.java | 53 +-
.../common/volume/TestVolumeSetDiskChecks.java | 75 +-
.../keyvalue/TestKeyValueBlockIterator.java | 8 +-
.../container/keyvalue/TestKeyValueContainer.java | 11 +-
.../keyvalue/TestKeyValueContainerCheck.java | 8 +-
.../TestKeyValueContainerMarkUnhealthy.java | 2 +-
.../container/keyvalue/TestKeyValueHandler.java | 8 +-
.../container/keyvalue/TestTarContainerPacker.java | 2 +-
.../container/keyvalue/helpers/TestChunkUtils.java | 4 +-
.../keyvalue/impl/CommonChunkManagerTestCases.java | 35 +-
.../keyvalue/impl/TestBlockManagerImpl.java | 2 +-
.../container/ozoneimpl/TestContainerReader.java | 11 +-
.../container/ozoneimpl/TestOzoneContainer.java | 19 +-
.../replication/TestReplicationSupervisor.java | 2 +-
.../replication/TestSimpleContainerDownloader.java | 4 +-
.../stream/TestDirstreamClientHandler.java | 139 +++
.../container/stream/TestStreamingServer.java | 50 +-
.../ozone/container/stream/package-info.java | 20 +
.../upgrade/TestDataNodeStartupSlvLessThanMlv.java | 96 ++
hadoop-hdds/dev-support/checkstyle/checkstyle.xml | 3 +
.../docs/content/design/namespace-support.md | 13 +-
.../docs/content/design/nonrolling-upgrade.md | 32 +
hadoop-hdds/docs/content/design/omprepare.md | 60 ++
.../docs/content/design/upgrade-dev-primer.md | 111 ++
.../docs/content/feature/PrefixFSO-Delete.png | Bin 0 -> 253792 bytes
.../docs/content/feature/PrefixFSO-Format.png | Bin 0 -> 270550 bytes
.../docs/content/feature/PrefixFSO-Rename.png | Bin 0 -> 227851 bytes
hadoop-hdds/docs/content/feature/PrefixFSO.md | 75 ++
hadoop-hdds/docs/content/feature/SCM-HA.md | 68 +-
.../feature/how-to-do-a-nonrolling-upgrade.md | 54 +
hadoop-hdds/docs/content/feature/scm-secure-ha.png | Bin 0 -> 46757 bytes
hadoop-hdds/docs/content/interface/O3fs.md | 12 +-
hadoop-hdds/docs/content/interface/O3fs.zh.md | 16 +-
hadoop-hdds/docs/content/interface/Ofs.md | 4 +-
hadoop-hdds/docs/content/recipe/SparkOzoneFSK8S.md | 6 +-
.../docs/content/recipe/SparkOzoneFSK8S.zh.md | 6 +-
hadoop-hdds/docs/pom.xml | 6 +-
hadoop-hdds/framework/pom.xml | 34 +-
.../java/org/apache/hadoop/hdds/NodeDetails.java | 5 +-
...inerLocationProtocolClientSideTranslatorPB.java | 127 ++-
.../SCMBlockLocationFailoverProxyProvider.java | 8 +-
.../SCMSecurityProtocolFailoverProxyProvider.java | 8 +-
.../scm/update/client/CRLClientUpdateHandler.java | 201 ++++
.../hadoop/hdds/scm/update/client/CRLStore.java | 34 +
.../hdds/scm/update/client/ClientCRLStore.java | 98 ++
.../scm/update/client/ClientUpdateHandler.java | 29 +
.../client/SCMUpdateClientConfiguration.java | 53 +
.../update/client/SCMUpdateServiceGrpcClient.java | 218 ++++
.../scm/update/client/UpdateServiceConfig.java | 44 +
.../hdds/scm/update/client/package-info.java | 22 +
.../hdds/scm/update/server/CRLClientInfo.java | 56 +
.../scm/update/server/SCMUpdateClientInfo.java | 67 ++
.../hdds/scm/update/server/package-info.java | 22 +
.../hdds/security/token/BlockTokenVerifier.java | 8 +
.../security/token/ContainerTokenGenerator.java | 58 ++
.../token/ContainerTokenSecretManager.java | 30 +-
.../certificate/authority/CertificateStore.java | 6 +
.../authority/PKIProfiles/DefaultProfile.java | 2 +-
.../x509/certificate/client/CertificateClient.java | 26 +-
.../client/DefaultCertificateClient.java | 65 ++
.../certificate/client/OMCertificateClient.java | 11 +-
.../hadoop/hdds/security/x509/crl/CRLCodec.java | 0
.../hadoop/hdds/security/x509/crl/CRLInfo.java | 222 ++++
.../hdds/security/x509/crl/CRLInfoCodec.java | 0
.../hadoop/hdds/server/events/EventExecutor.java | 5 +
.../hadoop/hdds/server/events/EventQueue.java | 44 +-
.../server/events/FixedThreadPoolExecutor.java | 134 +++
.../hdds/server/events/SingleThreadExecutor.java | 9 +
.../hadoop/hdds/server/http/ProfileServlet.java | 2 +-
.../apache/hadoop/hdds/utils/HddsServerUtil.java | 25 +
.../org/apache/hadoop/hdds/utils/LevelDBStore.java | 400 --------
.../hadoop/hdds/utils/LevelDBStoreIterator.java | 64 --
.../apache/hadoop/hdds/utils/MetadataStore.java | 233 -----
.../hadoop/hdds/utils/MetadataStoreBuilder.java | 151 ---
.../org/apache/hadoop/hdds/utils/RocksDBStore.java | 397 --------
.../hadoop/hdds/utils/RocksDBStoreIterator.java | 66 --
.../apache/hadoop/hdds/utils/db/ShortCodec.java | 43 +
.../hadoop/hdds/utils/db/cache/TableCache.java | 3 +
.../token/TestOzoneBlockTokenIdentifier.java | 2 +-
.../x509/certificate/authority/MockCAStore.java | 12 +
.../certificate/authority/TestDefaultCAServer.java | 2 +-
.../client/TestCertificateClientInit.java | 2 +-
.../client/TestDefaultCertificateClient.java | 4 +-
.../security/x509/keys/TestHDDSKeyGenerator.java | 2 +-
.../hdds/security/x509/keys/TestKeyCodec.java | 2 +-
.../hadoop/hdds/server/events/TestEventQueue.java | 62 +-
.../hadoop/hdds/utils/TestMetadataStore.java | 592 -----------
.../hadoop/hdds/utils/TestRocksDBStoreMBean.java | 230 -----
.../hdds/utils/db/TestTypedRDBTableStore.java | 2 +-
.../hadoop/hdds/utils/db/cache/TestTableCache.java | 2 +-
hadoop-hdds/hadoop-dependency-client/pom.xml | 10 +-
hadoop-hdds/hadoop-dependency-server/pom.xml | 10 +-
hadoop-hdds/hadoop-dependency-test/pom.xml | 15 +-
hadoop-hdds/interface-admin/pom.xml | 12 +-
.../src/main/proto/ScmAdminProtocol.proto | 59 +-
hadoop-hdds/interface-client/pom.xml | 24 +-
.../interface-client/src/main/proto/hdds.proto | 24 +-
.../dev-support/findbugsExcludeFile.xml | 3 +
hadoop-hdds/interface-server/pom.xml | 68 +-
.../src/main/proto/InterSCMProtocol.proto | 0
.../src/main/proto/SCMRatisProtocol.proto | 2 +-
.../src/main/proto/SCMUpdateProtocol.proto | 98 ++
.../proto/ScmServerDatanodeHeartbeatProtocol.proto | 41 +
.../src/main/proto/ScmServerProtocol.proto | 4 +-
.../src/main/proto/ScmServerSecurityProtocol.proto | 1 +
hadoop-hdds/pom.xml | 78 +-
hadoop-hdds/server-scm/pom.xml | 80 +-
.../hadoop/hdds/scm/SCMCommonPlacementPolicy.java | 31 +-
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 89 +-
.../block/DatanodeDeletedBlockTransactions.java | 6 -
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 2 +-
.../hdds/scm/block/DeletedBlockLogImplV2.java | 3 +-
.../scm/block/DeletedBlockLogStateManager.java | 4 +
.../scm/block/DeletedBlockLogStateManagerImpl.java | 30 +-
.../hdds/scm/block/PendingDeleteHandler.java | 41 -
.../hdds/scm/block/PendingDeleteStatusList.java | 85 --
.../hdds/scm/block/SCMBlockDeletingService.java | 26 +-
.../scm/container/CloseContainerEventHandler.java | 9 +
.../hdds/scm/container/ContainerReportHandler.java | 38 -
.../hdds/scm/container/ContainerStateManager.java | 4 +-
.../scm/container/ContainerStateManagerImpl.java | 2 +-
.../hdds/scm/container/ReplicationManager.java | 264 ++---
.../scm/container/balancer/ContainerBalancer.java | 406 +++++++-
.../balancer/ContainerBalancerConfiguration.java | 80 +-
.../balancer/ContainerBalancerMetrics.java | 130 +++
.../scm/container/states/ContainerStateMap.java | 10 +-
.../hdds/scm/crl/CRLStatusReportHandler.java | 87 ++
.../apache/hadoop/hdds/scm/crl/package-info.java | 22 +
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 33 +-
.../apache/hadoop/hdds/scm/ha/HASecurityUtils.java | 7 +-
.../hdds/scm/ha/MockSCMHADBTransactionBuffer.java | 4 +
.../org/apache/hadoop/hdds/scm/ha/RatisUtil.java | 22 +
.../org/apache/hadoop/hdds/scm/ha/SCMContext.java | 79 +-
.../hdds/scm/ha/SCMDBCheckpointProvider.java | 4 +-
.../hadoop/hdds/scm/ha/SCMHAManagerImpl.java | 1 +
.../hadoop/hdds/scm/ha/SCMHANodeDetails.java | 8 +
.../hadoop/hdds/scm/ha/SCMRatisServerImpl.java | 8 +-
.../apache/hadoop/hdds/scm/ha/SCMStateMachine.java | 69 +-
.../hadoop/hdds/scm/ha/SequenceIdGenerator.java | 29 +-
.../apache/hadoop/hdds/scm/ha/io/BooleanCodec.java | 3 +
.../org/apache/hadoop/hdds/scm/ha/io/Codec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/CodecFactory.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/EnumCodec.java | 3 +
.../hdds/scm/ha/io/GeneratedMessageCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/ListCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/LongCodec.java | 3 +
.../apache/hadoop/hdds/scm/ha/io/StringCodec.java | 3 +
.../apache/hadoop/hdds/scm/node/DatanodeInfo.java | 57 +-
.../hadoop/hdds/scm/node/DatanodeUsageInfo.java | 3 +
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 3 +
.../hdds/scm/node/HealthyReadOnlyNodeHandler.java | 70 ++
.../hdds/scm/node/NodeDecommissionManager.java | 31 +-
.../apache/hadoop/hdds/scm/node/NodeManager.java | 39 +
.../hadoop/hdds/scm/node/NodeStateManager.java | 216 +++-
.../apache/hadoop/hdds/scm/node/NodeStatus.java | 11 +-
.../scm/node/NonHealthyToHealthyNodeHandler.java | 53 -
.../node/ReadOnlyHealthyToHealthyNodeHandler.java | 56 +
.../hadoop/hdds/scm/node/SCMNodeManager.java | 170 +++-
.../hadoop/hdds/scm/node/SCMNodeMetrics.java | 3 +-
.../hadoop/hdds/scm/node/states/NodeStateMap.java | 10 +-
.../scm/pipeline/BackgroundPipelineCreator.java | 15 +
.../scm/pipeline/BackgroundPipelineCreatorV2.java | 26 +-
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 12 +-
.../hdds/scm/pipeline/PipelineManagerV2Impl.java | 25 +
.../hdds/scm/pipeline/PipelinePlacementPolicy.java | 38 +-
.../hdds/scm/pipeline/PipelineReportHandler.java | 17 +-
.../scm/pipeline/PipelineStateManagerV2Impl.java | 10 +-
.../hdds/scm/pipeline/SCMPipelineManager.java | 35 +-
.../scm/pipeline/WritableContainerFactory.java | 58 ++
.../scm/pipeline/WritableContainerProvider.java | 57 ++
.../pipeline/WritableRatisContainerProvider.java | 158 +++
...inerLocationProtocolServerSideTranslatorPB.java | 133 +++
.../hdds/scm/safemode/ContainerSafeModeRule.java | 56 +-
.../hdds/scm/safemode/DataNodeSafeModeRule.java | 8 +
.../scm/safemode/HealthyPipelineSafeModeRule.java | 85 +-
.../safemode/OneReplicaPipelineSafeModeRule.java | 70 +-
.../hdds/scm/safemode/SCMSafeModeManager.java | 41 +-
.../hadoop/hdds/scm/safemode/SafeModeExitRule.java | 9 +
.../scm/server/OzoneStorageContainerManager.java | 3 +
.../hdds/scm/server/SCMBlockProtocolServer.java | 2 +
.../hadoop/hdds/scm/server/SCMCertStore.java | 23 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 125 ++-
.../hadoop/hdds/scm/server/SCMConfigurator.java | 20 +
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 46 +-
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 31 +-
.../apache/hadoop/hdds/scm/server/SCMMXBean.java | 10 +
.../hdds/scm/server/SCMSecurityProtocolServer.java | 25 +-
.../hadoop/hdds/scm/server/SCMStorageConfig.java | 9 +-
.../hdds/scm/server/StorageContainerManager.java | 265 +++--
.../scm/server/upgrade/SCMUpgradeFinalizer.java | 161 +++
.../ScmHAUnfinalizedStateValidationAction.java | 72 ++
.../ScmOnFinalizeActionForDatanodeSchemaV2.java | 46 +
.../hdds/scm/server/upgrade/package-info.java | 23 +
.../hadoop/hdds/scm/update/server/SCMCRLStore.java | 51 +
.../scm/update/server/SCMCRLUpdateHandler.java | 151 +++
.../scm/update/server/SCMUpdateClientManager.java | 149 +++
.../hdds/scm/update/server/SCMUpdateHandler.java | 53 +
.../update/server/SCMUpdateServiceGrpcServer.java | 99 ++
.../scm/update/server/SCMUpdateServiceImpl.java | 119 +++
.../hdds/scm/update/server/package-info.java | 22 +
.../main/resources/webapps/scm/scm-overview.html | 8 +
.../scm/TestStorageContainerManagerHttpServer.java | 2 +-
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 136 ++-
.../hadoop/hdds/scm/block/TestBlockManager.java | 4 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 107 +-
.../command/TestCommandStatusReportHandler.java | 2 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 122 ++-
.../hdds/scm/container/SimpleMockNodeManager.java | 40 +-
.../container/TestCloseContainerEventHandler.java | 2 +-
.../scm/container/TestContainerManagerImpl.java | 2 +-
.../scm/container/TestContainerReportHandler.java | 4 +-
.../scm/container/TestContainerStateManager.java | 2 +-
.../TestIncrementalContainerReportHandler.java | 16 +-
.../hdds/scm/container/TestReplicationManager.java | 51 +-
.../scm/container/TestSCMContainerManager.java | 2 +-
.../container/balancer/TestContainerBalancer.java | 236 +++++
.../algorithms/TestContainerPlacementFactory.java | 15 +-
.../TestSCMContainerPlacementCapacity.java | 20 +-
.../TestSCMContainerPlacementRackAware.java | 29 +-
.../TestSCMContainerPlacementRandom.java | 23 +-
.../hdds/scm/crl/TestCRLStatusReportHandler.java | 137 +++
.../apache/hadoop/hdds/scm/ha/TestSCMContext.java | 9 +-
.../hadoop/hdds/scm/ha/TestSCMHAConfiguration.java | 3 +
.../hadoop/hdds/scm/ha/TestSCMServiceManager.java | 7 +-
.../hdds/scm/ha/TestSequenceIDGenerator.java | 3 +
.../hdds/scm/node/TestContainerPlacement.java | 23 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 57 +-
.../hdds/scm/node/TestNodeDecommissionManager.java | 37 +-
.../hdds/scm/node/TestNodeReportHandler.java | 39 +-
.../hadoop/hdds/scm/node/TestNodeStateManager.java | 57 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 550 +++++++++-
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 13 +-
.../hadoop/hdds/scm/node/TestStatisticsUpdate.java | 33 +-
.../hdds/scm/node/states/TestNodeStateMap.java | 14 +-
.../hdds/scm/pipeline/MockPipelineManager.java | 10 +
.../hdds/scm/pipeline/TestPipelineManagerImpl.java | 7 +-
.../scm/pipeline/TestPipelinePlacementPolicy.java | 56 +
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 167 ++-
.../safemode/TestHealthyPipelineSafeModeRule.java | 8 +-
.../TestOneReplicaPipelineSafeModeRule.java | 4 +-
.../hdds/scm/safemode/TestSCMSafeModeManager.java | 37 +-
.../scm/server/TestSCMBlockProtocolServer.java | 2 +-
.../scm/server/TestSCMSecurityProtocolServer.java | 4 +-
.../hdds/scm/update/server/MockCRLStore.java | 138 +++
.../server/TestSCMUpdateServiceGrpcServer.java | 301 ++++++
.../TestSCMHAUnfinalizedStateValidationAction.java | 128 +++
.../scm/upgrade/TestScmStartupSlvLessThanMlv.java | 75 ++
.../hadoop/hdds/scm/upgrade/package-info.java | 21 +
.../ozone/container/common/TestEndPoint.java | 54 +-
.../testutils/ReplicationNodeManagerMock.java | 23 +-
.../hadoop/ozone/scm/node/TestSCMNodeMetrics.java | 63 +-
hadoop-hdds/test-utils/pom.xml | 6 +-
.../org/apache/hadoop/test/GenericTestUtils.java | 365 -------
.../org/apache/hadoop/test/LambdaTestUtils.java | 814 ---------------
.../apache/hadoop/test/TimedOutTestsListener.java | 183 ----
.../org/apache/ozone/test/GenericTestUtils.java | 365 +++++++
.../org/apache/ozone/test/LambdaTestUtils.java | 814 +++++++++++++++
.../apache/ozone/test/TimedOutTestsListener.java | 183 ++++
.../java/org/apache/ozone/test/package-info.java | 22 +
hadoop-hdds/tools/pom.xml | 38 +-
.../hdds/scm/cli/ContainerBalancerCommands.java | 108 ++
.../scm/cli/ContainerBalancerStartSubcommand.java | 66 ++
.../scm/cli/ContainerBalancerStatusSubcommand.java | 45 +
.../scm/cli/ContainerBalancerStopSubcommand.java | 40 +
.../hdds/scm/cli/ContainerOperationClient.java | 78 +-
.../org/apache/hadoop/hdds/scm/cli/ScmOption.java | 14 +-
.../datanode/TestContainerBalancerSubCommand.java | 141 +++
.../scm/cli/datanode/TestListInfoSubcommand.java | 17 +-
hadoop-ozone/client/pom.xml | 14 +-
.../apache/hadoop/ozone/client/OzoneBucket.java | 313 +++++-
.../org/apache/hadoop/ozone/client/OzoneKey.java | 41 +-
.../hadoop/ozone/client/OzoneKeyDetails.java | 20 +
.../hadoop/ozone/client/OzoneMultipartUpload.java | 34 +-
.../client/OzoneMultipartUploadPartListParts.java | 34 +-
.../client/io/BlockOutputStreamEntryPool.java | 7 +-
.../hadoop/ozone/client/io/KeyInputStream.java | 51 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 38 +-
.../client/io/MultipartCryptoKeyInputStream.java | 3 +
.../hadoop/ozone/client/io/OzoneInputStream.java | 15 +-
.../ozone/client/protocol/ClientProtocol.java | 5 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 47 +-
.../hadoop/ozone/client/MockOmTransport.java | 12 +
.../hadoop/ozone/client/TestOzoneClient.java | 36 +-
hadoop-ozone/common/pom.xml | 26 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 32 +
.../hadoop/ozone/client/io/LengthInputStream.java | 17 +-
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 25 +
.../apache/hadoop/ozone/om/OmUpgradeConfig.java | 82 ++
.../hadoop/ozone/om/exceptions/OMException.java | 10 +-
.../ozone/om/ha/OMFailoverProxyProvider.java | 12 +
.../hadoop/ozone/om/helpers/OmDirectoryInfo.java | 264 +++++
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 76 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 184 +++-
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 6 +-
.../ozone/om/helpers/OmMultipartKeyInfo.java | 110 +-
.../hadoop/ozone/om/helpers/OmMultipartUpload.java | 32 +-
.../om/helpers/OmMultipartUploadListParts.java | 25 +-
.../hadoop/ozone/om/helpers/OzoneAclUtil.java | 25 -
.../hadoop/ozone/om/helpers/OzoneFSUtils.java | 122 +++
.../ozone/om/helpers/WithParentObjectId.java | 55 +
.../ozone/om/protocol/OzoneManagerProtocol.java | 105 ++
...OzoneManagerProtocolClientSideTranslatorPB.java | 188 +++-
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 2 +
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 20 +-
.../hadoop/ozone/security/acl/OzonePrefixPath.java | 67 ++
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 3 +-
.../main/resources/ozone-version-info.properties | 3 +-
.../java/org/apache/hadoop/ozone/TestOmUtils.java | 31 +
.../org/apache/hadoop/ozone/TestOzoneAcls.java | 2 +-
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 10 +-
.../ozone/om/helpers/TestOmMultipartKeyInfo.java | 5 +-
hadoop-ozone/csi/pom.xml | 22 +-
hadoop-ozone/datanode/pom.xml | 14 +-
hadoop-ozone/dev-support/checks/coverage.sh | 3 +-
hadoop-ozone/dev-support/checks/integration.sh | 2 +-
hadoop-ozone/dev-support/checks/unit.sh | 2 +-
hadoop-ozone/dev-support/docker/Dockerfile | 2 +-
.../intellij/runConfigurations/CsiServer.xml | 4 +-
.../intellij/runConfigurations/Datanode1.xml | 2 +-
.../intellij/runConfigurations/Datanode2.xml | 2 +-
.../intellij/runConfigurations/Datanode3.xml | 2 +-
.../intellij/runConfigurations/FreonStandalone.xml | 4 +-
.../intellij/runConfigurations/OzoneFsShell.xml | 2 +-
.../intellij/runConfigurations/OzoneManager.xml | 2 +-
.../runConfigurations/OzoneManagerInit.xml | 2 +-
.../intellij/runConfigurations/OzoneShell.xml | 2 +-
.../intellij/runConfigurations/Recon.xml | 4 +-
.../intellij/runConfigurations/S3Gateway.xml | 4 +-
.../runConfigurations/StorageContainerManager.xml | 2 +-
.../StorageContainerManagerInit.xml | 2 +-
.../dist/dev-support/bin/dist-layout-stitching | 3 +
.../dist/dev-support/bin/dist-tar-stitching | 6 +-
hadoop-ozone/dist/pom.xml | 75 +-
.../docker-image/docker-krb5/Dockerfile-krb5 | 35 -
.../common/docker-image/docker-krb5/README.md | 34 -
.../common/docker-image/docker-krb5/kadm5.acl | 20 -
.../common/docker-image/docker-krb5/krb5.conf | 41 -
.../common/docker-image/docker-krb5/launcher.sh | 25 -
.../src/main/compose/compatibility/docker-config | 1 +
.../dist/src/main/compose/ozone-csi/docker-config | 1 +
.../dist/src/main/compose/ozone-ha/docker-config | 2 +
.../dist/src/main/compose/ozone-mr/common-config | 1 +
.../main/compose/ozone-mr/hadoop27/docker-config | 4 +-
.../main/compose/ozone-mr/hadoop31/docker-config | 4 +-
.../main/compose/ozone-mr/hadoop32/docker-config | 4 +-
.../dist/src/main/compose/ozone-mr/test.sh | 6 +-
.../src/main/compose/ozone-om-ha/docker-config | 1 +
.../dist/src/main/compose/ozone-om-prepare/.env | 25 +
.../compose/ozone-om-prepare/docker-compose.yaml | 125 +++
.../main/compose/ozone-om-prepare/docker-config | 35 +
.../dist/src/main/compose/ozone-om-prepare/test.sh | 74 ++
.../src/main/compose/ozone-topology/docker-config | 1 +
.../src/main/compose/ozone/docker-compose.yaml | 10 +
.../dist/src/main/compose/ozone/docker-config | 1 +
hadoop-ozone/dist/src/main/compose/ozone/test.sh | 28 +-
.../src/main/compose/ozones3-haproxy/docker-config | 1 +
.../dist/src/main/compose/ozonesecure-ha/.env | 1 +
.../compose/ozonesecure-ha/docker-compose.yaml | 44 +-
.../src/main/compose/ozonesecure-ha/docker-config | 25 +-
.../compose/ozonesecure-ha/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../main/compose/ozonesecure-ha/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure-ha/keytabs/om.keytab | Bin 0 -> 278 bytes
.../compose/ozonesecure-ha/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure-ha/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure-ha/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure-ha/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../ozonesecure-ha/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure-ha/krb5.conf | 41 +
.../dist/src/main/compose/ozonesecure-mr/.env | 1 +
.../dist/src/main/compose/ozonesecure-mr/README.md | 2 +-
.../compose/ozonesecure-mr/docker-compose.yaml | 30 +-
.../src/main/compose/ozonesecure-mr/docker-config | 27 +-
.../compose/ozonesecure-mr/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../main/compose/ozonesecure-mr/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure-mr/keytabs/om.keytab | Bin 0 -> 278 bytes
.../compose/ozonesecure-mr/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure-mr/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure-mr/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure-mr/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../ozonesecure-mr/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure-mr/krb5.conf | 39 +
.../dist/src/main/compose/ozonesecure/.env | 1 +
.../main/compose/ozonesecure/docker-compose.yaml | 23 +-
.../src/main/compose/ozonesecure/docker-config | 24 +-
.../main/compose/ozonesecure/keytabs/HTTP.keytab | Bin 0 -> 144 bytes
.../src/main/compose/ozonesecure/keytabs/dn.keytab | Bin 0 -> 278 bytes
.../src/main/compose/ozonesecure/keytabs/om.keytab | Bin 0 -> 278 bytes
.../main/compose/ozonesecure/keytabs/recon.keytab | Bin 0 -> 296 bytes
.../main/compose/ozonesecure/keytabs/s3g.keytab | Bin 0 -> 434 bytes
.../main/compose/ozonesecure/keytabs/scm.keytab | Bin 0 -> 586 bytes
.../compose/ozonesecure/keytabs/testuser.keytab | Bin 0 -> 152 bytes
.../compose/ozonesecure/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/compose/ozonesecure/krb5.conf | 41 +
.../dist/src/main/compose/restart/docker-config | 1 +
hadoop-ozone/dist/src/main/compose/test-all.sh | 8 +-
hadoop-ozone/dist/src/main/compose/testlib.sh | 42 +-
hadoop-ozone/dist/src/main/compose/upgrade/.env | 21 -
.../dist/src/main/compose/upgrade/0.5.0/test.sh | 24 -
.../dist/src/main/compose/upgrade/1.0.0/test.sh | 24 -
.../dist/src/main/compose/upgrade/README.md | 95 +-
.../dist/src/main/compose/upgrade/compose/ha/.env | 26 +
.../compose/upgrade/compose/ha/docker-compose.yaml | 160 +++
.../main/compose/upgrade/compose/ha/docker-config | 43 +
.../src/main/compose/upgrade/compose/ha/load.sh | 28 +
.../src/main/compose/upgrade/compose/non-ha/.env | 21 +
.../upgrade/compose/non-ha/docker-compose.yaml | 137 +++
.../compose/upgrade/compose/non-ha/docker-config | 42 +
.../main/compose/upgrade/compose/non-ha/load.sh | 27 +
.../src/main/compose/upgrade/docker-compose.yaml | 134 ---
.../dist/src/main/compose/upgrade/docker-config | 38 -
hadoop-ozone/dist/src/main/compose/upgrade/test.sh | 34 +-
.../dist/src/main/compose/upgrade/testlib.sh | 135 ++-
.../src/main/compose/upgrade/upgrade_to_current.sh | 51 -
.../src/main/compose/upgrade/upgrade_to_release.sh | 51 -
.../manual-upgrade/0.5.0-1.1.0/callback.sh | 40 +
.../upgrade/upgrades/manual-upgrade/driver.sh | 56 +
.../non-rolling-upgrade/1.0.0-1.1.0/callback.sh | 79 ++
.../upgrade/upgrades/non-rolling-upgrade/driver.sh | 101 ++
.../dist/src/main/compose/versions/0.5.0.sh | 4 -
.../dist/src/main/compose/versions/1.0.0.sh | 4 -
.../dist/src/main/compose/versions/1.1.0.sh | 4 -
.../dist/src/main/compose/versions/README.md | 3 -
.../dist/src/main/compose/xcompat/docker-config | 1 +
.../definitions/ozone/definitions/persistence.yaml | 4 +-
.../examples/ozone-ha/datanode-statefulset.yaml | 2 +-
.../k8s/examples/ozone/datanode-statefulset.yaml | 2 +-
hadoop-ozone/dist/src/main/keytabs/HTTP.keytab | Bin 0 -> 432 bytes
hadoop-ozone/dist/src/main/keytabs/dn.keytab | Bin 0 -> 278 bytes
hadoop-ozone/dist/src/main/keytabs/hadoop.keytab | Bin 0 -> 146 bytes
hadoop-ozone/dist/src/main/keytabs/jhs.keytab | Bin 0 -> 142 bytes
hadoop-ozone/dist/src/main/keytabs/nm.keytab | Bin 0 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/om.keytab | Bin 0 -> 278 bytes
hadoop-ozone/dist/src/main/keytabs/recon.keytab | Bin 0 -> 296 bytes
hadoop-ozone/dist/src/main/keytabs/rm.keytab | Bin 0 -> 138 bytes
hadoop-ozone/dist/src/main/keytabs/s3g.keytab | Bin 0 -> 434 bytes
hadoop-ozone/dist/src/main/keytabs/scm.keytab | Bin 0 -> 586 bytes
hadoop-ozone/dist/src/main/keytabs/testuser.keytab | Bin 0 -> 450 bytes
.../dist/src/main/keytabs/testuser2.keytab | Bin 0 -> 154 bytes
.../dist/src/main/keytabs/update-keytabs.sh | 77 ++
hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 11 +-
.../license/bin/licenses/LICENSE-org.aspectj.html | 91 ++
.../LICENSE-org.reflections-reflections.txt | 14 +
hadoop-ozone/dist/src/main/license/jar-report.txt | 92 +-
.../dist/src/main/license/update-jar-report.sh | 2 +-
.../src/main/smoketest/admincli/container.robot | 11 +-
.../dist/src/main/smoketest/basic/links.robot | 10 +-
.../src/main/smoketest/basic/ozone-shell-lib.robot | 6 +-
.../dist/src/main/smoketest/cli/classpath.robot | 14 +-
.../dist/src/main/smoketest/commonlib.robot | 8 +-
.../main/smoketest/omha/om-cancel-prepare.robot | 43 +
.../dist/src/main/smoketest/omha/om-prepare.robot | 49 +
.../dist/src/main/smoketest/omha/om-prepared.robot | 36 +
.../src/main/smoketest/ozonefs/hadoopo3fs.robot | 2 +
.../dist/src/main/smoketest/ozonefs/setup.robot | 10 +-
.../main/smoketest/security/ozone-secure-fs.robot | 18 +-
.../dist/src/main/smoketest/spnego/web.robot | 2 +-
.../src/main/smoketest/upgrade/check-mlv.robot | 41 +
.../dist/src/main/smoketest/upgrade/finalize.robot | 31 +
.../dist/src/main/smoketest/upgrade/generate.robot | 33 +
.../dist/src/main/smoketest/upgrade/prepare.robot | 26 +
.../dist/src/main/smoketest/upgrade/validate.robot | 30 +
hadoop-ozone/dist/src/shell/ozone/ozone | 42 +-
hadoop-ozone/dist/src/shell/upgrade/1.0.0.sh | 23 +
.../upgrade/{v1 => 1.0.0}/01-migrate-scm-db.sh | 0
hadoop-ozone/dist/src/shell/upgrade/v1.sh | 23 -
.../dist/src/test/shell/compose_testlib.bats | 13 +
hadoop-ozone/dist/src/test/shell/failing1/test.sh | 17 +
.../fault-injection-test/mini-chaos-tests/pom.xml | 24 +-
.../apache/hadoop/ozone/MiniOzoneChaosCluster.java | 5 +-
.../fault-injection-test/network-tests/pom.xml | 6 +-
hadoop-ozone/fault-injection-test/pom.xml | 6 +-
hadoop-ozone/insight/pom.xml | 42 +-
.../hadoop/ozone/insight/BaseInsightPoint.java | 4 +-
hadoop-ozone/integration-test/pom.xml | 73 +-
.../ozone/TestDirectoryDeletingServiceWithFSO.java | 402 ++++++++
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 18 +-
.../fs/ozone/TestOzoneFileInterfacesWithFSO.java | 96 ++
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 511 +++++++++-
.../fs/ozone/TestOzoneFileSystemMissingParent.java | 6 +-
.../fs/ozone/TestOzoneFileSystemPrefixParser.java | 168 +++
.../fs/ozone/TestOzoneFileSystemWithFSO.java | 479 +++++++++
.../apache/hadoop/fs/ozone/TestOzoneFsHAURLs.java | 2 +-
.../hadoop/fs/ozone/TestRootedOzoneFileSystem.java | 234 ++++-
.../fs/ozone/TestRootedOzoneFileSystemWithFSO.java | 160 +++
.../ozone/contract/ITestOzoneContractCreate.java | 26 +-
.../ozone/contract/ITestOzoneContractDelete.java | 26 +-
.../contract/ITestOzoneContractGetFileStatus.java | 27 +-
.../fs/ozone/contract/ITestOzoneContractMkdir.java | 26 +-
.../fs/ozone/contract/ITestOzoneContractOpen.java | 27 +-
.../ozone/contract/ITestOzoneContractRename.java | 26 +-
.../ozone/contract/ITestOzoneContractRootDir.java | 26 +-
.../fs/ozone/contract/ITestOzoneContractSeek.java | 1 +
.../fs/ozone/contract/ITestOzoneContractUtils.java | 61 ++
.../hadoop/fs/ozone/contract/OzoneContract.java | 13 +
.../hadoop/hdds/scm/TestRatisPipelineLeader.java | 2 +-
.../hadoop/hdds/scm/TestSCMInstallSnapshot.java | 3 +-
.../apache/hadoop/hdds/scm/TestSCMSnapshot.java | 4 +
.../TestContainerStateManagerIntegration.java | 7 +-
.../metrics/TestSCMContainerManagerMetrics.java | 2 +-
.../hdds/scm/pipeline/TestLeaderChoosePolicy.java | 4 +-
.../hadoop/hdds/scm/pipeline/TestNodeFailure.java | 2 +-
.../hdds/scm/pipeline/TestPipelineClose.java | 2 +-
.../TestRatisPipelineCreateAndDestroy.java | 2 +-
.../safemode/TestSCMSafeModeWithPipelineRules.java | 2 +-
.../hadoop/hdds/upgrade/TestHDDSUpgrade.java | 1067 ++++++++++++++++++++
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 22 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 61 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 47 +-
.../hadoop/ozone/MiniOzoneOMHAClusterImpl.java | 2 +-
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 2 +-
.../ozone/TestContainerBalancerOperations.java | 112 ++
.../apache/hadoop/ozone/TestDelegationToken.java | 6 +-
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 8 +-
.../hadoop/ozone/TestMiniOzoneOMHACluster.java | 2 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 2 +
.../hadoop/ozone/TestSecureOzoneCluster.java | 6 +-
.../hadoop/ozone/TestStorageContainerManager.java | 38 +-
.../ozone/client/CertificateClientTestImpl.java | 14 +
.../ozone/client/rpc/Test2WayCommitInRatis.java | 2 +-
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 9 +-
.../rpc/TestCloseContainerHandlingByClient.java | 39 +-
.../rpc/TestContainerReplicationEndToEnd.java | 2 +-
.../client/rpc/TestContainerStateMachine.java | 2 +-
.../TestContainerStateMachineFailureOnRead.java | 2 +-
.../rpc/TestContainerStateMachineFailures.java | 2 +-
.../rpc/TestContainerStateMachineFlushDelay.java | 2 +-
.../client/rpc/TestDeleteWithSlowFollower.java | 10 +-
.../client/rpc/TestFailureHandlingByClient.java | 28 +-
.../rpc/TestFailureHandlingByClientFlushDelay.java | 7 +-
.../rpc/TestMultiBlockWritesWithDnFailures.java | 13 +-
.../client/rpc/TestOzoneAtRestEncryption.java | 6 +-
.../rpc/TestOzoneClientMultipartUploadWithFSO.java | 1024 +++++++++++++++++++
.../client/rpc/TestOzoneRpcClientAbstract.java | 8 +-
.../rpc/TestOzoneRpcClientForAclAuditLog.java | 2 +-
.../hadoop/ozone/client/rpc/TestReadRetries.java | 60 +-
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 8 +-
.../ozone/client/rpc/TestWatchForCommit.java | 2 +-
.../ozone/client/rpc/read/TestInputStreamBase.java | 5 +-
.../ozone/client/rpc/read/TestKeyInputStream.java | 71 +-
.../ozone/container/TestContainerReplication.java | 6 +-
.../apache/hadoop/ozone/container/TestHelper.java | 6 +-
.../commandhandler/TestBlockDeletion.java | 59 +-
.../TestCloseContainerByPipeline.java | 23 +-
.../commandhandler/TestCloseContainerHandler.java | 13 +-
.../commandhandler/TestDeleteContainerHandler.java | 12 +-
.../transport/server/ratis/TestCSMMetrics.java | 2 +-
.../container/metrics/TestContainerMetrics.java | 13 +-
.../ozoneimpl/TestOzoneContainerWithTLS.java | 2 +-
.../ozoneimpl/TestSecureOzoneContainer.java | 2 +-
.../hadoop/ozone/container/package-info.java | 22 +
.../container/server/TestContainerServer.java | 2 +-
.../server/TestSecureContainerServer.java | 8 +-
.../apache/hadoop/ozone/dn/DatanodeTestUtils.java | 40 +-
.../ozone/dn/TestDatanodeLayoutUpgradeTool.java | 4 +-
.../ozone/dn/ratis/TestDnRatisLogParser.java | 2 +-
.../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 4 +-
.../TestDatanodeHddsVolumeFailureDetection.java | 28 +-
.../TestDatanodeHddsVolumeFailureToleration.java | 135 +++
.../ozone/freon/TestHadoopDirTreeGenerator.java | 24 +-
.../freon/TestHadoopDirTreeGeneratorWithFSO.java | 36 +
.../ozone/freon/TestHadoopNestedDirGenerator.java | 4 +-
.../hadoop/ozone/fsck/TestContainerMapper.java | 7 +-
.../ozone/om/TestContainerReportWithKeys.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 136 ++-
.../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +-
.../hadoop/ozone/om/TestOMRatisSnapshots.java | 2 +-
.../hadoop/ozone/om/TestOMStartupWithLayout.java | 197 ++++
.../hadoop/ozone/om/TestOMUpgradeFinalization.java | 254 +++++
.../hadoop/ozone/om/TestObjectStoreWithFSO.java | 722 +++++++++++++
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 3 +-
.../hadoop/ozone/om/TestOmBlockVersioning.java | 4 +
.../ozone/om/TestOmStartupSlvLessThanMlv.java | 80 ++
.../ozone/om/TestOzoneManagerConfiguration.java | 2 +-
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 2 +-
.../ozone/om/TestOzoneManagerHAKeyDeletion.java | 5 +-
.../ozone/om/TestOzoneManagerHAMetadataOnly.java | 2 +-
.../ozone/om/TestOzoneManagerHAWithData.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerPrepare.java | 510 ++++++++++
.../hadoop/ozone/om/TestOzoneManagerRestart.java | 2 +-
.../ozone/om/TestOzoneManagerRocksDBLogging.java | 2 +-
.../hadoop/ozone/om/TestRecursiveAclWithFSO.java | 338 +++++++
.../apache/hadoop/ozone/om/TestScmSafeMode.java | 4 +-
.../hadoop/ozone/om/TestSecureOzoneManager.java | 4 +-
.../ozone/om/parser/TestOMRatisLogParser.java | 2 +-
.../hadoop/ozone/recon/TestReconAsPassiveScm.java | 5 +-
.../apache/hadoop/ozone/recon/TestReconTasks.java | 2 +-
.../ozone/recon/TestReconWithOzoneManager.java | 5 +-
.../ozone/recon/TestReconWithOzoneManagerHA.java | 10 +-
.../hadoop/ozone/scm/TestCloseContainer.java | 2 +-
.../hadoop/ozone/scm/TestFailoverWithSCMHA.java | 2 +-
.../ozone/scm/TestSCMInstallSnapshotWithHA.java | 3 +-
.../ozone/scm/TestStorageContainerManagerHA.java | 10 +-
.../hadoop/ozone/scm/TestXceiverClientManager.java | 2 +-
.../hadoop/ozone/scm/TestXceiverClientMetrics.java | 2 +-
.../scm/node/TestDecommissionAndMaintenance.java | 2 +-
.../hadoop/ozone/scm/node/TestQueryNode.java | 2 +-
.../TestSCMPipelineBytesWrittenMetrics.java | 2 +-
.../hadoop/ozone/shell/TestOzoneShellHA.java | 104 +-
hadoop-ozone/interface-client/pom.xml | 12 +-
.../src/main/proto/OmClientProtocol.proto | 114 +++
.../interface-client/src/main/proto/Security.proto | 1 +
hadoop-ozone/interface-storage/pom.xml | 32 +-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 52 +
.../ozone/om/codec/OmDirectoryInfoCodec.java | 60 ++
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 8 +
.../hadoop/ozone/om/codec/TestOmKeyInfoCodec.java | 5 +-
.../om/codec/TestOmMultipartKeyInfoCodec.java | 7 +-
.../ozone/om/codec/TestOmPrefixInfoCodec.java | 2 +-
.../ozone/om/codec/TestRepeatedOmKeyInfoCodec.java | 5 +-
.../ozone/om/codec/TestS3SecretValueCodec.java | 2 +-
.../ozone/om/codec/TestTransactionInfoCodec.java | 2 +-
hadoop-ozone/native-client/README.md | 4 +-
.../dev-support/findbugsExcludeFile.xml | 4 +
hadoop-ozone/ozone-manager/pom.xml | 103 +-
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 7 +-
.../hadoop/ozone/om/DirectoryDeletingService.java | 286 ++++++
.../org/apache/hadoop/ozone/om/KeyManager.java | 56 +
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 900 +++++++++++++++--
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 18 +-
.../apache/hadoop/ozone/om/OMStarterInterface.java | 2 +
.../java/org/apache/hadoop/ozone/om/OMStorage.java | 7 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 141 ++-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 362 ++++++-
.../hadoop/ozone/om/OzoneManagerPrepareState.java | 311 ++++++
.../hadoop/ozone/om/OzoneManagerStarter.java | 32 +-
.../hadoop/ozone/om/OzonePrefixPathImpl.java | 164 +++
.../java/org/apache/hadoop/ozone/om/ScmClient.java | 11 +
.../hadoop/ozone/om/TrashOzoneFileSystem.java | 146 ++-
.../apache/hadoop/ozone/om/TrashPolicyOzone.java | 40 +-
.../hadoop/ozone/om/codec/OMDBDefinition.java | 45 +-
.../ozone/om/ratis/OzoneManagerDoubleBuffer.java | 15 +-
.../ozone/om/ratis/OzoneManagerRatisServer.java | 28 +-
.../ozone/om/ratis/OzoneManagerStateMachine.java | 57 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 94 ++
.../hadoop/ozone/om/request/OMClientRequest.java | 55 +-
.../om/request/bucket/OMBucketCreateRequest.java | 34 +
.../om/request/bucket/OMBucketDeleteRequest.java | 1 +
.../request/bucket/OMBucketSetPropertyRequest.java | 20 +-
.../request/bucket/acl/OMBucketAddAclRequest.java | 5 +-
.../bucket/acl/OMBucketRemoveAclRequest.java | 1 +
.../om/request/file/OMDirectoryCreateRequest.java | 10 +-
.../file/OMDirectoryCreateRequestWithFSO.java | 314 ++++++
.../ozone/om/request/file/OMFileCreateRequest.java | 62 +-
.../request/file/OMFileCreateRequestWithFSO.java | 261 +++++
.../ozone/om/request/file/OMFileRequest.java | 745 ++++++++++++++
.../om/request/key/OMAllocateBlockRequest.java | 2 +-
.../request/key/OMAllocateBlockRequestWithFSO.java | 226 +++++
.../ozone/om/request/key/OMKeyCommitRequest.java | 37 +-
.../om/request/key/OMKeyCommitRequestWithFSO.java | 190 ++++
.../ozone/om/request/key/OMKeyCreateRequest.java | 24 +-
.../om/request/key/OMKeyCreateRequestWithFSO.java | 232 +++++
.../ozone/om/request/key/OMKeyDeleteRequest.java | 1 +
.../om/request/key/OMKeyDeleteRequestWithFSO.java | 205 ++++
.../ozone/om/request/key/OMKeyPurgeRequest.java | 1 +
.../ozone/om/request/key/OMKeyRenameRequest.java | 1 +
.../om/request/key/OMKeyRenameRequestWithFSO.java | 301 ++++++
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 285 +++---
.../ozone/om/request/key/OMKeysDeleteRequest.java | 1 -
.../om/request/key/OMPathsPurgeRequestWithFSO.java | 64 ++
.../om/request/key/acl/OMKeyAclRequestWithFSO.java | 167 +++
.../om/request/key/acl/OMKeyAddAclRequest.java | 1 +
.../request/key/acl/OMKeyAddAclRequestWithFSO.java | 151 +++
.../om/request/key/acl/OMKeyRemoveAclRequest.java | 1 +
.../key/acl/OMKeyRemoveAclRequestWithFSO.java | 161 +++
.../om/request/key/acl/OMKeySetAclRequest.java | 1 +
.../request/key/acl/OMKeySetAclRequestWithFSO.java | 155 +++
.../key/acl/prefix/OMPrefixAddAclRequest.java | 1 +
.../key/acl/prefix/OMPrefixRemoveAclRequest.java | 1 +
.../key/acl/prefix/OMPrefixSetAclRequest.java | 1 +
.../S3InitiateMultipartUploadRequest.java | 25 +-
.../S3InitiateMultipartUploadRequestWithFSO.java | 251 +++++
.../multipart/S3MultipartUploadAbortRequest.java | 58 +-
.../S3MultipartUploadAbortRequestWithFSO.java | 89 ++
.../S3MultipartUploadCommitPartRequest.java | 76 +-
.../S3MultipartUploadCommitPartRequestWithFSO.java | 87 ++
.../S3MultipartUploadCompleteRequest.java | 405 +++++---
.../S3MultipartUploadCompleteRequestWithFSO.java | 169 ++++
.../om/request/s3/security/S3GetSecretRequest.java | 1 +
.../security/OMCancelDelegationTokenRequest.java | 2 +-
.../security/OMGetDelegationTokenRequest.java | 2 +-
.../security/OMRenewDelegationTokenRequest.java | 1 +
.../om/request/upgrade/OMCancelPrepareRequest.java | 97 ++
.../request/upgrade/OMFinalizeUpgradeRequest.java | 116 +++
.../ozone/om/request/upgrade/OMPrepareRequest.java | 257 +++++
.../ozone/om/request/upgrade/package-info.java | 23 +
.../om/request/volume/OMVolumeDeleteRequest.java | 2 +
.../om/request/volume/OMVolumeSetOwnerRequest.java | 1 +
.../om/request/volume/OMVolumeSetQuotaRequest.java | 1 +
.../hadoop/ozone/om/response/CleanupTableInfo.java | 10 +-
.../file/OMDirectoryCreateResponseWithFSO.java | 103 ++
.../response/file/OMFileCreateResponseWithFSO.java | 94 ++
.../response/key/AbstractOMKeyDeleteResponse.java | 44 +
.../om/response/key/OMAllocateBlockResponse.java | 12 +
.../key/OMAllocateBlockResponseWithFSO.java | 67 ++
.../ozone/om/response/key/OMKeyCommitResponse.java | 15 +
.../response/key/OMKeyCommitResponseWithFSO.java | 75 ++
.../ozone/om/response/key/OMKeyCreateResponse.java | 14 +-
.../response/key/OMKeyCreateResponseWithFSO.java | 56 +
.../ozone/om/response/key/OMKeyDeleteResponse.java | 8 +
.../response/key/OMKeyDeleteResponseWithFSO.java | 100 ++
.../ozone/om/response/key/OMKeyRenameResponse.java | 11 +
.../response/key/OMKeyRenameResponseWithFSO.java | 77 ++
.../om/response/key/OMOpenKeysDeleteRequest.java | 1 +
.../response/key/OMPathsPurgeResponseWithFSO.java | 125 +++
.../om/response/key/acl/OMKeyAclResponse.java | 3 +
.../response/key/acl/OMKeyAclResponseWithFSO.java | 75 ++
.../S3InitiateMultipartUploadResponseWithFSO.java | 90 ++
.../multipart/S3MultipartUploadAbortResponse.java | 9 +-
.../S3MultipartUploadAbortResponseWithFSO.java | 57 ++
.../S3MultipartUploadCommitPartResponse.java | 3 +
...S3MultipartUploadCommitPartResponseWithFSO.java | 69 ++
.../S3MultipartUploadCompleteResponse.java | 41 +-
.../S3MultipartUploadCompleteResponseWithFSO.java | 85 ++
.../om/response/s3/security/package-info.java | 2 +-
.../response/upgrade/OMCancelPrepareResponse.java | 46 +
.../upgrade/OMFinalizeUpgradeResponse.java | 58 ++
.../om/response/upgrade/OMPrepareResponse.java | 60 ++
.../ozone/om/response/upgrade/package-info.java | 23 +
.../ozone/om/upgrade/BelongsToLayoutVersion.java | 33 +
.../om/upgrade/DisallowedUntilLayoutVersion.java | 35 +
.../hadoop/ozone/om/upgrade/OMLayoutFeature.java | 80 ++
.../ozone/om/upgrade/OMLayoutFeatureAspect.java | 114 +++
.../ozone/om/upgrade/OMLayoutVersionManager.java | 210 ++++
.../ozone/om/upgrade/OMUpgradeFinalizer.java | 50 +
.../hadoop/ozone/om/upgrade/OmUpgradeAction.java | 28 +
.../hadoop/ozone/om/upgrade/UpgradeActionOm.java | 37 +
.../hadoop/ozone/om/upgrade/package-info.java | 23 +
...OzoneManagerProtocolServerSideTranslatorPB.java | 13 +-
.../protocolPB/OzoneManagerRequestHandler.java | 80 +-
.../ozone/security/acl/OzoneNativeAuthorizer.java | 35 +-
.../hadoop/ozone/om/TestBucketManagerImpl.java | 10 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 6 +-
.../apache/hadoop/ozone/om/TestKeyManagerUnit.java | 21 +-
.../org/apache/hadoop/ozone/om/TestOMStorage.java | 2 +-
.../hadoop/ozone/om/TestOmMetadataManager.java | 9 +
.../ozone/om/TestOzoneManagerHttpServer.java | 2 +-
.../hadoop/ozone/om/TestOzoneManagerStarter.java | 28 +
.../apache/hadoop/ozone/om/TestTrashService.java | 4 +
.../hadoop/ozone/om/failover/TestOMFailovers.java | 2 +-
...tOzoneManagerDoubleBufferWithDummyResponse.java | 2 +-
...TestOzoneManagerDoubleBufferWithOMResponse.java | 7 +-
.../om/ratis/TestOzoneManagerRatisServer.java | 2 +-
.../om/ratis/TestOzoneManagerStateMachine.java | 108 +-
.../ozone/om/request/TestOMRequestUtils.java | 200 +++-
.../ozone/om/request/bucket/TestBucketRequest.java | 6 +
.../request/bucket/TestOMBucketCreateRequest.java | 4 +-
.../bucket/TestOMBucketCreateRequestWithFSO.java | 115 +++
.../bucket/TestOMBucketSetPropertyRequest.java | 12 +-
.../file/TestOMDirectoryCreateRequestWithFSO.java | 657 ++++++++++++
.../om/request/file/TestOMFileCreateRequest.java | 69 +-
.../file/TestOMFileCreateRequestWithFSO.java | 197 ++++
.../om/request/key/TestOMAllocateBlockRequest.java | 44 +-
.../key/TestOMAllocateBlockRequestWithFSO.java | 120 +++
.../ozone/om/request/key/TestOMKeyAclRequest.java | 81 +-
.../om/request/key/TestOMKeyAclRequestWithFSO.java | 81 ++
.../om/request/key/TestOMKeyCommitRequest.java | 86 +-
.../request/key/TestOMKeyCommitRequestWithFSO.java | 114 +++
.../om/request/key/TestOMKeyCreateRequest.java | 53 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 135 +++
.../om/request/key/TestOMKeyDeleteRequest.java | 40 +-
.../request/key/TestOMKeyDeleteRequestWithFSO.java | 139 +++
.../ozone/om/request/key/TestOMKeyRequest.java | 51 +-
.../TestS3InitiateMultipartUploadRequest.java | 24 +-
...estS3InitiateMultipartUploadRequestWithFSO.java | 145 +++
.../s3/multipart/TestS3MultipartRequest.java | 60 +-
.../TestS3MultipartUploadAbortRequest.java | 32 +-
.../TestS3MultipartUploadAbortRequestWithFSO.java | 69 ++
.../TestS3MultipartUploadCommitPartRequest.java | 70 +-
...tS3MultipartUploadCommitPartRequestWithFSO.java | 118 +++
.../TestS3MultipartUploadCompleteRequest.java | 118 ++-
...estS3MultipartUploadCompleteRequestWithFSO.java | 139 +++
.../upgrade/TestOMCancelPrepareRequest.java | 110 ++
.../request/volume/TestOMVolumeCreateRequest.java | 4 +-
.../request/volume/TestOMVolumeDeleteRequest.java | 2 +-
.../om/request/volume/TestOMVolumeRequest.java | 9 +-
.../volume/TestOMVolumeSetQuotaRequest.java | 2 +-
.../ozone/om/response/TestCleanupTableInfo.java | 26 +-
.../file/TestOMDirectoryCreateResponseWithFSO.java | 87 ++
.../file/TestOMFileCreateResponseWithFSO.java | 78 ++
.../response/key/TestOMAllocateBlockResponse.java | 37 +-
.../key/TestOMAllocateBlockResponseWithFSO.java | 85 ++
.../om/response/key/TestOMKeyCommitResponse.java | 64 +-
.../key/TestOMKeyCommitResponseWithFSO.java | 102 ++
.../om/response/key/TestOMKeyCreateResponse.java | 38 +-
.../key/TestOMKeyCreateResponseWithFSO.java | 76 ++
.../om/response/key/TestOMKeyDeleteResponse.java | 88 +-
.../key/TestOMKeyDeleteResponseWithFSO.java | 84 ++
.../ozone/om/response/key/TestOMKeyResponse.java | 25 +-
.../TestS3InitiateMultipartUploadResponse.java | 2 +-
...stS3InitiateMultipartUploadResponseWithFSO.java | 85 ++
.../s3/multipart/TestS3MultipartResponse.java | 191 +++-
.../TestS3MultipartUploadAbortResponse.java | 23 +-
.../TestS3MultipartUploadAbortResponseWithFSO.java | 85 ++
...S3MultipartUploadCommitPartResponseWithFSO.java | 226 +++++
...stS3MultipartUploadCompleteResponseWithFSO.java | 279 +++++
.../hadoop/ozone/om/upgrade/MockOmRequest.java | 32 +
.../ozone/om/upgrade/OMLayoutFeatureUtil.java | 66 ++
.../om/upgrade/TestOMLayoutFeatureAspect.java | 99 ++
.../ozone/om/upgrade/TestOMUpgradeFinalizer.java | 290 ++++++
.../ozone/om/upgrade/TestOMVersionManager.java | 195 ++++
.../TestOmVersionManagerRequestFactory.java | 111 ++
.../om/upgrade/TestOzoneManagerPrepareState.java | 283 ++++++
.../security/TestOzoneBlockTokenSecretManager.java | 4 +-
.../TestOzoneDelegationTokenSecretManager.java | 2 +-
.../ozone/security/TestOzoneManagerBlockToken.java | 2 +-
.../ozone/security/TestOzoneTokenIdentifier.java | 2 +-
.../security/acl/TestOzoneNativeAuthorizer.java | 7 +-
.../hadoop/ozone/security/acl/TestOzoneObj.java | 79 ++
.../hadoop/ozone/security/acl/TestParentAcl.java | 411 ++++++++
.../ozone/security/acl/TestRequestContext.java | 7 +-
.../hadoop/ozone/security/acl/TestVolumeOwner.java | 10 +-
hadoop-ozone/ozonefs-common/pom.xml | 42 +-
.../fs/ozone/BasicOzoneClientAdapterImpl.java | 34 +-
.../hadoop/fs/ozone/BasicOzoneFileSystem.java | 37 +
.../ozone/BasicRootedOzoneClientAdapterImpl.java | 28 +-
.../fs/ozone/BasicRootedOzoneFileSystem.java | 37 +
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 5 +-
.../apache/hadoop/fs/ozone/OzoneFSInputStream.java | 30 +-
hadoop-ozone/ozonefs-hadoop2/pom.xml | 16 +-
hadoop-ozone/ozonefs-hadoop3/pom.xml | 14 +-
hadoop-ozone/ozonefs-shaded/pom.xml | 14 +-
hadoop-ozone/ozonefs/pom.xml | 14 +-
hadoop-ozone/pom.xml | 142 +--
hadoop-ozone/recon-codegen/pom.xml | 10 +-
hadoop-ozone/recon/pom.xml | 43 +-
.../apache/hadoop/ozone/recon/ReconConstants.java | 7 +
.../hadoop/ozone/recon/ReconControllerModule.java | 18 +-
.../org/apache/hadoop/ozone/recon/ReconServer.java | 23 +-
.../ozone/recon/api/ClusterStateEndpoint.java | 3 +-
.../hadoop/ozone/recon/api/ContainerEndpoint.java | 17 +-
.../hadoop/ozone/recon/api/NodeEndpoint.java | 22 +-
.../ozone/recon/api/types/DatanodeMetadata.java | 28 +
.../hadoop/ozone/recon/api/types/NSSummary.java | 70 ++
.../hadoop/ozone/recon/codec/NSSummaryCodec.java | 81 ++
.../ozone/recon/fsck/ContainerHealthStatus.java | 10 +-
.../ozone/recon/fsck/ContainerHealthTask.java | 51 +-
.../ozone/recon/persistence/ContainerHistory.java | 3 +
.../hadoop/ozone/recon/scm/PipelineSyncTask.java | 46 +-
.../ozone/recon/scm/ReconContainerManager.java | 26 +-
.../ozone/recon/scm/ReconDeadNodeHandler.java | 79 ++
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 55 +-
.../scm/ReconStorageContainerManagerFacade.java | 32 +-
.../recon/spi/ContainerDBServiceProvider.java | 198 ----
.../recon/spi/ReconContainerMetadataManager.java | 193 ++++
.../recon/spi/ReconNamespaceSummaryManager.java | 37 +
.../recon/spi/StorageContainerServiceProvider.java | 6 +
.../spi/impl/ContainerDBServiceProviderImpl.java | 488 ---------
.../recon/spi/impl/ReconContainerDBProvider.java | 95 --
.../impl/ReconContainerMetadataManagerImpl.java | 458 +++++++++
.../ozone/recon/spi/impl/ReconDBDefinition.java | 13 +-
.../ozone/recon/spi/impl/ReconDBProvider.java | 117 +++
.../spi/impl/ReconNamespaceSummaryManagerImpl.java | 67 ++
.../impl/StorageContainerServiceProviderImpl.java | 8 +
.../ozone/recon/tasks/ContainerKeyMapperTask.java | 35 +-
.../ozone/recon/tasks/FileSizeCountTask.java | 5 +-
.../hadoop/ozone/recon/tasks/ReconTaskConfig.java | 4 +-
.../webapps/recon/ozone-recon-web/api/db.json | 20 +
.../src/views/datanodes/datanodes.tsx | 13 +-
.../ozone/recon/OMMetadataManagerTestUtils.java | 7 +-
.../hadoop/ozone/recon/ReconTestInjector.java | 20 +-
.../ozone/recon/api/TestContainerEndpoint.java | 15 +-
.../hadoop/ozone/recon/api/TestEndpoints.java | 23 +-
.../ozone/recon/api/TestOpenContainerCount.java | 427 ++++++++
.../recon/fsck/TestContainerHealthStatus.java | 6 +-
.../ozone/recon/fsck/TestContainerHealthTask.java | 97 +-
.../TestContainerHealthTaskRecordGenerator.java | 6 +-
.../recovery/TestReconOmMetadataManagerImpl.java | 9 +-
.../scm/AbstractReconContainerManagerTest.java | 31 +-
.../ozone/recon/scm/TestReconContainerManager.java | 5 +-
...TestReconIncrementalContainerReportHandler.java | 16 +-
.../ozone/recon/scm/TestReconNodeManager.java | 58 +-
.../ozone/recon/scm/TestReconPipelineManager.java | 23 +-
.../impl/TestContainerDBServiceProviderImpl.java | 386 -------
.../spi/impl/TestReconContainerDBProvider.java | 74 --
.../TestReconContainerMetadataManagerImpl.java | 388 +++++++
.../ozone/recon/spi/impl/TestReconDBProvider.java | 71 ++
.../impl/TestReconNamespaceSummaryManagerImpl.java | 96 ++
.../recon/tasks/TestContainerKeyMapperTask.java | 63 +-
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 5 +-
hadoop-ozone/s3gateway/pom.xml | 47 +-
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 5 +
.../org/apache/hadoop/ozone/s3/endpoint/S3Acl.java | 6 +
.../hadoop/ozone/s3/endpoint/S3BucketAcl.java | 6 +
.../apache/hadoop/ozone/s3/endpoint/S3Owner.java | 3 +
.../hadoop/ozone/s3/signature/SignatureInfo.java | 3 +
.../ozone/s3/TestEmptyContentTypeFilter.java | 5 +-
.../ozone/s3/TestVirtualHostStyleFilter.java | 2 +-
.../signature/TestAuthorizationV4HeaderParser.java | 2 +-
.../signature/TestAuthorizationV4QueryParser.java | 5 +-
.../hadoop/ozone/s3/util/TestOzoneS3Util.java | 2 +-
hadoop-ozone/tools/pom.xml | 50 +-
.../ozone/admin/om/CancelPrepareSubCommand.java | 53 +
.../admin/om/FinalizationStatusSubCommand.java | 64 ++
.../ozone/admin/om/FinalizeUpgradeSubCommand.java | 171 ++++
.../org/apache/hadoop/ozone/admin/om/OMAdmin.java | 44 +-
.../hadoop/ozone/admin/om/PrepareSubCommand.java | 173 ++++
.../admin/scm/FinalizationScmStatusSubcommand.java | 51 +
.../admin/scm/FinalizeScmUpgradeSubcommand.java | 156 +++
.../admin/scm/FinalizeUpgradeCommandUtil.java | 92 ++
.../apache/hadoop/ozone/admin/scm/ScmAdmin.java | 4 +-
.../ozone/audit/parser/common/DatabaseHelper.java | 7 +-
.../apache/hadoop/ozone/debug/DatanodeLayout.java | 8 +-
.../apache/hadoop/ozone/debug/PrefixParser.java | 236 +++++
.../ozone/debug/container/ContainerCommands.java | 14 +-
.../ozone/debug/container/ExportSubcommand.java | 3 +
.../ozone/debug/container/InfoSubcommand.java | 3 +
.../ozone/debug/container/ListSubcommand.java | 3 +
.../hadoop/ozone/freon/ChunkManagerDiskWrite.java | 4 +-
.../ozone/freon/ClosedContainerReplicator.java | 7 +-
.../apache/hadoop/ozone/freon/OmKeyGenerator.java | 5 +-
.../hadoop/ozone/freon/StreamingGenerator.java | 26 +-
.../freon/containergenerator/BaseGenerator.java | 3 +
.../containergenerator/GeneratorDatanode.java | 7 +-
.../freon/containergenerator/GeneratorOm.java | 11 +-
.../freon/containergenerator/GeneratorScm.java | 6 +-
.../hadoop/ozone/genesis/BenchMarkCRCBatch.java | 3 +
.../ozone/genesis/BenchMarkCRCStreaming.java | 3 +
.../ozone/genesis/BenchMarkContainerStateMap.java | 18 +-
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 4 +-
.../ozone/genesis/BenchMarkMetadataStoreReads.java | 70 --
.../genesis/BenchMarkMetadataStoreWrites.java | 62 --
.../ozone/genesis/BenchMarkOMKeyAllocation.java | 6 +-
.../ozone/genesis/BenchMarkOzoneManager.java | 13 +-
.../ozone/genesis/BenchMarkRocksDbStore.java | 119 ---
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 32 -
.../org/apache/hadoop/ozone/scm/cli/SQLCLI.java | 562 -----------
.../hadoop/ozone/audit/parser/TestAuditParser.java | 31 +-
.../TestGenerateOzoneRequiredConfigurations.java | 2 +-
.../tools/src/test/resources/testaudit.log | 3 +
.../tools/src/test/resources/testloadaudit.log | 21 +
pom.xml | 42 +-
1108 files changed, 50928 insertions(+), 12644 deletions(-)
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteArrayReader.java
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteBufferReader.java
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ByteReaderStrategy.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ha/RetriableWithFailOverException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfo.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLStatus.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutFeature.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutVersionManager.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSUpgradeAction.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/package-info.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/EntryConsumer.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/AbstractLayoutVersionManager.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/BasicUpgradeFinalizer.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/DefaultUpgradeFinalizationExecutor.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutFeature.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutVersionInstanceFactory.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutVersionManager.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/LayoutVersionManagerMXBean.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeActionHdds.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/UpgradeFinalizer.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/VersionFactoryKey.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/upgrade/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/upgrade/TestHDDSLayoutVersionManager.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/upgrade/test/MockComponent.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/InjectedUpgradeFinalizationExecutor.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestAbstractLayoutVersionManager.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestBasicUpgradeFinalizer.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestDefaultUpgradeFinalizationExecutor.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestLayoutVersionInstanceFactory.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestUpgradeFinalizerActions.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/upgrade/TestUpgradeUtils.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DataNodeLayoutVersion.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/DatanodeLayoutStorage.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/HDDSVolumeLayoutVersion.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/CRLStatusReportPublisher.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/FinalizeNewLayoutVersionCommandHandler.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/utils/StorageVolumeUtil.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeFactory.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolume.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/MetadataVolumeFactory.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolume.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/StorageVolumeFactory.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/stream/StreamingException.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DataNodeUpgradeFinalizer.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeMetadataFeatures.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/DatanodeSchemaV2FinalizeAction.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/ScmHAFirstUpgradeLayoutChangeAction.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/UpgradeUtils.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/upgrade/package-info.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/FinalizeNewLayoutVersionCommand.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestHddsVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestStorageVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/TestDirstreamClientHandler.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/stream/package-info.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDataNodeStartupSlvLessThanMlv.java
create mode 100644 hadoop-hdds/docs/content/design/nonrolling-upgrade.md
create mode 100644 hadoop-hdds/docs/content/design/omprepare.md
create mode 100644 hadoop-hdds/docs/content/design/upgrade-dev-primer.md
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Delete.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Format.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO-Rename.png
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO.md
create mode 100644 hadoop-hdds/docs/content/feature/how-to-do-a-nonrolling-upgrade.md
create mode 100644 hadoop-hdds/docs/content/feature/scm-secure-ha.png
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/CRLClientUpdateHandler.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/CRLStore.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/ClientCRLStore.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/ClientUpdateHandler.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/SCMUpdateClientConfiguration.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/SCMUpdateServiceGrpcClient.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/UpdateServiceConfig.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/client/package-info.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/CRLClientInfo.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateClientInfo.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/update/server/package-info.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/token/ContainerTokenGenerator.java
rename hadoop-hdds/{common => framework}/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLCodec.java (100%)
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfo.java
rename hadoop-hdds/{common => framework}/src/main/java/org/apache/hadoop/hdds/security/x509/crl/CRLInfoCodec.java (100%)
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/FixedThreadPoolExecutor.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/LevelDBStore.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/LevelDBStoreIterator.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/MetadataStore.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/MetadataStoreBuilder.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStore.java
delete mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreIterator.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/ShortCodec.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestMetadataStore.java
delete mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/TestRocksDBStoreMBean.java
rename hadoop-hdds/{interface-client => interface-server}/src/main/proto/InterSCMProtocol.proto (100%)
create mode 100644 hadoop-hdds/interface-server/src/main/proto/SCMUpdateProtocol.proto
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteStatusList.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/crl/CRLStatusReportHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/crl/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/HealthyReadOnlyNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NonHealthyToHealthyNodeHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/ReadOnlyHealthyToHealthyNodeHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableContainerFactory.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableContainerProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/SCMUpgradeFinalizer.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/ScmHAUnfinalizedStateValidationAction.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/ScmOnFinalizeActionForDatanodeSchemaV2.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMCRLStore.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMCRLUpdateHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateClientManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateServiceGrpcServer.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/SCMUpdateServiceImpl.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/update/server/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/balancer/TestContainerBalancer.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/crl/TestCRLStatusReportHandler.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/update/server/MockCRLStore.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/update/server/TestSCMUpdateServiceGrpcServer.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/TestSCMHAUnfinalizedStateValidationAction.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/TestScmStartupSlvLessThanMlv.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/upgrade/package-info.java
delete mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/GenericTestUtils.java
delete mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/LambdaTestUtils.java
delete mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/hadoop/test/TimedOutTestsListener.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/GenericTestUtils.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/LambdaTestUtils.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/TimedOutTestsListener.java
create mode 100644 hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/package-info.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerCommands.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStartSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStopSubcommand.java
create mode 100644 hadoop-hdds/tools/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestContainerBalancerSubCommand.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OmUpgradeConfig.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmDirectoryInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/WithParentObjectId.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzonePrefixPath.java
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/Dockerfile-krb5
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/README.md
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/kadm5.acl
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/krb5.conf
delete mode 100644 hadoop-ozone/dist/src/main/compose/common/docker-image/docker-krb5/launcher.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-om-prepare/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-om-prepare/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-om-prepare/docker-config
create mode 100755 hadoop-ozone/dist/src/main/compose/ozone-om-prepare/test.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-ha/krb5.conf
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure-mr/krb5.conf
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/keytabs/testuser2.keytab
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/krb5.conf
delete mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/.env
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/0.5.0/test.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/1.0.0/test.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/docker-config
create mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/load.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
create mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
delete mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/docker-compose.yaml
delete mode 100644 hadoop-ozone/dist/src/main/compose/upgrade/docker-config
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/upgrade_to_current.sh
delete mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/upgrade_to_release.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/0.5.0-1.1.0/callback.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/upgrades/non-rolling-upgrade/1.0.0-1.1.0/callback.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/upgrade/upgrades/non-rolling-upgrade/driver.sh
create mode 100755 hadoop-ozone/dist/src/main/keytabs/HTTP.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/dn.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/hadoop.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/jhs.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/nm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/om.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/recon.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/rm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/s3g.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/scm.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/testuser.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/testuser2.keytab
create mode 100755 hadoop-ozone/dist/src/main/keytabs/update-keytabs.sh
create mode 100644 hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-org.aspectj.html
create mode 100644 hadoop-ozone/dist/src/main/license/bin/licenses/LICENSE-org.reflections-reflections.txt
create mode 100644 hadoop-ozone/dist/src/main/smoketest/omha/om-cancel-prepare.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/omha/om-prepare.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/omha/om-prepared.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/check-mlv.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/finalize.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/generate.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/prepare.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/upgrade/validate.robot
create mode 100755 hadoop-ozone/dist/src/shell/upgrade/1.0.0.sh
rename hadoop-ozone/dist/src/shell/upgrade/{v1 => 1.0.0}/01-migrate-scm-db.sh (100%)
delete mode 100755 hadoop-ozone/dist/src/shell/upgrade/v1.sh
create mode 100644 hadoop-ozone/dist/src/test/shell/failing1/test.sh
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestDirectoryDeletingServiceWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileInterfacesWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemPrefixParser.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystemWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestRootedOzoneFileSystemWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/contract/ITestOzoneContractUtils.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestHDDSUpgrade.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestContainerBalancerOperations.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneClientMultipartUploadWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/dn/volume/TestDatanodeHddsVolumeFailureToleration.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestHadoopDirTreeGeneratorWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMStartupWithLayout.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestObjectStoreWithFSO.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupSlvLessThanMlv.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerPrepare.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestRecursiveAclWithFSO.java
create mode 100644 hadoop-ozone/interface-storage/src/main/java/org/apache/hadoop/ozone/om/codec/OmDirectoryInfoCodec.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerPrepareState.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzonePrefixPathImpl.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMAllocateBlockRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyDeleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRenameRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMPathsPurgeRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAddAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyRemoveAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeySetAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3InitiateMultipartUploadRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadAbortRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCommitPartRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCompleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMCancelPrepareRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMFinalizeUpgradeRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/OMPrepareRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/upgrade/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/file/OMDirectoryCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/file/OMFileCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMAllocateBlockResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyCommitResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyDeleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyRenameResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMPathsPurgeResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/acl/OMKeyAclResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3InitiateMultipartUploadResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadAbortResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCommitPartResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCompleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMCancelPrepareResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMFinalizeUpgradeResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMPrepareResponse.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/BelongsToLayoutVersion.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/DisallowedUntilLayoutVersion.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeature.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeatureAspect.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutVersionManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMUpgradeFinalizer.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OmUpgradeAction.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/UpgradeActionOm.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/bucket/TestOMBucketCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMDirectoryCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMAllocateBlockRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyAclRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCommitRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCreateRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyDeleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3InitiateMultipartUploadRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadAbortRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCommitPartRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCompleteRequestWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/upgrade/TestOMCancelPrepareRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/file/TestOMDirectoryCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/file/TestOMFileCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMAllocateBlockResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyCommitResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyCreateResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/key/TestOMKeyDeleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3InitiateMultipartUploadResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadAbortResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadCommitPartResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadCompleteResponseWithFSO.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/MockOmRequest.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeatureUtil.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeatureAspect.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMUpgradeFinalizer.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMVersionManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOmVersionManagerRequestFactory.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOzoneManagerPrepareState.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestOzoneObj.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/acl/TestParentAcl.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/NSSummary.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/codec/NSSummaryCodec.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconDeadNodeHandler.java
delete mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ContainerDBServiceProvider.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ReconContainerMetadataManager.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/ReconNamespaceSummaryManager.java
delete mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ContainerDBServiceProviderImpl.java
delete mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerDBProvider.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconContainerMetadataManagerImpl.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconDBProvider.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/ReconNamespaceSummaryManagerImpl.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestOpenContainerCount.java
delete mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestContainerDBServiceProviderImpl.java
delete mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerDBProvider.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconContainerMetadataManagerImpl.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconDBProvider.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/spi/impl/TestReconNamespaceSummaryManagerImpl.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/CancelPrepareSubCommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/FinalizationStatusSubCommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/FinalizeUpgradeSubCommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/PrepareSubCommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/FinalizationScmStatusSubcommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/FinalizeScmUpgradeSubcommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/FinalizeUpgradeCommandUtil.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/PrefixParser.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreReads.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkMetadataStoreWrites.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkRocksDbStore.java
delete mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/scm/cli/SQLCLI.java
create mode 100644 hadoop-ozone/tools/src/test/resources/testloadaudit.log
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org